summaryrefslogtreecommitdiffstats
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* devtool: ide_sdk: Use bitbake's python3 for generated scriptsEnguerrand de Ribaucourt2024-02-201-1/+1
| | | | | | | | | | | | | | | | | | The generated scripts use the sys.path configuration found inside bitbake. It can be a different python version than the one used on the host through the IDE. For instance, when running the generated script deploy_target_cmake-example-core2-64 from an eSDK generated on another machine, I got the following exception: AssertionError: SRE module mismatch We need to match the sys.executable to the sys.path. (From OE-Core rev: 45704319661570b45ef69fddd6b4b4fa22ca80d1) Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool; add support for python_mesonpy classTim Orling2024-02-201-0/+4
| | | | | | | | | | | | | | | | | * Add support to detect the "mesonpy" build-backend for recipetool create. * Add oe-selftest case for creating a recipe for "siphash24" from pypi. https://pypi.org/project/siphash24/ This is by far the simplest recipe using the mesonpy build backend. Upstream does not provide LICENSE file(s) and we do not detect the LICENSE so don't check for that result in the test. Likewise, upstream does not define HOMEPAGE, so skip that result. (From OE-Core rev: 256749322671d2f4ea994db671d73c4de10e1723) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/patch: Use git notes to store the filenames for the patchesPeter Kjellerstedt2024-02-192-11/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The old way of keeping track of the filenames for the patches that correspond to the commits was to add a special comment line to the end of the commit message, e.g., "%% original patch: <filename>", using a temporary git hook. This method had some drawbacks, e.g.: * It caused problems if one wanted to push the commits upstream as the comment line had to be manually removed. * The comment line would end up in patches if someone used git format-path rather than devtool finish to generate the patches. * The comment line could interfere with global Git hooks used to validate the format of the Git commit message. * When regenerating patches with `devtool finish --force-patch-refresh`, the process typically resulted in adding empty lines to the end of the commit messages in the updated patches. A better way of keeping track of the patch filenames is to use Git notes. This way the commit messages remain unaffected, but the information is still shown when, e.g., doing `git log`. A special Git notes space, refs/notes/devtool, is used to not intefere with the default Git notes. It is configured to be shown in, e.g., `git log` and to survive rewrites (i.e., `git commit --amend` and `git rebase`). Since there is no longer any need for a temporary Git hook, the code that manipulated the .git/hooks directory has also been removed. To avoid potential problems due to global Git hooks, --no-verify was added to the `git commit` command. To not cause troubles for those who have done `devtool modify` for a recipe with the old solution and then do `devtool finish` with the new solution, the code will fall back to look for the old strings in the commit message if no Git note can be found. While not technically motivated like above, the way to keep track of ignored commits is also changed to use Git notes to avoid having different methods to store similar information. (From OE-Core rev: f5e6183b9557477bef74024a587de0bfcc2b7c0d) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Make use of oe.patch.GitApplyTree.commitIgnored()Peter Kjellerstedt2024-02-192-8/+2
| | | | | | | | | | This makes use of the oe.patch.GitApplyTree.commitIgnored() function to create commits that shall be ignored by `devtool finish`. (From OE-Core rev: 4e1b9289450b5e7946bd5999c95a3ee214aab8a5) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-setup-build: add a tool for discovering config templates and setting up ↵Alexander Kanavin2024-02-193-0/+139
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | builds This is another piece of the puzzle in setting up builds from nothing without having to write custom scripts or use external tools. After layers have been fetched and placed into their respective locations by oe-setup-layers, one would surely want to proceed to the actual build, and here's how: 1. Without arguments the tool reads available layers from .oe-layers.json file (written out by oe-setup-layers or a fallback under scripts/), prints what templates it has found, and asks the user to select one, as seen below. This will land the user in a shell ready to run bitbake: ============================================= alex@Zen2:/srv/work/alex$ ./setup-build Available build configurations: 1. alex-configuration-gadget This configuration will set up a build for the purposes of supporting gadget. 2. alex-configuration-gizmo This configuration allows building a gizmo. 3. poky-default This is the default build configuration for the Poky reference distribution. Re-run with 'list -v' to see additional information. Please choose a configuration by its number: 1 Running: TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/configuration-gadget . /srv/work/alex/poky/oe-init-build-env /srv/work/alex/build-alex-configuration-gadget && /bin/bash You had no conf/local.conf file. This configuration file has therefore been created for you from /srv/work/alex/meta-alex/conf/templates/configuration-gadget/local.conf.sample You may wish to edit it to, for example, select a different MACHINE (target hardware). You had no conf/bblayers.conf file. This configuration file has therefore been created for you from /srv/work/alex/meta-alex/conf/templates/configuration-gadget/bblayers.conf.sample To add additional metadata layers into your configuration please add entries to conf/bblayers.conf. The Yocto Project has extensive documentation about OE including a reference manual which can be found at: https://docs.yoctoproject.org For more information about OpenEmbedded see the website: https://www.openembedded.org/ This configuration will set up a build for the purposes of supporting gadget. Please refer to meta-alex/README for additional details and available bitbake targets. ============================================== 2. It is also possible to list available configurations without selecting one using 'setup-build list' or to select and setup one non-interactively with 'setup-build setup'. 3. The full set of command line options is: $ ./setup-build --help usage: setup-build [-h] [--layerlist LAYERLIST] {list,setup} ... A script that discovers available build configurations and sets up a build environment based on one of them. Run without arguments to choose one interactively. positional arguments: {list,setup} list List available configurations setup Set up a build environment and open a shell session with it, ready to run builds. optional arguments: -h, --help show this help message and exit --layerlist LAYERLIST Where to look for available layers (as written out by setup-layers script) (default is /srv/work/alex/.oe-layers.json). $ ./setup-build list --help usage: setup-build list [-h] [-v] optional arguments: -h, --help show this help message and exit -v Print detailed information and usage notes for each available build configuration. $ ./setup-build setup --help usage: setup-build setup [-h] [-c configuration_name] [-b build_path] [--no-shell] optional arguments: -h, --help show this help message and exit -c configuration_name Use a build configuration configuration_name to set up a build environment (run this script with 'list' to see what is available) -b build_path Set up a build directory in build_path (run this script with 'list -v' to see where it would be by default) --no-shell Create a build directory but do not start a shell session with the build environment from it. 4. There's an an added hint in oe-setup-layers about how to proceed (as it is really not user-friendly to fetch the layer repos successfully and then exit without a word), and a symlink to the script from the top level layer checkout directory. 5. The selftest to check layer setup has been adjusted to run a basic check for template discovery and build setup. The revision of poky to be cloned has been bumped to 4.1, as that's the first version with a default template in a standard location. (From OE-Core rev: 1360b64e88cda7dddfb0eca6a64f70c13dafb890) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-setup-builddir: add support for configuration summariesAlexander Kanavin2024-02-191-0/+10
| | | | | | | | | They are handled exactly same as conf-notes.txt. (From OE-Core rev: 6fd2eb619693da4cb918b421f3e31072b48fa85a) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: add stronger indication for failed testsSimone Weiß2024-02-191-2/+2
| | | | | | | | | | | | | Do not only log that there has been an issue but add WARNING before for local runs. Hopefully this helps to avoid that people reading the log to quickly miss issues. Fixes [YOCTO #15389] (From OE-Core rev: 661c0a8fd8fe7bff61cea82778c25cf24d791267) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-setup-layers: write a list of layer paths into the checkout's top dirAlexander Kanavin2024-02-191-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is beneficial for setting up builds, as this list can be used to determine reliably where the actual layers are, and discover available configurations from them. Also adjust the selftest to check the presence of that file rather than any specific layer in a hardcoded location. Sample output (paths are written relative to the file for relocatability and ease of reading): { "layers": [ "meta-openembedded/meta-filesystems", "meta-openembedded/meta-gnome", "meta-openembedded/meta-initramfs", "meta-openembedded/meta-multimedia", "meta-openembedded/meta-networking", "meta-openembedded/meta-oe", "meta-openembedded/meta-perl", "meta-openembedded/meta-python", "meta-openembedded/meta-webserver", "meta-openembedded/meta-xfce", "poky/bitbake/lib/layerindexlib/tests/testdata/layer1", "poky/bitbake/lib/layerindexlib/tests/testdata/layer2", "poky/bitbake/lib/layerindexlib/tests/testdata/layer3", "poky/bitbake/lib/layerindexlib/tests/testdata/layer4", "poky/meta-poky", "poky/meta-selftest", "poky/meta-skeleton", "poky/meta-yocto-bsp", "poky/meta" ], "version": "1.0" } (From OE-Core rev: 82743f4f767f8016564be0d9d6c0d8fe9e067740) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-init-build-env: generate .vscode from templateAdrian Freihofer2024-02-191-0/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | Provide a reasonable default configuration for VSCode and the yocto.bitbake extension. The generated default configuration is generic and minimal. It's mostly supposed to prevent VSCode from OOM exceptions if the build directory is in the scope of the indexer plugins of VSCode. But it also configures the yocto-bitbake plugin to just work without manual user interaction. The configuration is only generated if VSCode is installed. Currently, VSCode is one of many popular editors for Yocto/OE. Removing the check would mean that the configuration would be generated by e.g. oe-selftest or for users not using VSCode. If it should prove useful, the check can be removed later. Customization for other layers is possible. A layer might provide it's own oe-setup-build-env script which calls the oe-setup-vscode script from poky with different folders. But it's also possible to override the oe-setup-vscode script by another layer with a custom implementation. (From OE-Core rev: 48829be7ab2edcbc2e4473f81cdaf35889d63f9c) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: _extract_source: Correct the removal of an old backup directoryPeter Kjellerstedt2024-02-181-6/+6
| | | | | | | | | Also correct the comment describing what is happening. (From OE-Core rev: 7d867753fba8d536bef4c72c7bea3f4ed26a1a95) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: standard: Add some missing whitespacePeter Kjellerstedt2024-02-181-12/+12
| | | | | | | | | Makes it a little bit easier when reading the code. (From OE-Core rev: 6a757f9648685448fd18507f6aaf4eed0a57579f) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: ide-sdk make deploy-target quickerAdrian Freihofer2024-02-181-5/+38
| | | | | | | | (From OE-Core rev: 3b63e7adb5596739b846396304ff815859ce6a74) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: new ide-sdk pluginAdrian Freihofer2024-02-184-0/+1790
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new devtool ide plugin provides the eSDK and configures an IDE to work with the eSDK. In doing so, bitbake should be used to generate the IDE configuration and update the SDK, but it should no longer play a role when working on the source code. The work on the source code should take place exclusively with the IDE, which, for example, calls cmake directly to compile the code and execute the unit tests from the IDE. The plugin works for recipes inheriting the cmake or the meson bbclass. Support for more programming languages and build tools may be added in the future. There are various IDEs that can be used for the development of embedded Linux applications. Therefore, devtool ide-sdk, like devtool itself, supports plugins to support IDEs. VSCode is the default IDE for this first implementation. Additionally, some generic helper scripts can be generated with --ide none instead of a specific IDE configuration. This can be used for any IDE that supports calling some scripts. There are two different modes supported: - devtool modify mode (default): devtool ide-sdk configures the IDE to manage the build-tool used by the recipe (e.g. cmake or meson). The workflow looks like: $ devtool modify a-recipe $ devtool ide-sdk a-recipe a-image $ code "$BUILDDIR/workspace/sources/a-recipe" Work in VSCode, after installing the proposed plugins Deploying the artifacts to the target device and running a remote debugging session is supported as well. This first implementation still calls bitbake and devtool to copy the binary artifacts to the target device. In contrast to compiling, installation and copying must be performed with the file rights of the target device. The pseudo tool must be used for this. Therefore bitbake -c install a-recipe && devtool deploy-target a-recipe are called by the IDE for the deployment. This might be improved later on. Executing the unit tests out of the IDE is supported via Qemu user if the build tool supports that. CMake (if cmake-qemu.bbclass is inherited) and Meson support Qemu usermode. - Shared sysroots mode: bootstraps the eSDK with shared sysroots for all the recipes passed to devtool ide-sdk. This is basically a wrapper for bitbake meta-ide-support && bitbake build-sysroots. The workflow looks like: $ devtool ide-sdk --share-sysroots a-recipe another-recipe vscode where/the/sources/are If the IDE and the build tool support it, the IDE gets configured to offer the cross tool-chain provided by the eSDK. In case of VSCode and cmake a cmake-kit is generated. This offers to use the cross tool-chain from the UI of the IDE. Many thanks to Enguerrand de Ribaucourt for testing and bug fixing. (From OE-Core rev: 3f8af7a36589cd05fd07d16cbdd03d6b3dff1f82) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate-cache-management: fix regex for 'sigdata' stamp filesAndré Draszik2024-02-171-1/+1
| | | | | | | | | | | | | Given file names like 6.4.do_fetch.sigdata.821b6c62f9f2bd8b7e1378656b8319697a21f6f6e4a351f98dc325a18ef7ed0f, I'm pretty sure we want to match the dot here, not any character. Fixes: 2fa1b25d7485 ("sstate-cache-management: Rewrite in python") in oe-core Fixes: b723fcaac52f ("sstate-cache-management: Rewrite in python") in poky (From OE-Core rev: 64281de7e352aaf48aa8c60b68633c54471fe353) Signed-off-by: André Draszik <andre.draszik@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest-send-results: add --debug optionTrevor Gamblin2024-02-161-6/+11
| | | | | | | | | | | Add a -d/--debug option to patchtest-send-results so that the contents of the constructed raw email can be checked without actually sending anything to the list or patch author. (From OE-Core rev: e50827d23fd3789e7a13df6918a06b198bbf4f95) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: Fix grammar in log outputTrevor Gamblin2024-02-161-1/+1
| | | | | | | (From OE-Core rev: ded6eb17dc423988cc975d72c36fe9ba6f28eab8) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: Correct appending of type=git-dependency to URIsPeter Kjellerstedt2024-02-161-2/+2
| | | | | | | | | | A missing space when using :append would lead to run-on URIs if there was no whitespace at the end of the original SRC_URI. (From OE-Core rev: 4de0c679e4b1a3cb394f348d625b97ad73f1efe7) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest-send-results: use Message-ID directlyTrevor Gamblin2024-02-151-1/+3
| | | | | | | | | | | | | | There's no need to use regex for extracting the Message-ID field from the patch email and mangle it by removing the angle brackets in the process. Pull it directly from the mbox so that Patchtest's replies have even fewer differences when compared to other replies. Also add a TODO so that it's clear this needs adjustment when full series support is added. (From OE-Core rev: 1ce51580a23869a9c0f409446ff717d471fb60db) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: log errors and failures at endSimone Weiß2024-02-151-2/+9
| | | | | | | | | | | | | | | At the moment, running patchtest locally will only print failures and errors to the log when the not passing test case is executed. This might lead to people overlooking issues with their patches, so print a log line at the end if testcases showed issues. This should make it more easy to spot then before. Fixes [YOCTO #15389] (From OE-Core rev: 84ca5a5f5a44de6ed4551ab08e58087aaa7e1369) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: python 3.12 regexAdrian Freihofer2024-02-1310-39/+39
| | | | | | | | | | | | | | | | | | All the regexes throw a warning like this: WARNING: scripts/lib/recipetool/create_buildsys.py:140: SyntaxWarning: invalid escape sequence '\s' proj_re = re.compile('project\s*\(([^)]*)\)', re.IGNORECASE) Python 3 interprets string literals as Unicode strings, and therefore \s is treated as an escaped Unicode character which is not correct. Declaring the RegEx pattern as a raw string instead of unicode is required for Python 3. (From OE-Core rev: 24b0ba00d4f0b4d9834f7693ecb6032dfc534a80) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest-send-results: Add 'References' headerTrevor Gamblin2024-02-101-0/+5
| | | | | | | | | | | | | | | | | Based on https://www.rfc-editor.org/rfc/rfc2822#section-3.6.4, to get proper threading there should be a 'References' field defined in Patchtest's response headers if its replies are going to thread properly on Patchwork. This wasn't there before, hence why were were seeing the responses in the right place on the mailing list but not Patchwork. Since we only want Patchtest to reply directly to the tested patch, the 'References' field should only need to contain the same ID as the 'In-Reply-To' one. (From OE-Core rev: f8a2af973d829c3c7f2dce39ce452daf74f31f52) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: don't dump stack traces if a toml parser can't be foundRoss Burton2024-02-101-6/+14
| | | | | | | | | | | | | | | | | If we can't find tomllib or tomli then we can just tell the user politely that we can't parse the pyproject.toml file, there's no need to dump exception stack traces. Move the parser exception handler to catch the actual parse, as otherwise it will never be used. Whilst here, also add some debug statements to make it clear what of the handlers is being called. (From OE-Core rev: 1a3ba4c312844d80ae382912b319e60ad8b30737) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: cleanup importsAdrian Freihofer2024-02-081-1/+1
| | | | | | | | | | - os is missing - glob is not used (From OE-Core rev: f560bc8972e97d2e497d7b976586e4a79ffc74ec) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: implement reproducible Disk GUIDAdithya Balakumar2024-02-081-0/+11
| | | | | | | | | | | GPT based disks have a disk guid apart from the 32-bit disk identifier. This commit implements reproducible disk guid by using SOURCE_DATE_EPOCH (if available) value as a random seed (From OE-Core rev: 150e079589e207fe174d2dceb40cd8f3d3972c5a) Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest.README: update mailing listTrevor Gamblin2024-02-061-5/+2
| | | | | | | | | | | The contributing info still lists the main Yocto list, but patchtest is in oe-core, so simplify the instructions and put the right mailing list in. (From OE-Core rev: 2cfad933c334b3d7f49bc8ca62f9d443428d4523) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest-send-results: properly parse test statusAlexis Lothoré2024-02-061-1/+4
| | | | | | | | | | | | | | | | | | | | patchtest-send-results currently search the word "FAIL" in the whole testresults file to decide whether it should send a report to patch submitter or not. This global search can lead to false positives, for example if the commit subject contains the word "FAIL" (as observed in [1]) Prevent those false positives by explicitely parsing the test status from each line. Each test result line, generated by the patchtest script, is expected to have the following format: <STATUS>: <some info, depending on the status> [1] https://lore.kernel.org/openembedded-core/0101018d79bfe020-06f2ce89-ea19-456b-92e7-66ee1c710fd1-000000@us-west-2.amazonses.com/ (From OE-Core rev: 3567c21af8ed65448f9325ee3fe85b8be839e1b5) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest-send-results: remove unused variableAlexis Lothoré2024-02-061-1/+0
| | | | | | | | | result_basename is declared but not used in the script (From OE-Core rev: c2ba125dc30fb1ef0bf96152863db22159f4b31c) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: refactor deploy-targetAdrian Freihofer2024-02-021-114/+122
| | | | | | | | | | | | | | Make the deploy function independent from d. This allows to call the function also from Python code not running in bitbake. This is needed to for the devtool ide plugin which will call the do_install task and the code from devtool deploy-target independently from a bitbake server. This allows a much quicker workflow. (From OE-Core rev: c8697d1132cbd4b2a2502b4c48e7d91fc18de786) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: refactor deploy to use exec_fakeroot_no_dAdrian Freihofer2024-02-021-4/+7
| | | | | | | | | | | This is a step towards a deploy function which can be called without passing the d variable. (From OE-Core rev: 3e8c41ec373a1766f68932d3b979d00e4e3bb3a9) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: refactor exec_fakerootAdrian Freihofer2024-02-021-1/+4
| | | | | | | | | | | | | Provide a function exec_fakeroot_no_d which does the same like exec_fakeroot does, but is usable independenlty from bitbake. This allows to use the fanction from scripts where the d variable is not available. (From OE-Core rev: 6f5980d96844559639fc3660bb975b0bdd1aa523) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: support plugins with pluginsAdrian Freihofer2024-02-021-2/+3
| | | | | | | | | | | Pass the plugins search path to plugins via context. This allows plugins to search for their own plugins at the same paths. (From OE-Core rev: e9c3c996f2e9a52462c4426f394798a211e53ae1) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: add support for multiple source in SRC_URIJulien Stephan2024-01-241-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [YOCTO #15162] when doing devtool modify, sources are extracted into a devtool temporary workdir. The main source is moved inside build/workspace/sources/${BPN}/ and local files are moved inside build/workspace/sources/${BPN}/oe-local-files. Secondary sources are currently not handled and are lost. Here is the output of devtool modify/build on bzip2 recipe: NOTE: bzip2: compiling from external source tree <...>/build/workspace/sources/bzip2 ERROR: bzip2-1.0.8-r0 do_install_ptest_base: ExecutionError('<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368', 1, None, None) ERROR: Logfile of failure stored in: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/log.do_install_ptest_base.3368 Log data follows: | DEBUG: Executing shell function do_install_ptest_base | NOTE: make -j 16 DESTDIR=<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest install-ptest | sed -n '/^runtest:/,/^install-ptest:/{/^install-ptest:/!p}' \ | ../../../../../../workspace/sources/bzip2/Makefile.am > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/Makefile | cp ../../../../../../workspace/sources/bzip2/sample1.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | cp ../../../../../../workspace/sources/bzip2/sample2.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | cp ../../../../../../workspace/sources/bzip2/sample3.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | cp ../../../../../../workspace/sources/bzip2/sample1.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | cp ../../../../../../workspace/sources/bzip2/sample2.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | cp ../../../../../../workspace/sources/bzip2/sample3.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/ | ln -s /usr/bin/bzip2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2 | cp: cannot stat '<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/git/commons-compress': No such file or directory | WARNING: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368:189 exit 1 from 'cp -r <...>/build/tmp/work/core2-64-poky-linux/bzip2/ 1.0.8/git/commons-compress <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2-tests/commons-compress' | WARNING: Backtrace (BB generated script): | #1: do_install_ptest, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 189 | #2: do_install_ptest_base, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 158 | #3: main, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 226 ERROR: Task (<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base) failed with exit code '1' NOTE: Tasks Summary: Attempted 776 tasks of which 765 didn't need to be rerun and 1 failed. Summary: 1 task failed: <...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base externalsrc class modify SRC_URI to keep only: * 'file', 'npmsw' and 'crate' sources * url with type parameter matching 'kmeta' or 'git-dependency' So by forcing to add type='git-dependency' on secondary sources, we ensure that when building the recipe, the secondary sources can be unpacked into WORKDIR. This allows recipes containing several sources to be built under a devtool context, but it has some limitations: * user would not be able to generate patches for the secondary sources * type="git-dependency" is added for secondary sources even on non git sources, so we may want to rename this parameter (From OE-Core rev: cfd5ee890163a3d975093359016dda104e7b71df) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Disregard version in URL for replaced modulesVyacheslav Yurkov2024-01-191-5/+17
| | | | | | | | | | | | | | | | Major module version is a part of name, but not necessary part of the actual URL (See https://go.dev/ref/mod#module-path). Nevertheless, name detection function can't handle that suffix, so get rid of it to determine component name. For replaced modules that name might be different that the actual module name defined in go.mod file. (From OE-Core rev: 0cccfa1041d48f0ae3a2dc89a129cf7884fc08f0) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Proceed even with a missing license fileVyacheslav Yurkov2024-01-191-0/+6
| | | | | | | | | | | Whenever the recipe uses a CLOSED license, the list is going to be empty. It's a discouraged practice not to have a license, but proceed anyway to finish recipe generation. (From OE-Core rev: 5ca920284d0946346f5b06f5e443c80d9d8b85ce) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Don't fail on local go modulesVyacheslav Yurkov2024-01-191-13/+23
| | | | | | | | | | Local modules are usually referenced with a 'replace' directive in go.mod file. If that's the case, remove them from populating SRC_URI. (From OE-Core rev: 9f220f61e3e44a650a46ee997b47f1d87b7c4ef0) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard: correctly escape \Alexander Kanavin2024-01-091-1/+1
| | | | | | | | | | | | python 3.12 points out that: SyntaxWarning: invalid escape sequence '\*' (From OE-Core rev: bafb4b4edb4fb7908fdda272b7b2c2cbdef4728b) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: fix exceptionJamin Lin2024-01-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Root Cause: initial_revs is an empty dictionary and do not have "." key. Traceback (most recent call last): File "scripts/devtool", line 349, in <module> ret = main() File "scripts/devtool", line 336, in main ret = args.func(args, config, basepath, workspace) File "scripts/lib/devtool/standard.py", line 922, in modify if not initial_revs["."]: KeyError: '.' Solution: check key exists, then get its value. (From OE-Core rev: fb0db5c48abb4d56233a175fdd349d18b972e452) Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: match .rootfs. in addition to -image- for rootfsMikko Rapeli2024-01-041-1/+1
| | | | | | | | | | | | | | | | | Also change path.exists() and !path.isdir() to a single path.isfile() which should be equal. Enables running tests against image recipes which are not called "bla-image" but plain "bla". Currently they fail with do_testimage/runqemu error: runqemu - ERROR - Unknown path arg /home/builder/src/base/build/tmp_qemuarm64/deploy/images/qemuarm64/img-qemuarm64.rootfs.wic Suggested-by: Richard Purdie <richard.purdie@linuxfoundation.org> (From OE-Core rev: 107d0db3dc3678e6f0ae4035f4c0b86c6b421168) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/runqemu: fix regex escape sequencesTrevor Gamblin2024-01-021-4/+4
| | | | | | | | | | | | | | | | | | | When invoking runqemu with Python 3.12, the following warning is encountered: |SyntaxWarning: invalid escape sequence '\.' This is because the interpreter scans the string before it is processed by the regex module, and it interprets the backslash as part of an escape sequence, but not a standard one. This will be registered as an error rather than a warning in future Python versions. To avoid the it, simply add an extra backslash so that Python doesn't misinterpret the string, while the regex parser still sees an escaped '.' character. (From OE-Core rev: 0e8a4142bb90a92d175df6b2537d24a372356f98) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: use straight print in check-upgrade-status outputChen Qi2024-01-021-1/+1
| | | | | | | | | | | | 'devtool check-upgrade-status' is for reporting upgradable status for recipes. The output should always be printed out. So we should just use 'print' instead of 'logger.info' as the latter will be suppressed if '-q' parameter is supplied to devtool. (From OE-Core rev: 2c7bf9c8a833bec13a1ebabdce30933cbe691108) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Drop shell sstate-cache-managementRichard Purdie2023-12-281-458/+0
| | | | | | | | Since there is a faster python version, drop the slower shell one. (From OE-Core rev: 0551fa2ae3dacf51825ff5c1d081e713a150ddd2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate-cache-management: Rewrite in pythonAlex Kiernan2023-12-281-0/+329
| | | | | | | | | | This (should be) a drop in replacement for sstate-cache-management.sh. (From OE-Core rev: 2fa1b25d7485bfbb92bcc33067beb6751218b36a) Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy: provide max_process to strip_execsClay Chang2023-12-231-1/+2
| | | | | | | | | | Use oe.utils.get_bb_number_threads to get max_process (From OE-Core rev: f0056dca0a44c374f1f0c5fccbf66ae88e0b1850) Signed-off-by: Clay Chang <clayc@hpe.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 imagesvenkata pyla2023-12-211-0/+14
| | | | | | | | | | | | | E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file system. hash_seed: creates reproducible directory indexes in the file system. Reference commit in e2fsprogs: e1f7100643a46456be107b33098f6034b0835e6d (From OE-Core rev: bb822ab75de0020572058090439b93cc56bbf7e0) Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: add python_maturin supportTim Orling2023-12-171-0/+1
| | | | | | | | | | | Add the new python_maturin PEP-517 backend Add selftest for 'pydantic-core' pypi package. (From OE-Core rev: 69b679380616a94a631681caa05d9bf7610f9372) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: Handle recipes with a menuconfig task correctlyPeter Kjellerstedt2023-12-161-1/+1
| | | | | | | | | | | | | | | | This avoids the following error when running `devtool modify` on a recipe that has a menuconfig task, but does not have KCONFIG_CONFIG_ENABLE_MENUCONFIG set. .../temp/run.do_configure.4163366: line 152: ${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') }: bad substitution WARNING: .../temp/run.do_configure.4163366:152 exit 1 from '[ ${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') } = True ]' (From OE-Core rev: e87e6fa84a0c4b5ac8e736dc62f6e08390ba2436) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: add qmp socket supportRoss Burton2023-12-151-0/+11
| | | | | | | | | | Add support for qmp sockets and defaults to unix:qmp.sock if unspecified (From OE-Core rev: 380631797f0d63124a8c21efa93ab672dbd79283) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: pypi: do not clobber SRC_URI checksumsTim Orling2023-12-091-6/+1
| | | | | | | | | | | | | | | | | The pypi change: "85a2a6f68af recipetool: create_buildsys_python: add pypi support" deleted all the SRC_URI variables, including the SRC_URI checksums. These are not generated by the pypi.bbclass (how could they be trusted?) Without the checksum(s), we are vulnerable to a man-in-the-middle attack and zero checks on the validity of the downloaded tarball from pypi.org. Fix by only setting S and SRC_URI to None. (From OE-Core rev: 560181a52111569f7bc57b09139b42510e0d0325) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: Make --no-extract work againPeter Kjellerstedt2023-12-081-1/+2
| | | | | | | | | | | | | | | | | | | | This avoids the following error when using --no-extract, introduced in commit 900129cbdf (devtool: add support for git submodules): Traceback (most recent call last): File ".../scripts/devtool", line 349, in <module> ret = main() File ".../scripts/devtool", line 336, in main ret = args.func(args, config, basepath, workspace) File ".../scripts/lib/devtool/standard.py", line 995, in modify for commit in commits[name]: KeyError: '.' (From OE-Core rev: 6a44fb10ef4b90ee24f470362007d56183abc593) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: upgrade: Update all existing checksums for the SRC_URIPeter Kjellerstedt2023-12-081-23/+28
| | | | | | | | | | | | | In addition to updating the sha256sum and removing the md5sum, update all other existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums (currently only sha256sum). (From OE-Core rev: 8ea8827ee49b7f0443b1c4bd47d1344a689d73a3) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-04-19 20:45:25 +0000