summaryrefslogtreecommitdiffstats
path: root/scripts/lib
Commit message (Collapse)AuthorAgeFilesLines
* devtool: add support for git submodulesJulien Stephan2023-12-014-125/+220
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding the support of submodules required a lot of changes on the internal data structures: * initial_rev/startcommit used as a starting point for looking at new / updated commits was replaced by a dictionary where the keys are the submodule name ("." for main repo) and the values are the initial_rev/startcommit * the extractPatches function now extracts patch for the main repo and for all submodules and stores them in a hierarchical way describing the submodule path * store initial_rev/commit also for all submodules inside the recipe bbappend file * _export_patches now returns dictionaries that contains the 'patchdir' parameter (if any). This parameter is used to add the correct 'patchdir=' parameter on the recipe Also, recipe can extract a secondary git tree inside the workdir. By default, at the end of the do_patch function, there is a hook in devtool that commits everything that was modified to have a clean repository. It uses the command: "git add .; git commit ..." The issue here is that, it adds the secondary git tree as a submodule but in a wrong way. Doing "git add <git dir>" declares a submodule but do not adds a url associated to it, and all following "git submodule foreach" commands will fail. So detect that a git tree was extracted inside S and correctly add it using "git submodule add <url> <path>", so that it will be considered as a regular git submodule (From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: tag all submodulesJulien Stephan2023-12-011-0/+3
| | | | | | | | | | | | In the case of a repository with submodules, we need to add the "devtool-base" and "devtool-patched" tag on all submodules in order to properly detect the added/removed/modified patches (From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish/update-recipe: restrict mode srcrev to recipes fetched from SCMJulien Stephan2023-12-011-1/+5
| | | | | | | | | | | | | | | | | | | | | | When specifying --mode / -m srcrev with devtool finish/update-recipe on recipes that are not fetched from a SCM repository we get the following error: Traceback (most recent call last): [..] File "<...>/poky/meta/lib/oe/patch.py", line 49, in runcmd raise CmdError(cmd, exitstatus >> 8, "stdout: %s\nstderr: %s" % (stdout, stderr)) oe.patch.CmdError: Command Error: 'sh -c 'git format-patch --no-signature --no-numbered INVALID -o /tmp/oepatchbj7pfmzj -- .'' exited with 0 Output: stdout: stderr: fatal: bad revision 'INVALID' Fix this by adding a check and abort with a proper error message. (From OE-Core rev: 5b77f60e85d07921ae5e808daa1fd7d8c3dc40ea) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix update-recipe dry-run modeJulien Stephan2023-11-301-5/+4
| | | | | | | | | | | | | | | | | When running devtool update-recipe with --mode=srcrev AND --append switch in dry-run, we get the following error: Traceback (most recent call last): [...] Exception: destpath should be set here Fix this by removing a misplaced else statement in _update_recipe_srcrev (From OE-Core rev: 85ba125703d5b442133fd7c470b915460ee68ac9) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: rawcopy: add support for zstd decompressionMalte Schmidt2023-11-271-1/+2
| | | | | | | | | | | Add support for zstd decompression in rawcopy plugin. zstd claims to reach higher, uniform decompression rates. (From OE-Core rev: 707471b71a417187f1e581d0c540e058825f7242) Signed-off-by: Malte Schmidt <malte.schmidt@weidmueller.com> Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Add handler to create go recipesLukas Funke2023-11-061-0/+751
| | | | | | | | [RP: Replace python 3.9 use of removeprefix()] (From OE-Core rev: 049afb933f42851d56d2e6522150a32bcccf2a6f) Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Ignore *.go files while scanning for licensesLukas Funke2023-11-061-1/+1
| | | | | | | (From OE-Core rev: 1765acc8bdc08e392daa8b77add8b4ce1a9e70b6) Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: group all regressions in regression reportAlexis Lothoré2023-11-051-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit c304fcbe0588b1078373558c2ddf36064bcdf214 introduced a grouping when listing regressions. This grouping has been added only for ptests. It has been observed that any other kind of tests could benefit from it. For example, current regression reports can show the following: 1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED [...] This output is not so useful in its current state and should be grouped per test type too. Enable grouping for all kind of tests, to make it llok like the following in reports: 5 regression(s) for oescripts oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED (From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: add python_hatchling supportTim Orling2023-10-301-1/+5
| | | | | | | | | | | | One of the newer PEP-517 backends to be added was python_hatchling.bbclass but it was not included in the recent improvements. Add selftest for 'jsonschema' pypi package. (From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S ↵Alexander Kanavin2023-10-271-1/+1
| | | | | | | | | | | | | | | | | | | | lockedsigs This was writing out locked-sigs.inc into cwd with every 'bitbake -S' invocation. When the intent is only to to get task stamps (-S none), or print the difference between them (-S printdiff), the file is unnecessary clutter. A couple of selftests/scripts were however relying on this, so they're adjusted to explicitly request the file. eSDK code calls dump_lockedsigs() separately via oe.copy_buildsystem.generate_locked_sigs() and so isn't affected. (From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create_buildsys_python: add PEP517 supportJulien Stephan2023-10-271-1/+267
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add basic support for PEP517 [1] for the 3 following backends that are supported by bitbake: - setuptools.build_meta - poetry.core.masonry.api - flit_core.buildapi If a pyproject.toml file is found, use it to create the recipe, otherwise fallback to the old setup.py method. Some projects can declare a minimal pyproject.toml file, and put all the metadata in setup.py/setup.cfg/requirements.txt .. theses cases are not handled. If a pyproject.toml file is found, assumes it has all necessary metadata. As for the old setup.py method, version numbers for dependencies are not handled. Some features may be missing, such as the extra dependencies. [YOCTO #14737] [1]: https://peps.python.org/pep-0517/ (From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create_buildsys_python: refactor code for futur PEP517 additionJulien Stephan2023-10-271-349/+371
| | | | | | | | | | | | | | | | In order to prepare the support for pyproject.toml (PEP517 [1]) enabled projects, refactor the code and move setup.py specific code into a specific class in order to allow sharing the PythonRecipeHandler class No functionnal changes expected [1]: https://peps.python.org/pep-0517/#source-tree (From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create_buildsys_python: prefix created recipes with python3-Julien Stephan2023-10-271-0/+5
| | | | | | | | | | | | | | | By convention, all python recipes start with "python3-" so update create_buildsys_python to do this This rule doesn't apply for packages already starting with "python" Update recipetool's selftest accordingly (From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create_buildsys_python: fix license noteJulien Stephan2023-10-271-1/+1
| | | | | | | | | | | | | | | | | License field of setup is not always standardized, so we usually use the classifier to determine the correct license format to use in the recipe. A warning note is added above the LICENSE field of the create recipe in case a license is provided in setup. But when the plugin is called, "LICENSE =" is not yet present so we can never display this note. Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##" to actually be able to display the note message (From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: make additional info more compactAlexis Lothoré2023-10-231-4/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since "matched" and "improved" tests are not as important as regressions, reduce the place they take in the regression report: - merge "matched" and "improved" tests, while removing the label - add a single line of additional info per pair Those changes make the "Matches and improvements" look like the following sample: oeselftest_almalinux-9.2_qemux86-64_20230910083156 oeselftest_almalinux-8.8_qemux86-64_20231018010951 -> +7 test(s) present oeselftest_almalinux-9.2_qemux86-64_20230911010538 oeselftest_debian-11_qemux86-64_20231017150459 oeselftest_debian-11_qemux86-64_20230910012927 oeselftest_debian-11_qemux86-64_20231017151319 -> +7 test(s) present [...] (From OE-Core rev: 6de4426d9a7da67deed7d3a3918892fb56238ff3) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: rearrange regressions report orderAlexis Lothoré2023-10-231-2/+6
| | | | | | | | | | | | | | Regressions reports currently reports matching pairs and improved pairs first, then regressions. Change order to print regressions first, which is the most valuable info in the report, and then print improvements and matches at the bottom. (From OE-Core rev: 599267467430e70fa4dc8ba6b2a8b126bf6da359) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: limit the number of changes displayed per testAlexis Lothoré2023-10-231-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | Most of the changes list generated in regression reports fall in one of the two following categories: - there is only a few (<10) changes listed and the info is valuable/relevant - the list is huge (> 100 ? 1000 ?) and basically tells us that the whole tests category suffers the same status (test missing, test failing, test skipped, etc) Prevent those huge, worthless lists by limiting the output for each test group: - current default limit is arbitrarily set to 50 - limit can still be overriden with a new "-l"/"--limit" flag, either with custom value, or with 0 to print the whole lists of changes - limit is applied per test family: currently it distinguishes only types of ptests, but it can be adapted to other kind of tests (From OE-Core rev: cec118406f3ad81cb4709f6e6ae1cef65799658e) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: fix a typo in help outputAndré Draszik2023-10-101-1/+1
| | | | | | | (From OE-Core rev: 1c35beb935d42e200370b54f9aa3848b940966dd) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/upgrade: check all git config locationsMarcus Flyckt2023-10-081-1/+1
| | | | | | | | | | | | | | | | | Fixes [YOCTO #15222] In some situations its more practical to keep git configuration at `/etc/gitconfig` instead of `$HOME/.gitconfig` (e.g., when mounting git configuration into a docker container). This change makes `devtool upgrade` consider any available git configuration instead of only checking `--global`. (From OE-Core rev: 30a9f7de45050c8bac49d4b37419cc2e067a75fa) Signed-off-by: Marcus Flyckt <marcus.flyckt@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/create_buildsys_python: use importlib instead of impChris Laplante2023-10-061-3/+2
| | | | | | | | | | | 'imp' was deprecated in Python 3.4 and removed in 3.12. The piece of importlib we use has been around since 3.3. (From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add console parameters to qemux86 canned-wksRichard Purdie2023-10-041-1/+1
| | | | | | | | | | | Currently our wic test images boot up without kernel output on the consoles which means we have no way to debug if anything goes wrong. Add the console parameters runqemu would have added if the kernel wasn't built into an image to improve our chances of debugging. (From OE-Core rev: 63ea1efdaaf1173ef4f2f69b5a3403afef4b556a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: bootimg-partition: Fix file name in debug messageDaniel Semkowicz2023-10-011-1/+1
| | | | | | | | | | Debug message about using custom configuration file includes file name with incorrect extension. Correct file name to "extlinux.conf". (From OE-Core rev: dd63e1520454b2d53a48b72eaae126059af9809b) Signed-off-by: Daniel Semkowicz <dse@thaumatec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/devtool: Ensure server knows about changed filesRichard Purdie2023-09-185-1/+6
| | | | | | | | | Bitbake has changed to require notification when metadata changes in the middle of tinfoil sessions. Add the required function calls at the places metadata is changed. (From OE-Core rev: e5574163ab49a8f51b2b34fd37acfd1cad9b7595) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/build_sdk: Drop unused importsRichard Purdie2023-09-181-1/+1
| | | | | | | | Drop some unused imports. (From OE-Core rev: 432446ef402ff42fe0c90172b77376fa5981524a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* build-sysroots: target or native sysroot population need to be selected ↵Alexander Kanavin2023-09-091-1/+2
| | | | | | | | | | | | | | | | | explicitly Running them in parallel is prone to races as postinsts from target sysroots rely on executables from native sysroots which may or may not be fully prepared yet. This was observed for example here: https://autobuilder.yoctoproject.org/typhoon/#/builders/146/builds/468/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/147/builds/467/steps/12/logs/stdio (From OE-Core rev: 38d7a2e45b883cf999a86af05bcc0eaa875bb47c) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add gpt-hybrid partition layoutJoshua Watt2023-09-023-17/+69
| | | | | | | | | | | | | | | | | | | | | | | Add support for formatting a disk with a hybrid MBR & GPT partition scheme. In this scheme, the primary partitioning method is GPT, but a valid MBR header is also written than can point to a subset of the GPT partitions on the disk (any partitions marked with the `--mbr` flag will be included in this MBR). The primary purpose of this method is to allow for SoCs that can only find a bootloader in an MBR partition to use GPT once the bootloader is running. As an example, older versions of the Raspberry Pi firmware can only parse MBR partitions to find a kernel (or other bootloader like u-boot), but once those have booted GPT partitions can be used. In addition to the partitions annotated with the `--mbr`, a "protective" GPT partition of type 0xEE is added, as the existence of such a partition is the indication to tooling that this a hybrid MBR and that the GPT partition table should be parsed instead. (From OE-Core rev: e50e4c2a5ada6947b3503ca4d8e9c30d359e8a5d) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipes/classes/scripts: Drop SRCPV usage in OE-CoreRichard Purdie2023-08-242-2/+2
| | | | | | | | Now that SRCPV isn't needed we can simplify things in a few places... (From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: fix wrong attempt to create file system in upartitioned regionsMarkus Niebel2023-08-211-1/+1
| | | | | | | | | | | | | | | | The kickstart parser defaults fstype to "vfat". This leads to an attempt to create an empty file system even for regions configured with "--no-table" if used without fstype when no --sourceparams given. The fix tests for fstype "none" or no_table in Partition prepare method. This will omit the file system creation an the potential error for small region with --no-table option. (From OE-Core rev: db771a4cd36bf291a8b68edfd905e03243f2c8b3) Signed-off-by: Markus Niebel <Markus.Niebel@ew.tq-group.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/report: Avoid divide by zeroRichard Purdie2023-08-151-1/+4
| | | | | | | | Avoid a divide by zero traceback if unfortunate test counts are encountered. (From OE-Core rev: c5aeea53dfacb53dedb8445cb3523dc3a8cb6dca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: bootimg-efi: Stop hardcoding VMA offsetsAlexandre Belloni2023-08-091-14/+49
| | | | | | | | | | | | | | | | Section VMA's are currently hardcoded. This doesn't work anymore starting with systemd-boot v254. Follow the actually solution to this which is documented here: https://wiki.archlinux.org/title/Unified_kernel_image#Manually This is also used by dracut. Later on, we may want to switch to ukify instead but this is not ready yet. (From OE-Core rev: 3abf99a6c6bde2fb8770f54dba609b35f6c6ee5a) Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: define custom string for "not found" test resultsAlexis Lothoré2023-08-041-0/+1
| | | | | | | | | | | | | | | | | | | Whenever a test result is present in base test result but absent from target test results, we have an entry in regression report looking like the following one: ptestresult.apr.testfile: PASSED -> None The "None" status may be a bit confusing, so replace it with a custom string which really states what "None" means in this context: ptestresult.apr.testfile: PASSED -> No matching test result (From OE-Core rev: b5f6da6e13f3484f51dfa07f6b3500aea7a21825) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: allow to replace test raw status with custom stringAlexis Lothoré2023-08-041-1/+7
| | | | | | | | | | | | | Add a STATUS_STRINGS dictionnary matching raw statuses to custom strings. Whenever a regression must be reported, raw status is searched in the custom statuses dict (key search is case insensitive). If no custom string is found, raw status is kept and used in regression report (From OE-Core rev: 9d22bfc9d0c4092dba1af0ee11a4c51b7b270786) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/scriptutils.py: add recipe_qa artifacts to exclusion list in ↵Alexander Kanavin2023-08-041-1/+1
| | | | | | | | | | | | | | | filter_src_subdirs() With newly added recipe_qa task, the function needs to filter out its output as well. The issue wouldn't appear in selftest because it's triggered by buildhistory and selftest disables that. (From OE-Core rev: da1284c990957827ee89685921c53b64490a32b2) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/upgrade: raise an error if extracting source produces more than one ↵Alexander Kanavin2023-08-041-0/+2
| | | | | | | | | | | | | | | | | | | directory This can happen if running unpack task produces unrelated files and directories (e.g. if recipe_qa or other tasks run); in this case it's better to stop, rather than allow devtool to continue and error out much later with a message that has nothing to do with where the problem originated. The idea here was to handle tarballs that don't contain a top level directory and thus the source tree is one level up; this basically never happens, and if it does we should find a less brittle way to handle such tarballs. (From OE-Core rev: b25e922b271794906b22450c7e6cc18fcab51ff8) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: do not run recipe_qa task when extracting sourceAlexander Kanavin2023-07-301-0/+2
| | | | | | | | | | | | | | | | | | | | The immediate reason is that the task is sstate enabled, and due to the way devtool sets up source extraction this causes bogus values to appear in tmp/sstate-control/index-* files. Also, recipe_qa may contain hard failures in the future, and it would be counterproductive for the failures to prevent devtool workflows (which may perhaps be about fixing the failure). An alternative would have been to not insert recipe_qa task before fetch task, but that would greatly delay the checks in larger bitbake builds, and it's better to run them as early as possible. (From OE-Core rev: 6f9db3f8de3dca9b0c87382561c70857d5b39e0f) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: add mention about new detected testsAlexis Lothoré2023-07-251-2/+14
| | | | | | | | | | | | | | | | Some regression reports show a lot of "PASSED->None" transitions. When such big lot of identical transitions are observed, it could be that tests are now failing, but it could also be that some tests has been renamed. To detect such case, add a log in regression report to report the number of new tests (i.e: tests that are present in target results but not in base result). This new log also allows to know about newly added tests bases (From OE-Core rev: 01b5cefd07e01c7407bc663842b8a8d502358a6d) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/resultutils: allow index generation despite corrupt jsonMichael Halstead2023-07-191-1/+5
| | | | | | | | | | | non-release indexes will continue to generate when test output is corrupted. (From OE-Core rev: 1a9157684a6bff8406c9bb470cb2e16ee006bbe9) Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: npm: Add support to handle peer dependenciesBELOUARGA Mohamed2023-06-291-0/+20
| | | | | | | | | | | | | | | | | NPM changed its manner to handle peer dependencies over its versions. Before NPM 3: NPM installs automatically peer dependencies between NPM 3 and 7: NPM shows a warning about peer dependencies After NPM 3: NPM reworked its manner how to handle peer dependencies The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot fetch them. in the same time peer dependencies are not direct dependencies, they should be installed as run time dependencies. (From OE-Core rev: f36021a749974ef3d4a6abe4d5429544a815071a) Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: npm: Add support for the new format of the shrinkwrap fileBELOUARGA Mohamed2023-06-291-5/+4
| | | | | | | | | | | The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly. The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format. (From OE-Core rev: 488d17c2af0c927ec66f0eee124bf6fc5b7f7c95) Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: npm: Remove duplicate function to not have future conflictsBELOUARGA Mohamed2023-06-291-11/+3
| | | | | | | | | | | | | | Npm packages do not have yocto friendly names. fore instance we can have names like "@example/npmPackage" npm fetcher has a function that convert these names to yocto friendly names. But in recipe tool we have an other function (duplicate). (From OE-Core rev: 18e5438de5389b58c8b6a548d4474128d510a28d) Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Fix inherit in created -native* recipesYoann Congal2023-06-281-0/+4
| | | | | | | | | | native and nativesdk classes are special and must be inherited last : put them at the end of the gathered classes to inherit. (From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix the wrong variable in srcuri_entryCharlie Wu2023-06-151-1/+1
| | | | | | | | | | | | | | | | | | | | devtool crashes when running "update-recipe" and append changes on the recipe. "$ devtool update-recipe -a <layer> <recipe>" Traceback (most recent call last): ... File "/ovss/ovss_quanta/poky/scripts/lib/devtool/standard.py", line 1636, in srcuri_entry return 'file://%s%s' % (basepath, paramstr) ^^^^^^^^ NameError: cannot access free variable 'basepath' where it is not associated with a value in enclosing scope The input variable 'fname' should have the same meaning as the variable 'basepath'. Modify the 'fname' to 'basepath' and solve the issue. (From OE-Core rev: c3231756bbc2cb5641204414ad3670d7f8607ed3) Signed-off-by: Charlie Wu <chiachiwu@google.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: introduce KCONFIG_CONFIG_ENABLE_MENUCONFIGMing Liu2023-06-091-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, uboot do_menuconfig task is breaking when UBOOT_CONFIG is chosen rather than UBOOT_MACHINE, it simply fails with the following errors: | make: *** No rule to make target 'menuconfig'. Stio. | Command failed. | Press any key to continue... this is due to the work directory of do_menuconfig is set to ${B} but not ${B}/$config. We should distinguish two situations: 1) When there is only one config item in UBOOT_CONFIG, do_menuconfig should work just like how it works for UBOOT_MACHINE. 2) When there are multiple config items in UBOOT_CONFIG, do_menuconfig should print out some information saying it's not supported other than just failing. This patch mainly aims to fix that by introducing a extra variable KCONFIG_CONFIG_ENABLE_MENUCONFIG, it would be set to 'false' for situation 2), and when it's set to 'true', then set KCONFIG_CONFIG_ROOTDIR correctly in uboot-config.bbclass to let do_menuconfig task work. DEVTOOL_DISABLE_MENUCONFIG could be replaced by this new variable KCONFIG_CONFIG_ENABLE_MENUCONFIG. (From OE-Core rev: f9e834e317880cf47dbb4f8285bc36d743beae5e) Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image_types: Fix reproducible builds for initramfs and UKI imgFrieder Paape2023-06-061-0/+2
| | | | | | | | | | | | | | | | | | | I've encountered issues reproducing initramfs and UKI image builds, which will be fixed with this patch. 1. initramfs There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation. The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes. 2. Unified Kernel Image '--preserve-dates' is required for a static 'Time/Date' entry. I've added '--enable-deterministic-archives' although in my case this didn't change anything. (From OE-Core rev: fd027729bafb4e085ba0949e38e724f3a8cad102) Signed-off-by: Frieder Paape <frieder@konvera.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add argument --hidden to default imagerLeon Anavi2023-05-253-0/+9
| | | | | | | | | | | | | | | | | | | Add argument --hidden to avoid MS Windows prompting to format partition after flashing to a USB stick, SD card on another media. Set Bit 0 (RequiredPartition) to mark that the partition is required for the platform to function on GUID Partition Table (GPT). The new argument simplifies setting RequiredPartition on GPT through a WKS file and the default imager plugin. Otherwise, without this feature, to achieve the same result a new imager plugin has to be implemented and set in WIC_CREATE_EXTRA_ARGS. (From OE-Core rev: 7a111ff58d7390b79e2e63c8059f6c25f40f8977) Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add support for proper kernel name to bootimg-pcbiosSergei Zhmylev2023-05-041-3/+3
| | | | | | | | | | Use appropriate kernel image name instead of hard-coded vmlinuz for bootimg-pcbios plugin. (From OE-Core rev: 8d9f00ba456fe76e0f4ef0e68ec64fc538c90d89) Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: if fixed-size is set then use that for mkdosfsRandolph Sapp2023-04-271-0/+7
| | | | | | | | | | | | | | | This is a bit of a compatibility issue more than anything. Some devices get upset if the FAT file system contains less blocks than the partition. The fixed-size argument is currently respected by the partition creation step but not by the file system creation step. Let's make it so the file system respects this value as well. (From OE-Core rev: 38e1a235f5eceade7c871f96dc97f6c384384c7b) Signed-off-by: Randolph Sapp <rs@ti.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use part_name when definedDit Kozmaj2023-04-131-10/+4
| | | | | | | | | | | | | | | So far part.label has been used to define GPT partition label even if part.part_name was defined. Fix the code to use part.part_name whenever available, as it makes sense to have a GPT partition label which is different from the contained filesystem label. (From OE-Core rev: 7704d5fc36eb065224792bf4d5543814eaa5fed3) Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com> Signed-off-by: Diego Rondini <diego.rondini@kynetics.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* filemap.py: enforce maximum of 4kb block sizeAndrew Geissler2023-03-271-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | The logic in this script validates that the length of data sections are evenly divisible by the block size. On most systems the block size is 4KB and all is good. Some systems though, such as ppc64le, have a block size larger then 4KB. For example on a POWER9 based ppc64le system, the block size is 64KB. This results in this script failing with errors like this when building wic images: |440, in _do_get_mapped_ranges | assert extent_len % self.block_size == 0 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | AssertionError In this case the data section size was 268KB and the block size was 64KB, resulting in the above assert failure. Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075 (From OE-Core rev: 1e23b803af6991fc20e4a4e88a0ef0541399e722) Signed-off-by: Andrew Geissler <geissonator@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/buildstats: handle top-level build_stats not being completeRoss Burton2023-03-251-0/+1
| | | | | | | | | | | | | | | | If we try to parse a buildstats directory which was either aborted or is still being built then the top-level build_stats file doesn't contain an elapsed value which causes an exception: UnboundLocalError: local variable 'elapsed' referenced before assignment Default both start and elapsed to 0 so that the parse succeeds. (From OE-Core rev: 701d985aa8f2e9c2b9c0736fa25b424f3701889e) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-01 20:29:12 +0000