summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake-config-build: add a plugin for config fragmentsAlexander Kanavin2024-12-132-0/+190
| | | | | | | | | | | | | | | | | | | | | | | | | | This allows fine-tuning local configurations with pre-frabricated configuration snippets in a structured, controlled way. It's also an important building block for bitbake-setup. The tool requires that each fragment contains a one-line summary, and one or more lines of description, as BB_CONF_FRAGMENT_SUMMARY style metadata. There are three (and a half) operations (list/enable/disable/disable all), and here's the 'list' output: alex@Zen2:/srv/storage/alex/yocto/build-64$ bitbake-config-build list-fragments NOTE: Starting bitbake server... Available fragments in selftest layer located in /srv/work/alex/poky/meta-selftest: Enabled fragments: selftest/test-fragment This is a configuration fragment intended for testing in oe-selftest context Unused fragments: selftest/more-fragments-here/test-another-fragment This is a second configuration fragment intended for testing in oe-selftest context (From OE-Core rev: fdb611e13bd7aa00360d3a68e4818ef5f05c8944) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rpm: replace use of rpm2cpio with rpm2archiveAlexander Kanavin2024-12-122-8/+8
| | | | | | | | | | | | | | rpm2cpio has been deprecated upstream, so this prepares for its eventual removal. rpm2archive produces a tar archive which can be uncompressed with tar executable from the host. (From OE-Core rev: ed824d3fb23f0c89d8dfdacb2c4ef0b7c21a5144) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rpm: update 4.19.1.1 -> 4.20.0Alexander Kanavin2024-12-121-0/+1
| | | | | | | | | | | | | | | | | | | | | With this release, as expected, deprecated openpgp support has been fully removed and replaced with sequoia: https://github.com/rpm-software-management/rpm-sequoia/ Alas, it's written in rust, and has no recipe, so for now signing rpm packages has to be disabled. Remove package name parameter to %prep in source archiving, as it's been dropped in the new rpm, and wasn't needed to begin with. Drop 0001-perl-disable-auto-reqs.patch (files removed upstream). (From OE-Core rev: 8c15b4577d5e554cc2dd5adfb88b816894b05a9a) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Fix failure when configuration contains BBLAYERS:appendOleksandr Hnatiuk2024-12-121-0/+1
| | | | | | | | | | | | | | | | | | | | | When used with `--newbuilddir` option, it replaces relative paths in BBLAYERS variable with absolute paths by evaluating the final value of the variable, converting paths and saving result by assignment (`BBLAYERS =`) at the end of bblayers.conf. This breaks tests when bblayers.conf contains BBLAYERS:append because :append statements are evaluated after all assignments and we end up with the appended layer added twice - first by evaluating final value of the variable, then again by bitbake evaluating BBLAYERS:append. The error is as follows: AssertionError: Command 'bitbake -e' returned non-zero exit status 1: ERROR: Found duplicated BBFILE_COLLECTIONS 'layer-name', check bblayers.conf or layer.conf to fix it. (From OE-Core rev: 516c8e8f722ed0eb85a715a23b5e68d05abe9275) Signed-off-by: Oleksandr Hnatiuk <ohnatiuk@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* spdx 3.0: Rework how SPDX aliases are linkedJoshua Watt2024-12-123-135/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The SPDX code needs to be able to look up an Element by its SPDX ID, locating the file that (should) contain the SPDX ID and opening it for parsing. Previously, the code would do this be hashing each Element SPDX ID and Alias, and the creating a symbolic link to the file that contains the element with a name of the hash. This worked well as it was possible to look up any arbitrary SPDX ID or alias by simply hashing it and following the symbolic link to get the file. However, the down side of this approach is that it creates a lot of symbolic links, since it will make one or two per Element in the document. This can be a problem when using SPDX_INCLUDE_SOURCES, for example. This change reworks this strategy so that the only Element that gets a symbolic link based on the hash is the singular SpdxDocument that is create for each file. All other Elements are assigned an alias with a special prefix that encodes the hash of SpdxDocument alias. Thus, when attempting to look up an arbitrary alias, the code sees the special prefix, extract the hash, opens the file based on the symlink with that hash name, then finds the matching Element in the file. This drastically reduces the number of symbolic links by making only one per file. This also means that the custom link extension can be removed since it is now superfluous. (From OE-Core rev: 838d64c09657ac53175737fc4e7fd6f01f3dcf47) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Fix stack trace generation in exception handlerMathieu Dubois-Briand2024-12-091-2/+4
| | | | | | | | | | | | | | | | | | Qemurunner exception handling code currently formats the stack trace using traceback.format_exception(), with parameters introduced in python 3.10. This will fail on platforms with an older python version. Change this to the old parameter order, still supported in current python versions. https://docs.python.org/3/library/traceback.html#traceback.format_exception Fixes [YOCTO #15675] (From OE-Core rev: 5f9ecf5f210e967594069f172728fd5b4d5b4f1d) Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: oeqa: spdx: Add tests for extra optionsJoshua Watt2024-12-052-23/+108
| | | | | | | | | | | | | | | Adds a test for several of the extra options provided by the SPDX classes. In particular, these are the options that can produce non-reproducible results, so are not enabled by default in OE core. This test takes care to configure the build so that the tests do run in a reproducible manner so that pre-built test objects can be pulled from sstate (From OE-Core rev: 14f534f15f7fe6362723d7f064d39783c5bd758f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: spdx: Fix SPDX_BUILD_HOSTJoshua Watt2024-12-051-1/+1
| | | | | | | | | | | | The call to make a new import for the SPDX_BUILD_HOST was incorrect, since it was asking for the key named "SPDX_BUILD_HOST", instead of the key name contained in that variable (From OE-Core rev: 250c2bcc557040b4cd67cecf25f8351638cff45d) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: sbom30: Fix agent referenceJoshua Watt2024-12-051-1/+1
| | | | | | | | | | | When a agent reference was being used, the code was not using the correct base variable to look up the agent (From OE-Core rev: f3f13f48e214b25cf302b8ce397b630f5aa283fa) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: sbom30: Fix import handlingJoshua Watt2024-12-051-1/+1
| | | | | | | | | | | Fixes a bug in the way that imports where being handled, due to a misspelled variable (From OE-Core rev: 10e2a1fbcf4828f3a6c7fd327976fefa3ffb252e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/gitarchive: Fix syntax warningRichard Purdie2024-12-051-1/+1
| | | | | | | | | | | The backslash characters cause syntax warnings, mark the strings are raw to avoid this. (From OE-Core rev: f717f61a37ed83618d054fc4017b5f5386fb2e3c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib: move buildstats.py and rootfspostcommands.py into oeChen Qi2024-12-052-0/+0
| | | | | | | | | | | These two files are the only ones that are left in meta/lib. They logically belong to meta/lib/oe, so move them there. (From OE-Core rev: c65dd0e3e463d6072b9364ac74e1fef0d998068f) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: spdx30_tasks: Fix supplied ByJoshua Watt2024-12-051-1/+1
| | | | | | | | | | Fixes the supplied by property in SPDX to use the correct name (From OE-Core rev: a33fb425262492f5e8f6cc5e7ec2adf072042817) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: oe: license: Add missing importJoshua Watt2024-11-281-0/+1
| | | | | | | | | | The code in this file uses oe.qa, but it was not imported resulting an an exception when a license error was detected (From OE-Core rev: e44cb7c5b7281d614ed51fdec06dad0a7211528a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/sbom30: Prefix aliases with "http://spdx.org/spdxdocs/"Joshua Watt2024-11-231-1/+3
| | | | | | | | | | | | While the SPDX ID aliases are primarily intended for internal use by the SPDX code, they are used in places where a valid SPDX ID is expected. In order to make sure that they still conform to this, prefix them with "http://spdx.org/spdxdocs/openembedded-alias" (From OE-Core rev: 5e0ff36e025f5e842fa90b8219b53257d65ea66a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib: sbom30: Fix build parametersJoshua Watt2024-11-231-1/+1
| | | | | | | | | | Fixes the build parameters code, since the build_parameter property was changed to be singular in SPDX 3.0.1 (From OE-Core rev: e6e1e4585198b819319bab61ac2676e82f177020) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: Return tag name and improve exclude handlingRichard Purdie2024-11-231-1/+3
| | | | | | | | | | | Tweak the gitarchive exclude handling not to error if excluded files don't match. Also return the tagname created so that other code can then use it. (From OE-Core rev: 1adba3430faffdf6217b6a00533a3b48a9388abc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Use single space indentation in json outputRichard Purdie2024-11-231-1/+1
| | | | | | | | | | | Using 4 space indentation in resulted in hundreds of megabytes of extra file size in general use. Reduce this to make filesizes more managable and reduce the processing cost. Some level of indentation and spacing does make the files more readable and allows use of git diff so we need to retain some of it. (From OE-Core rev: a274cdcaf852cca9497f0358f44dda99c06aacbe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Clean up pathnamesRichard Purdie2024-11-221-1/+1
| | | | | | | | | | | | | | | | | There are several problems with these paths. Firstly they contain full system paths which depend upon where the test was run. These are pretty pointless and just take up a lot of space making the results files large. Secondly, they contain the same path twice. The reference and target path will always be the same thing in two different locations. Strip off the prefix and remove the duplication. This does change the output data but that can't really be avoided. It does shrink the results data and makes it more readable. (From OE-Core rev: 81a44de36e864b08687451fd85aeba7c529fd7f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Drop rawlogsRichard Purdie2024-11-221-6/+0
| | | | | | | | | | | | | The "rawlogs" data consists of a long string of results data which is already in a structured data format. I can't see this is adding much value in duplciating the data but it does create a huge string with a lot of long problematic pathnames and inflates the results data size. I suggest we drop this data as obsolete and not necessary. (From OE-Core rev: 5b2c70fab2ffa409b861d83f048b65d458d03a90) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu/testimage: Ensure GNU tar is in test imagesRichard Purdie2024-11-212-1/+5
| | | | | | | | | | | | | | | | We've seen cases where a ptest (strace) has a 43GB sparse file in the test directory. busybox tar doesn't work well with this. The resulting 1.4GB archive takes hours to extract too. Ensure tar is added to our full images and use the sparse option to collect files for debugging. This stops crazy build hangs. Since tar is GPLv3, we have to exclude it from that test code. We don't boot any of those images so the debug collection code is safe there, at least for now. (From OE-Core rev: fefeb919696b6ac76f0997acfb0f612203ef7f1b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/ssh: Improve performance and log sizesRichard Purdie2024-11-211-12/+32
| | | | | | | | | | | | | | | | | | The current code is not fit for purpose when handling large files via ssh. In the strace ptest case, we can end up with a 1.4GB archive being transferred for which every byte is printed into the task logfile twice over. This is then sent over bitbake IPC which compounds the problems. Make the following improvements: * when the output is large (over 64kb), don't print it * use a bytearray for better concat performance since strings are slow for this * when there is no ssh output, say that * print periodic size status output rather than the data itself since this could be binary and/or large * fix the killed process message logic which appeared broken (From OE-Core rev: e7dd009a17dc902852983a82bce41bf78bb1e242) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: use HEAD commit as base for the selftest and not masterAlexander Kanavin2024-11-191-1/+1
| | | | | | | | | | | | | | | | Patchtest applies patches on top of poky master branch by default; this means selftest does the same, and any commits from the branch-under-test are then discarded. This can cause issues for example, if bitbake-server process started by selftest from the master branch tries to parse bitbake.conf from the branch under test: https://valkyrie.yoctoproject.org/#/builders/71/builds/460 (From OE-Core rev: 03c6b2e0277c00faf55c12c4d0b4b5e3a4898f8c) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rootfs-postcommands: Actually do the re-orderingRalph Siemsen2024-11-191-3/+3
| | | | | | | | | | | | | | | | | Previous commit added logic to move certain tasks to the end, but these had no effect, because the result of the make_last() function was not used to update the post_process_cmds variable. Also, once this is fixed, it becomes evident that the commands need to be joined using whitespace, otherwise they all run together, and cannot be executed as individual commands anymore. Fixes: 0ffff2c1f8 ("rootfs-postcommands: Try and improve ordering constraints") (From OE-Core rev: 2a4e8f06bac1bff0a167f775f7babab94b32732a) Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: add a test for bitbake "-e" and "-getvar" differenceYoann Congal2024-11-181-0/+18
| | | | | | | | | | This is a non-regression test for [YOCTO #15638] (From OE-Core rev: 22b508da24e0f7e5ad8ce4e090832bd0829963f0) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ping: don't bother trying to ping localhostRoss Burton2024-11-181-0/+7
| | | | | | | | | | | | | | | | | | | | If SLIRP is being used instead of TAP for networking to the guest then the target IP will be localhost. There's no point in pinging localhost to see if the target is up but whilst you'd think it is harmless, in some containers ping doesn't actually have enough rights to work: ping: socktype: SOCK_RAW ping: socket: Operation not permitted ping: => missing cap_net_raw+p capability or setuid? Look at the target address and if it's localhost or 127.0.0.* return immediately. (From OE-Core rev: a06ef43d2a50e16c32bd6edbdc7b32c3528687d5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: Remove passing test case from the exclude list in the Rust OE self-test.Deepesh Varatharajan2024-11-121-1/+0
| | | | | | | | | | | | | | We had previously excluded this test case due to failure in earlier versions. However, with the latest version of Rust, this test case is passing. As a result, we have removed it from the exclude list and added back into the test suite for execution. The rust-1.79 has been successfully tested with the current test inclusion. (From OE-Core rev: a33edee3efe67f81229496e03a098454dc7df051) Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa runtime uki.py: add testsMikko Rapeli2024-11-121-0/+16
| | | | | | | | | | If Unified Kernel Image was enabled via IMAGE_CLASSES, then target should also boot the same uki at runtime. (From OE-Core rev: 2ae651c869a19fd24b5a915451c1f99e1d7b32f0) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/container: fix IMAGE_FEATURES assignmentRoss Burton2024-11-061-3/+0
| | | | | | | | | | | | | | | | | | | | Assigning IMAGE_FEATURES="" in local.conf doesn't actually do anything useful, as bitbake.conf has IMAGE_FEATURES += EXTRA_IMAGE_FEATURES after the include of local.conf. In this test case, this results in IMAGE_FEATURES still using EXTRA_IMAGE_FEATURES, so the image contains the files installed by the post-install-logging feature. As the intention here is that container-test-image is as minimal as possible, move the IMAGE_FEATURES assignment into the image itself so it actually works, and update the expected file list to remove the post-install-logging files. (From OE-Core rev: c65349e1238ddc6634dfa4759c57e6168a355200) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducibility: restart bitbake for each buildYoann Congal2024-11-051-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since [0], the reproducibility test tries to save logs of failed builds. Using the memory persistent bitbake prevent this by concatenating logs in the single persistent instance (See [YOCTO #15641]). So, force a BitBake server restart for each build to ensure a new log file is created. This fixes this error (seen on AB [1]): 2024-11-02 08:16:00,816 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail. 2024-11-02 08:16:00,819 - oe-selftest - INFO - ... ERROR 2024-11-02 08:16:00,819 - oe-selftest - INFO - Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 321, in test_reproducible_builds self.copy_file(variables["BB_CONSOLELOG"], os.path.join(save_dir, "bitbake-%s.log" % name)) File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 216, in copy_file shutil.copyfile(source, dest) File "/usr/lib/python3.11/shutil.py", line 256, in copyfile with open(src, 'rb') as fsrc: ^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-python/build-st/tmp/log/cooker/qemux86-64/20241102071141.log' [0]: https://git.openembedded.org/openembedded-core/commit/?id=c78cc753843d4199443052e8deb0c9c3b7e4b580 [1]: https://valkyrie.yoctoproject.org/#/builders/87/builds/17/steps/40/logs/stdio (From OE-Core rev: e89bbc00ba16574d719b199c01ffbf37646f4f54) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runCmd: print stderr when that is a separate streamAlexander Kanavin2024-11-011-0/+2
| | | | | | | | | | | | | runCmd by default merges stderr into stdout, and only needs to print stdout when errors occur. When stderr is requested as a separate stream, and an error occurs, stderr is discarded, obscuring useful error messages. This changes the output to include both streams. (From OE-Core rev: 72c747b37ccdd486ddae06e3d0a99fb2b93643ba) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: wic_sector_size add wic-tools pathVince Chang2024-11-011-44/+48
| | | | | | | | | | Add wic-tools to the PATH to avoid failures when running the wic_sector_size test case on a build host that doesn’t have parted. (From OE-Core rev: 013dcdf75669421bc38d699263cb1e8d5b95d398) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: remove 2>/dev/null to help fail diagnosticVince Chang2024-11-011-21/+23
| | | | | | | | | | | | | | The wic test case previously used '2>/dev/null' to suppress error messages. This commit updates the code to use 'stderr=subprocess.PIPE' when calling runCmd(). Refer: https://lists.openembedded.org/g/openembedded-core/topic/109308684 (From OE-Core rev: bd26d999a0ba1107ee5629a8e238f4fe945e9be5) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: add "Inactive-Upstream"Katariina Lounento2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The list of valid statuses (`upstream_status_literal_valid_status`) was missing "Inactive-Upstream", which caused patchtest to fail the test test_patch.TestPatch.test_upstream_status_presence_format for patches containing lines like: +Upstream-Status: Inactive-Upstream [lastrelease: 2013 lastcommit: 2013] with the error: FAIL: test Upstream-Status presence: Upstream-Status is in incorrect format (test_patch.TestPatch.test_upstream_status_presence_format) "Inactive-Upstream" is documented in the Yocto Project and OpenEmbedded Contributor Guide [1]: Inactive-Upstream [lastcommit: when (and/or) lastrelease: when] The upstream is no longer available. This typically means a defunct project where no activity has happened for a long time — measured in years. To make that judgement, it is recommended to look at not only when the last release happened, but also when the last commit happened, and whether newly made bug reports and merge requests since that time receive no reaction. It is also recommended to add to the patch description any relevant links where the inactivity can be clearly seen. `upstream_status_nonliteral_valid_status` only seems to be used in logging and the value was copied verbatim from the aforementioned documentation. After this change all upstream status options documented in [1] are covered. [1] https://docs.yoctoproject.org/5.0.3/contributor-guide/recipe-style-guide.html#patch-upstream-status (From OE-Core rev: a39e6e6854b60fd10f21c6c3f1e7d676e12ba9ee) Signed-off-by: Katariina Lounento <katariina.lounento@vaisala.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: fix malformed cve status description with : charactersPeter Marko2024-11-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When CPE is not provided and character ":" is in cve status description, current code takes only last part of split function. This works only if there is no ":" in description, otherwise it drops the other split parts. Do a new split of the original string to take the whole description unchanged. This fixes following entries from world build of poky+meta-oe+meta-python: tiff-4.6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2015-7313 CVE_STATUS: fixed-version: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue description: //security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue corrected: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue gnupg-2.5.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2022-3219 CVE_STATUS: upstream-wontfix: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993 description: //dev.gnupg.org/T5993 corrected: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35325 CVE_STATUS: upstream-wontfix: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303 description: //github.com/yaml/libyaml/issues/303 corrected: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35326 CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 description: //github.com/yaml/libyaml/issues/302 corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35328 CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 description: //github.com/yaml/libyaml/issues/302 corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302 cpio-2.15-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-7216 CVE_STATUS: disputed: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html description: //lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html corrected: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html openssh-9.9p1-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-51767 CVE_STATUS: upstream-wontfix: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. description: //bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. corrected: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1. cups-2.4.10-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2021-25317 CVE_STATUS: not-applicable-config: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply. description: root, so this doesn't apply. corrected: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply. unzip-1_6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2008-0888 CVE_STATUS: fixed-version: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source description: //bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source corrected: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source syslog-ng-4.7.0-r0 do_cve_check: CVE_STATUS with 6 parts for CVE-2022-38725 CVE_STATUS: cpe-incorrect: cve-check wrongly matches cpe:2.3:a:oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 description: syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 corrected: cve-check wrongly matches cpe:2.3:a:oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32 (From OE-Core rev: cc33dd9176726cb4b2d2f142ed1bc655da8e0a9f) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: do not skip cve status description after :Peter Marko2024-11-011-1/+1
| | | | | | | | | | | | | | | | Correct maxsplit parameter from 5 to 4 to not drop text if description contains ":". Example: >>> "detail: cpe:vendor:product:description:cont".split(':', 5) ['detail', ' cpe', 'vendor', 'product', 'description', 'xxx'] >>> "detail: cpe:vendor:product:description:cont".split(':', 4) ['detail', ' cpe', 'vendor', 'product', 'description:xxx'] (From OE-Core rev: 3c4d8ca41ac0b429af92bf0ea84f1dfd0cda9e1f) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package: remove PACKAGE_SNAP_LIB_SYMLINKSRoss Burton2024-11-011-20/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was added to OpenEmbedded in 2011[1] for the micro distro[2] and subsequently ported to oe-core[3]. This feature aims to improve runtime link speed by removing intermediate symlinks, i.e. libstdc++.so.6 is not a symlink but the actual library. However, there are several issues here: - The meta-micro distribution has been unmaintained since 2012[4] - The original creator of this function has confirmed that it is not in use anymore because the renaming doesn't affect the -dev packages, so on-target development or use of SDKs isn't possible. Whilst the latter is possible to fix, the performance impact of removing a single symlink resolution is negliable at best and the packaging code is already very complex. So, remove this functionality as it is known to be broken, isn't tested, and removing it reduces complexity. [1] https://git.openembedded.org/openembedded/commit/?id=cf7114179ead8ddff8f66e84d630811920ac9add [2] https://git.openembedded.org/openembedded/tree/conf/distro/micro.conf [3] https://git.openembedded.org/openembedded-core/commit/?id=600dbb7cb384c2290af38b993a9bea3a4dfc4494 [4] https://git.openembedded.org/meta-micro/ (From OE-Core rev: 9bfdf4c9eff16804588942a13ac7484059197f99) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/sstatetests: run CDN mirror check only onceAlexander Kanavin2024-11-011-2/+0
| | | | | | | | | | | The first no-fail check was an attempt to work around the old CDN's instability (and it didn't really help); it should not be necessary with the new CDN, and only delays a-full completion. (From OE-Core rev: 7f75c42b7fcf60a9ca58d3ded9047df675d76dc2) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Add SPDX 3.0 include source case for work-shareHongxu Jia2024-11-011-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | Build gcc and check gcc-14.2.0/README in objset is available $ oe-selftest -r spdx.SPDX30Check.test_gcc_include_source ... 2024-10-26 01:24:57,063 - oe-selftest - INFO - test_gcc_include_source (spdx.SPDX30Check.test_gcc_include_source) 2024-10-26 01:28:24,204 - oe-selftest - INFO - The spdxId of gcc-14.2.0/README in gcc.spdx.json is http://spdx.org/spdxdocs/gcc-f2eaeb0d-b54b-53ba-899a-8c36c21139bf/88d5068ffd41e5ea6b4e0dd390b23bf499bb2b6674a41e09eaf2a887eced16c8/sourcefile/42 2024-10-26 01:28:26,369 - oe-selftest - INFO - ... ok 2024-10-26 01:28:33,315 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-26 01:28:33,316 - oe-selftest - INFO - Ran 1 test in 216.457s 2024-10-26 01:28:33,316 - oe-selftest - INFO - OK 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS: 2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS - spdx.SPDX30Check.test_gcc_include_source: PASSED (209.31s) 2024-10-26 01:28:45,260 - oe-selftest - INFO - SUMMARY: 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 216.457s 2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) (From OE-Core rev: ccd6dde301dc8c45c8f901ebd4676b488d638b08) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-{2.2,3.0}: support SPDX include source for work-share directoryHongxu Jia2024-11-011-26/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Originally, while SPDX_INCLUDE_SOURCES = "1" [1], there is bug in scan for gcc, libgcc in which the sources locates in work-share directory. Copy source from ${WORKDIR} to ${SPDXWORK} did not satisfy the situation while ${S} was not included in ${WORKDIR} This commit aim to support SPDX include source for work-share directory 1. If is_work_shared_spdx, Copy source from ${S} to ${SPDXWORK}, normally the dest dir in ${SPDXWORK} has the same basename dir of ${S}; but for kernel source, rename basename dir 'kernel-source' to ${BP} (${BPN}-${PV}) 2. For SPDX source copy, do hard link copy to save copy time 3. Move do_patch to no work shared situation along with do_unpack 4. Tweak task do_create_spdx dependencies to assure the patched source in work share is ready for SPDX source copy 5. Remove bb.data.inherits_class('kernel', d) from is_work_shared_spdx, the kernel source locates in 'work-shared', test kernel.bbclass is not necessary [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES (From OE-Core rev: 64454b1956a9b50d6c89a3f3d7c594c1272cb289) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-{2.2,3.0}: fix do_create_spdx dependency while spdx include sourcesHongxu Jia2024-11-011-0/+4
| | | | | | | | | | | | | | | | Call function ${@create_spdx_source_deps(d)} or ${create_spdx_source_deps(d)} along with addtask not working, use task do_create_spdx flag 'depends' to instead Move function create_spdx_source_deps to spdx-common.bbclass for both of create-spdx-2.2.bbclass and create-spdx-3.0.bbclass (From OE-Core rev: dda4b7ccd416ef370634babd150e944f2aa9116e) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/spdx30_tasks.py: improve debug log in add_package_filesHongxu Jia2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | While SPDX_INCLUDE_SOURCES = "1" [1], there are mess of `Adding file' in debug log ''' DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/sources-unpack/COPYING to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native DEBUG: Adding file tmp/work/x86_64-linux/gettext-minimal-native/0.22.5/spdx/3.0.1/work/gettext-0.22.5/.tarball-version to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/e2c2366654a818397af8b8ddb45fda88c2c71aa2d71695861f82376a658d8e66/document/gettext-minimal-native ''' Summary the total number other than print for each file. ''' DEBUG: Added 7201 files to http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/f5e0e04913ac4c595be791fc001d545a77519ed6ee8c743deef721ca0898bc94/document/gettext-minimal-native ''' [1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES (From OE-Core rev: a2f1498f3db44f34599b86221b688e1abf08a3c7) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/sbom30.py: correct typoHongxu Jia2024-11-011-1/+1
| | | | | | | | | | | The isinstance expected 2 arguments (From OE-Core rev: 038dcdca760404a44e0bcef389b4b60903f14a1f) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib/oe/sbom30.py: correct python list comprehensionHongxu Jia2024-11-011-2/+2
| | | | | | | | | | | | | | | | | | | The python list comprehension is not right for list: >>> license_text_map = {'LicenseRef-FSF-Unlimited': 'http://spdx.org/spdxdocs/gettext-minimal-native-1fa0d5cb-2bb8-5631-9fab-cd219801733f/8d31e22acc4a8979f24dc24042692fb548fc8fc8d85d775ddac406abb122ceea/license-text/FSF-Unlimited'} >>> license_text = ((k, license_text_map[k]) for k in sorted(license_text_map.keys())) >>> print(license_text) <generator object <genexpr> at 0x7f8575173270> >>> [(k,v) for k, v in license_text] [] Change the () to [] to make it a list instead of a generator expression. (From OE-Core rev: e13c54645bf716ddfb19606fd212cc349b570ad4) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Reviewed-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducibility: continue testing in case of build failureYoann Congal2024-11-011-9/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current reproducibility test stops all build tasks when a single task fails (default BitBake behavior). This means that a single build failure prevents the reproducibility of other packages from being tested, which is not ideal. To address this, we now use the --continue option of BitBake during test builds, allowing the build process to proceed even when some tasks fail. The failure cases are handled as gracefully as possible. In the event of a build failure: - The entire reproducibility test will be considered a failure. - The complete BitBake log will be saved in the "saved output" directory to facilitate debugging. On the autobuilder, this log should be available at https://autobuilder.yocto.io/pub/repro-fail/. - The last 20 lines of the log, which should show the failing recipes, will be displayed in the oe-selftest console. - If BitBake fails to create the deployment directory, it will be manually created to allow the comparison process to proceed. Here is what the output looks like when testing reproducibility of bash, a failing recipe (hello-fail) and a non-reproducible recipe (hello-norepro): <snip> 2024-10-01 23:09:33,977 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds) 2024-10-01 23:10:39,093 - oe-selftest - INFO - Non-reproducible packages will be copied to <snip>/yocto/repro_output/oe-reproducible-20241001-ng43_hzd 2024-10-01 23:10:39,094 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... 2024-10-01 23:18:12,378 - oe-selftest - ERROR - Bitbake failed! but keep going... Log: 2024-10-01 23:18:12,379 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1: 2024-10-01 23:18:12,379 - oe-selftest - INFO - 2024-10-01 23:18:12,379 - oe-selftest - INFO - ... (last 20 lines of output) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running task 976 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_package_qa) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 342 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx_setscene) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 343 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_spdx_setscene) 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Started 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded 2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build) 2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 841 didn't need to be rerun and 1 failed. 2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Generating JSON CVE summary 2024-10-01 23:18:12,380 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/log/cve/cve-summary.json 2024-10-01 23:18:12,380 - oe-selftest - INFO - 2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: 1 task failed: 2024-10-01 23:18:12,380 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile 2024-10-01 23:18:12,380 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.3221257 2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code. 2024-10-01 23:18:12,380 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail. 2024-10-01 23:18:12,382 - oe-selftest - INFO - Building reproducibleB-extended (sstate NOT allowed)... 2024-10-01 23:46:58,451 - oe-selftest - ERROR - Bitbake failed! but keep going... Log: 2024-10-01 23:46:58,463 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1: 2024-10-01 23:46:58,463 - oe-selftest - INFO - 2024-10-01 23:46:58,463 - oe-selftest - INFO - ... (last 20 lines of output) 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: Running task 978 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx) 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Started 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_deb: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Succeeded 2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_rpm: Succeeded 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build) 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 2 didn't need to be rerun and 1 failed. 2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Generating JSON CVE summary 2024-10-01 23:46:58,464 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/log/cve/cve-summary.json 2024-10-01 23:46:58,464 - oe-selftest - INFO - 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: 1 task failed: 2024-10-01 23:46:58,464 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile 2024-10-01 23:46:58,464 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.4136075 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There were 5 WARNING messages. 2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code. 2024-10-01 23:46:58,467 - oe-selftest - ERROR - reproducibleB-extended build failed. Trying to compute built packages differences but the test will fail. 2024-10-01 23:46:58,481 - oe-selftest - INFO - Checking deb packages for differences... 2024-10-01 23:46:58,586 - oe-selftest - INFO - Reproducibility summary for deb: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,588 - oe-selftest - INFO - Checking ipk packages for differences... 2024-10-01 23:46:58,658 - oe-selftest - INFO - Reproducibility summary for ipk: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,659 - oe-selftest - INFO - Checking rpm packages for differences... 2024-10-01 23:46:58,691 - oe-selftest - INFO - Reproducibility summary for rpm: same=52 different=1 different_excluded=0 missing=0 total=53 unused_exclusions=[] 2024-10-01 23:46:58,692 - oe-selftest - INFO - Running diffoscope 2024-10-01 23:46:59,765 - oe-selftest - INFO - ... FAIL 2024-10-01 23:46:59,766 - oe-selftest - INFO - Traceback (most recent call last): File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds self.fail('\n'.join(fails)) AssertionError: Bitbake reproducibleA failure Bitbake reproducibleB-extended failure The following deb packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb The following ipk packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk The following rpm packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm 2024-10-01 23:46:59,769 - oe-selftest - INFO - ====================================================================== 2024-10-01 23:46:59,770 - oe-selftest - INFO - FAIL: test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds) 2024-10-01 23:46:59,770 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-01 23:46:59,770 - oe-selftest - INFO - Traceback (most recent call last): File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds self.fail('\n'.join(fails)) AssertionError: Bitbake reproducibleA failure Bitbake reproducibleB-extended failure The following deb packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb The following ipk packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk The following rpm packages are different and not in exclusion list: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm 2024-10-01 23:46:59,770 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-10-01 23:46:59,770 - oe-selftest - INFO - Ran 1 test in 2246.039s 2024-10-01 23:46:59,770 - oe-selftest - INFO - FAILED 2024-10-01 23:46:59,770 - oe-selftest - INFO - (failures=1) 2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS: 2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (2245.79s) 2024-10-01 23:47:03,203 - oe-selftest - INFO - SUMMARY: 2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2246.040s 2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0) => Test failed but hello-norepro is displayed as non-reproducible. The testresult.json contains: { "oeselftest_debian-12_qemux86-64_20240930000424": { "configuration": { <snip> }, "result": { "reproducible": { "files": { "package_deb": { "different": [ { "reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb", "test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb" } ], "different_excluded": [], "missing": [], "same": [ <snip> { "reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb", "test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb" }, <snip> ] }, "package_ipk": { <same as deb> }, "package_rpm": { <same as deb> } } }, "reproducible.ReproducibleTests.test_reproducible_builds": { "duration": 2146.5671875476837, "log": "Traceback (most recent call last):\n File \"<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py\", line 380, in test_reproducible_builds\n self.fail('\\n'.join(fails))\nAssertionError: Bitbake reproducibleA failure\nBitbake reproducibleB-extended failure\nThe following deb packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nThe following ipk packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk\nThe following rpm packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm\n", "status": "FAILED" }, "reproducible.rawlogs": { "log": "DIFFERENT: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nSAME: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb\n<snip>" } } } } => "reproducible.ReproducibleTests.test_reproducible_builds".status is correctly "FAILED" but the reproducibility of bash and hello-norepro is tested. (From OE-Core rev: c78cc753843d4199443052e8deb0c9c3b7e4b580) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Update the BB_HASHSERVE_UPSTREAMJose Quaresma2024-10-312-2/+2
| | | | | | | (From OE-Core rev: ec7b024725fc93b7fef881d41343c8b77d97a57c) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: Remove passing test cases from the exclude list in the Rust OE self-test.Deepesh Varatharajan2024-10-291-3/+0
| | | | | | | | | | | | | | | | | There were some rust tests which were FAILing only for mips architectures. Hence, they were added to the exclusion list to avoid the failures when running rust oe-selftest for mips architectures.The current yocto framework disabled the rust oe-selftest for mips architectures.Hence, these tests have been added back to the test framework. The rust-1.79 has been successfully tested on X86, X86_64, ARM, AARCH64 and PPC Architectures with the current test inclusions. (From OE-Core rev: e1ec1ad75feb1321228bf3b52251e1aecfdb8746) Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: wic: add test for WIC_SECTOR_SIZEVince Chang2024-10-291-0/+50
| | | | | | | | | | Add test for WIC_SECTOR_SIZE=4096. Verified it on ext4 file system. (From OE-Core rev: 69a1f18ce18d67881d7379ccc19ca001a35acb68) Signed-off-by: Vince Chang <vince_chang@aspeedtech.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* do_package/sstate/sstatesig: Change timestamp clamping to hash output onlyRichard Purdie2024-10-281-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code was changing the timestamps of the files in the do_package output, particularly the files added for debug sources. This was to do two things: a) make do_package sstate more reproducible b) ensure better hash equivalence matching Unfortuately the debug source files are hardlinks into the source tree for efficiency so touching these, touches a lot of files in ${B} and ${S}. This causes unpredictable effects if compile is run again for example, or could cause compiling in the install task. The hash equivalence matching is of key importance but we can mimic that using clamping of the file timestamps in the depsig output used to generate the hashes. This patch drops the global timestamp clamping, instead allowing the files to retain their creation timestamps into sstate. This makes do_package sstate slightly less reproducibile. We could clamp the sstate timestamps but that would lead to two different sets of timestamps depending on whether the data came from sstate or not. I'd prefer to have consistent code behaviour, rather than differing behavhour depending on whether data came from sstate or not. If we wanted to have reproducibiliy and fix the "corruption" of S/B and have consistent codepaths, the only other option would be two copies of the sources, which could end up huge and seems the least desireable option. This patch therefore drops the timestamp clamping in the sstate files and tweaks the depsig data generation to clamp the timestamps for do_package instead since this seems the best compromise. I validated that rpm/deb/ipk files still generate correctly as before. (From OE-Core rev: 475759fdab7200488b2a568b2ba1aa31a456d113) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/package: Copy locale licenseJoshua Watt2024-10-251-0/+4
| | | | | | | | | | | When creating split locales, copy the license from LICENSE:${PN}-locale if set, otherwise leave it unspecified (which will result in falling back to LICENSE) (From OE-Core rev: 34b844d3a920574d81affbdd2096ffd7dea14067) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-01-31 11:51:04 +0000