summaryrefslogtreecommitdiffstats
path: root/meta/classes
Commit message (Collapse)AuthorAgeFilesLines
* buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usageRichard Purdie2024-09-041-20/+19
| | | | | | | | | | | | | | | | We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of postfuncs. Finally get around to doing that which should make the buildhistory code a little more readable. Unfortunately ordering the buildhistory function calls after the sstate ones is difficult without coding that into the sstate class. This patch does that to ensure everything functions as expected until we can find a better way. This is still likely preferable than the generic sstate postfuncs support since the function flow is much more readable. (From OE-Core rev: c9e2a8fa2f0305ef1247ec405555612326f798f8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: remove the TEXT format supportMarta Rybczynska2024-08-251-115/+1
| | | | | | | | | | | | | | | Remove the TEXT format support, as the JSON format offers more functions. Users who do automation should have migrated already. Support of both formats makes the code more complex than necessary. Users can convert JSON files to TEXT files with cve-json-to-text.py in scripts/ (From OE-Core rev: 05ef4f2a7b225c8d230eaca8d333ffb921729d79) Signed-off-by: Marta Rybczynska <marta.rybczynska@ygreky.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* vex.bbclass: add a new classMarta Rybczynska2024-08-201-0/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "vex" class generates the minimum information that is necessary for VEX generation by an external CVE checking tool. It is a drop-in replacement of "cve-check". It uses the same variables from recipes to make the migration and backporting easier. The goal of this class is to allow generation of the CVE list of an image or distribution on-demand, including the latest information from vulnerability databases. Vulnerability data changes every day, so a status generated at build becomes out-of-date very soon. Research done for this work shows that the current VEX formats (CSAF and OpenVEX) do not provide enough information to generate such rolling information. Instead, we extract the needed data from recipe annotations (package names, CPEs, versions, CVE patches applied...) and store for later use in the format that is an extension of the CVE-check JSON output format. This output can be then used (separately or with SPDX of the same build) by an external tool to generate the vulnerability annotation and VEX statements in standard formats. (From OE-Core rev: 6352ad93a72e67d6dfa82e870222518a97c426fa) Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com> Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: annotate CVEs during analysisMarta Rybczynska2024-08-201-98/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add status information for each CVE under analysis. Previously the information passed between different function of the cve-check class included only tables of patched, unpatched, ignored vulnerabilities and the general status of the recipe. The VEX work requires more information, and we need to pass them between different functions, so that it can be enriched as the analysis progresses. Instead of multiple tables, use a single one with annotations for each CVE encountered. For example, a patched CVE will have: {"abbrev-status": "Patched", "status": "version-not-in-range"} abbrev-status contains the general status (Patched, Unpatched, Ignored and Unknown that will be added in the VEX code) status contains more detailed information that can come from CVE_STATUS and the analysis. Additional fields of the annotation include for example the name of the patch file fixing a given CVE. We also use the annotation in CVE_STATUS to filter out entries that do not apply to the given recipe (From OE-Core rev: 452e605b55ad61c08f4af7089a5a9c576ca28f7d) Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com> Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: encode affected product/vendor in CVE_STATUSMarta Rybczynska2024-08-201-12/+12
| | | | | | | | | | | | | | | | | | CVE_STATUS contains assesment of a given CVE, but until now it didn't have include the affected vendor/product. In the case of a global system include, that CVE_STATUS was visible in all recipes. This patch allows encoding of affected product/vendor to each CVE_STATUS assessment, also for groups. We can then filter them later and use only CVEs that correspond to the recipe. This is going to be used in meta/conf/distro/include/cve-extra-exclusions.inc and similar places. (From OE-Core rev: abca80a716e92fc18d3085aba1a15f4bac72379c) Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: fix BB_GENERATE_MIRROR_TARBALLS checkingEnrico Jörns2024-08-091-1/+1
| | | | | | | | | | | | | | The variable 'have_mirror_tarballs' is used as a boolean while it is actually the result of 'd.getVar('BB_GENERATE_MIRROR_TARBALLS')' and thus a string. Fix this by converting it into a boolean before using i t. (From OE-Core rev: f6185d51b84d8d2eb578dbd322c1c61537efabf6) Signed-off-by: Enrico Jörns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdpx: Avoid loading of SPDX_LICENSE_DATA into global configRichard Purdie2024-08-072-16/+15
| | | | | | | | | | | | | Loading a load of json files into a memory structure and stashing in a bitbake variable is relatively anti-social making bitbake -e output hard to read for example as well as other potential performance issues. Defer loading of that data until it is actually needed/used in a funciton where it is now passed as a parameter. (From OE-Core rev: 6f21cc9598178288784ff451ab3c40b174c0ef3e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: Restoring files from preserve listPedro Ferreira2024-08-071-0/+23
| | | | | | | | | | | | | This fix will ensure that, when we activate feature `BUILDHISTORY_RESET`, files marked to keep on feature `BUILDHISTORY_PRESERVE` will indeed exist is buildhistory final path since they are moved to buildhistory/old but not restored at any point. (From OE-Core rev: 9f68a45aa238ae5fcdfaca71ba0e7015e9cb720e) Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: Fix intermittent package file list creationPedro Ferreira2024-08-071-6/+3
| | | | | | | | | | | | | | | | | | The directory that buildhistory_list_pkg_files writes to during do_package is created by do_packagedata so a clean buildhistory doesn't have files-in-package written during the first build since packagedata happens after do_package. Ensure the output package folder is created to avoid missing files-in-package.txt files. Also it ensures that in case of `find` fails we leave with a hard error instead of hiding the error on the for loop. (From OE-Core rev: 8de9b8c1e199896b9a7bc5ed64967c6bfbf84bea) Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve_check: Use a local copy of the database during buildsRichard Purdie2024-08-051-3/+4
| | | | | | | | | Rtaher than trying to use a sqlite database over NFS from DL_DIR, work from a local copy in STAGING DIR after fetching. (From OE-Core rev: 03596904392d257572a905a182b92c780d636744) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix ↵Richard Purdie2024-07-262-145/+1
| | | | | | | | | | | | | | | | | | | | | | | tarball SPDX manifests Currently, "tarball" sdk based recipes don't generate SPDX manifests as they don't include the rootfs generation classes. Split the SPDX 3.0 image class into two so the SDK components can be included where needed. To do this, introduce an SDK_CLASSES variable similar to IMAGE_CLASSES which the SDK code can use. Migrate testsdk usage to this. Also move the image/sdk spdx classes to classes-recipe rather than the general classes directory since they'd never be included on a global level. For buildtools-tarball, it has its own testsdk functions so disable the class there as a deferred inherit would overwrite it. (From OE-Core rev: 662396533177b72cc1d83e95841b27f7e42dcb20) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHSMark Hatle2024-07-263-8/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When a create-spdx-* classes is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib. When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as: ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found for package libilp32-libgcc-dbg, False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12: sstate:libilp32-libgcc::14.1.0:r0::12: Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-* to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations. (From OE-Core rev: f1499c36c1054fc90f7b7268cc95285f2eca72f7) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrcMark Hatle2024-07-181-1/+2
| | | | | | | | | | | | | While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. (From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9) Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Switch default spdx version to 3.0Joshua Watt2024-07-161-1/+1
| | | | | | | | | Changes the default SPDX version to 3.0 (From OE-Core rev: beef4c1a3e9fd8fa9cdbbb61e1bc3931012735b3) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-3.0: Move tasks to libraryJoshua Watt2024-07-162-1154/+27
| | | | | | | | | | Move the bulk of the python code in the SPDX 3.0 classes into a library file (From OE-Core rev: aed6f8c1c2e291bde4d7172742790fa535b2fc7d) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/spdx-common: Move to libraryJoshua Watt2024-07-164-205/+40
| | | | | | | | | Moves the bulk of the code in the spdx-common bbclass into library code (From OE-Core rev: 3f9b7c7f6b15493b6890031190ca8d1a10f2f384) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-3.0: Add classesJoshua Watt2024-07-163-1/+1463
| | | | | | | | | | Adds a class to generate SPDX 3.0 output and an image class that is used when generating images (From OE-Core rev: b63f6f50458fc6898e4deda5d6739e7bf3639c15) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Handle empty packagesJoshua Watt2024-07-131-41/+42
| | | | | | | | | | When combining an SPDX document, the package list might be empty (e.g. a baremetal image). Handle this case instead of erroring out (From OE-Core rev: baf4e360f6e65a5e9aff2def69d2a720f38f92b2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: fix typosUlrich Ölmann2024-07-041-7/+7
| | | | | | | | | Fix some simple typos found while looking through the code. (From OE-Core rev: c23cb2d89fcd014d1f08944468c7e2d8a7f3e8b0) Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Handle SemVer License List VersionsJoshua Watt2024-07-021-4/+9
| | | | | | | | | | | | SPDX transitioned the license list to use SemVer visioning, (e.g. "MAJOR.MINOR.MICRO"), but SPDX 2 only allows "MAJOR.MINOR". For maximum compatibility, only keep the first two version numbers and discard the rest which allows it to work with either scheme (From OE-Core rev: 8757a5eaeaf2b9d7345212d003e5622289b123e6) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/spdx-common: Move SPDX_SUPPLIERJoshua Watt2024-06-272-7/+8
| | | | | | | | | | | Move the SPDX_SUPPLIER variable to create-spdx-2.2 since it's format only has meaning in SPDX 2.2 (SPDX 3 uses SPDX_PACKAGE_SUPPLIER with a different format) (From OE-Core rev: 628c1e04072178d2c8095e53d5f6600c45a2679f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Introduce CVE_CHECK_MANIFEST_JSON_SUFFIXAleksandar Nikolic2024-06-241-2/+4
| | | | | | | | | | | | | The variable contains the suffix of the CVE JSON manifest file. By default, this variable is set to 'json', so the current behavior is not changed, but enables developers to use some other suffix, e.g., cve.json (similar to spdx.json). (From OE-Core rev: d99eee76923659c0b95bf9ef415ae5d44f736d01) Signed-off-by: Aleksandar Nikolic <an010@live.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Fix work-shared checking for kernel recipesBenjamin Szőke2024-06-181-1/+2
| | | | | | | | | | | | Source dir can be a symbolic link in some BSP's linux kernel recipe which points to work-shared path (like linux-fslc in meta-freescale). Change to use os.path.realpath() in order to get real path of source dir. (From OE-Core rev: 9191aa685418af32f003e067ef7c5737a271e3a5) Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/spdx-common: Return empty list from extract_licensesJoshua Watt2024-06-121-1/+1
| | | | | | | | | | This is nicer as the normal return type is a list, so the calling code doesn't have to deal with a None sometimes and a list others. (From OE-Core rev: e200aa9cc6ceb8ca58ef239a1a5565287b38ce55) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/spdx-common: Add SPDX version to pathJoshua Watt2024-06-122-2/+5
| | | | | | | | | | Since multiple versions of SPDX are available, use the version in the path (From OE-Core rev: 35c061a4514905b3ebbb7f0633a584927519445c) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/spdx-common: Move common SPDX to new classJoshua Watt2024-06-122-249/+266
| | | | | | | | | | Moves SPDX code that can be shared between different SPDX versions into a common class (From OE-Core rev: 769a390adc9fc0b52978abe0f19f885967af0117) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multilib.bbclass: replace deprecated e.data with dKai Kang2024-06-062-56/+56
| | | | | | | | | | | | | | Replace deprecated e.data with d in multilib.bbclass and multilib_global.bbclass. Remove event check in function multilib_virtclass_handler_vendor in multilib_global.bbclass. The function flag 'eventmask' has been set with 'bb.event.ConfigParsed', so no need to check the event any more. (From OE-Core rev: 8d87662d676a2b9ef921cb49ea6edc07b37410e2) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siteconfig: Drop siteconfig class/code/supportRichard Purdie2024-05-311-39/+0
| | | | | | | | | | | The siteconfig code was only used for 5 cache values. The complexity added to sstate to support this code was considerable and the runtime much more significant than any benefit the cache files would have added. Drop the support for this which was only used minimally for ncurses and zlib. (From OE-Core rev: f3766dc038f7ba9780ddaf5eb8d27385ea31d7d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Drop oe-local-files and simplifyRichard Purdie2024-05-211-63/+11
| | | | | | | | | | | | | | | | | | | | The only real reason for oe-local-files was to support S = WORKDIR. With changes to drop support for that, it makes sense to simplify devtool and to try and make both the code and the processes/workflows simpler. This patch drops support for S = WORKDIR, removes oe-local-files and then updates the test cases to match this new situation. At the code level, we assume we can always now track code changes using git and that things committed into git are handled as patches (as before) but delta against HEAD is saved as specific file level changes to the recipe. One test is disabled as it is no longer approproate. It is being keped until we can make WORKDIR != UNPACKDIR at which point it should be revisited. (From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Fix SPDX dependencies for ABI Safe recipesjoshua Watt2024-05-091-4/+10
| | | | | | | | | | | | | | | | | | SPDX Documents can currently get into a state where they reference SPDX IDs that do not exist (locally). The reason for this is that some tasks/recipes are marked as ABI safe, and thus are excluded from taskhash. This means that when SPDX creates a document, it will refer to the SPDX ID in the dependency at the time when the document is created, but if the dependency changes and gets a new SPDX ID, the document will not rebuild to reference the new SPDX ID, causing it to be dangling. Fix this by using the new field in BB_TASKDEPDATA to skip dependencies that are not part of the taskhash calculation. (From OE-Core rev: 8393557b5229c7a0cea18a586848384bff1d7424) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx-2.2: Fix SPDX Namespace Prefixjoshua Watt2024-05-021-1/+1
| | | | | | | | | | | | According to the SPDX documentation, it should be "spdxdocs" not "spdxdoc" [YOCTO #15398] (From OE-Core rev: d1b25413ced62dc2927dae57b8d16e67d15dc220) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Log if CVE_STATUS set but not reported for componentSimone Weiß2024-02-241-0/+3
| | | | | | | | | | | Log if the CVE_STATUS is set for a CVE, but the cve is not reported for a component. This should hopefully help to clean up not needed CVE_STATUS settings. (From OE-Core rev: 013d531a84fa08b6ae8a47bdf3ba1fa8f18ba270) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd.bbclass: Fix order of postinst-useradd-*Piotr Łobacz2024-02-241-26/+45
| | | | | | | | | | | | | | | | | | | | | | | | | postinst-useradd-* haven't been running in order of dependency. This patch is reworked from Piotr Łobacz's patch and fixes: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904 basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-* scripts with any order. Sometimes this means, for example a useradd postinst will attempt to run without the corresponding group postinst causing errors. This patch ensures that we first run groupadd, then useradd and then group membership. [RP: Tweaked to avoid removing previous fixes and for whitespace/style issues Also ensure the scripts are changed to execute with -e to highlight errors] (From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> Signed-off-by: Jan Górski <j.gorski@welotec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd.bbclass: Fix missing space when appending vardeps.Siong W.LIM2024-02-191-1/+1
| | | | | | | (From OE-Core rev: c4658401f1331026dc47a859dd665aab74918bba) Signed-off-by: Siong <wslim87@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multilib_global.bbclass: fix parsing error with no kernel module splitChen Qi2024-02-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | The problem could be reproduced with the following settings: MACHINE = "qemux86-64" KERNEL_SPLIT_MODULES = "0" require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32" The error message is as below: bb.data_smart.ExpansionError: Failure expanding variable KERNEL_VERSION_PKG_NAME, expression was ${@legitimize_package_name(d.getVar('KERNEL_VERSION'))} which triggered exception TypeError: expected string or bytes-like object The variable dependency chain for the failure is: KERNEL_VERSION_PKG_NAME -> RPROVIDES:kernel-modules This is because multilib_virtclass_handler_global function in multilib_global.bbclass deletes KERNEL_VERSION. So we need to handle such situation. We'll also need to delete KERNEL_VERSION_PKG_NAME to avoid this parsing error. (From OE-Core rev: 43dd497bc161ac44faecfdff052db03679dbb4f8) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDSAlexander Kanavin2024-01-271-0/+1
| | | | | | | | | | | | | | | | Otherwise, PACKAGE_WRITE_DEPS would contain non-multilib variants of dependencies even when building multilib items, resulting in sysroots being populated with entirely wrong versions of them. This hasn't been noticed until now through sheer (bad) luck, I think, except in the cpio recipe, but the previous commit shows that the issues did occur, quietly. Every other recipe in oe-core and meta-oe does not prepend the multilib prefix. (From OE-Core rev: 234965cb88ccfa9c3a357928f7155b119044e8fc) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sysroot user management postinsts: run with /bin/sh -e to report errors when ↵Alexander Kanavin2024-01-271-1/+1
| | | | | | | | | | | | | | | | | | | they happen This exposes the following failure in a multilib setup, when everything up to do_package_write_rpm is in sstate, but do_package_write_rpm is not (there's a similar fail for lib64-man-db, and the failures themselves will be fixed separately in the next commit): Exception: subprocess.CalledProcessError: Command '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot/usr/bin/postinst-base-passwd' returned non-zero exit status 1. Subprocess output: install: cannot stat '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/share/base-passwd/passwd.master': No such file or directory (From OE-Core rev: 5ffa333db28bc5d8e440c983fdf95589d332461d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* externalsrc: fix task dependency for do_populate_licJulien Stephan2024-01-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | do_populate_lic dependencies are defined inside license.bbclass such as: addtask populate_lic after do_patch before do_build but externalsrc deletes the do_patch task, so the only dependency left for do_populate_lic is "before do_build" On a devtool context, when doing devtool modify, sources are extracted inside build/workspace/sources/${BPN}/ and local files inside build/workspace/sources/${BPN}/oe-local-files When building the recipe after a devtool modify, do_unpack is called again to unpack (possibly modified) local files from build/workspace/sources/${BPN}/oe-local-files into ${WORKDIR}. Since the only left dependency for do_populate_lic is do_build, the do_populate_lic can be called BEFORE do_unpack. Most of the time this is not a problem, because license files are generally located inside ${S}, which corresponds to build/workspace/sources/${BPN} (and is already unpacked), but this can lead to an issue if recipe sets LIC_FILES_CHKSUM to look for files in ${WORKDIR} (example from init-ifupdown_1.0.bb): LIC_FILES_CHKSUM = "file://${WORKDIR}/copyright;md5=3dd6192d306f582dee7687da3d8748ab" So devtool modify init-ifupdown && bitbake init-ifupdown gives the following error: WARNING: init-ifupdown-1.0-r0 do_populate_lic: Could not copy license file <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright to <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/license-destdir/qemux86_64/init-ifupdown/copyright: [Errno 2] No such file or directory: '<...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright' ERROR: init-ifupdown-1.0-r0 do_populate_lic: QA Issue: init-ifupdown: LIC_FILES_CHKSUM points to an invalid file: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright [license-checksum] ERROR: init-ifupdown-1.0-r0 do_populate_lic: Fatal QA errors were found, failing task. ERROR: Logfile of failure stored in: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/temp/log.do_populate_lic.838584 ERROR: Task (<...>/poky/meta/recipes-core/init-ifupdown/init-ifupdown_1.0.bb:do_populate_lic) failed with exit code '1' Fix this by forcing the do_populate_lic task to run after do_unpack (From OE-Core rev: ea6a0cccdd274534809df62a0a196bf83489a1e5) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Unlink vendor dir laterVyacheslav Yurkov2024-01-191-1/+1
| | | | | | | | | | Vendor directory might still be required during install stage, so defer the removal until later stage. (From OE-Core rev: f4538e3884a0cfa07e16e6a2c986271cc7322dd8) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Handle modules from the same repoVyacheslav Yurkov2024-01-191-5/+4
| | | | | | | | | | | Take into account module version when populating vendor directory, because a module with the same URL but with a different version tag could be used as an indirect dependency. (From OE-Core rev: 8f6320c0858941b2441e290ef3586b48c2700cd1) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: go-vendor: Reference local modulesVyacheslav Yurkov2024-01-191-1/+17
| | | | | | | | | | | Create symlinks for local modules, which are usually not referenced in the SRC_URI, but still expected to be found in the vendor directory during the build. (From OE-Core rev: 16da5d9ad448aafd8b5fd63480727bd1b09ec9f1) Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/recipes: Switch to use inherit_deferRichard Purdie2024-01-181-1/+1
| | | | | | | | | | | | | | | | | Now that bitbake supports the use of inherit_defer, switch all conditional (variable based) inherits to use this instead. This leads to more a more deterministic user experience since there is no longer an immediate expansion and later changes to the variables in question (e.g. a bbappend) are accounted for. This patch tries to ensure the behaviour before/after remains as unchanged as it reasonably can, e.g. by always inherting populate_sdk_base. native and nativesdk continue to need to be inherited last, hence being used with inherit_defer in a handful of very specific cases. (From OE-Core rev: 451363438d38bd4552d5bcec4a92332f5819a5d4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* chrpath.bbclass: fix Darwin supportEtienne Cordonnier2024-01-151-1/+1
| | | | | | | | | | | | | | | Ported from the meta-darwin layer: The call to out.split("\n") expects a string, thus the parameter text=True is needed (otherwise Popen returns a bytes object). Note that "text" is just a more readable alias for universal_newlines. (From OE-Core rev: 0abaa7bf7f7d9a5ac96e6fdbe99334cb2fb0e4db) Signed-off-by: Dominik Schnitzer <dominik@snap.com> Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Drop tarfile module to improve performanceRobert Yang2023-12-231-12/+16
| | | | | | | | | | | | | | | | | | | * The tarfile module doesn't support xz options or environment varible XZ_DEFAULTS, this makes do_ar_patched incrediblely slow when the file is large, for example, chromium-x11 is about 3GB: - "bitbake chromium-x11 -car_patched" hasn't been done after 3 hours on my host, I checked the partial tar.xz file is only 1.5GB, so maybe more than 6 hours is required to complete the task. - Now only less than 4 minutes is needed on the same host. * Need add xz to HOSTTOOLS when archiver.bbclass is enabled and compression is xz. (From OE-Core rev: 6548354f049b173e8d443bc547d35c9d9fc05259) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx-2.2: combine spdx can try to write before dir creationJeremy A. Puhlman2023-12-171-1/+1
| | | | | | | | | | | | | | | | | | | On occasion a file is attmpeded to be opened prior to the creation of the spdx_workdir. Create the directory before the open, just in case. File: '/build/layers/poky/meta/classes/create-spdx-2.2.bbclass', lineno: 1081, function: combine_spdx 1077: ) 1078: 1079: image_spdx_path = spdx_workdir / (rootfs_name + ".spdx.json") 1080: *** 1081: with image_spdx_path.open("wb") as f: 1082: doc.to_json(f, sort_keys=True, indent=get_json_indent(d)) (From OE-Core rev: bb9f2a9c0ff5dcdeaf1a0beb6a614d0d022a2481) Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Improve work-shared checkingRobert Yang2023-12-131-4/+2
| | | | | | | | | | | | There are other recipes except the listed ones which use work-shared, improve the checking to make other recipes such as llvm-project-source work with do_ar_patched. (From OE-Core rev: 5fbb4ca8da4f4f1ea426275c45634802dcb5a575) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd: Fix useradd do_populate_sysroot dependency bugRichard Purdie2023-12-101-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | If a task is adde which has a dependency on the do_populate_sysroot task of the recipe, it will cause it to be installed into the sysroot (similar to do_addto_recipe_sysroot). This fails since the postinst script is an overlapping file: Exception: FileExistsError: [Errno 17] File exists: 'tmp/sysroots-components/all/useraddbadtask/usr/bin/postinst-useradd-useraddbadtask' -> 'tmp/work/all-poky-linux/useraddbadtask/1.0/recipe-sysroot/usr/bin/postinst-useradd-useraddbadtask' The copy written out at do_prepare_recipe_sysroot time is just for debug so rename it, meaning there are no longer overlapping files and the installation can be successful, removing the error. [YCOTO #14961] With the bug fixed, enable the test. (From OE-Core rev: 564339afb73fc52a66c1a08437587cad1c4d46e7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd: Fix issues with useradd dependenciesEilís 'pidge' Ní Fhlannagáin2023-12-081-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | If recipe A requires the useradd actions of recipe B we need to ensure that recipe B is part of the recipe A dependancy chain. In order to do that, we introduce USERADD_DEPENDS. This makes sure that the do_populate_sysroot_setscene of recipe B exists for recipe A in case of a missing TMPDIR. This requires changes made in runqueue.py by RP. This commit along with the runqueue fixes effects: Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens) Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails Bug 13279 - Make sure users/groups exist for package_write_* tasks Bug 15084 - For some reason using of same user in two recipes does not work properly I've included the start of self-testing for useradd by adding tests for 13419 (which ends up testing 13904, 13279, 15084 by virtue of them all having the same root cause) (From OE-Core rev: b47f2352376bd16b7e7087b4dab143403e67e094) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "cve-check: Modify judgment processing using "=" in version comparison"Ross Burton2023-12-051-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change introduced a warning if version comparisons failed, but this is far too common an issue in data that we don't control, so this shouldn't cause a warning: WARNING: automake-native-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4539 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4644 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2011-0715 WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2011-0715 WARNING: automake-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2003-0577 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-0982 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-1284 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2006-3355 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2007-0578 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2007-0578 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2009-1301 WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2009-1301 This reverts commit a1989e4197178c2431ceca499e0b4876b233b131. (From OE-Core rev: c7c7dbdd5474002cfd9ec24864e77a0df2b790ea) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Modify judgment processing using "=" in version comparisonMatsunaga-Shinji2023-12-021-0/+1
| | | | | | | | | | | | | | | | Judgment processing of vulnerable using "=" compares characters as strings rather than numbers, and misjudges "cases that do not match in strings but do match in numbers" as "Patched". (e.g. PV = "1.2.0" and Vulnerabilities Affected Versions (registered with NVD) = "1.2") Therefore, if the comparison operator used in the judgment processing of vulnerable is "=", add numeric comparison processing. (From OE-Core rev: a1989e4197178c2431ceca499e0b4876b233b131) Signed-off-by: Shinji Matsunaga <shin.matsunaga@fujitsu.com> Signed-off-by: Shunsuke Tokumoto <s-tokumoto@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-15 14:25:03 +0000