summaryrefslogtreecommitdiffstats
path: root/meta/classes-global
Commit message (Collapse)AuthorAgeFilesLines
* meta/meta-selftest: Fix variable assignment whitespaceRichard Purdie2025-02-011-1/+1
| | | | | | | | | | Recipes are much more readable with whitespace around the assignment operators. Fix various assignments in OE-Core to show this is definitely the preferred formatting. (From OE-Core rev: 30ea609d3357fb3de911f2f6a5e6856c151b976a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sanity: Add test for functional c++ toolchainChristos Gavros2025-01-271-0/+25
| | | | | | | | | | | | Users reported issues caused by missing the right libstdc++-version-dev. A new function 'check_cpp_toolchain' added in sanity.bbclass to test linking libstdc++ [YOCTO #15712] (From OE-Core rev: 611c1a26212dfbfe8d0640d9fefe5df49f7b69b8) Signed-off-by: Christos Gavros <gavrosc@yahoo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/recipes: Switch virtual/XXX-gcc to virtual/cross-cc (and c++/binutils)Richard Purdie2025-01-212-13/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea of the base class dependency is to say "yes, I need a C cross compiler" and this was never meant to be gcc specific. Looking at the codebase, whilst we code triplets into this, it does overcomplicate things as there are only ever limited, "target", "sdk" and the class extended versions like mutlilib. After much thought, we can simplify this to virtual/cross-cc and virtual/nativesdk-cross-cc. This lets us remove the "gcc" specific element as well as removing the over complicated triplet usage. At the same time, change the much less widely used "g++" variant to "c++" for similar reasons and remove the triplet from virtual/XXX-binutils too. Backwards compatibility mappings could be left but are just going to confuse things in future so we'll just require users to update. This simplification, whilst disruptive for any toolchain focused layers, will make improved toolchain selection in the future much easier. Since we no longer have overlapping variables, some code for that can just be removed. The class extension code does need to start remapping some variables but not the crosssdk target recipe names. This patch is in two pieces, this one handles the renaming with the functional changes separate in a second for easier review even if this breaks bisection. (From OE-Core rev: 4ccc3bc8266c327bcc18c9a3faf7536210dfb9f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* kernel-yocto: make kernel commits reproducibleEnrico Jörns2025-01-211-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The git commit hashes for the kernel checkout are not reproducible under certain conditions: - If the git repository is initialized on an archive (rather than a git), the initial git commit not only has the current user name set, it also uses the current system time as committer and author date. This will affect the initial git hash and thus all subsequent ones. - The patches applied by the kern-tools have a valid author and date. However, their committer again depends on the user building the BSP. This is an issue, for example, if one compiles a kernel with CONFIG_LOCALVERSION_AUTO enabled where the commit hash lands into the kernel and thus the package version. This not only makes the package version non-reproducible, but also leads to version mismatches between kernel modules built against a fresh kernel checkout and the kernel retrieved from the sstate cache. The class uses 'check_git_config' from utils.bbclass, but this only sets the git user and only if none existed before. Thus it doesn't really help here. Since in Git the committer information can be set only from the environment variables GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL, and GIT_COMMITTER_DATE, we introduce a helper function to set those and apply the author settings in the same way. As values simply use PATCH_GIT_USER_NAME, PATCH_GIT_USER_EMAIL (from patch.bbclass) and SOURCE_DATE_EPOCH. For convenience, put the new helper 'reproducible_git_committer_author' into utils.bbclass next to 'check_git_config' so others can use it, too. Using this helper in kernel-yocto.bbclass makes the committer and author date/name/email for the initial commit reproducible, as well as the committer name/email for the patches applied with kern-tools. For debugging purpose, allow disabling the reproducibility features by setting KERNEL_DEBUG_TIMESTAMPS to "1". Suggested-by: Felix Klöckner <F.Kloeckner@weinmann-emt.de> (From OE-Core rev: aab4517b4649917abd519ea85a20fd9d51bf3d99) Signed-off-by: Enrico Jörns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/logging: Added the bbverbnote command to the logging setBastien JAUNY2025-01-201-0/+10
| | | | | | | | | | | Fixes [YOCTO #15688] Reviewed-by: Yoann Congal <yoann.congal@smile.fr> (From OE-Core rev: 48e4b8c06e503868404b99bf45a46a52794baa5a) Signed-off-by: Bastien JAUNY <bastien.jauny@smile.fr> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Ensure errors cause task failureRichard Purdie2025-01-081-0/+4
| | | | | | (From OE-Core rev: c1491ef7afc8722927c568b0ac0ddbbc84837ca3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_rpm: set a timestamp handling policy with the new macro ↵Bin Lan2025-01-041-1/+1
| | | | | | | | | | | | | | | | %build_mtime_policy There is the following warning when the do_package_write_rpm task is executed when bitbake linux-yocto: warning: %clamp_mtime_to_source_date_epoch is deprecated, please use %build_mtime_policy After RPM is upgraded to 4.20.0, the %clamp_mtime_to_source_date_epoch macro is now deprecated in favor of %build_mtime_policy set to clamp_to_source_date_epoch. (From OE-Core rev: fa9b9e599bbf3ac73c7a9b1d15ccb0e0bbbe4c11) Signed-off-by: Bin Lan <bin.lan.cn@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sanity.bbclass: Lower the length of TMPDIR limit to 400Changqing Li2025-01-031-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When TMPDIR has length 410, bitbake uninative-tarball will fail at do_populate_sdk with error: Failed to open backup database: "TMPDIR/work/x86_64-nativesdk-pokysdk-linux/uninative-tarball/1.0/sdk/image/var/lib/dnf/history.sqlite" sqlite3 has a hardcode max pathname limit 512, refer [1], and there is a check, refer [2], pathname + 8 > max path name limit(512) is not allowed. So the pathname length need to <= 504. While, length of "TMPDIR/work/x86_64-nativesdk-pokysdk-linux/uninative-tarball/1.0/sdk/image/var/lib/dnf/history.sqlite" is 505, so error happened. We had met similar issue [3], and refer the comments, upstream reject the change of the hardcode length, see [4][5]. This patch try to improve and workaround this issue by lower max TMPDIR length to 400, at least for distro name length <=13, the build can success. [1] https://github.com/sqlite/sqlite/blob/master/src/os_unix.c#L180 [2] https://github.com/sqlite/sqlite/blob/master/src/pager.c#L4777 [3] https://bugzilla.yoctoproject.org/show_bug.cgi?id=12374 [4] https://sqlite.org/cgi/src/tktview/c060923a5422590b3734 [5] https://sqlite.org/forum/forumpost/0b1b8b5116 (From OE-Core rev: da1c0998bdaf5def7ae94f44a33b9a505edc99c1) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base/features_check: Make PARSE_ALL_RECIPES and SOURCE_MIRROR_FETCH booleanRobert Yang2024-12-231-3/+4
| | | | | | | | | | So that value "0" can turn them off. (From OE-Core rev: 1a2afcd0773c5ec5946d1dfc27bde585c52a5724) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/utility-tasks: port do_listtasks to use bb.build.listtasksRoss Burton2024-12-121-12/+11
| | | | | | | | | | | Instead of iterating every datastore value by hand to find tasks, use the new bb.build.listtasks() function (bitbake 185c4b) (From OE-Core rev: 466c3ed0d01bc70caa29d5eb8bb99f7d0e6e710c) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rpm: update 4.19.1.1 -> 4.20.0Alexander Kanavin2024-12-121-1/+1
| | | | | | | | | | | | | | | | | | | | | With this release, as expected, deprecated openpgp support has been fully removed and replaced with sequoia: https://github.com/rpm-software-management/rpm-sequoia/ Alas, it's written in rust, and has no recipe, so for now signing rpm packages has to be disabled. Remove package name parameter to %prep in source archiving, as it's been dropped in the new rpm, and wasn't needed to begin with. Drop 0001-perl-disable-auto-reqs.patch (files removed upstream). (From OE-Core rev: 8c15b4577d5e554cc2dd5adfb88b816894b05a9a) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base.bbclass: fix RRECOMMENDS extend issue for multilibChangqing Li2024-12-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During handle PACKAGECONFIG, DEPENDS/RDEPENDS/RRECOMMENDS is appended by calling function appendVar according to the PACKAGECONFIG setting: PACKAGECONFIG[f1] = "\ --with-f1, \ --without-f1, \ build-deps-for-f1, \ runtime-deps-for-f1, \ runtime-recommends-for-f1, \ packageconfig-conflicts-for-f1" [snip of base.bbclass] pkgconfigflags = d.getVarFlags("PACKAGECONFIG") or {} if pkgconfigflags: ... appendVar('DEPENDS', extradeps) appendVar('RDEPENDS:${PN}', extrardeps) appendVar('RRECOMMENDS:${PN}', extrarrecs) ... [snip of base.bbclass] After following commit in meta-oe, [fe3ba7ac5c cryptsetup: demote udev runtime dependency to recommendation] 'udev lvm2' changed from RDEPENDS to RRECOMMENDS, but RRECOMMENDS is not handled in this process, and causes following error: $bitbake lib32-cryptsetup ERROR: QA Issue: lib32-cryptsetup package lib32-cryptsetup - suspicious values 'udev lvm2' in RRECOMMENDS [multilib] In layer openembedded-core, we do have very few recipes that have runtime-recommends-for-f1 in PACKAGECONFIG setting, but f1 is not enabled in PACKAGECONFIGS, so not report error when multilib is enabled. (From OE-Core rev: 2f539a1318a80d3ad5d0d43a9ba4070465e402af) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes-global/insane: Handle case where RPROVIDER is also a providerJoshua Watt2024-12-121-8/+13
| | | | | | | | | | | | | | | | | | | The check to see if a provider of a given package is listed first checks for an exact match of the provider name. However, if this match existed, but didn't match in the task dependencies, it would not continue to look for other providers of package. This would manifest if one (non-virtual) recipe package RPROVIDES the name of a package produced by another recipe. Fix this, and also clean up the code to make it more readable by using a function to check if a runtime dependency is in the task dependencies. In addition, if no provider is found, list all the possible providers instead of the last one that was looked at. (From OE-Core rev: f13de6ab616eb1e38960a2296111febe2a9f4a28) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib: move buildstats.py and rootfspostcommands.py into oeChen Qi2024-12-051-2/+2
| | | | | | | | | | | These two files are the only ones that are left in meta/lib. They logically belong to meta/lib/oe, so move them there. (From OE-Core rev: c65dd0e3e463d6072b9364ac74e1fef0d998068f) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-check-layer: expand to cover all required QA checksDenys Dmytriyenko2024-12-031-0/+28
| | | | | | | | | | | | | | insane.bbclass now defines CHECKLAYER_REQUIRED_TESTS list with required QA checks that are becoming mandatory for Yocto Project Compatible layers. Update yocto-check-layer.bbclass in order to catch when packages from such layers try to skip any of the required QA checks. (From OE-Core rev: 9c3ba88628853b20fb4c98c99cf3fe8349024016) Signed-off-by: Denys Dmytriyenko <denys@konsulko.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Split ERROR_QA into CHECKLAYER_REQUIRED_TESTSRichard Purdie2024-11-281-19/+18
| | | | | | | | | | | | | | | We'd like to start requiring some set of checks are enabled for Yocto Project Compatible Status and to pass yocto-check-layer. Start by splitting ERROR_QA into two sets, the ones we think can be required and the ones we know have challenges to implement (e.g. with prebuilt binaries). To change the required list, the YP TSC would need to approve. (From OE-Core rev: ce2e42ace2d15fb6745437cf0a7f07d28398ca12) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_rpm: restrict rpm to 4 threadsAlexander Kanavin2024-11-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TL;DR version: with this, and the previous compression level changes I am seeing drastic speedups in package_write_rpm completion times: webkitgtk goes from 78 seconds to 37 seconds glibc-locale goes from 399 seconds to 58 seconds (!) The long version: rpm uses multithreading for two purposes: - spawning compressors (which are nowadays themselves multi-threaded, so the feature is not as useful as it once was) - parallel file classification While the former behaves well on massively parallel CPUs (it was written and verified here :), the latter was then added by upstream and only benchmarked on their very old, slow laptop, apparently: https://github.com/rpm-software-management/rpm/commit/41f0e214f2266f02d6185ba11f797716de8125d4 On anything more capable it starts showing pathologic behavior, presumably from spawning massive amount of very short-lived threads, and then having to synchronize them. For example classifying glibc-locale takes 5m20s with 256 threads (default on my machine!) 1m49s with 64 threads 59s with 16 threads 48s with 8 threads Even a more typical recipe like webkitgtk is affected: 47s with 256 threads 32s with 64 threads 27s with 16 or 8 threads I have found that the optimal amount is actually four: this also means that only four compressors are running at a time, but as they're themselves using threads, and typical recipes are dominated by just two or three large packages, this does not affect overall completion time. (From OE-Core rev: 286d456e71ee2730c197ce394d6be2c7eeced18d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_rpm: use zstd's default compression levelAlexander Kanavin2024-11-211-1/+1
| | | | | | | | | | | | | | | | | | | | zstd uses 3 by default (and ZSTD_COMPRESSION_LEVEL is set to that), while 19 is the highest and slowest. It's not clear why 19 was picked to begin with, possibly I copy-pasted it from rpm's examples without thinking: https://git.yoctoproject.org/poky/commit/?h=master-next&id=4a4d5f78a6962dda5f63e9891825c80a8a87bf66 This brings significant speedups in rpm's compression step: for example compressing webkitgtk takes 11s instead of 36s. The rpm size increases from 175648k to 234860k. I think it's a worthy default tradeoff. (From OE-Core rev: f1c7d76361425c85d68ad1f61be1e7ff05df4bab) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: rewrite sstate_archive_package in pythonRoss Burton2024-11-181-27/+26
| | | | | | | | | | | | | | As sstate_archive_package just calls tar, writing the function in shell is actually more complex and opaque than the equivalent python. Don't check for zstd vs pzstd, we have pzstd in HOSTTOOLS so it will always be available. (From OE-Core rev: c9ac5d9d94f254292cf3cafdf273dd6b61d3baa7) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* do_package/sstate/sstatesig: Change timestamp clamping to hash output onlyRichard Purdie2024-10-281-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code was changing the timestamps of the files in the do_package output, particularly the files added for debug sources. This was to do two things: a) make do_package sstate more reproducible b) ensure better hash equivalence matching Unfortuately the debug source files are hardlinks into the source tree for efficiency so touching these, touches a lot of files in ${B} and ${S}. This causes unpredictable effects if compile is run again for example, or could cause compiling in the install task. The hash equivalence matching is of key importance but we can mimic that using clamping of the file timestamps in the depsig output used to generate the hashes. This patch drops the global timestamp clamping, instead allowing the files to retain their creation timestamps into sstate. This makes do_package sstate slightly less reproducibile. We could clamp the sstate timestamps but that would lead to two different sets of timestamps depending on whether the data came from sstate or not. I'd prefer to have consistent code behaviour, rather than differing behavhour depending on whether data came from sstate or not. If we wanted to have reproducibiliy and fix the "corruption" of S/B and have consistent codepaths, the only other option would be two copies of the sources, which could end up huge and seems the least desireable option. This patch therefore drops the timestamp clamping in the sstate files and tweaks the depsig data generation to clamp the timestamps for do_package instead since this seems the best compromise. I validated that rpm/deb/ipk files still generate correctly as before. (From OE-Core rev: 475759fdab7200488b2a568b2ba1aa31a456d113) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/package: Check incompatible licenses at packaging timeJoshua Watt2024-10-252-5/+0
| | | | | | | | | | | | | | | | | | | Instead of checking for incompatible licenses in the anonymous python and setting '_exclude_incompatible-', (re)check all the packages in populate_packages(). This ensures that all packages are processed, even dynamically generated ones. The use of the '_exclude-incompatible-' variable set in base.bbclass has been the mechanism used for per-packages licenses since it was added as a feature (although with different names for the variable throughout history). However, since this misses dynamic packages, calling oe.license.skip_incompatible_package_licenses() a second time on the actual final package set is a better solution. (From OE-Core rev: 1816c5a5e724a7f907c9afe4a47b6699ea081ace) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/license: Move package license skip to libraryJoshua Watt2024-10-251-31/+4
| | | | | | | | | | Moves the code that skips packages with incompatible licenses to the library code so that it can be called in other locations (From OE-Core rev: 86eb409e3c1b30110869ec5a0027ae2d48bbfe7f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes-global/license: Move functions to library codeJoshua Watt2024-10-252-170/+5
| | | | | | | | | | | | | | Moves several of the functions in license.bbclass to be library code New function dependencies were manually verified using bitbake-dumpsigs to ensure that bitbake identified the same dependencies even though they are now in library code (although the new function names mean that the task hashes still change) (From OE-Core rev: 0333e04e353991260c5f67a72f80f3ab9dcf526a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Ensure package_qa tasks run in builds when expectedRichard Purdie2024-10-241-0/+2
| | | | | | | | | | | | | Currently, if you "bitbake XXX" and XXX depends on something else, the do_package_qa teask for that something may not run. Users would generally expect it to have though. Add in the missing dependency to ensure that do_build does trigger the right package_qa tasks. (From OE-Core rev: e0beb64c6d3cf1d649f79a8704fb25cdf83b4a8b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sanity.bbclass: skip check_userns for non-local uidChangqing Li2024-10-221-0/+2
| | | | | | | | | | | | | Bitbake preserve network for non-local uid, refer [1], so check_userns is not needed for non-local uid [1] https://git.openembedded.org/bitbake/commit/?id=4eafae7904bae6e5c6bc50356e8a9077f2e207fa (From OE-Core rev: 808d0cece22bcbee15236717e158da247cbedaf1) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: avoid race condition when DEBIAN/CONTROL entries are removedRoss Burton2024-10-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race condition when iterating directories which are being altered whilst iterating, which is something that can and does happen when do_package_qa runs at the same time as eg do_package_write_ipkg (the opkg metadata is written inside the build tree). The race is that naive code will list a directory contents and then stat() each name to determine if its a directory or file. The classic failure that we see is that CONTROL/ is found on a listdir but deleted by the time the stat happens, so is incorrectly listed as a file (because it is not a directory). Since Python 3.5, os.walk() uses scandir() instead of listdir() which mitigates this race by returning the file type alongside the name, so a stat is no longer needed to identify the type. However, cachedpath.walk() was copied from Python before this, so it uses listdir() and has this race condition. Since I changed insane to use cachedpath.walk()[1] I inadvertently reintroduced this race. I believe there's actually no need to use cachedpath.walk() and a logical fix is to simply use os.walk(): With os.walk() each directory is listed and categorised in a single os.scandir() as the underlying syscall, getdents64, returns the type. However, cachedpath.walk() uses os.listdir() which ignores the type field returned and has to do a stat() on every file to determine the type. Thus, we should switch users of cachedpath.walk() to os.walk(): there's no real gain in what is effectively just a prefetch for the stat cache, but depending on what the calling code does may result in more stat() calls than needed. In the future we may want to redesign cachedpath to reimplement walk so that it can also cache the DirEntry instances as returned by scandir() as that will avoid needing to call stat() at all in many cases. However I believe we should instead use a caching pathlib.Path instance instead. [1] cad3c8 insane: use oe.cachedpath.CachedPath instead of os.path (From OE-Core rev: 22e4486d65e4874bf48d89160d69118f318278e8) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Don't used cachedpath for os.lstat()Richard Purdie2024-10-141-3/+2
| | | | | | | | | | | The functions behave slightly differently to the functions they're caching and the use in insane.bbclass isn't compatible. For now, to avoid build failures, switch back to the stat calls. We may be able to improve cachedpath or change the call sites. (From OE-Core rev: fa771ae887ab5152f043748cf3419735831bcf7b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_rpm: Check if file exists before open()Pavel Zhukov2024-10-111-4/+12
| | | | | | | | | | | | | | | | Exception handler tries to read() /etc/passwd file in sysroot and if file doesn't exist for any reason then it raises FileNotFoundError exception which mask the original source of the problem and makes debugging of the issue more difficult. Fixes: Exception: FileNotFoundError: [Errno 2] No such file or directory: '/codebuild/output/src1899304708/src/build/tmp-container/work/core2-64-oe-linux/emqx-bin/4.3.12/recipe-sysroot/etc/passwd' (From OE-Core rev: 4ad9a0e0b11eb7bc5a3dd45fc8945e094ea949e9) Signed-off-by: Pavel Zhukov <pavel@zhukoff.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: rewrite package_qa_check_archRoss Burton2024-10-111-16/+25
| | | | | | | | | | Reorder and comment the architecture checks to make it clearer what they are actually checking. (From OE-Core rev: 78db9e79e1a307ffb8436e26656bfb98efb513bc) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: use oe.cachedpath.CachedPath instead of os.pathRoss Burton2024-10-111-14/+28
| | | | | | | | | | | | | | | | | | | The insane QAPATHTESTs make many os.stat() calls, the majority of which are redundant with caching as the initial sweep does a stat() on every entry to determine if it is a file or a directory, and from then on each test that does further stat()s is redundant as the tree doesn't change. Switch os.stat() and friends (os.path.isfile(), etc) to use a common oe.cachedpath.CachedPath() instance that is shared between all of the functions, meaning only one stat is done. In my test case of ltp:do_package_qa, this reduces the time taken from 44s to 37s. (From OE-Core rev: cad3c889439fd6a007debd6f2f6578f4a1e16c9c) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: optimise test in package_qa_check_shebang_sizeRoss Burton2024-10-111-2/+1
| | | | | | | | | | Check whether the elf is not None first, before doing os.stat() calls on disk. Also don't check anything that isn't a file, not just FIFOs. (From OE-Core rev: 38454a2675f38c7db55efcb67bbb8b9fef7e0bf1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: micro-optimise the sweep of pkgfilesRoss Burton2024-10-111-16/+11
| | | | | | | | | | | | | | Don't actively do more work: - Exit early if there are no packages being generated - Don't iterate repeatedly when removing CONTROL and DEBIAN - Extend a list with another list instead of appending item by item - Remove unused variables (From OE-Core rev: 79ffb8896d570dd935d3aea9d28ee20b52e1674a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: tidy up objdump preloading in package_qa_walk()Ross Burton2024-10-111-10/+11
| | | | | | | | | | | | | | Move the prepopulate function out of global scope, and access the dictionary once instead of repeatedly. This still results in each ELF being opened twice, but this avoids opening all of the files at once and the ELFFile.open() call is fairly fast. (From OE-Core rev: cda3647b32703f43c4fe2af3bab977e5698633f6) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: check for RUNPATH as well as RPATHRoss Burton2024-10-111-4/+4
| | | | | | | | | | | | | | Since oe-core 66f8a7 merged in 2023[1], ld sets DT_RUNPATH instead of DT_RPATH when -rpath is specified, which we don't check for. Update the insane tests to look at both RPATH and RUNPATH. [1] oe-core 66f8a745668a067d8d763fa2af3e65f26c9c1ebe (From OE-Core rev: d6c5076d179a3d5ebb74b719ec4d523c197c1918) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: only parse ELFs if they're files, not symlinksRoss Burton2024-10-111-19/+2
| | | | | | | | | | | This reduces the number of files that need to be swept by not scanning eg the library symlinks, and means we can remove the explicit islink() checks in many of the tests. (From OE-Core rev: aa9ec4b5c719bf610ad953095d1111e4c257747e) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sanity: check for working user namespacesRoss Burton2024-09-131-0/+24
| | | | | | | | | | | | | | | | If user namespaces are not available (typically because AppArmor is blocking them), alert the user. We consider network isolation sufficiently important that this is a fatal error, and the user will need to configure AppArmor to allow bitbake to create a user namespace. [ YOCTO #15592 ] (From OE-Core rev: b6af956fe6e876957a49d4abf425e8c789bf0459) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Remove dependency on TCLIBC from QA testRichard Purdie2024-09-051-0/+1
| | | | | | | | | | The TCLIBC value is already encoded into build paths through the triplet so no need to encode it here where it can cause problems for allarch output that span multiple libcs. (From OE-Core rev: ea8c7a457a79589c35ca80b2f265799164855674) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Make do_recipe_qa and do_populate_lic non-arch specificRichard Purdie2024-09-051-2/+2
| | | | | | | | | | | The sstate functions currently pull in STAMP and SSTATE_PKG which end up pulling in DEFAULTTUNE and other variables. The location on disk encodes all the "architecture" information we need so clean up the dependencies of these tasks and make them non-architecture specific. (From OE-Core rev: 65df61ccff6781906449bfea386a8dd13112a51c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* license: Fix directory layout issuesRichard Purdie2024-09-051-2/+8
| | | | | | | | | | | | | | | | There are several issues: a) pointless empty directories were being created as a path wasn't fixed in a previous commit b) SSTATE_PKGARCH wasn't being captured into the task signature since it is in the ignore list by default. We want to capture the absolute value, not the dependencies c) with those issues fixed, cross/native issues became apparent so those need to be fixed too. (From OE-Core rev: f68aed302a0e4b86fb8c16a6ef4e7295bed48b86) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Drop SSTATEPOSTINSTFUNC supportRichard Purdie2024-09-041-6/+0
| | | | | | | | | | | This was deprecated with the introduction of postfunc support for tasks in general and only used by buildhistory. Now that usage has been removed, drop the code from sstate.bbclass. Any other users should be able to use postfuncs too. (From OE-Core rev: 74e08170a5584d83f5f03cd8a71978b5e0895c1d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usageRichard Purdie2024-09-041-1/+4
| | | | | | | | | | | | | | | | We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of postfuncs. Finally get around to doing that which should make the buildhistory code a little more readable. Unfortunately ordering the buildhistory function calls after the sstate ones is difficult without coding that into the sstate class. This patch does that to ensure everything functions as expected until we can find a better way. This is still likely preferable than the generic sstate postfuncs support since the function flow is much more readable. (From OE-Core rev: c9e2a8fa2f0305ef1247ec405555612326f798f8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* abi-version/ssate: Bump to avoid systemd hash corruption issueRichard Purdie2024-09-011-1/+1
| | | | | | | | | | | | Unfortunately some recent patches caused non-deterministic output. One input hash lead to both good and bad output and whilst that patch has been fixed, the problematic hash 'cross' linkage remains. Bump to a new sstate and hash equivalence version to avoid this and work from a clean slate. (From OE-Core rev: 639e42b9c14dff516688964dba4ab25bba7b8a55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Remove redundant returnsOla x Nilsson2024-09-011-3/+0
| | | | | | | | | | Some redundant return statements were left over from insane: Drop oe.qa.add_message usage (From OE-Core rev: 1e49635f802b04acad14115640ce9fcd63cc32a7) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Allow ERROR_QA to use 'contains' hash optimisations for do_package_qaRichard Purdie2024-08-291-34/+36
| | | | | | | | | | | | | | | | | | | We want ERROR_QA to operate using the "contains" optimizations which means accessing the variable only using the contains function. To do this, remove usage of ALL_QA, open coding the few references to check both WARN_QA and ERROR_QA. Move the function table generation to a separate function where we can exclude the ERROR_QA and WARN_QA variables since they are handled by the handle_error() function calls. Ensure all the chain of functions to the handle_error calls is correctly recognised in the variable dependencies. (From OE-Core rev: 384e9a6b2e7943b6a3ade1215ed79351c78a0b0d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Further simplify codeRichard Purdie2024-08-291-27/+14
| | | | | | | | | Now handle_error is used, we can further simplify the QA test execution as we don't need seperate function lists for warnings and errors. (From OE-Core rev: 6896c9fcfc57f007c0ce15f7804e79b6b88f5ded) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Add missing vardepsexcludeRichard Purdie2024-08-291-0/+2
| | | | | | | | | | | | If we improve the function dependency visibility in insane.bbclass, it exposes some dependencies which were previously not seen causing variances in the do_package_qa task checksums. Update vardepsexclude in a couple of test cases to ensure the sstate hash selftests pass and the taskhashes don't vary when we don't expect them too. (From OE-Core rev: 9b6dae2771ed86bd2946548004f4da58e8c0b44c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Drop oe.qa.add_message usageRichard Purdie2024-08-291-138/+87
| | | | | | | | | | | | | | | Drop the oe.qa.add_message() usage in favour of oe.qa.handle_error() which has code allowing it to be optimised with contains usage. The patch also drops unused return values which we stopped using a while ago and drops the now unneeded function parameters, generally leading to cleaner code. The code should be functionally equivalent. (From OE-Core rev: 9b2eea9fd4eab4f5e12e955738db22091b91f698) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Optimise ERROR_QA/WARN_QA references in do_populate_sysrootRichard Purdie2024-08-291-3/+2
| | | | | | | | | | The new contains code can't inspect variable references in handle_error() calls. Expand what is effectively a hardcoded reference anyway so the code can optimise it. (From OE-Core rev: 51f767d92efb3daeb4aa3b91d72e6d2993cb0f46) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: split guess_license functionEnguerrand de Ribaucourt2024-08-231-4/+4
| | | | | | | | | | | | | | | The npm recipetool handler redefines the license code the could be unified. In order to do this refactoring, extract the bits we'll need into separate functions. guess_license() is renamed to find_licenses() and is split into find_license_files() and match_licenses(). (From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2) Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* license_image.bbclass: Rename license-incompatible to license-exceptionPeter Kjellerstedt2024-08-211-1/+1
| | | | | | | | | | | | | | | | There is currently both an incompatible-license and a license-incompatible QA message. This is very confusing. However, license-incompatible is only used to output a message when a package is included in an image despite it having a license that is normally incompatible (by using the INCOMPATIBLE_LICENSE_EXCEPTIONS variable). To better match how it is used and to distinguish it from incompatible-license, rename it to license-exception. (From OE-Core rev: d309eed66f5a4a4bce082536e51207fe65725fab) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-07-24 19:09:52 +0000