linux/poky.git/meta/lib/oe/sstatesig.py, branch mickledore-next

sstatesig: Improve output hash calculation

2023-02-15T10:21:34+00:00

Symbolic links to the files are included during the output hash calculation but symlinks to the directories are missed. So if the new symlink to a directory was the only change made, then the output hash won't change, and the Hash Equivalence server may change unihash. In the next run bitbake may use an older package from sstate-cache. To fix this followlinks=True flag could be set for os.walk but it can lead to infinite recursion if link points to a parent directory of itself. Also, all files from a directory to which symlink points would be included in depsig file. Therefore another solution was applied, I added code that will loop through directories and process those that are symlinks. (From OE-Core rev: ee729163f31f26b1462a47e1e53f7a0f9de9b464) Signed-off-by: Mateusz Marciniec Signed-off-by: Tomasz Dziendzielski Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

lib/sstatesig: Drop OEBasic siggen

2022-12-11T23:20:14+00:00

We're now used to using hashes as part of the task hashes and the sstate code relies on this. The older OEBasic hash approach therefore wouldn't work and can be removed. (From OE-Core rev: 3667e589ba16eb261cfd72c2b11429f482c239f6) Signed-off-by: Richard Purdie

sstatesig: emit more helpful error message when not finding sstate manifest

2022-11-20T08:29:07+00:00

Since oe-core commit 64b89f3c8fc31842256c482a3039d90d3f12c1cc ("sstatesig.py: make it fatal error when sstate manifest isn't found") errors like: | Manifest [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot not found in imx8mm_dummy cortexa53-mx8mm cortexa53 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')? are fatal now and cannot be ignored but must be debugged. Unfortunately, the currently emitted error message is a bit imprecise with telling the reader what has actually gone wrong. This commit: * adds the word 'sstate' to the error message to clarify the scope we are dealing with ('sstate manifests', since there are other manifests, too) * does not randomly print the last manifest file searched for as THE manifest file that could not be found Instead, we print the name of the task the sstate was searched for * adds the word 'multilib' to variant to make clear which variant we are talking about * adds a separate line noting the searched pkgarchs and adds explicitly mentions this word ('pkgarchs') * prints a list of ALL manifest file locations attempted * removes the '?' at the end of the message since such errors indeed leave the question of what is the cause but the error message itself is more like a statement. The result for the exact same issue as noted above then looks as follows: | The sstate manifest for task 'dbus:populate_sysroot' (multilib variant '') could not be found. | The pkgarchs considered were: imx8mm_dummy, cortexa53-mx8mm, cortexa53, armv8a-crc, armv8a, aarch64, allarch, x86_64_x86_64-nativesdk. | But none of these manifests exists: | [..]/tmp/sstate-control/manifest-imx8mm_dummy-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-mx8mm-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-crc-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-aarch64-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-allarch-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot (From OE-Core rev: 735ec126ec219c7cb89cb05b0e433201bb7f59eb) Signed-off-by: Enrico Jorns Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

sstatesig: skip the rm_work task signature

2022-11-14T16:19:43+00:00

We can skip the rm_work task signature to avoid running the task when we remove some tasks from the dependencie chain. The inject_rm_work handler on the rm_work bbclass triggers the rm_work task running for any signature change in the dependencie chain of the task do_build of each recipe. i.e INHERIT:remove = "create-spdx" will trigger the do_rm_work when we collect the sstate cache with INHERIT = "create-spdx" (From OE-Core rev: 292305700e39d0ebd64763f5032c39ace5005fad) Signed-off-by: Jose Quaresma Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

lib: Add copyright statements to files without one

2022-08-12T11:00:43+00:00

Where there isn't a copyright statement, add one to make it explicit. Also add license identifiers as MIT if there isn't one. (From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040) Signed-off-by: Richard Purdie

sstatesig: Include all dependencies in SPDX task signatures

2022-07-19T09:54:20+00:00

SDPX generation involves looking through BB_TASKDEPDATA for dependencies, then linking to the generated documents for those dependencies. These document links use a checksum to validate the document, which means that if a upstream document changes, all downstream documents must be regenerated to get the new checksum, otherwise the compendium of documents produced by the build will have broken links; therefore all dependent task should be included in the signature (even from "ABI safe" recipes). (From OE-Core rev: 5fe543b9ceec971cf0297ff0ae3b0ccc4703cece) Signed-off-by: Joshua Watt Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

lib/sstatesig: Fix find_siginfo to match sstate filename generation

2022-04-29T16:54:25+00:00

sstate filename generation was changed a while ago and taskname has to be passed into core functions for the correct filename to be generated. Update find_siginfo to match those changes and pass in taskname via SSTATE_CURRTASK. Thanks to Gregory Lumen for spotting. [YOCTO #14774] (From OE-Core rev: 51d7310b6cf8ef9033d461868c07f686656070ba) Signed-off-by: Richard Purdie Signed-off-by: Luca Ceresoli Signed-off-by: Richard Purdie

sstate: do not add TARGET_ARCH to pkgarch for cross recipes.

2022-03-02T18:43:24+00:00

This is redundant (target arch is already in PN), and breaks compiling a cross-canadian toolchain, as that needs populating the sysroot with two different native-hosted toolchains built from cross recipes. Inserting TARGET_ARCH allows only one or the other. (From OE-Core rev: 33fc1792cd782feb8dbb4285e3006bb588f7978f) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

classes/native: Propagate dependencies to outhash

2022-01-17T11:49:11+00:00

Native task outputs are directly run on the build system during the build after being built. Even if the output of a native recipe doesn't change, a change in one of its dependencies may cause a change in the output it generates (e.g. rpm output depends on the output of its dependent zstd library). This can cause poor interactions with hash equivalence, since this recipe's output-changing dependency is "hidden" and downstream tasks only see that this recipe has the same outhash and therefore is equivalent. This can result in different output in different cases and issues with reproducible builds in parcular (e.g. rpm compression changes for the same content). To resolve this, unhide the output-changing dependency by adding it's unihash to this tasks outhash calculation. Unfortunately, we don't know specifically know which dependencies are output-changing, so we have to add all of them. [YOCTO #14685] (From OE-Core rev: d6c7b9f4f0e61fa6546d3644e27abe3e96f597e2) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable

2021-10-16T16:41:59+00:00

We want things to be reproduicble and the variable doesn't really change much any more. Drop the remaining uses and make those code paths always active. (From OE-Core rev: d15fb02c7ee7da50e322d74bc6a545234e20c7f3) Signed-off-by: Richard Purdie