summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest/cases/reproducible.py
Commit message (Collapse)AuthorAgeFilesLines
* rust: rustdoc reproducibility issue fix - disable PGOSundeep KOKKONDA2023-12-211-2/+0
| | | | | | | | | | | | | | | | | | | The PGO (Profile-guided Optimization) collect data about the typical execution of a program and then use this data to inform optimizations such as inlining, machine-code layout, register allocation, etc. This optimization is by default disabled in rust sources but enabled in Yocto and causing the reproducibility issue in rustdoc binary. To fix the issue this optimization is set to it's default 'false'. More about the optimization: https://doc.rust-lang.org/rustc/profile-guided-optimization.html With the reproducibility issue fixed, we can enable the reproducibility tests again. (From OE-Core rev: 189c266378c8c4a918cb205b3888577c7ce76856) Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: Split rustdoc into a separate packageSundeep KOKKONDA2023-11-271-1/+1
| | | | | | | | | | | The 'rustdoc' is moved to a separate 'rust-rustdoc' package. This is a workaround to test if the main rust binary is reproducible even if rustdoc isn't. (From OE-Core rev: dfed7e63e199009ecca8e2e34f6a823c6e1677e0) Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Allow packages exclusion via configRichard Purdie2023-11-151-2/+4
| | | | | | | | | | | | | | | | | | | | | | OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known not-reproducible packages to make the reproducible test fail. For example, in local.conf: OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible" To quote the original exclusion commit (4864764667097b32a3ee1935e2c52df732d55632) > selftest/reproducible: add an exclusion list for items that are not yet reproducible > [...] > Non-reproducible excluded packages are not given to diffoscope and do not cause a > failure, but still saved side-by-side with non-reproducible failing ones to make > investigation easier. NB: Patch was written by Richard with small fixes and a commit message from Yoann. (From OE-Core rev: bea0e3f5b8b383ae8590b874a5878f67832d6929) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Split a long lineYoann Congal2023-11-151-1/+9
| | | | | | | | | | | No other change. Just cosmetic to avoid tripping the maximum line length of patchtest when the next element will be added to the list. (From OE-Core rev: 7c401fe421c2085f2e7fc14589ca44ec1791db03) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible: Exclude rust for now againRichard Purdie2023-10-131-0/+2
| | | | | | | | | | | | Unfortunately there is still an issue with rustdoc not being reproducible so part of the problem has been fixed but not everything. Add the exclusion back until this has been addressed to avoid autobuilder failures and long diffoscope analysis time. (From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: reproducibility issue fixSundeep KOKKONDA2023-10-111-2/+0
| | | | | | | | | | | | | | | | | | | | The '--remap-path-prefix' option removes all references to build directory structure in the debug information within the compiled output for Cargo dependencies and the project's binary. However, some references to build directories remains in the final binary in .rustc section in the form of compressed metadata and this makes the build output dependent on the folder structure of the computer it's compiled on. So, for reproducible builds, use the configuration option 'remap-debuginfo = true' along with the '--remap-path-prefix'. [YOCTO# 14875] (From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a) Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducible: Avoid oe-selftest startup delaysRichard Purdie2023-09-221-12/+15
| | | | | | | | | | Currently the bb_get_var calls trigger multiple "bitbake -e" executions which slow the start of oe-selftest for any test. Rework the code to avoid these delays. (From OE-Core rev: c9ea3570f5dbdaf8c61e445ae9de759a0b71e21f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Update config to match ongoing changesRichard Purdie2023-08-111-2/+1
| | | | | | | | | | | | | | | We can't have systemd here any longer without usrmerge. We don't really want to enable the latter since having separate usr will likely result in a class of reproducibility and host contamination issues that enabling it might hide. Also drop INHIBIT_PACKAGE_STRIP since we generally don't build with that and the debug binaries should be generated regardless. I suspect this is legacy from older issues. (From OE-Core rev: 7b7411788e805fa067dd672c9771dcaf2af918a0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Allow chose the package managerJose Quaresma2023-06-171-1/+5
| | | | | | | | | | This is a follow-up of 76e5fcb2 that also allow users to chose the package manager using OEQA_REPRODUCIBLE_TEST_PACKAGE (From OE-Core rev: 3d414d85b44077bac57aba36707b0fc699a73e97) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest reproducible.py: support different build targetsMikko Rapeli2023-06-151-2/+6
| | | | | | | | | | | | | | | | | Allow users to set different build reproducibility targets than the defaults using OEQA_REPRODUCIBLE_TEST_TARGET and OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS variables in local.conf. Fixing all issues from "world" builds is not possible in some complex build environments with lots of layers. Limiting the focus to a smaller subset allows using this test to detect and fix build reproduction issues incrementally. (From OE-Core rev: c66bebbce5995e386a1a4d055a914a39b6ee518d) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Allow native/cross reuse in testRichard Purdie2023-06-071-2/+2
| | | | | | | | | | | | We don't compare reproducibility of the native/cross components, only the target ones. With the long build times of rust-native, the test now takes crazy lengths of time so this tweak should allow us to reuse native/cross artefacts from sstate whilst still testing the target output is reproducible. (From OE-Core rev: b494d83c639a877cefeb7cbab6d37195e492f059) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducible: Split different packages from missing packages ↵Jose Quaresma2023-03-281-3/+7
| | | | | | | | | | | | | output When we provide sstate_targets recipes they are reported as missing which is not quite correct because they are proveided to cut build/debugging time. (From OE-Core rev: fd70a564d6946fa460638dd04ce2daecf4566cf3) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Exclude rust/rust-dbg for now until we can fixRichard Purdie2022-08-081-0/+2
| | | | | | | | | | | | There looks to be a reproducibility issue left in one of the rust libraries. It doesn't appear to be a string issue but some binary problem. Disable rust from the reproducibility testing until we can get to the bottom of the issue (allowing wider testing of all the other improvements). (From OE-Core rev: a261333f6591ea94afc567dee04a2e3c6d5059cf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: remove unused importsRoss Burton2022-04-011-2/+0
| | | | | | | (From OE-Core rev: 7ef7b03eeefc0a9911fd62c73e346fa5aeeb09eb) Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Rename LICENSE_FLAGS variableSaul Wold2022-02-211-1/+1
| | | | | | | (From OE-Core rev: 5c5b3bc563059ba728dc9724656cc69669f8e25f) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust-llvm: apply the same reproducibility patch as for llvm properAlexander Kanavin2022-01-111-4/+0
| | | | | | | (From OE-Core rev: a845029df7ea8c1bd987ffac3902d4521742debb) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ruby: disable rdoc due to non-reproducibilityAlexander Kanavin2022-01-111-3/+0
| | | | | | | | | | | | | The issue is reported upstream: https://bugs.ruby-lang.org/issues/18456 Otherwise I do not feel that further investigation of rdoc's parser to find out the source of non-determinism is worth the time. (From OE-Core rev: 208021f7212a8a790c350ccca720695c5bcbb1ca) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: reproducible: Set maximum report sizeJoshua Watt2022-01-071-3/+7
| | | | | | | | | | | Diffoscope can end up running for a very long time if there are a lot of changes. To put a limit on how long it can run, cap the maximum report size at 250 MB by default. (From OE-Core rev: 52d5f76f54eac384f9480dffe96df089d9ee8f33) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: drop go items from exception listAlexander Kanavin2022-01-051-13/+0
| | | | | | | | | With this changeset, go becomes reproducible \0/ (From OE-Core rev: 03ada343ecbbbe3822a8fd74e678b01be6b9511a) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible: Merge code into base.bbclassRichard Purdie2021-10-161-1/+0
| | | | | | | | | | Reproducibility is here to stay and needs to be part of our default workflow. Move the remaining code to base.bbclass so it is always a first class citizen and it is clear people need to be mindful of it. (From OE-Core rev: abb0671d2cebfd7e8df94796404bbe9c7f961058) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/reproducibile: Exclude rust packagesRichard Purdie2021-08-261-1/+5
| | | | | | | | | rust-llvm-liblto and rust-llvm-staticdev sometimes vary in contents. Exclude them from the test for now until we can work on and resolve the issues. (From OE-Core rev: 60c12da5aae2534c972df851f26e4523fed03afc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Convert to new override syntaxRichard Purdie2021-08-021-1/+1
| | | | | | | | | | | | This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". (From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Drop ovmf exclusionRichard Purdie2021-03-201-1/+0
| | | | | | | | The hardcoded path issue was fixed, drop the exclusion. (From OE-Core rev: 58e02c83dcf4d6c40c206a0371ef5d7f8db2b272) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meson: Work around determinism issueRichard Purdie2021-03-161-1/+0
| | | | | | | | | | The meson determinism issue is the same frozenset issue we encountered with python itself. Remove the problematic pyc file until upstream work out the best way to address the issue (as with core python). (From OE-Core rev: ad00107dc02b3211f5d1a7fe889f538ee2bc064d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* go-dep: RemoveKhem Raj2021-03-151-1/+0
| | | | | | | | | | | | | go-dep was an effort for dependency management before go modules, which since 2020 has been deprecated in favor of go modules. Since its not developed any longer and go mdules is officially supported, this should be retired from OE-core as well. (From OE-Core rev: 1e7ed44d87034446f1d07692c9378c3b0a8a9dd3) Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Otavio Salvador <otavio.salvador@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducibile: remove perf from exclusionsBruce Ashfield2021-03-141-1/+0
| | | | | | | | | | We have fixes for perf reproducibility, so we can drop it from the exclusion list. (From OE-Core rev: 2989779fa244ec2c2c714c27f67753eefeeaf4af) Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* python3-cython: Remove build paths from debug sourcesRichard Purdie2021-03-021-1/+0
| | | | | | | | | | | | There are build paths which are in generated sources injected into ${PN}-src. These are in generated files and don't influence the binary output. As such we can correct them at do_package time by tweaking in an appropriately injected function. This fixes the reproducbility of the recipe. (From OE-Core rev: cc1de818247c9ae329fdc0536658b3968a4caea2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: Fixing determinism issuesRichard Purdie2021-03-021-1/+0
| | | | | | | | | | | | | | Add a patch adding sorting to a couple of points in the Makefiles which removes most of the determinism issues in ltp. Build swapon before the main build to ensure libswapon.o is built deterministically as it races with swapoff. All issues reported on the upstream mailing list. (From OE-Core rev: 0f51f9a37e5d058bce28cfe7b9a32a895f83c091) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add core-image-weston to reproducible build testsJan-Simon Moeller2021-03-021-1/+1
| | | | | | | | | | | | | This change adds core-image-weston to the reproducible build test. For this we also need to update the static-group and static-passwd. Besides the world package build which includes the wayland package, this adds a wayland/weston-based image to the reproducible build test. (From OE-Core rev: ffc4de47988ccf7568eecc8a27e8964beeaaacfb) Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lttng: Fix reproducibility issuesRichard Purdie2021-03-021-2/+0
| | | | | | | | | | | Add a hack to hardcode in specific rpaths which we then remove, allowing the build to be reproducible. Strip build patches out of one of the test scripts too. (From OE-Core rev: 18299a114c66280ba2d00becc3fae235d3a6cbd7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu: Determinism fixesRichard Purdie2021-03-021-1/+0
| | | | | | | | | | | | Add a patch to address two determinism issues and allow reproducible builds. Also strip full paths we don't need out of some ptest files, we can use the installed binaries. (From OE-Core rev: 14e7cc6c6040b983024bec56fbd32d9101507ff0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Add ability to pull some objects from sstateRichard Purdie2021-03-021-2/+14
| | | | | | | | | | | | | | | | | | | | When debugging why a single recipe doesn't reproduce, its a pain to wait for the world to rebuild from scratch. Update the selftest to allow this to be configured, for example you could set targets as ['perf'] and sstate_targets as ['virtual/kernel'] and then it should only be rebuilding perf in the test rather than things like the toolchain (parts of the kernel may be unavoiable as they're not in sstate). Can be run as: OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds to save diffoscope output. (From OE-Core rev: 132a17d02f29711572e14a2f38a841323fbb6df6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* systemd-bootchart: Disable LTO to fix reproducibilityRichard Purdie2021-03-011-2/+1
| | | | | | | | | LTO likely doesn't buy us much here, disable it to allow the binaries to be reproducible. (From OE-Core rev: b4b5e52b5bd98deb0855700e4aae6228d52e5a83) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* efivar: Fix reproducibility issueRichard Purdie2021-03-011-1/+0
| | | | | | | | | Add sorting to the globbing within the Makefile to make the output reproducible. (From OE-Core rev: c43ae151f572786818fe048233b4bbfd6b0ba2cf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* swig: Fix reproducibility issueRichard Purdie2021-03-011-1/+0
| | | | | | | | | Remove hardcoding the build configuration into the help/version output from swig to make the binaries reproducible. (From OE-Core rev: 7ed28ae9717ea9dad4e131012186d5f08e8f0bec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* syslinux: Fix reproducibility issuesRichard Purdie2021-03-011-1/+0
| | | | | | | | | Add sorting to wildcard expansion in the makefile to make builds reproducible. (From OE-Core rev: 5541ba76ccc0c416f315bc0dc14a20a33059bd5f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gstreamer1.0-python: Set internal python library path correctyRichard Purdie2021-03-011-1/+0
| | | | | | | | | | The library hardcodes paths to the python library internally and currently these are build paths. Fix this to use the correct target path and fix reproducibility in the process. (From OE-Core rev: 4c190e8c9a2b53de3def1065fb6e479b72d040dc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* apr-util: Fix CFLAGS used in buildRichard Purdie2021-03-011-1/+0
| | | | | | | | | We need to use CFLAGS with the correct WORKDIR in them, replace those in the sysroot file with the ones appropriate to the current recipe. (From OE-Core rev: 45edf189961aff1858be9bb7b63116073c0a0c10) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* libid3tag: Fix reproducibility issueRichard Purdie2021-02-271-1/+0
| | | | | | | | | Configure was swallowing our cflags meaning the resulting binaries were not reproducible. Tweak configure not to do that and fix reproducibility. (From OE-Core rev: 1395269a6fae5cec01d721e2ad52bd7cb8fc38ce) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* igt-gpu-tools: Fix reproducibility issueRichard Purdie2021-02-271-1/+0
| | | | | | | | | Add a configuration option to pass in srcdir, removing hard coded build paths from the binaries. (From OE-Core rev: d7e92e3a22f0c87aff4f452b51f50ec417d2949b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bootchart2: Fix manpage reproducibility issueRichard Purdie2021-02-271-1/+0
| | | | | | | | | | | The compressed docs contained a timestamp of the original file which meant the SDE clamping during package creation didn't work. The benefits of compression are minor, decompress the files to avoid the reproducibility issues. (From OE-Core rev: a3b59a67bea16899b57a0d187120c988495da4d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Don't call sync between each file compareRichard Purdie2021-02-261-1/+1
| | | | | | | | | | Calling sync between each file compare is horrible performance wise as we compare thousands of files. We don't care about IO latency here so disable. (From OE-Core rev: 25f78abd8bbeb201fd9452e7983e015027954948) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* libhandy: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | (From OE-Core rev: ba559d5e07cbe92ee22308a1b5b8afc558c662dd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* libsecret: Improve determimismRichard Purdie2021-02-261-2/+0
| | | | | | (From OE-Core rev: 28175c44e7402397a24a00a4b056eef39b0a03fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parted: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | | | | We don't use tbe BUILDINFO line of host information in the Makefile so remove it for reproducibility. (From OE-Core rev: a9742595fa90d4977fdd8129a4fe4932ddb96a18) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gst-devtools: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | (From OE-Core rev: d221abc411b49b219b89cb38a56b2901d0bd5529) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* epiphany: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | | | | | We don't want to encide BUILD_ROOT into target packages. This is used for build time tests but in our case those would be on target anyway do use the target paths. (From OE-Core rev: 2818dc90bd62487879e7da2c68357b39282415b0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gtk-doc: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | | | | config_data was including a build system path to pkg-config, fix this. (From OE-Core rev: 7b8f56224c12a40f4f0aca3f612198deb933af4d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gcr: Fix reproducibility issueRichard Purdie2021-02-261-1/+0
| | | | | | (From OE-Core rev: b53133572de9e5bdf77dd44831a86ec34bf41b2c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cups: Fix reproducibility issuesRichard Purdie2021-02-261-1/+0
| | | | | | | | | configure inspects the host's /etc/group for these configuration options, fix this to the correct values by using configure options. (From OE-Core rev: f16f9c727569414cd52862dcba18d8e423f4e961) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-02 14:01:33 +0000