linux/poky.git/meta/classes/reproducible_build.bbclass, branch yocto-3.3.5

reproducible_build: Work around caching issues

2021-10-29T10:17:21+00:00

SOURCE_DATE_EPOCH can be expanded early in the parsing process before the class extensions are applied. This can mean the directory pointed to for the SDE can be incorrect until later in parsing. Cache the file name in the cached value and allow it to dynamically update. This isn't ideal but avoding expansion of the variable likely isn't possible and I'm not sure how else to handle this. This works around the issue until a better solution can be found. (From OE-Core rev: 7e3752dcce2236a4d1ad046b1df2edf8ee5619b4) Signed-off-by: Richard Purdie (cherry picked from commit 44dc97cd1223e4d2b635669627ec5f796838d42d) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

classes/reproducible_build: Use atomic rename for SDE file

2021-06-20T08:43:39+00:00

If an existing source date epoch file was found during do_unpack, it was deleted and a new one would be written in its place. This causes a race with check-before-use code in get_source_date_epoch_value. Resolve the problem by making do_unpack write the new source date epoch to a temporary file, then do an atomic rename to ensure it's always present, and change the check-before-use code to use a EAFP exception instead of checking for file existence. [YOCTO #14384] (From OE-Core rev: 8b2fd4e5e0841b81b4f709b061b655e2266dd4da) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie (cherry picked from commit 0b5e3b33187bf78a2d62cc886463e4b27d6bd228) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

reproducible: Improve SOURCE_DATE_EPOCH_FALLBACK handling

2021-02-24T00:19:38+00:00

Ensure the fallback value if used is written to the SDE file and hence stored in sstate, reducing any confusion within the code over '0' values. Bump the HASHEQUIV_VERSION since we've had a ton of trouble with ensuring this rolls out correctly on the autobuilder so others may too, take a clean slate for it. (From OE-Core rev: 53ffffa32b76330835287dfc05d4e4796841af08) Signed-off-by: Richard Purdie

reproducible_builds: SOURCE_DATE_EPOCH should not be 0

2021-02-24T00:19:38+00:00

A SOURCE_DATE_EPOCH of 0 might be misinterpreted by namely rpm as no SDE. e.g.: char *srcdate = getenv(SOURCE_DATE_EPOCH); if (srcdate && rpmExpandNumeric(%{?clamp_mtime_to_source_date_epoch})) Solve this by: - providing a SOURCE_DATE_EPOCH_FALLBACK variable != 0 - changing defaults to use it - using SOURCE_DATE_EPOCH_FALLBACK also when reading 0 out of sstate-cache (From OE-Core rev: 9a1dde74e794362399193dc3f81c9685a83d0776) Signed-off-by: Jan-Simon Möller Signed-off-by: Richard Purdie

classes/reproducible: Move to library code

2020-07-22T11:45:56+00:00

Moves most of the python code used for dealing with the source date epoch to library code. (From OE-Core rev: a7ede90955bc0c8bec1cbb3cab498ef2583b2f4e) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

reproducibile_build: Fix SDE file generation when unpack reruns

2020-03-08T08:21:11+00:00

Currently, if an existing TMPDIR is rebuilt, do_fetch/do_unpack can rerun but SDE would remain unchanged. This leads to different results compared to a fresh build. An example change which triggered this is: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cb4e69e6346a9fbeebf83a5d5397cacbd41d48b5 Instead, delete any existing SDE and recalculate if we're reunning. Also rename and drop the do_ prefix since these are for tasks, not functions. (From OE-Core rev: 5d579fc2fe71637fc6e071aa66542befa39ac8bb) Signed-off-by: Richard Purdie

classes/reproducible_build: Read SDE file later

2020-02-03T00:10:10+00:00

Defers the resolution of the SOURCE_DATE_EPOCH until the variable needs to be actually realized with a value. The previous method of loading the value in anonymous python had issues because it could occur before other anonymous python functions that affect the location of the epoch file, such as when a recipe uses AUTOINC/AUTOREV or allarch.bbclass. Also adds more logging to help diagnose issues in the future. [YOCTO #13763] (From OE-Core rev: b3313a10a3eb93f0a3710a35de0404fb49cd6202) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

classes/reproducible_build: Create SDE destination

2019-10-02T19:22:25+00:00

Creates the source data epoch file destination directory when restoring from sstate (From OE-Core rev: f5259a62d444dfc221a9ba592dbca35564f794ac) Signed-off-by: Richard Purdie

classes/reproducible_build: Move SDE deploy to another directory

2019-09-27T12:02:16+00:00

The deployment of the source date epoch file had a race condition where any task attempting to read from the file would race with creation of the sstate archive for the do_deploy_source_date_epoch task. The creation of the sstate archive requires moving the directory to a temporary location, then moving it back. This means that the file disappears for a short period of time, which will cause a failure if any other task is running and trying to open the file to get the current source date epoch. The solution is to copy the source date epoch file to a separate directory when deploying so the file never disappears. When the file is restored from sstate, it is moved to the correct location after being extracted. [YOCTO #13501] (From OE-Core rev: ac27d12fe5480e9b8cc93de6a32bf9631c52d7f4) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

reproducible: Don't look for youngest file when no source tarball

2018-11-23T23:35:19+00:00

Some packages (eg. init-ifupdown) take their source files entirely from openembedded-core, that is, they download no source tarball. These recipes either don't use S at all (ie. it is empty at unpack time), or they set S = WORKDIR (as in init-ifupdown). Looking at the file timestamps in the WORKDIR causes a non-reproducible SOURCE_DATE_EPOCH, as files taken from file:// URIs do not have reproducible timestamps. If S == WORKDIR, we are better to assume that there is no source tarball, and to fall back to a fixed timestamp for the SOURCE_DATE_EPOCH. This makes the init-ifupdown build reproducible. (From OE-Core rev: d395bad0179037eb5d0fa4d921985c87ae13f3a4) Signed-off-by: Douglas Royds Signed-off-by: Richard Purdie