| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When debugging why a single recipe doesn't reproduce, its a pain
to wait for the world to rebuild from scratch. Update the selftest
to allow this to be configured, for example you could set
targets as ['perf'] and sstate_targets as ['virtual/kernel']
and then it should only be rebuilding perf in the test rather than
things like the toolchain (parts of the kernel may be unavoiable
as they're not in sstate).
Can be run as:
OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
to save diffoscope output.
(From OE-Core rev: 8b38cd1b36f3e227a63c0aa3955f3f5ab172c509)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 132a17d02f29711572e14a2f38a841323fbb6df6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Calling sync between each file compare is horrible performance wise
as we compare thousands of files. We don't care about IO latency here
so disable.
(From OE-Core rev: dd893279f352e290c23577a0f91df1b8c6232f10)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 25f78abd8bbeb201fd9452e7983e015027954948)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The reproducible build tests can take a long time, so having more
logging messages at various points in the build can help debug where the
build is taking a long time.
(From OE-Core rev: 60a9e095297b756175e4a414bbee297b74547a7a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6b792afe8759d62af8e713b86dad8f6721961a05)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The SSTATE_MIRRORS variable was misspelled, which allowed the "clean"
test build to pull from the mirror.
(From OE-Core rev: f81fa4cf7ff1e48f7499f2cb5e2c7d7f9ea438be)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e42497bd84d0bb370a9f7b0448bff29f01fd1b0c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Includes jquery in the output directory to make the diffoscope output
easier to navigate
(From OE-Core rev: ff7c386455848e497ef40f9fee2be87d5171e203)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
If there are differing packages and they are being saved for review,
automatically run diffoscope on them and include the output in the saved
output. The output is currently done in HTML format since these are
typically published on a webpage by the autobuilder.
(From OE-Core rev: 6e1d5e8b58f0940ba6dfd99536159dd974e6f24c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding newline characters between the packages in the failure output
massively improves readability.
Also ensure to output ipk failures when there are deb failures by
calling self.fail() at the end, else sometimes only partial differences
are returned.
(From OE-Core rev: 6e2e0480852177db75a6108d77c99c92c4e9950f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not strip packages when testing reproducible builds. In some cases,
stripped data differs between builds, but then gets removed. However,
the contents affect the generation of the GCC build-id, which then
differs in the resulting ELF files, even though the data that caused
this is no longer there. Inhibit stripping so that their causes can be
evaluated.
(From OE-Core rev: 0e23c6faf8169a333ae29064553705b5d7d6b589)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If OEQA_DEBUGGING_SAVED_OUTPUT is set in the environment, use this location
to store reproducibile build failure output. This aids debugging on the
YP autobuilder in particular.
Use a date in the directory name to make it easier to find failure output.
Also clean up empty directories as they're unnecessary distracting noise.
(From OE-Core rev: 138ed4aa96ec5069f9f7fd02994a42452dbccc2d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
Previous refactoring broke the case where save_dir was set. Fix this.
(From OE-Core rev: 9bfa04aa1f1a0b38ffa51f0211f3a46a90e17b0a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
A typo was preventing the extra data about the reproducible build from
being reported in the test results
(From OE-Core rev: 7c653337360dd5e49809b930f22a999c1d6e09ee)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a flag to control if the reproducible QA test should allow building
from sstate or not. Building from sstate may not be reproducible
depending on how the sstate is populated.
(From OE-Core rev: c24b71e9ec2e518bd24da20ddbb1406910b1b92e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Adds core-image-sato and core-image-full-cmdline to the list of images
that the reproducible build test builds.
(From OE-Core rev: d6c62cc0305ab082e990041da104fedb5798e0e4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds an option that can be used to copy the offending packages to a temp
directory for later evaluation. This is useful on the Autobuilder to
investigate failures.
(From OE-Core rev: 54b29bae78d1711074fbd18f0350ef0b83b555d1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now that opkg-build can build reproducible ipkgs, we can also add those to the
test case.
[ YOCTO #13513 ]
(From OE-Core rev: 3806d38e6194a42534381993fcfcc3ff39550a13)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to take advantage of multiprocess execution of tests the
extraresults must be passed through the TestResult. With changes to how
oeqa/core handles test cases the extraresults attribute of the testcase
is passed to the TestResult, with passing across process boundaries
handled automatically.
(From OE-Core rev: 6a1b0c2003a0b4a1983f9494440e6ea02dc25585)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Preserves the SSTATE_MIRROR variable for the first build in the
reproducible test. This patch is intended to test the theory that using
the Yocto autobuilder sstate mirror will still be reproducible. The
autobuilder always does clean builds and never rebuilds recipes that
were built previously, thus building with the mirror but not
sharing tmpdir and sstate with previous builds should be reproducible.
There is no guarantee that all sstate caches are populated from clean
builds so this patch cannot be submitted to master, and I'm not
bothering sending it to the mailing list.
(From OE-Core rev: 6b8e0077339a89cb01aa40c1b367a4e41a638892)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Perform two clean builds without sstate instead of one partial rebuild
with sstate and one clean build without. There are some classes of
reproducibility issues that this solves, and while we would like to
resolve them in the long term the direction to do so is not currently
clear.
(From OE-Core rev: e97c529bfa4e1d0038ea44f15ee3298003daf981)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
[YOCTO #13323]
(From OE-Core rev: 3fc08155b72f711c48f9a667539305eea869b5e9)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Records the results of the reproducibility analysis in the JSON test
results file. This makes it easier to do post-test analysis on the
packages.
[YOCTO #13324]
(From OE-Core rev: 6c357c8eee322163da96df676a54ff70a0bdc911)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Runs each package class reproducibility test in a separate sub-test.
This allows the other sub tests to still run in the event that one
fails.
(From OE-Core rev: f0095e6c1b915e1e6ded111f4bf77fff0362e29a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cleans up the output from the reproducible build before building to
ensure consistent results. Note that the output put is purposely left
after around after the build so that non-reproducible packages can be
diffed.
(From OE-Core rev: f67bb34c8e7950779920b696f8502dbaed9eded5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds an initial test for reproducible builds to the OE selftest. This
initial test builds core-image-minimal using sstate, then does a clean
build without sstate in another build directory, and finally does a
binary comparison of the resulting package files between the two builds.
The test is currently always skipped since it doesn't pass yet, but it
can easily be enabled locally
(From OE-Core rev: 2e591bdf93ec9e59b900562263dfe8e72b163baa)
Signed-off-by: Joshua Watt <JPEWHacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|