linux/poky.git/bitbake/lib/bb/siggen.py, branch 3.5_M2

bitbake: siggen: Ensure dumpsig output is deterministic

2021-12-09T10:56:49+00:00

Currently the console output for signature dumps e.g. with bitbake-dumpsig isn't deterministic. Add some sorting to improve that. (Bitbake rev: a663440b1623f97c9c169df5566e429fbc932a53) Signed-off-by: Richard Purdie

bitbake: fetch2/checksum/siggen: Fix taskhashes not tracking file directories

2021-11-08T22:01:19+00:00

Currently if you have something like: SRC_URI = "file://foobar;subdir=${S}" and a file like: foobar/1/somefile and then move it to: foobar/2/somefile the task checksums don't reflect/notice this. The file-checksum fields encode two pieces of data, the file path and whether or not the file exists. Changing the code which uses these fields is problematic. We can however add a "/./" path element which means "include the bit after the marker in the checksum" which the path walking code can use to mark which bits of the path are visible to the fetcher. I'm not convinced this is great design but it does appear to work. (Bitbake rev: b4975d2ecf615ac4c240808fbc5a3f879a93846b) Signed-off-by: Richard Purdie

bitbake: siggen: Fix type conversion issues

2021-10-17T10:57:27+00:00

The switch to using json has messed up the type handling as the code does assume that set()s are present. Add a decoder to reconstruct the set() objects. Also fix the change of tuples to lists for the file checksums and fix an existing type bug where dicts insteads of lists was used. Drop some old siginfo format handling code which is now long since obsolete. (Bitbake rev: 2d704842c0928f8dbe78fd081042aa7280af96be) Signed-off-by: Richard Purdie

bitbake: siggen: Change file format of siginfo files to use zstd compressed json

2021-10-14T21:37:43+00:00

Since OE is about to change to zstd compression of sstate, it would make it timely to convert the siginfo files from pickle which isn't reproducible to json which is both reproducible and also human readable. At the same time add zstd compression. This makes the siginfo files smaller, reprodubicle and easier to debug. Backwards compatibility mixing the two formats hasn't been supported since in reality if sstate changes at the same time, files will be in one format or the new one but comparing mixed formats won't make much sense. Since json doesn't support sets, we translate them into lists in the files themselves. We only use sets in bitbake since it makes things easier in the internal code, sorted lists are fine for the file format. [YOCTO #13973] (Bitbake rev: 22c18494c9072788e6e26eb73de70378ae5c5bf5) Signed-off-by: Richard Purdie

bitbake: siggen: Fix sorting in diff output

2021-09-26T13:37:34+00:00

The diff output isn't deterministic at the moment as the sets can have differing ordering. Sort the output so it is consistent. (Bitbake rev: 117830c1d7ef3e53052fa326e1ca62c5c3946c45) Signed-off-by: Richard Purdie

bitbake: bitbake: Switch to using new override syntax

2021-08-02T14:44:10+00:00

This change updates the datastore to use the new override syntax using colons instead of underscores exclusively. It is expected layers would have to be converted to work with bitbake after this change. Supporting mixed syntax isn't possible, it is only feasible to have one internal representation of overrides. Whilst we can't warn for every possible override that may be set in the old format, show errors for _append/_prepend/_remove since those should never be present. (Bitbake rev: 7dcf317cc141dc980634f8c18bfa84f83e57206a) Signed-off-by: Richard Purdie

bitbake: lib/bb: Add bb.utils.rename() helper function and use for renaming

2021-05-06T10:04:59+00:00

os.rename can fail for example an incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than trying to fix every call site, add a wrapper in bb.utils for renames. We can then handle cross device failures and fall back to shutil.move. The reason os.rename is still used is because shutil.move is too slow for speed sensitive sections of code. [YOCTO #14301] (Bitbake rev: c5c4e49574ab2a65e06298a0a77bb98b041cf56b) Signed-off-by: Devendra Tewari Signed-off-by: Richard Purdie

bitbake: hashserv: Use generic ConnectionError

2021-04-27T14:12:57+00:00

The Python built-in ConnectionError type can be used instead of a custom HashConnectionError type. This will make code refactoring simpler. (Bitbake rev: 8a796c3d6d99cfa8ef7aff0ae55bb0f23bbbeae1) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

bitbake: logging: Make bitbake logger compatible with python logger

2021-02-10T23:48:16+00:00

The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: lib/bb: Don't treat mc recipe (Midnight Commander) as a multiconfig target

2021-02-06T09:12:00+00:00

When we run `devtool build mc` recipe's task dependencies are expanded to "mc:do_populate_sysroot" where "mc" name is treated as multiconfig and "do_package_sysroot" as multiconfigname. | ERROR: Multiconfig dependency mc:do_populate_sysroot depends on | nonexistent multiconfig configuration named do_populate_sysroot (Bitbake rev: 3ce4b2caccfe608a54dff159459f3687ea610597) Signed-off-by: Tomasz Dziendzielski Signed-off-by: Richard Purdie