linux/poky.git/bitbake, branch 4.2_M2

bitbake: bb/utils: include SSL certificate paths in export_proxies

2023-01-26T21:50:31+00:00

bb.utils.export_proxies() is a poor-man's alternative for the environment setup code in bb/fetch2, but it's used in several places where recipes want to download manually (such as cve-update-db-native). Notably, export_proxies() doesn't pass on the SSL certificate paths from the original environment, so if SSL_CERT_FILE needs to be set (for example, in a buildtools environment) then proxies work but SSL doesn't. In an ideal world export_proxies and the same logic in fetch2 would merge, but until then we can add the SSL_CERT_ variables and duplicate the basic logic: check the datastore first and then the original environment for variables. Also remove the return value as nothing ever checked it. [ YOCTO #15000 ] (Bitbake rev: c19035e8e71c419c5688a86bfc9c946c96f638e8) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: cache/codeparser: Switch to a new BB_CACHEDIR variable for cache location

2023-01-26T21:50:31+00:00

Currently the codeparser cache is set from CACHE, which is typically in bitbake.conf which means we can't read/write any cache until it is found/read. We may well have python expressions to parse before that happens. The net result is suboptimal functioning of the codeparser cache since it will often be invalidated by data that is never written. This patch changes the codeparser and filechecksum caches to use BB_CACHE as their setting and defaults it to ${TOPDIR}/cache. The patch doesn't change where the "persistent" data such as prserver and hash-equiavalance resides (PERSISTENT_DIR) or where the metadata parsing cache resists (still currently CACHE). I've left those for a later patch. The patch does ensure data parsed by the core datastore parsing calls is written back since this is now much more useful after this change. (Bitbake rev: ee89ade5b5a4cf9c53f336d8b800e06fbe436628) Signed-off-by: Richard Purdie

bitbake: toaster: update fixtures

2023-01-26T21:50:31+00:00

* Update to latest langdale 4.1.2 and kirkstone 4.0.6 * Re-instate dunfell and update to 3.1.22 - drop comments about bitbake crash (Bitbake rev: f90c45df5cf4640e0714fde13e311db3327f9fd4) Signed-off-by: Tim Orling Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: bitbake-getvar: Add a quiet command line argument

2023-01-26T21:50:31+00:00

bitbake-getvar does not have a way to silence bitbake server's logger and that makes the tool hard to use for text processing. This is especially true when one wants to get a bitbake value to be piped to some other utility and instead we get uncontrolled logging messages or warnings together with bitbake's variable value. Example without quiet: bitbake-getvar --value MACHINE NOTE: Starting bitbake server... qemux86-64 With quiet: bitbake-getvar --value MACHINE --quiet qemux86-64 (Bitbake rev: af354e975d0b4c26d0e91e3c82946b093bc11b45) Signed-off-by: Paulo Neves Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: bitbake: fix deprecated threading.Thread.setDaemon

2023-01-24T21:59:44+00:00

Deprecated in Python 3.10: https://docs.python.org/3/whatsnew/3.10.html#deprecated https://github.com/python/cpython/pull/25174 Fixes warnings like: ...bitbake/lib/bb/ui/uievent.py:68: DeprecationWarning: setDaemon() is deprecated, set the daemon attribute instead self.t.setDaemon(True) (Bitbake rev: 323f6ce27a1bfd7159e72f29684674ff495dedee) Signed-off-by: Tim Orling Signed-off-by: Richard Purdie

bitbake: data_smart: Small optimisation to _findVar()

2023-01-24T21:59:44+00:00

Some users of _findVar don't need the override data and even getVarFlag doesn't need it in some common cases (parsing=True). Rearrange the code as the current overridedata approach doesn't need to be in the _findVar code anyway. This removes some search overhead from a critical path. (Bitbake rev: fcb64e1138a20eb19560af3fc5d1fa748cc9cf34) Signed-off-by: Richard Purdie

bitbake: data: Add support for new BB_HASH_CODEPARSER_VALS for cache optimisation

2023-01-24T21:59:44+00:00

Currently the codeparser cache ends up being extended for every parse run since there are values in the functions such as the result of os.getpid() from LOGFIFO in OE-Core. Digging into that issue, there are also lots of similar but different functions being parsed where the change might just be a path to WORKDIR, a change in PN or PV or something like DATE/TIME. There is no reason we have to use these changing values when computing the dependenies of the functions. Even with a small tweak like: BB_HASH_CODEPARSER_VALS = "LOGFIFO=/ T=/ WORKDIR=/ DATE=1234 TIME=1234 PV=0.0-1 PN=nopn" the cache is reduced from ~4.6MB, increasing by ~300kb for every parse run to around 1.3MB and remaining static for oe-core and meta-oe. In my local build, admittedly heavily experimented with, the cache had grown to 120MB. The benefits of doing this are: * faster load time for bitbake since the cache is smaller to read from disk and load into memory * being able to skip saving the cache upon shutdown * lower memory footprint for bitbake * faster codeparser data lookups (since there is less data to search) We only use these special values when passing code fragments to the codeparser to parse so the real variable values should otherwise be used in the hash data. The overall effect of this change, combined with others to avoid saving unchanged cache files can be ~2s on a ~16s parse on my local system and results in a more responsive feeling bitbake. It also allows parsing performance to be investigated more consistently. (Bitbake rev: f24bbaaddb36f479a59a958e7fc90ef454c19473) Signed-off-by: Richard Purdie

bitbake: cooker: Fix parsing race around cache handling

2023-01-24T21:59:44+00:00

When draining the result queue from the parsing processes, cache objects can be created even if they are then immediately destroyed. The reset in the sync code needs to happen after any objects have been created. Change the ordering to fix this. This ordering has caused various cache errors, particularly when interrupting parsing with Ctrl+C. (Bitbake rev: f45a94e6720dacf7f51ac147c115a6f608769093) Signed-off-by: Richard Purdie

bitbake: cache: Only write files if we have data

2023-01-24T21:59:44+00:00

By writing the cache files only if there is data to write, we can save a bit of time. (Bitbake rev: abeff1f80bb1c690b92d535d472dff9df7a56067) Signed-off-by: Richard Purdie

bitbake: cooker: Fix siggen recipe cache race issue

2023-01-24T21:59:44+00:00

We need to reset the cache before the start() call which starts the parsing processs, tweak the code to ensure this is the case. (Bitbake rev: 4712d6237df5d2188238294fbdb9f107b068a3a7) Signed-off-by: Richard Purdie