linux/poky.git/bitbake/lib/bb/cooker.py, branch yocto-5.2.4

bitbake: Use a "fork" multiprocessing context

2025-09-17T22:32:27+00:00

Python 3.14 changes the default multiprocessing context from "fork" to "forkserver"; however bitbake heavily relies on "fork" to efficiently pass data to the child processes. As such, make "fork" context in the bb namespace and use it in place of the normal multiprocessing module. Note that multiprocessing contexts were added in Python 3.4, so this should be safe to use even before Python 3.14 [YOCTO #15858] (Bitbake rev: 0d9d8d0863e82e986c33c08064ce9a99224a06d5) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: cooker: Add debug for parsing being complete

2025-03-11T10:53:13+00:00

We have a "parsing started" event in the cooker deamon log but we don't currently log the corresponding "parsing complete" event which is confusing. Add this so that the logs are more logical. (Bitbake rev: 1aa491c1f1211bf9faab712c321b66629fb7be66) Signed-off-by: Richard Purdie

bitbake: lib/bb/cooker.py: produce an error when BB_HASHSERVE_UPSTREAM cannot be used due to missing python module

2025-03-06T11:16:46+00:00

Previously if the websockets module was not installed bitbake would only print a warning and continue, resulting in a degraded user experience due to inability to use the configured sstate server. Let's consider that as fatal misconfiguration, so that users can address the issue properly and not wonder why builds are taking forever. (Bitbake rev: cfba2a9fca9dfa3b05ec9040fe0cb8143ac04af7) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: cooker: Make cooker 'skiplist' per-multiconfig/mc

2024-12-17T11:48:25+00:00

Previously, the cooker skiplist was shared across multiconfigs (including default ''). If you had a recipe that was incompatible with several multiconfigs for different reasons, then the displayed reason (i.e. the "ERROR: Nothing PROVIDES" and "* was skipped" messages) might vary across invocations of bitbake. This was caused by the random order in which recipes are parsed under different multiconfig contexts, with each skip reason overwriting the previously assigned reason. I hit this specificially when using COMPATIBLE_MACHINE, but COMPATIBLE_HOST (or anything using bb.parse.SkipRecipe) would have done it too. (Bitbake rev: c51f01a35ed9a928402eab0899598b5c59602eef) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie

bitbake: cooker: use enum for cooker state to improve readability

2024-12-06T23:33:54+00:00

enum was introduced in Python 3.4 (Bitbake rev: 35b71a94f8757fcca830f972a42edab1dd000c16) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie

bitbake: Remove custom exception backtrace formatting

2024-11-28T00:06:24+00:00

Removes the code in bitbake to show custom backtrace formatting for exceptions. In particular, the bitbake exception code prints function arguments, which while helpful is a security problem when passwords and other secrets can be passed as function arguments. As it turns out, the handling of the custom serialized exception stack frames was pretty much made obsolete by d7db75020ed ("event/msg: Pass formatted exceptions"), which changed the events to pass a preformatted stacktrack list of strings, but the passing of the serialized data was never removed. Change all the code to use the python traceback API to format exceptions instead of the custom code; conveniently traceback.format_exception() also returns a list of stack trace strings, so it can be used as a drop in replacement for bb.exception.format_exception() (Bitbake rev: 2cda75a185aaf8f657f072dac34f8cef9d75f63a) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: cooker: Drop support for BB_DANGLINGAPPENDS_WARNONLY

2024-11-21T12:16:28+00:00

DANGLINGAPPENDS_WARNONLY is a very poorly designed interface and is commonly abused. The challenge is that once it is set, by any layer, it applies everywhere. Some layers rely on this to get notification they need to update bbappend files and having the behaviour change from inclusion of an antisocial layer is not good. In addition, showing warnings as an accepted thing on the console devalues them and trains the user to ignore them. I want to steer us away from this mindset. We could extend the functionality and make it apply only to certain layers, or only to certain appends but then we've basically re-invented BBMASK. Given all the above, we should drop support for BB_DANGLINGAPPENDS_WARNONLY and direct anyone with issues to BBMASK instead. https://lists.openembedded.org/g/openembedded-architecture/message/2029 [YOCTO #14870] (Bitbake rev: fca9c9e3cb6f8e9f99bf51dc5e8a8d83f4c84c69) Signed-off-by: Richard Purdie

bitbake: cooker: Sort pn-buildlist

2024-11-05T11:17:17+00:00

So that we can compare the different pn-buildlist easily. (Bitbake rev: 529043117a7c62feb45bc891658a412cc8dd7e3f) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: cookerdata: Separate out data_hash and hook to tinfoil

2024-08-13T14:47:55+00:00

Within tinfoil, the user can write to the configuration data but it won't cause the data_hash checksum to be re-written, meaning cached parsing data can be reused when it would now be incorrect. Abstract out the data_hash code and add it to the invalidateCaches command, called by tinfoil.modified_files() meaning that tinfoil can instruct bitbake to update the caches and re-parse if necessary. Also move the data_hash entirely into databuilder and drop the copy in cooker as obsolete and not needed. (Bitbake rev: d9ee77829f693ce75348fa64f406fcecfe4989aa) Signed-off-by: Richard Purdie

bitbake: bitbake: Drop older python version compatibility code

2024-05-31T15:56:25+00:00

cooker: We can call multiprocessing close() unconditionally and tweak a comment give 3.8 is now the minimum version. lib/bb: We can drop the logger addition code only needed before 3.6 asyncrpc/hashserv: Since the minimum version is 3.8, we can drop the conditional code. (Bitbake rev: 16f4386400f88ba50605307961c248bef09895c1) Signed-off-by: Richard Purdie