linux/poky.git/bitbake/lib/bb/cooker.py, branch scarthgap-5.0.15

bitbake: Use a "fork" multiprocessing context

2025-09-17T22:36:44+00:00

Python 3.14 changes the default multiprocessing context from "fork" to "forkserver"; however bitbake heavily relies on "fork" to efficiently pass data to the child processes. As such, make "fork" context in the bb namespace and use it in place of the normal multiprocessing module. Note that multiprocessing contexts were added in Python 3.4, so this should be safe to use even before Python 3.14 [YOCTO #15858] (Bitbake rev: 15d7448e04aa78c827d2cef9eb1a62bd6e0dd119) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: cooker: Make cooker 'skiplist' per-multiconfig/mc

2025-01-24T15:59:38+00:00

Previously, the cooker skiplist was shared across multiconfigs (including default ''). If you had a recipe that was incompatible with several multiconfigs for different reasons, then the displayed reason (i.e. the "ERROR: Nothing PROVIDES" and "* was skipped" messages) might vary across invocations of bitbake. This was caused by the random order in which recipes are parsed under different multiconfig contexts, with each skip reason overwriting the previously assigned reason. I hit this specificially when using COMPATIBLE_MACHINE, but COMPATIBLE_HOST (or anything using bb.parse.SkipRecipe) would have done it too. (Bitbake rev: 7dde14582bfd104c6da26e3f5ecf2ef37a1494ce) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: Remove custom exception backtrace formatting

2024-12-06T13:50:24+00:00

Removes the code in bitbake to show custom backtrace formatting for exceptions. In particular, the bitbake exception code prints function arguments, which while helpful is a security problem when passwords and other secrets can be passed as function arguments. As it turns out, the handling of the custom serialized exception stack frames was pretty much made obsolete by d7db75020ed ("event/msg: Pass formatted exceptions"), which changed the events to pass a preformatted stacktrack list of strings, but the passing of the serialized data was never removed. Change all the code to use the python traceback API to format exceptions instead of the custom code; conveniently traceback.format_exception() also returns a list of stack trace strings, so it can be used as a drop in replacement for bb.exception.format_exception() (Bitbake rev: c25e7ed128b9fd5b53d28d678238e2f3af52ef8b) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Process unihashes in parallel at init

2024-06-06T13:53:49+00:00

Improve the runqueue init code to call unihash queries in parallel since this is faster and more efficient, particularly on slower links with longer round trip times. The call to the function from cooker is unneeded since that function calls prepare() and hence this functionality will already have run, so drop that obsolete call. (Bitbake rev: e0486054c7a4c637446c18608e9983cc8dc4d7fe) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: cooker: Handle ImportError for websockets

2024-05-29T14:50:00+00:00

Handles ImportError when creating a hash equivalence to ping the server. This notifies user earlier with a more precise error if websockets can't be used, and also prevents passing a known bad upstream value to the local server (Bitbake rev: 93190565fd1251e8f47d9a6291739f8b8ed5ec87) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie (cherry picked from commit aa80b3cfc5d16dfba13ca7fb9b78bae179ce3b74) Signed-off-by: Steve Sakoman

bitbake: cooker: Use hash client to ping upstream server

2024-05-02T15:07:25+00:00

The cooker attempts to connect to the upstream hash equivalent server to warn the user early if it is misconfigured. However, this was making the assumption that it was a raw TCP connection and failed when attempting to use a websocket upstream server. Fix this by creating an hash client and using the ping API to check the server instead of using a raw socket. (Bitbake rev: 8f90d10f9efc9a32e13f6bd031992aece79fe7cc) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: cooker: Avoid eventlog variable listing lockups

2023-11-27T22:52:08+00:00

If the event log is enabled and parsing the metadata triggers log messages, the event code and deadlock. Iterating the variables inside the event handling code causes this. SOURCE_DATE_EPOCH triggers a python function which calls bb.debug() and can trigger a lockup as one example. Move the code around and add it to the BuildStarted events explictly. This does mean runs without builds no longer get variables added to the eventlog however we can look into a more targetted version of data if/as/where neded. (Bitbake rev: 4135a617ae16d509362b5bf56378139cdc0876d2) Signed-off-by: Richard Purdie

bitbake: cooker: Avoid sideeffects for autorev from getAllKeysWithFlags

2023-11-23T12:06:06+00:00

If we expand the variable AUTOREV in OE-Core, it triggers side effects in the fetcher. The situation isn't ideal and needs improvement but this breaks and is blocking enabling BB_DEFAULT_EVENTLOG. Hack around the issue for now so we unblock things until we can work out a better plan for how to improve AUTOREV support. (Bitbake rev: cb9b6530f3d12c56a8b48847af2e7461924205d2) Signed-off-by: Richard Purdie

bitbake: cooker: Add support for BB_DEFAULT_EVENTLOG

2023-11-23T12:06:06+00:00

Currently it is only possible to specify an eventlog on the bitbake commandline. Add a variable that can be used in bitbake.conf so that we can log data by default more easily. (Bitbake rev: ee174b231897a53cdde0f68769518342e53210cf) Signed-off-by: Richard Purdie

bitbake: asyncrpc: Add option to set log level when running as a process

2023-11-14T23:39:53+00:00

When running an asyncrpc server as a subprocess, it is often desired to run it with a lower logging level since the normal logging of clients connecting and disconnecting is not desired. As such, add an option to set the logging level of the server when running as a subprocess and set the level to WARNING when starting a local hashserver or PRserver (Bitbake rev: 61dac7b99ad6d2a858f85d8ed1b5524d558be6c8) Signed-off-by: Joshua Watt Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie