linux/poky.git/bitbake/bin/bitbake-worker, branch master

The poky repository master branch is no longer being updated.

2025-11-07T13:31:53+00:00

You can either: a) switch to individual clones of bitbake, openembedded-core, meta-yocto and yocto-docs b) use the new bitbake-setup You can find information about either approach in our documentation: https://docs.yoctoproject.org/ Note that "poky" the distro setting is still available in meta-yocto as before and we continue to use and maintain that. Long live Poky! Some further information on the background of this change can be found in: https://lists.openembedded.org/g/openembedded-architecture/message/2179 Signed-off-by: Richard Purdie

bitbake: utils: Refactor filemode variable conversion to a function

2025-07-01T07:49:37+00:00

We have other places in the code where we need to take filemode/mask information from a bitbake variable and turn it into a real python number. Turn this internal code into public API in bb.utils and add some tests for it. (Bitbake rev: d89e30fb2fb15b09f2cb95c4e5aa9f749ca257ea) Signed-off-by: Richard Purdie

bitbake: bitbake-server/worker: Hide os.fork() deprecation warning

2025-03-11T14:17:57+00:00

We're fairly careful in bitbake about how we handle fork() calls and believe our code to be safe. The upstream deprecation warning is problematic as it can appear in log output as a WARNING, breaking tests. It also tends to alarm users. Hide the warning for now to avoids the test failures. (Bitbake rev: c636bd629896f56e5f3d4030da3d1f130590afc6) Signed-off-by: Richard Purdie

bitbake: bitbake-worker/cooker: Increase default pipe size

2024-11-21T12:16:28+00:00

The default pipe size is 64kb on builds, which can be inefficient for larger log files from workers. Increase the pipe size to 512kb since build systems have decent amounts of memory and this is a more efficient way of batching the data. Tweak the default read sizes to match the pipe size for efficiency. Since the contstant is only present in python 3.10 onwards, add some compatibility code. (Bitbake rev: 69c14e46600ba5ae9703f67704ab2548875ae6d7) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Improve bytearray truncation performance

2024-11-21T12:16:28+00:00

If there are large amounts of data being transferred to the cooker from the worker, recreating the bytearray becomes inefficient as it happens for every pipesize block of data, defaulting to 64kb. Instead we can use the deletion API for bytearrays to make this more efficient and avoid the object recreation. We noticed this with a strace ptest image taking days to complete the build after having 6GB of data in the testimage log. Whilst there are other issues there, making this code more efficient doesn't hurt. (Bitbake rev: a4a72b7edb368f352784c856a647236a887010dd) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Fix silent hang issue caused by unexpected stdout content

2024-03-22T16:26:46+00:00

This patch addresses an issue in bitbake-worker where stdout, reserved for status reporting, is improperly accessed by child processes. The problem occurs during the execution of parseRecipe, which calls anonymous functions. If these functions use print-like operations, they can inadvertently output data to stdout. This unexpected data can cause the runqueue to hang silently, if the stdout buffer is flushed before exec_task is executed. To prevent this, the patch redirects stdout to /dev/null and ensures it is flushed prior to the execution of exec_task. (Bitbake rev: 08f3e677d6af27a41a918aaa9da9c1c9b20a0b95) Signed-off-by: Yang Xu Signed-off-by: Richard Purdie

bitbake: bitbake-worker: allow '=' in environment variable values

2024-03-22T16:26:45+00:00

Limit the split to key & value (2 items) instead of the n items one can get if there are '=' characters in the value. Fixes [YOCTO #15447] (Bitbake rev: 86315961829ab1d137a0265cc246c44d3929e1fb) Signed-off-by: david d zuhn Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Fix bug where umask 0 was not being applied to a task

2024-03-22T16:26:45+00:00

* In the current implementation, "umask" variable is initially set to None and overwritten with user-specified value. However, in the worker implementation, a faulty if clause would only check whether the variable contains a value that evaluates to True, and not whether the variable is defined, so the value of 0 would lead to umask not being changed. This bug makes it impossible to have a task set its umask to value 0, for any possible reason it may want to. Fix this bug by extending the condition checked in the worker implementation. (Bitbake rev: 19f9df6c750c592316a0fa18165b68636281fe3e) Signed-off-by: Sava Jakovljev Signed-off-by: Richard Purdie

bitbake: bitbake-worker: add header with length of message

2023-11-06T16:45:11+00:00

The IPC mechanism between runqueue.py and bitbake-worker is currently not scalable: The data is sent with the format pickled-data, and bitbake-worker has no information about the size of the message. Therefore, the bitbake-worker is calling select() and read() in a loop, and then calling "self.queue.find(b"")" for each chunk received. This does not scale, because queue.find has a linear complexity relative to the size of the queue, and workerdata messages get very big e.g. for builds which reference a lot of files in SRC_URI. The number of chunks varies, but on my test system a lot of chunks of 65536 bytes are sent, and each iteration takes 0.1 seconds, making the transfer of the "workerdata" data very slow (on my test setup 35 seconds before this fix, and 1.5 seconds after this fix). This commit adds a 4 bytes header after , so that bitbake-worker knows how many bytes need to be received, and does not need to constantly search the whole queue for . (Bitbake rev: 595176d6be95a9c4718d3a40499d1eb576b535f5) Signed-off-by: Etienne Cordonnier Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: bitbake-worker/runqueue: Avoid unnecessary bytes object copies

2023-09-22T06:45:47+00:00

declaring queue=b"" creates an object of types bytes(). bytes() is an immutable object, and therefore doing "self.queue = self.queue + r" creates a new object containing "self.queue" concatenated with "r". On my test setup, we are passing 180MB of data of "workerdata" to the bitbake-worker, so those copies significantly slow down the initialization of the bitbake-worker. Rather use bytearray() which a mutable type, and use extend() to avoid copies. In my test setup, byterray.extend() is 10.000 times faster than copying the queue, for a queue size of 180MB. (Bitbake rev: 2302b5316091dff189e6c3f546341b2274ed9d0a) Signed-off-by: Etienne Cordonnier Signed-off-by: Richard Purdie