linux/poky.git/bitbake/bin/bitbake-worker, branch yocto-4.0.14

bitbake: bitbake-worker/runqueue: Avoid unnecessary bytes object copies

2023-10-18T15:13:24+00:00

declaring queue=b"" creates an object of types bytes(). bytes() is an immutable object, and therefore doing "self.queue = self.queue + r" creates a new object containing "self.queue" concatenated with "r". On my test setup, we are passing 180MB of data of "workerdata" to the bitbake-worker, so those copies significantly slow down the initialization of the bitbake-worker. Rather use bytearray() which a mutable type, and use extend() to avoid copies. In my test setup, byterray.extend() is 10.000 times faster than copying the queue, for a queue size of 180MB. (Bitbake rev: 9993a89e5b97dda5f3657e5a7cc3a4fa94ff7111) Signed-off-by: Etienne Cordonnier Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: bin/utils: Ensure locale en_US.UTF-8 is available on the system

2023-04-26T14:03:21+00:00

Get rid of the duplicate code and add extra check that the locale en_US.UTF-8 is available on the system. This new helper method is now located right above the method filter_environment() which sets LC_ALL environment variable to 'en_US.UTF-8'. [YOCTO #10165] (Bitbake rev: 0c6f86b60cfba67c20733516957c0a654eb2b44c) Signed-off-by: Frank de Brabander Signed-off-by: Richard Purdie (cherry picked from commit a4ce040a6fd540a1cac52f808f909f9fcf8c961c) Signed-off-by: Steve Sakoman

bitbake: bitbake: Add copyright headers where missing

2022-10-11T20:57:28+00:00

Where copyright headers were not present, add them to make things clear. (Bitbake rev: e591325b2bd901c381003deb96a7b32a7148e93e) Signed-off-by: Richard Purdie (cherry picked from commit 1aa338a216350a2751fff52f866039343e9ac013) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/cooker: Fix source revision handling with floating upstreams

2022-02-17T17:51:03+00:00

Where a git url uses a tag instead of a full source revision, breakage can currently occur in builds. Issues include: * the revision being looked up in multiple tasks (fetch and unpack) * the risk a different revision may be obtained in those tasks * that some tasks may not be allowed to access the network * that a revision may not be consistent throughout a given build * rerunning a specific task may given inconsistent results To fix this, stop the workers from cleaning out the source revision store. This should only be done in the cooker itself (based on current policy). Also, where the code "sees" an upstream access, mark the recipe as not to be cached. The reparse re-triggers the upstream lookup by the server. Add a test to ensure that if get_srcrev isn't called, the user is told they're using a configuration that is known to break. (Bitbake rev: 4b5eed1626709ef3dc06b32fd55d40a2a6edd179) Signed-off-by: Richard Purdie

bitbake: bitbake: bitbake-worker: Preserve network non-local uid

2022-02-01T07:31:48+00:00

The NIS can't work when network is dissable, so preserve network for it, the error is like: do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable Note, enable nscd on the build machine might be a solution, but that isn't reliable since it depends on whether the network function has been cached or not. (Bitbake rev: 4eafae7904bae6e5c6bc50356e8a9077f2e207fa) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Add/support network task flag

2022-01-12T21:10:24+00:00

This patch changes behaviour. After this change any task which does not have the network flag set will have networking disabled on systems that support that (kernel version dependent). Add a "network" task specific flag which then triggers networking to be enabled for this task, it is otherwise disabled. This needs to happen before we enter the fakeroot environment of the task due to the need for the real uid/gid which we save in the parent process. (Bitbake rev: 0746b6a2a32fec4c18bf1a52b1454ca4c04bf543) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Add debug when unpickle fails

2021-10-17T10:57:27+00:00

We occasionally see bitbake-worker failing and from the logs, an unpickle error occurs. Add more debug so we can further debug this next time it fails. [YOCTO #14595] (Bitbake rev: fe8105cc06beca8240b76ea366a1eff5aa9c5412) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Set BB_CURRENTTASK earlier

2021-10-14T21:37:43+00:00

For some debugging, BB_CURRENTTASK is set too late to be useful as it isn't present in some event handlers for example. There is no other way to know which task is actually running so set the value earlier. (Bitbake rev: 7daaaaa27f55b5a458656857c6d61a51b34a62fe) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Handle pseudo shutdown in Ctrl+C case

2021-10-08T15:45:06+00:00

If the build is interrupted, handle the shutdown of pseudo even in this case to avoid data corruption inside docker containers. [YOCTO #14555] (Bitbake rev: a2a04c6fe94bc56efcff299c669a151746e35916) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Allow shutdown/database flush of pseudo server at task exit

2021-09-26T13:37:34+00:00

We have a problem where pseudo server processes exist after bitbake exits and hold the pseudo database in memory. In a docker container, the processes will be killed as the container is destroyed with no warning and no opportunity to write the data to disk. This leads to permissions/inode corruptions and data loss. Send a shutdown message to pseudo which in new versions of pseudo will flush the database, thereby fixing some of the issues people using docker containers see. (Bitbake rev: a07a971b40acd3eee12e203d2cfa3e49f56109f6) Signed-off-by: Richard Purdie