linux/poky.git/bitbake/lib/bb/runqueue.py, branch scarthgap-5.0.15

bitbake: bitbake: runqueue: Verify mcdepends are valid

2025-08-04T14:55:06+00:00

In order to avoid a potentially confusing backtrace, check that the mcdepend is valid when we add it. Add a test case to ensure invalid configurations are caught and trigger an error. [RP: Reworked test case to simplify and improve code] (Bitbake rev: 9f6f049870e0ec829e171fe91ec8f7a092ddd2ab) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie Signed-off-by: Chris Laplante Signed-off-by: Steve Sakoman

bitbake: runqueue: Fix scenetask processing performance issue

2024-12-09T14:54:14+00:00

Analysis shows that "bitbake core-image-ptest-all" spends a lot of time in scenequeue_updatecounters and much of it is rebuilding a set which doens't change. Reorder the code to avoid that performance glitch. (Bitbake rev: f40a3a477d5241b697bf2fb030dd804c1ff5839f) Signed-off-by: Richard Purdie (cherry picked from commit 923c19b9713e398d8e66e6d4422dfd4c18a03486) Signed-off-by: Steve Sakoman

bitbake: runqueue: Optimise setscene loop processing

2024-12-09T14:54:14+00:00

Rather than looping through things we looped through on the previous execution, start looping where we left off for setscene processing. This gives speed improvements depending on the kind of build being executed. (Bitbake rev: 28569e9796d4b34d7b77b4f79074ab7854850386) Signed-off-by: Richard Purdie (cherry picked from commit 00f4d932e3af0eeb333339cbe942010fc76dee0f) Signed-off-by: Steve Sakoman

bitbake: runqueue: Fix performance of multiconfigs with large overlap

2024-12-09T14:54:14+00:00

There have been complaints about the performance of large multiconfig builds for a while. The key missing data point was that the builds needed to have large overlaps in sstate objects. This can be simulated by building the same things with just different TMPDIRs. In runqueue/bitbake terms this equates to large numbers of deferred tasks. The issue is that the expensive checks in the setscene loop were hit every time through runqueue's execute function before the check on deferred tasks. This leads to task execution starvation as that only happens once per iteration. Move the skip check earlier in the function which speeds things up enormously and should improve performance of such builds for users. (Bitbake rev: db083cfe9e33c9fd7ffeead7b8c6023a5d581976) Signed-off-by: Richard Purdie (cherry picked from commit 9c6c506757f2b3e28c8b20513b45da6b4659c95f) Signed-off-by: Steve Sakoman

bitbake: bb: Use namedtuple for Task data

2024-06-06T13:53:49+00:00

Task dependency data is becoming unwieldy with the number of indices it contains. Convert it to use a named tuple instead, which allows members to be indexed by a named property or an index (which allows it to retain backward compatibility). (Bitbake rev: bc7c44affe8e3fa94e92c3bcb8ad85bf11963779) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Improve rehash get_unihash parallelism

2024-06-06T13:53:49+00:00

Improve the rehash code to query unihashes in parallel since this is more efficient on slower links. (Bitbake rev: 682fb42420eebf18f8a98bd3992baf9034dac5ac) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Process unihashes in parallel at init

2024-06-06T13:53:49+00:00

Improve the runqueue init code to call unihash queries in parallel since this is faster and more efficient, particularly on slower links with longer round trip times. The call to the function from cooker is unneeded since that function calls prepare() and hence this functionality will already have run, so drop that obsolete call. (Bitbake rev: e0486054c7a4c637446c18608e9983cc8dc4d7fe) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: siggen/runqueue: Report which dependencies affect the taskhash

2024-06-06T13:53:49+00:00

Report which task dependencies in BB_TASKDEPDATA are included in the taskhash. This allows tasks to identify which tasks dependencies may change without the task re-running. Knowing this information is important for tasks that want to transfer information from dependencies (such as SPDX) (Bitbake rev: 853423661779023763a87462b623b6e9ff2798b2) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Allow rehash loop to exit in case of interrupts

2024-06-06T13:53:49+00:00

The initial hash serve loop exits in the case where interrupts are present but probably checks a bit too often. Tweak that and also allow the slow rehash loop to break on interrupt, improving bitbake Ctrl+C response. (Bitbake rev: ca2f63d84b90cbd99936c9d7815d912d9019a594) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Add timing warnings around slow loops

2024-06-06T13:53:49+00:00

With hashserve enabled, there are two slow paths/loops, one at initial runqueue generation and also during the rehash process when new outhashes are found. Add timing information at the hashserve log level for when these loops take longer than 30s or 60s overall. This will leave evidence in the logs when things are running particularly slowly. (Bitbake rev: 9ee503c79936b13f1d45f9e43211f77a528cdbfa) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman