linux/poky.git/bitbake, branch styhead-5.1.3

bitbake: bblayers/query: Fix using "removeprefix" string method

2025-02-21T14:34:58+00:00

The minimum Python version required for Yocto 5.0 is 3.8 which causes failure in poky/bitbake/lib/bblayers/query.py when listing layers by using command "bitbake-layers show-recipes -f --bare --mc MC" for the given multiconfig MC. The reason for that failure is the use of "removeprefix" string method which got introduced in Python 3.9. This patch replaces the "removeprefix" method with an equivalent solution supported by Python 3.8. (Bitbake rev: 58e5c70a0572ff5994dc181694e05cd5d3ddaf66) Signed-off-by: Joerg Schmidt Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: cooker: Make cooker 'skiplist' per-multiconfig/mc

2025-02-12T14:29:33+00:00

Previously, the cooker skiplist was shared across multiconfigs (including default ''). If you had a recipe that was incompatible with several multiconfigs for different reasons, then the displayed reason (i.e. the "ERROR: Nothing PROVIDES" and "* was skipped" messages) might vary across invocations of bitbake. This was caused by the random order in which recipes are parsed under different multiconfig contexts, with each skip reason overwriting the previously assigned reason. I hit this specificially when using COMPATIBLE_MACHINE, but COMPATIBLE_HOST (or anything using bb.parse.SkipRecipe) would have done it too. (Bitbake rev: a8578d12dfe5a3c92119bfee224595a392d6b0dd) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie (cherry picked from commit c51f01a35ed9a928402eab0899598b5c59602eef) Signed-off-by: Steve Sakoman

bitbake: tests/fetch: Fix git shallow test failure with git >= 2.48

2025-01-29T14:20:43+00:00

>From git 2.48 release notes: """ When "git fetch $remote" notices that refs/remotes/$remote/HEAD is missing and discovers what branch the other side points with its HEAD, refs/remotes/$remote/HEAD is updated to point to it. """ This means with git 2.48 onwards, there is a mystery "HEAD" revision appearing in some of our shallow clone tests. We can avoid this by using the same canonicalization as used for the reference revisions. This resolves autobuilder failures on the Fedora 40 workers. (Bitbake rev: f4b2ae97a8dd745b463e5e10fbc8d30d29d223c5) Signed-off-by: Richard Purdie (cherry picked from commit c83444d1210740e27b1744d3aa7c5cad4e28db2f) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Fix scenetask processing performance issue

2024-12-09T14:25:53+00:00

Analysis shows that "bitbake core-image-ptest-all" spends a lot of time in scenequeue_updatecounters and much of it is rebuilding a set which doens't change. Reorder the code to avoid that performance glitch. (Bitbake rev: 1452e74cc4ddfaadc6537a049877a66cec253c8d) Signed-off-by: Richard Purdie (cherry picked from commit 923c19b9713e398d8e66e6d4422dfd4c18a03486) Signed-off-by: Steve Sakoman

bitbake: runqueue: Optimise setscene loop processing

2024-12-09T14:25:53+00:00

Rather than looping through things we looped through on the previous execution, start looping where we left off for setscene processing. This gives speed improvements depending on the kind of build being executed. (Bitbake rev: 5465f9b6e3b4748e563efc53657af96f02a41c7a) Signed-off-by: Richard Purdie (cherry picked from commit 00f4d932e3af0eeb333339cbe942010fc76dee0f) Signed-off-by: Steve Sakoman

bitbake: runqueue: Fix performance of multiconfigs with large overlap

2024-12-09T14:25:53+00:00

There have been complaints about the performance of large multiconfig builds for a while. The key missing data point was that the builds needed to have large overlaps in sstate objects. This can be simulated by building the same things with just different TMPDIRs. In runqueue/bitbake terms this equates to large numbers of deferred tasks. The issue is that the expensive checks in the setscene loop were hit every time through runqueue's execute function before the check on deferred tasks. This leads to task execution starvation as that only happens once per iteration. Move the skip check earlier in the function which speeds things up enormously and should improve performance of such builds for users. (Bitbake rev: 5392a2bf6d82626e11eec5578eb4de53ba09f1fe) Signed-off-by: Richard Purdie (cherry picked from commit 9c6c506757f2b3e28c8b20513b45da6b4659c95f) Signed-off-by: Steve Sakoman

bitbake: fetch2: use persist_data context managers

2024-11-30T13:41:59+00:00

Python 3.13 emits a ResourceWarning for unclosed sqlite3 `Connection`s. See https://docs.python.org/3/whatsnew/3.13.html#sqlite3 The previous commit fixed persist_data's context manager to close the connection, but we were never actually using `with` in the first place. This change is not necessary on 'master' because persist_data was removed. (Bitbake rev: 9602a684568910fd333ffce907fa020ad3661c26) Signed-off-by: Chris Laplante (cherry picked from commit 61f803c7d92a012b62837b0cdae4789a394b260e) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: persist_data: close connection in SQLTable exit

2024-11-30T13:41:59+00:00

A Connection's __exit__ does not close the connection, but rather deals with transactions. See https://docs.python.org/3/library/sqlite3.html#how-to-use-the-connection-context-manager This change is not necessary on 'master' because persist_data was removed. (Bitbake rev: a5640a2527cf1995e2afb80c95f303dea52c73b3) Signed-off-by: Chris Laplante (cherry picked from commit b0fba0f5af555ff970406b5cf3bf8fb113d953d1) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: fetch/wget: Increase timeout to 100s from 30s

2024-11-30T13:41:59+00:00

Testing shows the worst case CDN response time can be up to 100s. The wget fetcher is used for accessing sstate from the CDN so increase our timeouts there to match our worst case repsonse times. (Bitbake rev: 70b09f62bc5622fa64bb668cdad610bd48bf7bb9) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: Remove custom exception backtrace formatting

2024-11-30T13:41:59+00:00

Removes the code in bitbake to show custom backtrace formatting for exceptions. In particular, the bitbake exception code prints function arguments, which while helpful is a security problem when passwords and other secrets can be passed as function arguments. As it turns out, the handling of the custom serialized exception stack frames was pretty much made obsolete by d7db75020ed ("event/msg: Pass formatted exceptions"), which changed the events to pass a preformatted stacktrack list of strings, but the passing of the serialized data was never removed. Change all the code to use the python traceback API to format exceptions instead of the custom code; conveniently traceback.format_exception() also returns a list of stack trace strings, so it can be used as a drop in replacement for bb.exception.format_exception() (Bitbake rev: 1f7889c2f5e192fb108b51883aac8685ccb187f2) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

linux/poky.git/bitbake, branch styhead-5.1.3

bitbake: bblayers/query: Fix using "removeprefix" string method

bitbake: cooker: Make cooker 'skiplist' per-multiconfig/mc

bitbake: tests/fetch: Fix git shallow test failure with git >= 2.48

bitbake: runqueue: Fix scenetask processing performance issue

bitbake: runqueue: Optimise setscene loop processing

bitbake: runqueue: Fix performance of multiconfigs with large overlap

bitbake: fetch2: use persist_data context managers

bitbake: persist_data: close connection in SQLTable __exit__

bitbake: fetch/wget: Increase timeout to 100s from 30s

bitbake: Remove custom exception backtrace formatting

bitbake: persist_data: close connection in SQLTable exit