linux/poky.git/bitbake/lib/bb, branch yocto-4.0.14

bitbake: bitbake-worker/runqueue: Avoid unnecessary bytes object copies

2023-10-18T15:13:24+00:00

declaring queue=b"" creates an object of types bytes(). bytes() is an immutable object, and therefore doing "self.queue = self.queue + r" creates a new object containing "self.queue" concatenated with "r". On my test setup, we are passing 180MB of data of "workerdata" to the bitbake-worker, so those copies significantly slow down the initialization of the bitbake-worker. Rather use bytearray() which a mutable type, and use extend() to avoid copies. In my test setup, byterray.extend() is 10.000 times faster than copying the queue, for a queue size of 180MB. (Bitbake rev: 9993a89e5b97dda5f3657e5a7cc3a4fa94ff7111) Signed-off-by: Etienne Cordonnier Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: runqueue: Fix deferred task/multiconfig race issue

2023-07-12T15:11:37+00:00

If there are several multiconfigs in play for example a non-multiconfig with a task with one hash and then three multiconfigs for the same task, different architectures but the same hash (different to the non-mc), the three mcs will be deferred until after the non-mc task but then will all run together and race against each other. Change the code to re-enable deferred tasks one at a time. This way, if they do race, they won't run in parallel against each other. (Bitbake rev: 907416ee1062f87f5844ab0638b54616abfc1a22) Signed-off-by: Richard Purdie (cherry picked from commit 9523e28658ad7fb446645b590608dfac2812afd3) Signed-off-by: Steve Sakoman

bitbake: bin/utils: Ensure locale en_US.UTF-8 is available on the system

2023-04-26T14:03:21+00:00

Get rid of the duplicate code and add extra check that the locale en_US.UTF-8 is available on the system. This new helper method is now located right above the method filter_environment() which sets LC_ALL environment variable to 'en_US.UTF-8'. [YOCTO #10165] (Bitbake rev: 0c6f86b60cfba67c20733516957c0a654eb2b44c) Signed-off-by: Frank de Brabander Signed-off-by: Richard Purdie (cherry picked from commit a4ce040a6fd540a1cac52f808f909f9fcf8c961c) Signed-off-by: Steve Sakoman

bitbake: fetch/git: Fix local clone url to make it work with repo

2023-04-11T10:31:52+00:00

The "git clone /path/to/git/objects_symlink" couldn't work after the following change: https://github.com/git/git/commit/6f054f9fb3a501c35b55c65e547a244f14c38d56 But repo command manages the git repo as symlinks, so check whether the objects is an symlink to fix the problem: * Nothing is changed if git/objects is not a symlink * Use "git clone file://" when git/objects is a symlink (Bitbake rev: 2802adb572eb73a3eb2725a74a9bbdaafc543fa7) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit a0d8108eba8d542707740d00c66c1c5f5b963f18) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: cookerdata: Drop dubious exception handling code

2023-03-14T15:05:30+00:00

This code appears to be dangerous, it swallows exceptions, turning them into "handled" versions which then show no errors to the user. This is a pretty poor user experience and I can't see why this code should be swallowing such things. Drop the worst bits of code. (Bitbake rev: a29ffcc12df24737e702e198aeb8f6892884932b) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: cookerdata: Improve early exception handling

2023-03-14T15:05:30+00:00

Martin Jansa reported that if you put a syntax error into an imported module such as qa.py in OE, no error is shown. Part of the issue appears to be that the catch_parse_error() decorator only catches certain exceptions and SyntaxError isn't one of them. As far as I can tell we should remove all the special cases and use the more advanced code in all cases, not just expansion errors. I confirmed this now prints a proper error message for a qa.py syntax error. (Bitbake rev: e66012bfda60ffe1658473e25879aa67909ae65f) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: cookerdata: Remove incorrect SystemExit usage

2023-03-14T15:05:30+00:00

Calling SystemExit doesn't work well with server/client usage since the string isn't printed to the right place. Use bb.fatal() instead which prints the right log output and raises and handled exception which then shows correctly on the UI. (Bitbake rev: b9ae7164d9e744e8eb9aaff79218f57233a449b7) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: utils: Allow to_boolean to support int values

2023-03-14T15:05:30+00:00

Some variables may be set as: X = 1 as well the more usual X = "1" so add support to to_boolean to handle this case. (Bitbake rev: e7df13a61911b7431802af2b4d7472b2aaf346fa) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: show SRCREV and git repo in error message about fixed SRCREV

2023-02-23T22:23:07+00:00

* in recipe with 17 git repos in SRC_URI I've accidentally pasted one SRCREV to be one character shorter and because fetcher uses: if not ud.revisions[name] or len(ud.revisions[name]) != 40 or (False in [c in "abcdef0123456789" for c in ud.revisions[name]]): to decide which SRCREV values are fixed SRCREVs this one was considered as tag or branch name, because it was only 39 chars long The original error message wasn't very helpful as it doesn't show which repo or which SRCREV was considered missing: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) with SRCPV included in PV as error recomments it's a bit better: bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '0a92994d729ff76a58f692d3028ca1b64b145d9' in upstream git repository in git ls-remote output for github.com/Maratyszcza/FP16 The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> T with this change the first error will read: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '0a92994d729ff76a58f692d3028ca1b64b145d9' for repo 'github.com/Maratyszcza/FP16' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) (Bitbake rev: 9bbdedc0ba7ca819b898e2a29a151d6a2014ca11) Signed-off-by: Martin Jansa Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: bitbake-diffsigs: break on first dependent task difference

2023-02-17T15:05:08+00:00

compare_sigfiles() recursively calculates differences on all dependent tasks with changed hashes. This is done in arbitrary/alphabetical order, and only the last of those results is returned, while everything else is discarded. This changes the behavior to instead return the first difference and not calculate any more, which significantly speeds up diffs of tasks with many dependencies. (Bitbake rev: 89f13cd4a927a73de98998c27082c63b07671525) Signed-off-by: Adriaan Schmidt Signed-off-by: Richard Purdie (cherry picked from commit ea6a676c9aa2864c2eff40eea41ba09ce903a651) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie