linux/poky.git/bitbake/bin/bitbake-worker, branch mickledore

bitbake: data: Evaluate the value of export/unexport/network flags

2023-02-17T18:02:04+00:00

Currently the export/unexport/network flags are acted on only by presence or lack of in the data store. This is deliberate due to performance reasons since a given recipe shell task might export ~80-90 variables and this means expanding flags for every one of them. This does catch users unaware since values which expand to nothing don't work e.g. ${@""} and setting values to "0" wouldn't work either. The downside to this patch is slow down in parsing speed of around 4%. The network flag is easier to change and doesn't affect performance, it was made to operate the same as export/unexport for consitency reasons. (Bitbake rev: 3d96c07f9fd4ba1a84826811d14bb4e98ad58846) Signed-off-by: Richard Purdie

bitbake: lib/bb: Update thread/process locks to use a timeout

2023-01-05T11:50:17+00:00

The thread/process locks we use translate to futexes in Linux. If a process dies holding the lock, anything else trying to take the lock will hang indefinitely. An example would be the OOM killer taking out a parser process. To avoid bitbake processes just hanging indefinitely, add a timeout to our lock calls using a context manager. If we can't obtain the lock after waiting 5 minutes, hard exit out using os._exit(1). Use _exit() to avoid locking in any other places trying to write error messages to event handler queues (which also need locks). Whilst a bit harsh, this should mean we stop having lots of long running processes in cases where things are never going to work out and also avoids hanging builds on the autobuilder. (Bitbake rev: d2a3f662b0eed900fc012a392bfa0a365df0df9b) Signed-off-by: Richard Purdie

bitbake: siggen: Add dummy dataCaches from task context/datastore

2022-12-17T08:52:28+00:00

One of the challenges in maintaining the code is that it sometimes uses a datacaches structure and sometimes a datastore. Rather than continue the current dual API madness, have the worker contexts create a dummy datacaches structure with the entries we need. Whilst this does need to be kept in sync with the real structure, that doesn't change and this allows the code to be simplified. With this new approach, we can unify the stamps dependency code again. (Bitbake rev: c6d325fc9b53e9d588ab273ee3c2a99a70fba42c) Signed-off-by: Richard Purdie

bitbake: bin/utils: Ensure locale en_US.UTF-8 is available on the system

2022-12-08T10:49:53+00:00

Get rid of the duplicate code and add extra check that the locale en_US.UTF-8 is available on the system. This new helper method is now located right above the method filter_environment() which sets LC_ALL environment variable to 'en_US.UTF-8'. [YOCTO #10165] (Bitbake rev: a4ce040a6fd540a1cac52f808f909f9fcf8c961c) Signed-off-by: Frank de Brabander Signed-off-by: Richard Purdie

bitbake: worker/runqueue: Reduce initial data transfer in workerdata

2022-11-20T08:31:28+00:00

When setting up the worker we were transfering large amounts of data which aren't needed until task execution time. Defer the fakeroot and taskdeps data until they're needed for a specific task. This will duplicate some information when executing different tasks for a given recipe but as is is spread over the build run, it shouldn't be an issue overall. Also take the opportunity to clean up the silly length argument lists that were being passed around at the expense of extra dictionary keys. (Bitbake rev: 3a82acdcf40bdccd933c4dcef3d7e480f0d7ad3a) Signed-off-by: Richard Purdie

bitbake: cache/cookerdata: Move recipe parsing functions from cache to databuilder

2022-11-20T08:31:28+00:00

When 'NoCache' was written, databuilder/cookerdata didn't exist. It does now and the recipe parsing functionality contained in NoCache clearly belongs there, it isn't a cache function. Move those functions, renaming to match the style in databuilder but otherwise not changing functionality for now. Fix up the callers to match (which make it clear this is the right move). (Bitbake rev: 783879319c6a4cf3639fcbf763b964e42f602eca) Signed-off-by: Richard Purdie

bitbake: BBHandler/cooker: Implement recipe and global classes

2022-08-12T10:49:29+00:00

We have some confusion for users since some classes are meant to work in the configuration space (or "globally") and some are meant to be selected by recipes individually. The cleanest way I could find to clarify this is to create "classes-global" and "classes-recipe" directories which contain the approproate classes and have bitbake switch scope between them at the appropriate point during parsing. The existing "classes" directory is always searched as a fallback. Once a class is moved to a specific directory, it will no longer be found in the incorrect context. A good example from OE is that INHERIT += "testimage" will no longer work but IMAGE_CLASSES += "testimage" will, which makes the global scope cleaner by only including it where it is useful and intended to be used (images). (Bitbake rev: f33ce7e742f46635658c400b82558cf822690b5e) Signed-off-by: Richard Purdie

bitbake: bitbake: Add copyright headers where missing

2022-08-12T10:49:29+00:00

Where copyright headers were not present, add them to make things clear. (Bitbake rev: 1aa338a216350a2751fff52f866039343e9ac013) Signed-off-by: Richard Purdie

bitbake: fetch2/cooker: Fix source revision handling with floating upstreams

2022-02-17T17:51:03+00:00

Where a git url uses a tag instead of a full source revision, breakage can currently occur in builds. Issues include: * the revision being looked up in multiple tasks (fetch and unpack) * the risk a different revision may be obtained in those tasks * that some tasks may not be allowed to access the network * that a revision may not be consistent throughout a given build * rerunning a specific task may given inconsistent results To fix this, stop the workers from cleaning out the source revision store. This should only be done in the cooker itself (based on current policy). Also, where the code "sees" an upstream access, mark the recipe as not to be cached. The reparse re-triggers the upstream lookup by the server. Add a test to ensure that if get_srcrev isn't called, the user is told they're using a configuration that is known to break. (Bitbake rev: 4b5eed1626709ef3dc06b32fd55d40a2a6edd179) Signed-off-by: Richard Purdie

bitbake: bitbake: bitbake-worker: Preserve network non-local uid

2022-02-01T07:31:48+00:00

The NIS can't work when network is dissable, so preserve network for it, the error is like: do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable Note, enable nscd on the build machine might be a solution, but that isn't reliable since it depends on whether the network function has been cached or not. (Bitbake rev: 4eafae7904bae6e5c6bc50356e8a9077f2e207fa) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie