linux/poky.git/bitbake/lib/bb/event.py, branch 5.2_M2

bitbake: Remove custom exception backtrace formatting

2024-11-28T00:06:24+00:00

Removes the code in bitbake to show custom backtrace formatting for exceptions. In particular, the bitbake exception code prints function arguments, which while helpful is a security problem when passwords and other secrets can be passed as function arguments. As it turns out, the handling of the custom serialized exception stack frames was pretty much made obsolete by d7db75020ed ("event/msg: Pass formatted exceptions"), which changed the events to pass a preformatted stacktrack list of strings, but the passing of the serialized data was never removed. Change all the code to use the python traceback API to format exceptions instead of the custom code; conveniently traceback.format_exception() also returns a list of stack trace strings, so it can be used as a drop in replacement for bb.exception.format_exception() (Bitbake rev: 2cda75a185aaf8f657f072dac34f8cef9d75f63a) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: bitbake: event: Inject empty lines to make code match lineno in filename

2024-01-10T13:55:33+00:00

So that we can get the correct error messages. * In python 3.10.9, the error message was: ERROR: Unable to register event handler 'defaultbase_eventhandler': File "/path/to/poky/meta/classes-global/base.bbclass", line 4 # SPDX-License-Identifier: MIT ^^^^^ SyntaxError: invalid syntax This is hard to debug since the error line number 4 is incorrect, but nothing is wrong with the code in line 4. * Now the error message and lineno is correct: ERROR: Unable to register event handler 'defaultbase_eventhandler': File "/path/to/poky/meta/classes-global/base.bbclass", line 256 an error line ^^^^^ SyntaxError: invalid syntax And no impact on parsing time: * Before: $ rm -fr cache tmp; time bitbake -p real 0m27.254s * Now: $ rm -fr cache tmp; time bitbake -p real 0m27.200s (Bitbake rev: c212933d9c786806852c87f188250a4f0a14c048) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: bitbake: cooker: add a new function to retrieve task signatures

2023-09-28T11:38:26+00:00

adding a new command in cooker to compute and get task signatures this commit also add the associated command and event needed to get the signatures using tinfoil (Bitbake rev: 05c15162de90c41dad67e37a95ec9fdb440a7864) Signed-off-by: Julien Stephan Signed-off-by: Richard Purdie

bitbake: event: add bb.event.ParseError

2023-04-13T11:01:45+00:00

Add bb.event.ParseError to let error-report can catch this kind of error. (Bitbake rev: 316524ab59a5e738c25e062923ee5717d88ae5c7) Signed-off-by: Mingli Yu Signed-off-by: Richard Purdie

bitbake: event/cooker/runqueue: Add ability to interrupt longer running code

2023-02-23T12:12:18+00:00

Bitbake is now able to understand when a UI wants it to stop the current processing. There are some long running functions which currently have no mechanism to interrupt them however. This patch adds a call, bb.event.check_for_interrupts(d) which can be placed in long running code and allows an internal state flag within bitbake to be checked. If set, that flag will trigger an exit. This means that Ctrl+C can be made to work in a variety of places where it currently would not. Long running event handlers in OE-Core can also then benefit from this new approach with the addition of the function call as well. (Bitbake rev: b7ed7e9a815c4e10447fd499508be3dbb47f06e8) Signed-off-by: Richard Purdie

bitbake: lib/bb: Update thread/process locks to use a timeout

2023-01-05T11:50:17+00:00

The thread/process locks we use translate to futexes in Linux. If a process dies holding the lock, anything else trying to take the lock will hang indefinitely. An example would be the OOM killer taking out a parser process. To avoid bitbake processes just hanging indefinitely, add a timeout to our lock calls using a context manager. If we can't obtain the lock after waiting 5 minutes, hard exit out using os._exit(1). Use _exit() to avoid locking in any other places trying to write error messages to event handler queues (which also need locks). Whilst a bit harsh, this should mean we stop having lots of long running processes in cases where things are never going to work out and also avoids hanging builds on the autobuilder. (Bitbake rev: d2a3f662b0eed900fc012a392bfa0a365df0df9b) Signed-off-by: Richard Purdie

bitbake: event: Always use threadlock

2022-12-31T17:05:17+00:00

With the move to a server idle thread, we always need threading. The existing accessor functions could end up turning this off! I was going to hold the lock whilst changing it, check if the value was already set, cache the result and also fix the event code to always release the lock with a try/finally. Instead, disable the existing functions and use a with: block to handle the lock, keeping things much simpler. (Bitbake rev: 645c9d3b50e55f69b222cc338373cdfd91d524ce) Signed-off-by: Richard Purdie

bitbake: event: Add enable/disable heartbeat code

2022-12-31T17:05:17+00:00

Currently heartbeat events are always generated by the server whilst it is active. Change this so they only appear when builds are running, which is when most code would expect to be executed. This removes a number of races around changes in the datastore which can happen outside of builds. (Bitbake rev: 8c36c90afc392980d999a981a924dc7d22e2766e) Signed-off-by: Richard Purdie

bitbake: event: builtins fix for 'd' deletion

2022-12-29T00:07:06+00:00

I've been seeing event handlers where 'd' seems to disappear half way through event handler execution. This is problematic when multiple threads are active since this code assumes single threading. The easiest fix is to change the handler function calls to contain d as a parameter as we do elsewhere for other functions. This will break any non-text handlers but I was only able to spot one of those in runqueue. It will also break handlers than call functions that assume 'd' is in the global namespace but those failures should be obvious and we can fix those to pass d around. This solution avoids manipulating builtins which was always a horrible thing to do anyway and solves the issue without needing locking, thankfully. (Bitbake rev: 1e12f0a4b592dacd006d370ec29cd71d2a44312e) Signed-off-by: Richard Purdie

bitbake: event.py: ignore exceptions from stdout and sterr operations in atexit

2022-06-11T09:30:56+00:00

When atexit functions run, stdout and stderr operations may fail, e.g. when output is piped to less but has been exited by the user. This removes error print from output of "bitbake -e sqlite3 | less" if user presses "q" before bitbake has finished processing: [Errno 32] Broken pipeError in atexit._run_exitfuncs: Traceback (most recent call last): File "/home/builder/src/poky/bitbake/lib/bb/event.py", line 135, in print_ui_queue sys.stdout.flush() (Bitbake rev: 35167536c163eb6b7653cbcaad9f65b834d3e2f8) Signed-off-by: Mikko Rapeli Signed-off-by: Richard Purdie