linux/poky.git/bitbake/lib/bb/server, branch hardknott

bitbake: server/process: Disable gc around critical section

2022-04-16T22:03:46+00:00

The python gc can trigger whilst we're holding the event stream lock and when cleaning up objects, they can trigger warnings. This translates into a new event which would then need the lock and we can deadlock. Disable gc whilst we hold that lock to avoid this unfortunate and problematic situation. (Bitbake rev: d01d8326331cfe59208674cfc53aa26c0028b313) Signed-off-by: Richard Purdie (cherry picked from commit 96a6303949cefd469bcf5ed250ff512271354357) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: server/xmlrpcserver: Add missing xmlrpcclient import

2022-04-05T15:54:06+00:00

This avoids backtraces when starting toaster or using bitbake in remote mode. (Bitbake rev: 1d583db85ba8fa3242e231af4d94274527c051a6) Signed-off-by: Richard Purdie

bitbake: bitbake: correct deprecation warning in process.py

2021-11-24T10:08:28+00:00

(Bitbake rev: 35ca8e9fcc20a8ee9efb1356a957e054ad5532d8) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie (cherry picked from commit aff52fe21a0b27f6302555c1e52a864550eb46ce) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: cooker/process: Fix typos in exiting message

2021-09-24T09:13:32+00:00

(Bitbake rev: 284ca139b3a2ce61cef91b3076fd8bb544461c16) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie (cherry picked from commit 1ff1ea3880d293b14ce0fc65e3bc4c938d587a2f) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: server: Fix early parsing errors preventing zombie bitbake

2021-09-09T09:19:42+00:00

If the client process never sends cooker data, the server timeout will be 0.0, not None. This will prevent the server from exiting, as it is waiting for a new client. In particular, the client will disconnect with a bad "INHERIT" line, such as: INHERIT += "this-class-does-not-exist" Instead of checking explicitly for None, check for a false value, which means either 0.0 or None. (Bitbake rev: c7b506d11df78cfc4610db6578745eaa6220b13a) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie (cherry picked from commit 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: server/process: Handle error in heartbeat funciton in OOM case

2021-05-20T20:40:07+00:00

We've seen cases where an OOM error causes bitbake server to hang: 9171 02:21:09.127810 Command Completed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/bin/bitbake-server", line 51, in bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 550, in execServer server.run() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 108, in run ret = self.main() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 242, in main ready = self.idle_commands(.1, fds) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 370, in idle_commands bb.event.fire(heartbeat, self.cooker.data) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 216, in fire fire_class_handlers(event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 123, in fire_class_handlers execute_handler(name, handler, event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 93, in execute_handler ret = handler(event) File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 182, in defaultrun_buildstats write_host_data(os.path.join(bsdir, "host_stats"), e, d, "interval") File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 160, in write_host_data output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child restore_signals, start_new_session, preexec_fn) OSError: [Errno 12] Cannot allocate memory We need to wrap the calls in the same high level wrapper as idle function calls and trigger an exit upon an unhandled exception. (Bitbake rev: 0c83e98b0b4129e43e05129074fff60b22a9f2eb) Signed-off-by: Richard Purdie

bitbake: bitbake-server: ensure server timeout is a float

2021-04-27T14:22:58+00:00

bitbake-server is spawned by process.py and passes the arguments it is given to ProcessServer. There's some type confusion here: bitbake-server is called with a string representation of the timeout, which may be None. If the timeout is not set, pass 0 instead of None. Inside bitbake-server a ProcessServer is created which expects the timeout to be a float not a string, so always float() the value. [ YOCTO #14350 ] (Bitbake rev: f2cfb9f6710808ea37aecb6c34c62f92191e1d4b) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit c93ae1f861208f6d39fd15c84fbcd0e2b54331f5) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: process: Show command exceptions in the server log as well

2020-10-11T12:44:26+00:00

There are autobuilder logs where the server commands are failing but we have no debug info in the server log. Improve this to try and understand what is failing. (Bitbake rev: 04d3a79226c9ea448b22f4efbab33876a72c9bdb) Signed-off-by: Richard Purdie

bitbake: server/process: Note when commands complete in logs

2020-09-05T10:45:18+00:00

Its hard to tell from the server logs whether commands complete or not (or how long they take). Add extra info to allow more debugging of server timeouts. (Bitbake rev: 56285ada585ec1481449522282b335bcb5a2671e) Signed-off-by: Richard Purdie

bitbake: server/process: Prefix the log data with pid/time information

2020-09-05T10:45:18+00:00

Knowing which process printed which messages and the timestamp of the message is useful for debugging, so add this. Ensure the log parsing isn't affected by using search() instead of match(). (Bitbake rev: 1d043666710df1fa9d9586fd974c0371dd1514b0) Signed-off-by: Richard Purdie