linux/poky.git/bitbake/lib/bb/server, branch yocto-4.1.2

bitbake: server/process: Fix logging issues where only the first message was displayed

2022-06-25T20:14:07+00:00

I realised only the first logging message was being displayed in a given parsing process. The reason turned out to be the UI handler failing with a "pop from empty list". The default handler was then lost and no further messages were processed. Fix this by catching the exception correctly in the connection writer code. (Bitbake rev: d3e64f64525187f1409531a0bd99df576e627f7f) Signed-off-by: Richard Purdie

bitbake: server/process: Avoid tracebacks at exit

2022-06-10T12:40:02+00:00

In theory this should have been worked around but is still occurring. Add it to the list of things to ignore when bitbake is shutting down. Traceback (most recent call last): File "/usr/lib64/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 698, in startCallbackHandler event = self.reader.get() File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 722, in get res = self.reader.recv_bytes() File "/usr/lib64/python3.9/multiprocessing/connection.py", line 221, in recv_bytes buf = self._recv_bytes(maxlength) File "/usr/lib64/python3.9/multiprocessing/connection.py", line 426, in _recv_bytes return self._recv(size) File "/usr/lib64/python3.9/multiprocessing/connection.py", line 384, in _recv chunk = read(handle, remaining) TypeError: an integer is required (got type NoneType)' (Bitbake rev: 7a28ac4fe478bee1e52e84412da9626495f9c6c7) Signed-off-by: Richard Purdie

bitbake: server/process: Remove daemonic thread usage

2022-06-08T20:53:15+00:00

We're seeing UI deadlocks occasionally and this is possibly due to the use of a daemonic thread in the UI event queue processing. This thread could terminate holding a threading Lock() which would cause issues for the process when exitting. Change the shutdown process to handle this more cleanly. (Bitbake rev: f5ad8349a5dbff9824a89f5708cfd011d61888c9) Signed-off-by: Richard Purdie

bitbake: server/process: Avoid risk of exception deadlocks

2022-06-08T20:53:15+00:00

The open coded lock acquire/release in the UI event handler doesn't cover the case an exception occurs and if one did, it could deadlock the code. Switch to use 'with' statements which would handle this possibility. We have seen deadlocks in the UI at exit this so this removes a possible cause. (Bitbake rev: bd12792f28efd2f03510653ec947ebf961315272) Signed-off-by: Richard Purdie

bitbake: server/process: Drop unused import

2022-04-21T20:00:35+00:00

(Bitbake rev: 543315e6463f15ca7ab2b4ef3e8ed41bb4207ccf) Signed-off-by: Richard Purdie

bitbake: server/process: Disable gc around critical section

2022-04-03T16:51:26+00:00

The python gc can trigger whilst we're holding the event stream lock and when cleaning up objects, they can trigger warnings. This translates into a new event which would then need the lock and we can deadlock. Disable gc whilst we hold that lock to avoid this unfortunate and problematic situation. (Bitbake rev: 96a6303949cefd469bcf5ed250ff512271354357) Signed-off-by: Richard Purdie

bitbake: cooker/process: Fix signal handling lockups

2022-03-30T12:05:03+00:00

If a parser process is terminated while holding a write lock, then it will lead to a deadlock (see https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.terminate). With SIGTERM, we don't want to terminate holding the lock. We also don't want a SIGINT to cause a partial write to the event stream. I tried using signal masks to avoid this but it doesn't work, see https://bugs.python.org/issue47139 Instead, add a signal handler and catch the calls around the critical section. We also need a thread lock to ensure other threads in the same process don't handle the signal until all the threads are not in the lock. (Bitbake rev: a40efaa5556a188dfe46c8d060adde37dc400dcd) Signed-off-by: Richard Purdie

bitbake: server/process: Correct a typo in a comment

2022-03-28T12:33:28+00:00

(Bitbake rev: b4a157b2fe2fb481ffa40e0f32659d05dd6320c2) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

bitbake: server/process: Move threads left debug to after cooker shutdown

2022-03-26T09:27:43+00:00

This debug is useful but the cooker shutdown or post_serve() may have cleanup left so run after those. (Bitbake rev: 1463fc0448d1a6a7265806a4a8b165b610dfb43f) Signed-off-by: Richard Purdie

bitbake: server/xmlrpcserver: Add missing xmlrpcclient import

2022-03-08T09:45:57+00:00

This avoids backtraces when starting toaster or using bitbake in remote mode. (Bitbake rev: bf723f2cb5d288ca730e4f029110b36380420a01) Signed-off-by: Richard Purdie