linux/poky.git/bitbake/lib/bb/server, branch langdale

bitbake: server/process: Add bitbake.sock race handling

2023-01-16T11:08:39+00:00

We've seen cases where the bitbake.sock file appears to disappear but the server continues to hold bitbake.lock. The most likely explaination is that some previous build directory was moved out the way, a server there kept running, eventually exited and removed the sock file from the wrong directory. To guard against this, save the inode information for the sock file and check it before deleting the file. The new code isn't entirely race free but should guard against what is a rare but annoying potential issue. (Bitbake rev: 52b6b099a47555811b8d0b311f62af712dd6eb8e) Signed-off-by: Richard Purdie (cherry picked from commit b02ebbffdae27e564450446bf84c4e98d094ee4a) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: process: log odd unlink events with bitbake.sock

2023-01-16T11:08:39+00:00

Log when the socket file already exists and is removed before recreating a new socket. Log when unlinking the socket file failed. (Bitbake rev: 9779fad4d9e2540b24bb91cfa38fc1984402bef2) Signed-off-by: Frank de Brabander Signed-off-by: Richard Purdie (cherry picked from commit cfd7c9899f988bab6d9fe7bbfbdb60603fb5ed34) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: server/process: Fix logging issues where only the first message was displayed

2022-06-25T20:14:07+00:00

I realised only the first logging message was being displayed in a given parsing process. The reason turned out to be the UI handler failing with a "pop from empty list". The default handler was then lost and no further messages were processed. Fix this by catching the exception correctly in the connection writer code. (Bitbake rev: d3e64f64525187f1409531a0bd99df576e627f7f) Signed-off-by: Richard Purdie

bitbake: server/process: Avoid tracebacks at exit

2022-06-10T12:40:02+00:00

In theory this should have been worked around but is still occurring. Add it to the list of things to ignore when bitbake is shutting down. Traceback (most recent call last): File "/usr/lib64/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 698, in startCallbackHandler event = self.reader.get() File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 722, in get res = self.reader.recv_bytes() File "/usr/lib64/python3.9/multiprocessing/connection.py", line 221, in recv_bytes buf = self._recv_bytes(maxlength) File "/usr/lib64/python3.9/multiprocessing/connection.py", line 426, in _recv_bytes return self._recv(size) File "/usr/lib64/python3.9/multiprocessing/connection.py", line 384, in _recv chunk = read(handle, remaining) TypeError: an integer is required (got type NoneType)' (Bitbake rev: 7a28ac4fe478bee1e52e84412da9626495f9c6c7) Signed-off-by: Richard Purdie

bitbake: server/process: Remove daemonic thread usage

2022-06-08T20:53:15+00:00

We're seeing UI deadlocks occasionally and this is possibly due to the use of a daemonic thread in the UI event queue processing. This thread could terminate holding a threading Lock() which would cause issues for the process when exitting. Change the shutdown process to handle this more cleanly. (Bitbake rev: f5ad8349a5dbff9824a89f5708cfd011d61888c9) Signed-off-by: Richard Purdie

bitbake: server/process: Avoid risk of exception deadlocks

2022-06-08T20:53:15+00:00

The open coded lock acquire/release in the UI event handler doesn't cover the case an exception occurs and if one did, it could deadlock the code. Switch to use 'with' statements which would handle this possibility. We have seen deadlocks in the UI at exit this so this removes a possible cause. (Bitbake rev: bd12792f28efd2f03510653ec947ebf961315272) Signed-off-by: Richard Purdie

bitbake: server/process: Drop unused import

2022-04-21T20:00:35+00:00

(Bitbake rev: 543315e6463f15ca7ab2b4ef3e8ed41bb4207ccf) Signed-off-by: Richard Purdie

bitbake: server/process: Disable gc around critical section

2022-04-03T16:51:26+00:00

The python gc can trigger whilst we're holding the event stream lock and when cleaning up objects, they can trigger warnings. This translates into a new event which would then need the lock and we can deadlock. Disable gc whilst we hold that lock to avoid this unfortunate and problematic situation. (Bitbake rev: 96a6303949cefd469bcf5ed250ff512271354357) Signed-off-by: Richard Purdie

bitbake: cooker/process: Fix signal handling lockups

2022-03-30T12:05:03+00:00

If a parser process is terminated while holding a write lock, then it will lead to a deadlock (see https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.terminate). With SIGTERM, we don't want to terminate holding the lock. We also don't want a SIGINT to cause a partial write to the event stream. I tried using signal masks to avoid this but it doesn't work, see https://bugs.python.org/issue47139 Instead, add a signal handler and catch the calls around the critical section. We also need a thread lock to ensure other threads in the same process don't handle the signal until all the threads are not in the lock. (Bitbake rev: a40efaa5556a188dfe46c8d060adde37dc400dcd) Signed-off-by: Richard Purdie

bitbake: server/process: Correct a typo in a comment

2022-03-28T12:33:28+00:00

(Bitbake rev: b4a157b2fe2fb481ffa40e0f32659d05dd6320c2) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie