linux/poky.git/bitbake/lib/bb/server, branch sumo

bitbake: process: Rewrite multiple connection handling

2019-02-25T22:27:46+00:00

If the bitbake server recieved multiple connections, it currently closes ones it can't handle (while its dealing with another). This is rather antisocial behaviour which causes clients to quickly run through their retries and abort. Instead, queue any other connections until the current one is closed. This way the client can decide when it wants to stop waiting for the server. If the client is gone by the time we handle it, we handle that gracefully. This also fixes a number of bugs in the connection handling where connections which did drop early were badly handled causing tracebacks in the logs. Also, handle queue incomming connections in a loop to ensure that the main client handling doesn't starve that piece of the system. This code was stress tested by running 50 connection attempts in parallel at once, ensuring the code correctly handled them. (Bitbake rev: f675293d86504db66442532258d99b26333742bf) Signed-off-by: Richard Purdie

bitbake: process: Handle EWOULDBLOCK in socket connect

2019-02-25T22:27:46+00:00

Now that we set a timeout for the socket, it can return EWOULDBLOCK if a signal or other event happens to wake up even if we don't timeout. If this happens, retry the connection, else we simply see it quickly loop through the retries and abort the connection in a very short interval. (Bitbake rev: f770d6a332812031682dc6bef1a2a84da52a4c32) Signed-off-by: Richard Purdie

bitbake: process.py: Set socket timeout to 10 seconds

2019-02-25T22:27:46+00:00

The current value of 2 seconds has shown to be short in wider testing. (Bitbake rev: c4a940991f261959eb08273d2250d3866b868938) Signed-off-by: Richard Purdie

bitbake: server/process: Ensure socket has a timeout set

2019-02-25T22:27:46+00:00

We're seeing hangs in oe-selftest where server startup and shutdown are racing. The assumption was a connect would timeout however no timeout is set which can leave processes hanging. Set a short timeout for the connection to avoid this. (Bitbake rev: e53c1009356cc49c57d3b9af1e3dda6927acd78d) Signed-off-by: Richard Purdie

bitbake: server/process: Increase server startup time delay

2019-02-25T22:27:46+00:00

On loaded production systems we've seen bitbake server take over 40s to start up. Increase the timeout to 90s which tries to avoid failures in selftests. The delays come from setting up the inotify watches (31s) so can't really be avoided. After 5s delay we now warn the user we're waiting for 90s so the interactive exeperience shouldn't be much changed and its very unlikely the user would see that anyway. (Bitbake rev: 64055fdb0d8485b40ba710a762e9a0af5c4540b4) Signed-off-by: Richard Purdie

bitbake: server/process: Show last 60 lines of the log if the server didn't start

2019-02-25T22:27:46+00:00

We're seeing issues where the server doesn't start with no logs as to why. Allow the server to print the last 60 log lines just in case this shows us something useful about what is failing. (Bitbake rev: da54a3fef9cee308dfa87eea9b9638796d734abd) Signed-off-by: Richard Purdie

bitbake: server/process: Show the last 60 log lines, not the last 10

2019-02-25T22:27:46+00:00

10 log lines may not capture any full traceback, increase the number of lines to 60 which covers most tracebacks. (Bitbake rev: 340a7d91b5030e4d62680f427286fc419509879b) Signed-off-by: Richard Purdie

bitbake: server/process: print a message when no logfile

2019-02-25T22:27:46+00:00

[YOCTO #12898] There might be no bitbake-cookerdaemon.log, print a message for debugging. (Bitbake rev: cc32e703a8e68107353702edb230296503f5dad0) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: server/process: Make lockfile handling clearer

2019-02-25T22:27:46+00:00

This simplifies the code and makes it easier to read but has the same functionality. (Bitbake rev: 21dcec291d58ab7ac5d2c07186c68e01d85e6f65) Signed-off-by: Richard Purdie

bitbake: process: Flush key output to logs

2018-12-06T10:28:05+00:00

Small tweak to ensure these items are printed into the log since there is other logging code which looks for the header and this makes it clearer the server did start but is slow somewhere in startup. (Bitbake rev: e698ff1817e3536211f40af161c563d15e2ef3b5) Signed-off-by: Richard Purdie