linux/poky.git/bitbake/lib/bb/server/process.py, branch uninative-2.9

bitbake: server/process: Add extra logfile flushing

2020-08-13T07:07:00+00:00

There is the possibility data is being lost from the logfile due to data buffering. Add in a couple of extra flush calls to ensure data is being written out before the lock file is dropped. Possible fix for [YOCTO #14000]? (Bitbake rev: 596ea229a87d26a8e970c7ee77179519ad081fef) Signed-off-by: Richard Purdie

bitbake: server/process: Pass timeout/xmlrpc parameters directly to the server

2020-08-13T07:07:00+00:00

Further cleanup, just pass these settings directly. (Bitbake rev: ac2284357f1fc7044dac9c146fad218fc9906412) Signed-off-by: Richard Purdie

bitbake: server/process: Simplfy idle callback handler function

2020-08-13T07:07:00+00:00

Having the idle callbacks abstracted via the configuration object makes no sense. Its like this for historical reasons from the multiple server backends but we don't need this now so simplfy. (Bitbake rev: e56c49717355c9493b07d5fc80981a95ad8a0ec8) Signed-off-by: Richard Purdie

bitbake: server/process: Remove pointless process forking

2020-08-13T07:07:00+00:00

We already call bb.daemonize.createDaemon() in BitBakeServer so the extra multiprocessing.Process() appears to be totally unneeded and just an extra layer of forking which confuses things. Remove it. (Bitbake rev: d214e55c45f9733b3289138feec0ae3361a4a48b) Signed-off-by: Richard Purdie

bitbake: server/process: Account for xmlrpc connections

2020-07-28T11:40:11+00:00

UI control can happen via the xmlrpc connection. Account for this when timing out UI connections. This was causing issues for toaster on systems where it couldn't parse the metadata within the timeout. (Bitbake rev: fa85a8263971c25e67fa3b421c686a90e46acd87) Signed-off-by: Richard Purdie

bitbake: server/process: Fix UI first connection tracking

2020-07-28T11:40:11+00:00

We're only meant to be doing UI connection timeouts on the first connection but haveui changes for each connection. We need to add a specific variable to track this correctly and get the intended behaviour. (Bitbake rev: e7c387c2e2fb2cc3ca1dc9d2029362909c326d72) Signed-off-by: Richard Purdie

bitbake: server/process: Fix note reference -> info

2020-07-21T15:57:43+00:00

Its bb.note or logger.info, this avoids a backtrace. (Bitbake rev: 82c534ca1a1313de067b0d79c79857e89fa2764a) Signed-off-by: Richard Purdie

bitbake: server/process: Ensure UI-less servers don't sit in infinite loops

2020-07-13T11:53:22+00:00

If server startup is broken for some reason (e.g. lockfile issues) and no UI connection is made, the server will just sit inifinitely waiting. Add a timeout upon startup in the non-memory resident case so that such infinite waits are avoided. In the memory resident case, the server wouldn't have shut down in the first place or will timeout according to configuration. Since any race may mean the socket file is no longer present, ensure the unlink doesn't fault upon exit, thus ensuring any hashequiv or PRServ is removed from memory, allowing all processes to exit cleanly in such scenarios. (Bitbake rev: 39888b750df12478e8bdea6727cca112dce1df85) Signed-off-by: Richard Purdie

bitbake: server/process: Fix a rare lockfile race

2020-07-12T21:38:34+00:00

We're seeing rare occasional races on the autobuilder as if two server processes have the lockfile at the same time. We need to be extremely careful this does not happen. I think there is a potential race in this shutdown code since we delete the lockfile, then call unlockfile() which also tries to delete it. This means we may remove a lock file now held by another process if we're unlucky. Since unlockfile removes the lockfile when it can, just rely on that and remove any possible race window. An example cooker-deamonlog: --- Starting bitbake server pid 2266 at 2020-07-11 06:17:18.210777 --- Started bitbake server pid 2266 Entering server connection loop Accepting [] ([]) Processing Client Connecting Client Running command ['setFeatures', [2]] Running command ['updateConfig', XXX] Running command ['getVariable', 'BBINCLUDELOGS'] Running command ['getVariable', 'BBINCLUDELOGS_LINES'] Running command ['getSetVariable', 'BB_CONSOLELOG'] Running command ['getSetVariable', 'BB_LOGCONFIG'] Running command ['getUIHandlerNum'] Running command ['setEventMask', XXXX] Running command ['getVariable', 'BB_DEFAULT_TASK'] Running command ['setConfig', 'cmd', 'build'] Running command ['getVariable', 'BBTARGETS'] Running command ['parseFiles'] --- Starting bitbake server pid 8252 at 2020-07-11 06:17:28.584514 --- Started bitbake server pid 8252 --- Starting bitbake server pid 13278 at 2020-07-11 06:17:31.330635 --- Started bitbake server pid 13278 Running command ['dataStoreConnectorCmd', 0, 'getVar', ('BBMULTICONFIG',), {}] Running command ['getRecipes', ''] Running command ['clientComplete'] Processing Client Disconnecting Client No timeout, exiting. Exiting where it looks like there are two server processes running which should not be. In that build there was a process left sitting in memory with its bitbake.sock file missing but holding the lock (not sure why it wouldn't timeout/exit). (Bitbake rev: e1a7c1821483031b224a1570bfe834da755219cc) Signed-off-by: Richard Purdie

bitbake: server/process: Increase timeout for commands

2020-07-08T11:12:45+00:00

We're running into this timeout on loaded autobuilders in situations where things should otherwise succeed. Log a note in these cases and continue to try for longer. (Bitbake rev: e567743e70f426786ae54dcb5ab550748d9266e4) Signed-off-by: Richard Purdie