linux/poky.git/bitbake/lib/bb/server, branch 3.2_M2

bitbake: server/process: Account for xmlrpc connections

2020-07-28T11:40:11+00:00

UI control can happen via the xmlrpc connection. Account for this when timing out UI connections. This was causing issues for toaster on systems where it couldn't parse the metadata within the timeout. (Bitbake rev: fa85a8263971c25e67fa3b421c686a90e46acd87) Signed-off-by: Richard Purdie

bitbake: server/process: Fix UI first connection tracking

2020-07-28T11:40:11+00:00

We're only meant to be doing UI connection timeouts on the first connection but haveui changes for each connection. We need to add a specific variable to track this correctly and get the intended behaviour. (Bitbake rev: e7c387c2e2fb2cc3ca1dc9d2029362909c326d72) Signed-off-by: Richard Purdie

bitbake: server/process: Fix note reference -> info

2020-07-21T15:57:43+00:00

Its bb.note or logger.info, this avoids a backtrace. (Bitbake rev: 82c534ca1a1313de067b0d79c79857e89fa2764a) Signed-off-by: Richard Purdie

bitbake: server/process: Ensure UI-less servers don't sit in infinite loops

2020-07-13T11:53:22+00:00

If server startup is broken for some reason (e.g. lockfile issues) and no UI connection is made, the server will just sit inifinitely waiting. Add a timeout upon startup in the non-memory resident case so that such infinite waits are avoided. In the memory resident case, the server wouldn't have shut down in the first place or will timeout according to configuration. Since any race may mean the socket file is no longer present, ensure the unlink doesn't fault upon exit, thus ensuring any hashequiv or PRServ is removed from memory, allowing all processes to exit cleanly in such scenarios. (Bitbake rev: 39888b750df12478e8bdea6727cca112dce1df85) Signed-off-by: Richard Purdie

bitbake: server/process: Fix a rare lockfile race

2020-07-12T21:38:34+00:00

We're seeing rare occasional races on the autobuilder as if two server processes have the lockfile at the same time. We need to be extremely careful this does not happen. I think there is a potential race in this shutdown code since we delete the lockfile, then call unlockfile() which also tries to delete it. This means we may remove a lock file now held by another process if we're unlucky. Since unlockfile removes the lockfile when it can, just rely on that and remove any possible race window. An example cooker-deamonlog: --- Starting bitbake server pid 2266 at 2020-07-11 06:17:18.210777 --- Started bitbake server pid 2266 Entering server connection loop Accepting [] ([]) Processing Client Connecting Client Running command ['setFeatures', [2]] Running command ['updateConfig', XXX] Running command ['getVariable', 'BBINCLUDELOGS'] Running command ['getVariable', 'BBINCLUDELOGS_LINES'] Running command ['getSetVariable', 'BB_CONSOLELOG'] Running command ['getSetVariable', 'BB_LOGCONFIG'] Running command ['getUIHandlerNum'] Running command ['setEventMask', XXXX] Running command ['getVariable', 'BB_DEFAULT_TASK'] Running command ['setConfig', 'cmd', 'build'] Running command ['getVariable', 'BBTARGETS'] Running command ['parseFiles'] --- Starting bitbake server pid 8252 at 2020-07-11 06:17:28.584514 --- Started bitbake server pid 8252 --- Starting bitbake server pid 13278 at 2020-07-11 06:17:31.330635 --- Started bitbake server pid 13278 Running command ['dataStoreConnectorCmd', 0, 'getVar', ('BBMULTICONFIG',), {}] Running command ['getRecipes', ''] Running command ['clientComplete'] Processing Client Disconnecting Client No timeout, exiting. Exiting where it looks like there are two server processes running which should not be. In that build there was a process left sitting in memory with its bitbake.sock file missing but holding the lock (not sure why it wouldn't timeout/exit). (Bitbake rev: e1a7c1821483031b224a1570bfe834da755219cc) Signed-off-by: Richard Purdie

bitbake: server/process: Increase timeout for commands

2020-07-08T11:12:45+00:00

We're running into this timeout on loaded autobuilders in situations where things should otherwise succeed. Log a note in these cases and continue to try for longer. (Bitbake rev: e567743e70f426786ae54dcb5ab550748d9266e4) Signed-off-by: Richard Purdie

bitbake: lib: remove unused imports

2020-01-19T13:31:05+00:00

removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews Signed-off-by: Richard Purdie

bitbake: bitbake: server/process: Handle BBHandledException to avoid unexpected exceptions

2019-08-16T09:05:32+00:00

The parseBaseConfiguration() raises bb.BBHandledException(), but BitBakeServer() didn't handle it, so we always got unexpected exceptions when there were errors. For example: === Case 1: * Add "print "hello"' in base.bbclass' def oe_import() function def oe_import(d): print "hello" [snip] $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Last 60 lines of server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 21 print "hello" ^ SyntaxError: Missing parentheses in call to 'print' During handling of the above exception, another exception occurred: During handling of the above exception, another exception occurred: During handling of the above exception, another exception occurred: [snip] Now it looks like: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: Error in compiling python function in /buildarea1/lyang1/poky/meta/classes/base.bbclass, line 21: The code lines resulting in this error were: 0001:def oe_import(d): *** 0002: print "hello" 0003: import sys 0004: 0005: bbpath = d.getVar("BBPATH").split(":") 0006: sys.path[0:0] = [os.path.join(dir, "lib") for dir in bbpath] SyntaxError: Missing parentheses in call to 'print' (base.bbclass, line 21) === Case 2: * Add 'HOSTTOOLS += "hello"' to conf/local.conf: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): [snip] During handling of the above exception, another exception occurred: [snip] ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello The error message is printed by bb.fatal() which raises bb.BBHandledException(), but BitBakeServer() doesn't handle it, so we got it. Now it looks like: ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello No unexpected exceptions anymore. [YOCTO #13267] (Bitbake rev: 6e6865e6371dbd31a136eae64cc5b1fa5f5bee33) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: bitbake: Drop duplicate license boilerplace text

2019-05-04T09:44:10+00:00

With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie

bitbake: bitbake: Add initial pass of SPDX license headers to source code

2019-05-04T09:44:04+00:00

This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie