summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/server
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: process: Show command exceptions in the server log as wellRichard Purdie2020-10-111-0/+1
| | | | | | | | | | There are autobuilder logs where the server commands are failing but we have no debug info in the server log. Improve this to try and understand what is failing. (Bitbake rev: 04d3a79226c9ea448b22f4efbab33876a72c9bdb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Note when commands complete in logsRichard Purdie2020-09-051-0/+1
| | | | | | | | | | Its hard to tell from the server logs whether commands complete or not (or how long they take). Add extra info to allow more debugging of server timeouts. (Bitbake rev: 56285ada585ec1481449522282b335bcb5a2671e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Prefix the log data with pid/time informationRichard Purdie2020-09-051-2/+2
| | | | | | | | | | Knowing which process printed which messages and the timestamp of the message is useful for debugging, so add this. Ensure the log parsing isn't affected by using search() instead of match(). (Bitbake rev: 1d043666710df1fa9d9586fd974c0371dd1514b0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure we don't keep looping if some other server ↵Richard Purdie2020-09-051-1/+20
| | | | | | | | | | | | | | | is started Showing "leftover process" messages when a new server has started and is being used by some UI is horrible. Compare the PID data from the lockfile to avoid this (and the ton of confusing log data it generates). Also, move the time.sleep() call to be after the first lock attempt, which reduces noise in the logs significantly. (Bitbake rev: ce1897a31afb5a14997bc3d2f459b90d43eecb7d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Don't show tracebacks if the lockfile is removedRichard Purdie2020-09-051-0/+6
| | | | | | | | | lsof/fuser error if the file doesn't exist. It can be deleted by something else so ignore this if it happens and loop. (Bitbake rev: b100d22ce37b7548b50e59a71802bcc903acd6ea) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure logging is flushedRichard Purdie2020-09-051-21/+23
| | | | | | | | | | | | | The cookerlog output goes to a file and its misleading to look at it and not have it up to date with what the cooker is actually doing. Ensure written data is flushed. Ultimately this should be using python's logging but that is for another day, we need simple fixes right now. (Bitbake rev: d95c3dd2b8ac50423976a7baf0a51e9580871761) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Avoid printing binary strings for leftover processesRichard Purdie2020-09-051-1/+1
| | | | | | | | | | The binary string printed into the output is ugly, parse this so the linebreaks come out in the logs and make them much more readable (I was misssing the information initially despite looking for it). (Bitbake rev: c2dd8bb434d5738fedf9019651074b90affff3b2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process.py: Handle SystemExit exception to eliminate backtraceMark Hatle2020-09-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | With an invalid layer, the desired error output should be: ERROR: The following layer directories do not exist: ERROR: /this_path_does_not_exist ERROR: Please check BBLAYERS in .../build-invalid-layer/conf/bblayers.conf Instead we were met with a backtrace: Traceback (most recent call last): File "/scratch1/fray/xilinx/poky/bitbake/bin/bitbake", line 36, in <module> cookerdata.CookerConfiguration())) ... File "/scratch1/fray/xilinx/poky/bitbake/lib/bb/cookerdata.py", line 267, in parseBaseConfiguration self.data = self.parseConfigurationFiles(self.prefiles, self.postfiles) File "/scratch1/fray/xilinx/poky/bitbake/lib/bb/cookerdata.py", line 358, in parseConfigurationFiles sys.exit(1) SystemExit: 1 (Bitbake rev: 3a2503c785a5cd9dca0dc68c3aec31b4bec7684b) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process/knotty: Improve early exception handlingRichard Purdie2020-09-021-1/+6
| | | | | | | | | | | | | | | | | | | | The new server startup code means exceptions can happen when we aren't setup to show them to the user correctly, leading to ugly tracebacks. Add in some special case handling of BBHandledException to at least ensure that common case doesn't traceback and the user sees meaningful output. In the future, the logging setup can likely be improved, as can the way runCommand handles exceptions, they all should likely become real exceptions again on the UI side. [YOCTO #14022] [YOCTO #14033] (Bitbake rev: 6059d0e77f60ddb679049bd34478f41b1ab7995d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix typo in code causing tracebacksRichard Purdie2020-08-261-1/+1
| | | | | | (Bitbake rev: 14caa3d4e5615252b9453162183980044d896d2f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Avoid bb.utils.timeoutRichard Purdie2020-08-261-24/+27
| | | | | | | | | | | | | | I have a suspicion based on process traces that the flock() call is no longer interrupted by SIGALRM and hence the timeout doesn't work. We were relying on EINTR triggering around syscalls but python is likely protecting us from that in modern versions. Re-implement this code with a different mechanism which doesn't have that potential issue. (Bitbake rev: 8eb52afdfd4c3e6478d4f8cc56e99def3f1c924c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Use sys.executable for bitbake-serverRichard Purdie2020-08-261-1/+1
| | | | | | | | | | Using sys.executable ensures we're using the same python binary for the server as the client, which is probably advisable. "bitbake-server" is left as the process name as its more distinctive in process listings. (Bitbake rev: 387a339b330e8122a62a148249beb3f064dd4e3d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: fix most undefined code picked up by pylintFrazer Clews2020-08-251-0/+1
| | | | | | | | | | Correctly import, and inherit functions, and variables. Also fix some typos and remove some Python 2 code that isn't recognised. (Bitbake rev: b0c807be5c2170c9481c1a04d4c11972135d7dc5) Signed-off-by: Frazer Clews <frazerleslieclews@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add bitbake-server and exec() a new server processRichard Purdie2020-08-251-39/+55
| | | | | | | | | | | | Trying to have a new python process forked off an original doesn't work out well and ends up having race issues. To avoid this, exec() a new bitbake server process. This starts with a fresh python interpreter and resolves various atexit and other multiprocessing issues once and for all. (Bitbake rev: 9501dd6fdd7a7c25cbfa4464cf881fcf8c049ce2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Log extra threads at exitRichard Purdie2020-08-251-0/+3
| | | | | | | | | Dump info into the logs if there are extra threads left at process exit time. (Bitbake rev: 1c9496797b753e67351bd5cb98ef2b8e9435d51e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main/server/process: Drop configuration object passingRichard Purdie2020-08-251-4/+5
| | | | | | | | | | | | | | | | | | | The first thing the UIs do is update the server config from the UI. We can just rely upon that and start the server with a standard config, removing the need to pass the confusing configuration object around as well as configParams, which contains a similar copy of some of the data. This makes memory resident bitbake work the same way as the normal mode, removing the opportunity for some class of bugs. The xmlrpcinterface and server_timeout values are passed in at server startup time now and there no longer a second option in the configuration which is effective ignored once the server starts. (Bitbake rev: 783a03330802e83c525c55522e3ee2a933bded3a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Move the socket code to server process onlyRichard Purdie2020-08-251-17/+15
| | | | | | | | | The sock object isn't used client side so we can just created it in the server process and save passing around the fd/object. (Bitbake rev: ee5d2c92dcce89ccb701e028ffc6419eb315f5ce) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Defer configuration init to after UI connectionRichard Purdie2020-08-251-17/+21
| | | | | | | | | | | | | Currently we end up parsing the base configuration multiple times as initially, the right settings haven't come from the UI. We can defer this until later in startup using runCommand as a trigger. The advantage to doing this is improved startup times and ultimately we should be able to avoid the double parse of the base configuration. (Bitbake rev: 3caa43b665604475d2c87ba505efb0b9fca9c2e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add extra logfile flushingRichard Purdie2020-08-131-13/+22
| | | | | | | | | | | | There is the possibility data is being lost from the logfile due to data buffering. Add in a couple of extra flush calls to ensure data is being written out before the lock file is dropped. Possible fix for [YOCTO #14000]? (Bitbake rev: 596ea229a87d26a8e970c7ee77179519ad081fef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Pass timeout/xmlrpc parameters directly to the serverRichard Purdie2020-08-131-4/+5
| | | | | | | | Further cleanup, just pass these settings directly. (Bitbake rev: ac2284357f1fc7044dac9c146fad218fc9906412) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Simplfy idle callback handler functionRichard Purdie2020-08-131-2/+1
| | | | | | | | | | Having the idle callbacks abstracted via the configuration object makes no sense. Its like this for historical reasons from the multiple server backends but we don't need this now so simplfy. (Bitbake rev: e56c49717355c9493b07d5fc80981a95ad8a0ec8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Remove pointless process forkingRichard Purdie2020-08-131-3/+2
| | | | | | | | | | We already call bb.daemonize.createDaemon() in BitBakeServer so the extra multiprocessing.Process() appears to be totally unneeded and just an extra layer of forking which confuses things. Remove it. (Bitbake rev: d214e55c45f9733b3289138feec0ae3361a4a48b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Account for xmlrpc connectionsRichard Purdie2020-07-281-1/+1
| | | | | | | | | | UI control can happen via the xmlrpc connection. Account for this when timing out UI connections. This was causing issues for toaster on systems where it couldn't parse the metadata within the timeout. (Bitbake rev: fa85a8263971c25e67fa3b421c686a90e46acd87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix UI first connection trackingRichard Purdie2020-07-281-1/+3
| | | | | | | | | | We're only meant to be doing UI connection timeouts on the first connection but haveui changes for each connection. We need to add a specific variable to track this correctly and get the intended behaviour. (Bitbake rev: e7c387c2e2fb2cc3ca1dc9d2029362909c326d72) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix note reference -> infoRichard Purdie2020-07-211-1/+1
| | | | | | | | Its bb.note or logger.info, this avoids a backtrace. (Bitbake rev: 82c534ca1a1313de067b0d79c79857e89fa2764a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure UI-less servers don't sit in infinite loopsRichard Purdie2020-07-131-4/+15
| | | | | | | | | | | | | | | | | | | | If server startup is broken for some reason (e.g. lockfile issues) and no UI connection is made, the server will just sit inifinitely waiting. Add a timeout upon startup in the non-memory resident case so that such infinite waits are avoided. In the memory resident case, the server wouldn't have shut down in the first place or will timeout according to configuration. Since any race may mean the socket file is no longer present, ensure the unlink doesn't fault upon exit, thus ensuring any hashequiv or PRServ is removed from memory, allowing all processes to exit cleanly in such scenarios. (Bitbake rev: 39888b750df12478e8bdea6727cca112dce1df85) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix a rare lockfile raceRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're seeing rare occasional races on the autobuilder as if two server processes have the lockfile at the same time. We need to be extremely careful this does not happen. I think there is a potential race in this shutdown code since we delete the lockfile, then call unlockfile() which also tries to delete it. This means we may remove a lock file now held by another process if we're unlucky. Since unlockfile removes the lockfile when it can, just rely on that and remove any possible race window. An example cooker-deamonlog: --- Starting bitbake server pid 2266 at 2020-07-11 06:17:18.210777 --- Started bitbake server pid 2266 Entering server connection loop Accepting [<socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([]) Processing Client Connecting Client Running command ['setFeatures', [2]] Running command ['updateConfig', XXX] Running command ['getVariable', 'BBINCLUDELOGS'] Running command ['getVariable', 'BBINCLUDELOGS_LINES'] Running command ['getSetVariable', 'BB_CONSOLELOG'] Running command ['getSetVariable', 'BB_LOGCONFIG'] Running command ['getUIHandlerNum'] Running command ['setEventMask', XXXX] Running command ['getVariable', 'BB_DEFAULT_TASK'] Running command ['setConfig', 'cmd', 'build'] Running command ['getVariable', 'BBTARGETS'] Running command ['parseFiles'] --- Starting bitbake server pid 8252 at 2020-07-11 06:17:28.584514 --- Started bitbake server pid 8252 --- Starting bitbake server pid 13278 at 2020-07-11 06:17:31.330635 --- Started bitbake server pid 13278 Running command ['dataStoreConnectorCmd', 0, 'getVar', ('BBMULTICONFIG',), {}] Running command ['getRecipes', ''] Running command ['clientComplete'] Processing Client Disconnecting Client No timeout, exiting. Exiting where it looks like there are two server processes running which should not be. In that build there was a process left sitting in memory with its bitbake.sock file missing but holding the lock (not sure why it wouldn't timeout/exit). (Bitbake rev: e1a7c1821483031b224a1570bfe834da755219cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Increase timeout for commandsRichard Purdie2020-07-081-1/+3
| | | | | | | | | | We're running into this timeout on loaded autobuilders in situations where things should otherwise succeed. Log a note in these cases and continue to try for longer. (Bitbake rev: e567743e70f426786ae54dcb5ab550748d9266e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: remove unused importsFrazer Clews2020-01-192-6/+0
| | | | | | | | | | removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: server/process: Handle BBHandledException to avoid ↵Robert Yang2019-08-161-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | unexpected exceptions The parseBaseConfiguration() raises bb.BBHandledException(), but BitBakeServer() didn't handle it, so we always got unexpected exceptions when there were errors. For example: === Case 1: * Add "print "hello"' in base.bbclass' def oe_import() function def oe_import(d): print "hello" [snip] $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Last 60 lines of server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 21 print "hello" ^ SyntaxError: Missing parentheses in call to 'print' <The first exception> During handling of the above exception, another exception occurred: <Tracebacks> <The second exception> During handling of the above exception, another exception occurred: <Tracebacks> <The third exception> During handling of the above exception, another exception occurred: <Tracebacks> [snip] Now it looks like: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: Error in compiling python function in /buildarea1/lyang1/poky/meta/classes/base.bbclass, line 21: The code lines resulting in this error were: 0001:def oe_import(d): *** 0002: print "hello" 0003: import sys 0004: 0005: bbpath = d.getVar("BBPATH").split(":") 0006: sys.path[0:0] = [os.path.join(dir, "lib") for dir in bbpath] SyntaxError: Missing parentheses in call to 'print' (base.bbclass, line 21) === Case 2: * Add 'HOSTTOOLS += "hello"' to conf/local.conf: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): <Tracebacks> [snip] During handling of the above exception, another exception occurred: [snip] <Tracebacks> ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello The error message is printed by bb.fatal() which raises bb.BBHandledException(), but BitBakeServer() doesn't handle it, so we got it. Now it looks like: ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello No unexpected exceptions anymore. [YOCTO #13267] (Bitbake rev: 6e6865e6371dbd31a136eae64cc5b1fa5f5bee33) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie2019-05-044-50/+0
| | | | | | | | | | With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-044-0/+8
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add missing exception raiseRichard Purdie2019-02-061-0/+1
| | | | | | | | | | | The intent of the code was to catch one kind of error, it was actually swallowing all exceptions and looping indefinitely. Fix it to work as intended. This explains some mystery hangs we've been seeing. (Bitbake rev: d73dbc3580faa1225d95ae4cefac4879ca3c1b2f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Rewrite multiple connection handlingRichard Purdie2018-12-261-22/+34
| | | | | | | | | | | | | | | | | | | | | | | | If the bitbake server recieved multiple connections, it currently closes ones it can't handle (while its dealing with another). This is rather antisocial behaviour which causes clients to quickly run through their retries and abort. Instead, queue any other connections until the current one is closed. This way the client can decide when it wants to stop waiting for the server. If the client is gone by the time we handle it, we handle that gracefully. This also fixes a number of bugs in the connection handling where connections which did drop early were badly handled causing tracebacks in the logs. Also, handle queue incomming connections in a loop to ensure that the main client handling doesn't starve that piece of the system. This code was stress tested by running 50 connection attempts in parallel at once, ensuring the code correctly handled them. (Bitbake rev: 02845a561b38658ac3edf5cc9c34625ed860d34f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Handle EWOULDBLOCK in socket connectRichard Purdie2018-12-261-1/+8
| | | | | | | | | | | | | Now that we set a timeout for the socket, it can return EWOULDBLOCK if a signal or other event happens to wake up even if we don't timeout. If this happens, retry the connection, else we simply see it quickly loop through the retries and abort the connection in a very short interval. (Bitbake rev: d5b0a9a302ac0eafa4f797ac15ea77db87e82b3c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process.py: Set socket timeout to 10 secondsRichard Purdie2018-12-261-1/+1
| | | | | | | | | The current value of 2 seconds has shown to be short in wider testing. (Bitbake rev: 8a1f2fcf35d61d83bbafa8fa3ae215fd5f51728b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure socket has a timeout setRichard Purdie2018-12-151-0/+2
| | | | | | | | | | | We're seeing hangs in oe-selftest where server startup and shutdown are racing. The assumption was a connect would timeout however no timeout is set which can leave processes hanging. Set a short timeout for the connection to avoid this. (Bitbake rev: f02114cb70e8f6f1d32e19c02b758fe0aadecd19) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Increase server startup time delayRichard Purdie2018-12-081-1/+4
| | | | | | | | | | | | | | | | | On loaded production systems we've seen bitbake server take over 40s to start up. Increase the timeout to 90s which tries to avoid failures in selftests. The delays come from setting up the inotify watches (31s) so can't really be avoided. After 5s delay we now warn the user we're waiting for 90s so the interactive exeperience shouldn't be much changed and its very unlikely the user would see that anyway. (Bitbake rev: 7a4e105093c3080c1087cd06abc2883852a23e28) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Flush key output to logsRichard Purdie2018-12-051-0/+4
| | | | | | | | | | Small tweak to ensure these items are printed into the log since there is other logging code which looks for the header and this makes it clearer the server did start but is slow somewhere in startup. (Bitbake rev: 7c0b84cac892744225fa0462f918ea9a79b356cb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Handle short readsRichard Purdie2018-12-051-3/+3
| | | | | | | | | Its possible the read may return a smaller number of characters. Remove the possibility by using a single character to signal the server is ready. (Bitbake rev: 767c9596d129d21ddf5d2e00c55f0a0525c641dc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Make lockfile handling clearerRichard Purdie2018-12-051-4/+6
| | | | | | | | | This simplifies the code and makes it easier to read but has the same functionality. (Bitbake rev: a0b9cfaf2d03fd047a79d32e668001718d02c4bf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix ConnectionRefusedError tracebacksRichard Purdie2018-12-011-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Improve connetion refused error handling: NOTE: Retrying server connection... NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection... (Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 471, in connectProcessServer sock.connect(os.path.basename(sockname)) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 502, in connectProcessServer os.close(i) TypeError: an integer is required (got type NoneType) ) WARNING: /home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> logger.info("Retrying server connection... (%s)" % traceback.format_exc()) (Bitbake rev: 6998fd816ceb0034c852a8fb994901fdf1975cfd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Show the last 60 log lines, not the last 10Richard Purdie2018-11-271-2/+2
| | | | | | | | | 10 log lines may not capture any full traceback, increase the number of lines to 60 which covers most tracebacks. (Bitbake rev: b8538c2088c199157404712a60a6740c926ce312) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Show last 60 lines of the log if the server didn't ↵Richard Purdie2018-11-271-0/+6
| | | | | | | | | | | | start We're seeing issues where the server doesn't start with no logs as to why. Allow the server to print the last 60 log lines just in case this shows us something useful about what is failing. (Bitbake rev: c8c80b404e38fe96f65d6314cd95f4069319f3d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: print a message when no logfileRobert Yang2018-11-191-0/+4
| | | | | | | | | | | [YOCTO #12898] There might be no bitbake-cookerdaemon.log, print a message for debugging. (Bitbake rev: 4adc582d2df7fdb9e51c4ebb5e66bbd21165b4dc) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/server: Avoid UnboundLocalError tracebackRichard Purdie2018-11-141-3/+3
| | | | | | | | | | | | | Traceback (most recent call last): File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 490, in connectProcessServer if command_chan_recv: UnboundLocalError: local variable 'command_chan_recv' referenced before assignment (Bitbake rev: ad79fadd855f5c10242ed17e9e0f3eb0274f26d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix unclosed socket warnings upon server connection ↵Richard Purdie2018-11-071-8/+7
| | | | | | | | | | | | | | | | | | | | | | | refused Extend the server error handling to avoid: Reconnecting to bitbake server... NOTE: Retrying server connection... (Traceback (most recent call last): File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 457, in connectProcessServer sock.connect(os.path.basename(sockname)) ConnectionRefusedError: [Errno 111] Connection refused ) WARNING: /home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> logger.info("Retrying server connection... (%s)" % traceback.format_exc()) NOTE: Starting bitbake server... (Bitbake rev: afee3f594e1510051a0b18e430e92549caf72fa2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Various server startup logging fixesRichard Purdie2018-09-051-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | There were various problems in the server startup loggin: a) stdout/stderr were not being flushed before forking which could potentially duplicate output b) there were separate buffers for stdout/stderr leading to confusing logs where the entries could be reordered. This was particularly confusing due to the separator the logs use to idendify new messages c) an fd wasn't being closed during server startup meaning if the server failed to start, the closed fd wasn't detected as it was held open by the other reference d) If the pipe was detected as being closed, the code incorrectly retried server startup e) The event code would remap stdout/stderr without flushing them, leading to lose log messages (Bitbake rev: 0594faa0b52ce5dbd948d836d88617d38d9862d1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Always place the server logfile in the build directoryuninative-2.2Richard Purdie2018-07-251-2/+3
| | | | | | | | | | Currently the bitbake-cookerdaemon.log is placed into cwd. This seems like a bad idea, we can place it in the build directory alongside the lockfile that represents the server instead. (Bitbake rev: 1620dbc48ffb2a882371cf9174a7b12648befc8a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: don't stop file notifier when cooker is shutdownRobert Yang2018-02-141-0/+2
| | | | | | | | | | | | | | | | | | | | | It should be live/exited with server rather than cooker, fixed: $ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quilt [snip] ERROR: When reparsing /workspace1/lyang1/poky/meta/recipes-connectivity/openssl/openssl_1.0.2m.bb.do_package, the basehash value changed from c216f7f4fdd3cf4a0b10b975a636426c to d5a8e9431ab261381752d7a64c7b2fa9. The metadata is not deterministic and this needs to be fixed. [snip] This is because the server doesn't know local.conf is changed since the notifiers are stopped, so it doesn't reparse, and then we would get the errors, let the notifiers live/exited with server can fix the problem. (Bitbake rev: a6a641cb9c5f3abe901b150da915372e295383d7) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-24 04:11:13 +0000