summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/server
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: server/process: Increase runCommand timeoutRichard Purdie2017-09-251-1/+1
| | | | | | | | | | | | We were seeing cases where we could hit the 5s timeout on large/fast machines running many different tasks at once. Increase this to 30s since the main connection timeout path should no longer hit this slow path. [YOCTO #12116] (Bitbake rev: 131d4b8a5834781a93ed41e2967d8dcd4d80f29a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: ensure server failure log is limited to current sessionPaul Eggleton2017-09-121-3/+25
| | | | | | | | | | | | | | | | Printing the last 10 lines of bitbake-cookerdaemon.log when the server fails to start can sometimes result in printing the output from a previous run, which could lead the user completely down the wrong path in terms of the cause of the failure. Use a known start text containing the time which we can then look for when scanning through the log, and then grab the last 10 lines of that part instead. Fixes [YOCTO #11903]. (Bitbake rev: 567f2cf1bc455b4f3cfb1cbd7f25145360b05a62) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process/cooker: Improve readypipe handlingRichard Purdie2017-09-121-4/+13
| | | | | | | | | | | | | Issues in start are not being correctly detected by the current readypipe code. Change it to use specific "ready" or "fail" messages to correctly determine the correct failure mode and avoid bitbake seeming to hang (it does currently timeout eventually). [YOCTO #12062] (Bitbake rev: 60d4791e3dd05729d2a2adf6f3b203c80d466a73) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Change to consistent prefile/postfile handlingRichard Purdie2017-08-311-6/+0
| | | | | | | | | | | | | | Currently the original prefile and postfile passed when starting bitbake server are 'sticky'. With the new memory resident model this doesn't make sense as the server the system is started with isn't special. This patch changes the code so the prefile/postfile are used if specified on the commandline and not used otherwise. This makes the behaviour much more predictable and expected and as an added bonus simplifies the code. (Bitbake rev: 638d366234fad78f283d3a13a12b07cb0ccbe914) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Clean up connection retry logicRichard Purdie2017-08-241-4/+4
| | | | | | | | | | | | | | | | | | Its possible for a connection to connect to the server as its shutting down but before its removed the socket file. This patch: a) Removes the socket file earlier to avoid connections. b) Handles EOFError in initial connections gracefully. These occur if the socket is closed during the server shutdown. c) Ensure duplicate events aren't shown on the console. This makes debugging these issues very very confusing. With these changes the backtrace that was concerning users is hidden and the server works as expected with a reconnect when it catches it in a bad state. (Bitbake rev: f45196cf84669723382730944dddc7eaf50826f2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Ensure we call select() to know which fds to readRichard Purdie2017-08-221-1/+1
| | | | | | | | | | | | | | | | | | | There is an interesting bug in the current code where a sync command is not seen until the current async command completes, by which time the UI may have shut down. The reason is that if there are idle commands, we may not end up sleeping in the select call at all, partiularly under heavy load like parsing. Fix this by calling select with a zero timeout so that we see active fds and know to read from them. This fixes various problems toaster was having with the recent server changes. [YOCTO #11898] (Bitbake rev: bbcce58e824b2793abf50efa52db158ae16e23e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Increase server startup timeoutRichard Purdie2017-08-161-1/+1
| | | | | | | | | | We're seeing the server fail to start within 8s on heavily loaded autobuilders so increase this timeout to 30s which should be more than enough time. (Bitbake rev: 8d4c120ec46d6d7a54947c64d33e18cb60b60505) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Improve client disconnectsRichard Purdie2017-08-161-15/+23
| | | | | | | | | | There have been cases where the server could loop indefinitely and incorrectly handle client disconnects. In the EOFError case, ensure a full disconnect happens in the alternative disconnect path to avoid this. (Bitbake rev: 5e267f14bb0155889615f567a920af4a37eb3c6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure we don't loop on client EOFErrorRichard Purdie2017-08-131-2/+2
| | | | | | | | | | | | | | The server currently crashes if we hit an EOFError due to controllersock still being in ready and the continue meaning ready isn't re-evaluated. Setting the value to False can mean the shutdown code doesn't handle the situation cleanly. Clear ready to avoid the crash/loop instead and handle any OSError whilst we're in here. (Bitbake rev: 2bc47e887c3b41417edaa89a8708c223fd2085de) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Handle BB_SERVER_TIMEOUT = -1 for no server timeoutRobert Yang2017-08-091-1/+2
| | | | | | | | | Make BB_SERVER_TIMEOUT = -1 mean no unload forever. (Bitbake rev: 923d5567be6fd9969ca74c166f36817ec09305e3) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Fix disconnect when BB_SERVER_TIMEOUTRobert Yang2017-08-091-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed: $ export BB_SERVER_TIMEOUT=10000 $ bitbake --server-only $ bitbake --status-only [snip] File "/buildarea/lyang1/poky/bitbake/lib/bb/server/process.py", line 472, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) OSError: [Errno 9] Bad file descriptor And: $ export BB_SERVER_TIMEOUT=10000 $ bitbake --server-only -B localhost:-1 $ bitbake --status-only # Everything is fine in first run $ bitbake --status-only [snip] File "/buildarea/lyang1/poky/bitbake/lib/bb/server/process.py", line 472, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) OSError: [Errno 9] Bad file descriptor This was because self.controllersock was not set to False, so it still ran sock.recvmsg() when sock was closed. And also need set command_channel to Flase, otherwise the self.command_channel.get() will always run when EOF, and cause infinite loop. (Bitbake rev: 7b739a38601b053d9bea4df2c0b44a952ab670c4) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Add some extra server startup logsRichard Purdie2017-07-311-1/+4
| | | | | | | | | | We have cases where the server is being started but we're not seeing any messages from it. Add some earlier logging so we can try and better understand where issues may be occurring. (Bitbake rev: 10a6a48c0c5a2fe2051e90143e66075356853971) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Reorder server command processing and handle EOFErrorRichard Purdie2017-07-311-13/+18
| | | | | | | | | | | | | If the connection control socket and the command channel close together, we can race and hit EOFError exceptions before we close the channel. Reorder the code to handle this in the correct order and ignore the EOFError exceptions as they mean the client is disconnecting and shouldn't terminate the server. (Bitbake rev: 974281ed72d8366baa16ee85f7e93970689b5956) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Clean up server communication timeout errorsRichard Purdie2017-07-301-12/+3
| | | | | | | | | | | | | | | | | | | | | This timeout path was commonly hit due to errors starting the server. Now we have a better way to handle that, the retry logic can be improved and cleaned up. This patch: * Makes the timeout 5s rather than intervals of 1s with a message. Paul noted some commands can take around 1s to run on a server which has just been started on a loaded system. * Allows a broke connection to exit immediately rather than retrying something which will never work. * Drops the Ctrl+C masking, we shouldn't need that anymore and any issues would be better handled in other ways. This should make things clearer and less confusing for users and is much cleaner code too. (Bitbake rev: 8633b7cd03cfaba3e0359aa5da22fc76b66768c7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Don't leak open pipes upon reconnectionRichard Purdie2017-07-301-1/+18
| | | | | | | | | If we reconnect to the server, stop leaking pipes and clean up after ourselves. (Bitbake rev: f41e4e971e807157be68cf4496580494b8b60643) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process/cooker: Allow UI process to know if the cooker was started ↵Richard Purdie2017-07-301-2/+18
| | | | | | | | | | | | | | | | | | | successfully Currently if the server fails to start, the user sees no error message and the server will be repeatedly attempted to be started until some longer timeouts expire. There are error messages in the cookerdeamon log but nobody thinks to look there. Add in a pipe which can be used to tell the starting process whether the cooker did actually start or not. If it fails to start, no further attempts can be made and if present, the log file can be shown to the user. [YOCTO #11834] (Bitbake rev: 57000d44beb1aeba37dfc70782b0d6418943acc5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Move socket keep alive into BitBakeProcessServerConnectionRichard Purdie2017-07-301-4/+6
| | | | | | | | | This cleans up the socket keep alive into better class structured code and adds cleanup of the open file descriptors upon shutdown. (Bitbake rev: 77fd3a3a29a569e212374b27aea742ddbaafcdd5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Allow BBUIEventQueue to exit cleanlyRichard Purdie2017-07-301-3/+8
| | | | | | | | | Currently the monitoring thread exits with some error code or runs indefinitely. Allow closure of the pipe its monitoring to have the thread exit cleanly/silently. (Bitbake rev: 930d077637928213e13a07c78fee3bf7a8c37ebf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Ensure ConnectionReader/Writer have fileno() and close() ↵Richard Purdie2017-07-301-0/+9
| | | | | | | | | | | methods Expose the underlying close() and fileno() methods which allow connection monitoring and cleanup. (Bitbake rev: f79187f4ebfad7969be47b429995e7f7a3e33c1e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Change timeout warning to a noteRichard Purdie2017-07-241-1/+1
| | | | | | | | | | | The warning message currently shown can occur more frequently than previously if a previous bitbake server is shutting down and we're reconnecting to a new server. Change it to a note message to match the higher level connection logging retry messages and so as not to interfer with selftests. (Bitbake rev: b7514340cd6a2753eb217b059229bb279c3849ec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/process: Drop server_main functionRichard Purdie2017-07-241-1/+21
| | | | | | | | | | | | | | Now that there is only one server, this abstraction is no longer needed and causes indrection/confusion. The server shutdown is also broken with the cooker post_server calls happening too late, leading to "lock held" warnings in the logs if PRServ is enabled. Remove the abstraction and put the shutdown calls in the right order with respect to the locking. (Bitbake rev: c0ddde7cf680225127d6285685652b905ed176c3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process.py: fix self.bitbake_lock.write()Robert Yang2017-07-211-2/+2
| | | | | | | | | | | | | | | | There is no global var "configuration", so the old code hang at self.bitbake_lock.write(), and nothing wrote to bitbake.lock. I didn't figure out why it hang (but not print errors). Reproducer: $ bitbake -B localhost:-1 world -k Check bitbake.log, there was nothing, now fixed. (Bitbake rev: dbdd9010663cd8dcb328e1b6fb40e3a777293bc5) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server: Rework the server API so process and xmlrpc servers coexistRichard Purdie2017-07-215-701/+705
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the way bitbake server works quite radically. Now, the server is always a process based server with the option of starting an XMLRPC listener on a specific inferface/port. Behind the scenes this is done with a "bitbake.sock" file alongside the bitbake.lock file. If we can obtain the lock, we know we need to start a server. The server always listens on the socket and UIs can then connect to this. UIs connect by sending a set of three file descriptors over the domain socket, one for sending commands, one for receiving command results and the other for receiving events. These changes meant we can throw away all the horrid server abstraction code, the plugable transport option to bitbake and the code becomes much more readable and debuggable. It also likely removes a ton of ways you could hang the UI/cooker in weird ways due to all the race conditions that existed with previous processes. Changes: * The foreground option for bitbake-server was dropped. Just tail the log if you really want this, the codepaths were complicated enough without adding one for this. * BBSERVER="autodetect" was dropped. The server will autostart and autoconnect in process mode. You have to specify an xmlrpc server address since that can't be autodetected. I can't see a use case for autodetect now. * The transport/servetype option to bitbake was dropped. * A BB_SERVER_TIMEOUT variable is added which allows the server to stay resident for a period of time after the last client disconnects before unloading. This is used if the -T/--idle-timeout option is not passed to bitbake. This change is invasive and may well introduce new issues however I believe the codebase is in a much better position for further development and debugging. (Bitbake rev: 72a3dbe13a23588e24c0baca6d58c35cdeba3f63) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server: Remove base classes and inline codeRichard Purdie2017-07-213-86/+80
| | | | | | | | | In preparation for rewriting this code, expand the relatively useless base classes into the code itself. (Bitbake rev: a1c6151420d86bac658c08ae714647062edd6ef2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/command: Allow UI to request the UI eventhander IDRichard Purdie2017-07-211-1/+6
| | | | | | | | | | | | The UI may want to change its event mask however to do this, it needs the event handler's ID. Tweak the code to allow this to be stored and add a command to query it. Use the new command in the process server backend. (Bitbake rev: f8cf2cb58b80ce74f756a11a9773b6b0e78d51ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/server/process: Handle EINTR on idle_commands selectAníbal Limón2017-07-181-1/+6
| | | | | | | | | | | | | If a signal is sent like SIGWINCH the select could be interrupted so ignore the InterruptError like in XMLRPC server [1]. [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/server/xmlrpc.py#n307 (Bitbake rev: 96bb174325493764718c61c12c943c37b882cd61) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/xmlrpc: Add Heartbeat event supportRichard Purdie2017-07-081-0/+26
| | | | | | | | | | | | | | When heartbeat event support was added it was only added to process.py. Add it to server/xmlrpc too. There is duplicated code however since we're likely to combine the server abstractions soon its not worth worrying about now. This ensures the backends have the same event support. [YOCTO #10741] (Bitbake rev: 00bf2e60222767b4dee84fb3f958732a83544e80) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Queue offline events for the UIRichard Purdie2017-07-082-2/+2
| | | | | | | | | | | Messages printed when no UI is connected (e.g. memres) are currently lost. Use the existing queue mechanism to queue these until a UI attaches, then replay them. This isn't ideal but better than the current situation of losing them entirely. (Bitbake rev: cb241fb8544dfb05646dbae1a1b04e17878a466c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix waitEvent() calls with 0 timeoutRichard Purdie2017-07-081-0/+2
| | | | | | | | | | | | | | | You might think Queue.Queue.get(True, 0) would return an event immediately if present and otherwise return. It doesn't, it immediately "times out" and returns with nothing from the queue. The behaviour we want is not to wait but return anything present which is what .get(False) does so map to this. This fixes some odd behaviour observed in some of the tinfoil selftests. (Bitbake rev: 412bfab8721ea317898a1974f6a7a0d0bea763df) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/server/process.py: ProcessEventQueue add close of _writer pipeAníbal Limón2017-01-231-1/+3
| | | | | | | | | | | | Call explicity close in _writer to avoid fd leakage because isn't called on Queue.close() [YOCTO #10873] (Bitbake rev: 12dfac442d2d5674198485fbeb708a01d215c576) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar calls (take 2)Joshua Lock2016-12-161-1/+1
| | | | | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (a follow on patch to fix up a few recent introductions) (Bitbake rev: f9961fd5beb31d5ab9656a5be59f7ab3effef2f0) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: don't change UI process signal handler on terminatePaul Eggleton2016-12-141-1/+0
| | | | | | | | | | | | | | | | | | | | | On terminating the connection to the server, we were disabling SIGINT - and this is executed on the UI side. I'm not sure whether the intention here was to undo the SIGINT disabling we did in the server, and it was just a mistake that it disabled rather than restored and it's run on the wrong side, or whether we wanted to stop the user from breaking out of the shutdown code - the commit message provides no clues either way. Regardless, we do not want to permanently disable Ctrl+C here - it's legitimate to terminate the connection to the server and then re-establish it within the same process; at least currently, devtool modify by virtue of using tinfoil in two separate parts of the code does this, and the result of this disabling is that during the second tinfoil usage we can potentially be parsing all recipes without the ability to easily interrupt the process. (Bitbake rev: 58c60a951229dcbd8253863fb24228d046c23f6e) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/xmlrpc: send back 503 response with correct encodingPaul Eggleton2016-12-141-1/+1
| | | | | | | | | | | | If you send back a string here you get "TypeError: 'str' does not support the buffer interface" errors in bitbake-cookerdaemon.log and "IncompleteRead(0 bytes read, 22 more expected)" errors on the client side. (Bitbake rev: 0d659a7dfe5fb096f8aa4380320f9e2a464b3cb5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker process: fire heartbeat event at regular time intervalsPatrick Ohly2016-12-071-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | The intended usage is for recording current system statistics from /proc in buildstats.bbclass during a build and for improving the BB_DISKMON_DIRS implementation. All other existing hooks are less suitable because they trigger at unpredictable rates: too often can be handled by doing rate-limiting in the event handler, but not often enough (for example, when there is only one long-running task) cannot because the handler does not get called at all. The implementation of the new heartbeat event hooks into the cooker process event queue. The process already wakes up every 0.1s, which is often enough for the intentionally coarse 1s delay between heartbeats. That value was chosen to keep the overhead low while still being frequent enough for the intended usage. If necessary, BB_HEARTBEAT_EVENT can be set to a float specifying the delay in seconds between these heartbeat events. (Bitbake rev: 7cf22ea057d28c54bd98dc1ab7a43402a29ff1f5) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: implement idle timeout for xmlrpc serverEd Bartosh2016-07-211-3/+16
| | | | | | | | | | | | | | Idle timeout can be specified either by -T/--idle-timeout option or by sessing BBTIMEOUT environment variable. Bitbake xmlrpc server will unload itself when timeout exprired, i.e. when server is idle for more than <idle timeout> seconds. [YOCTO #5534] (Bitbake rev: 5fa0b3a19e7d0f40790f737abeddf499d53f1f6a) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: xmlrpc: implement check of connection to serverEd Bartosh2016-07-201-0/+17
| | | | | | | | | | | | | | | | | Implemented check_connection function. The purpose of this function is to check if bitbake server is accessible and functional. To check this this function tries to connect to bitbake server and run getVariable command. This API is going to be used to implement autoloading of bitbake server. [YOCTO #5534] (Bitbake rev: 1a18f5ceb478f766b53850451549333f655621ea) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: xmlrpc: add parameter use_builtin_typesEd Bartosh2016-06-021-1/+1
| | | | | | | | | | | | Added use_builtin_types parameter to XMLRPCProxyServer.__init__ to fix this error: ERROR: Could not connect to server 0.0.0.0:37132 : __init__() got an unexpected keyword argument 'use_builtin_types' (Bitbake rev: ceb6e5bd33a25c45c2afe1559b9394c466db8a92) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Convert to python 3Richard Purdie2016-06-022-27/+29
| | | | | | | | | Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix missing log messages issueRichard Purdie2016-05-141-4/+4
| | | | | | | | | | | | | | | | | Currently if the server dies, its possible that log messages are never displayed which is particularly problematic if one of those messages is the exception and backtrace the server died with. Rather than having the event queue exit as soon as the server disappears, we should pop events from the queue until its empty before exiting. This patch tweaks that code so that even if the server is dead and we're going to exit, we return any events left in the pipe. This makes debugging certain failures much easier. (Bitbake rev: 29f6ade68fb2b506a23a7eb3a00cdcffa291b362) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: xmlrpc: set single use mode differentlyEd Bartosh2016-03-242-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently xmlrpc server implicitly sets itself into single use mode when bitbake server is started with anonymous port (0) or no port is provided in command line. In this mode bitbake shuts down xmlrpc server after build is done. This assumption is incorrect in some cases. For example Toaster uses bitbake in this mode and expects xmlrpc server to stay in memory. Till recent changes single use mode was always unset due to the bug. When the bug was fixed it broke toaster builds as Toaster couldn't communicate with bitbake server in single use mode. Reimplemented logic of setting single use mode. The mode is explicity set when --server-only command line parameter is not provided to bitbake. It doesn't depend on the port number anymore. [YOCTO #9275] [YOCTO #9240] [YOCTO #9252] (Bitbake rev: afc0dd5c532684f6201b1e12bbf4c226ea19062d) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Try connecting 4 times before giving upLucas Dutra Nunes2016-03-091-4/+6
| | | | | | | | | | Instead of trying one time with a timeout of 20 seconds try 4 times with a timeout of 5 seconds, to account for a slow server start. (Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596) Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: xmlrpc: fix bug in setting XMLRPCServer.single_useEd Bartosh2016-03-091-3/+1
| | | | | | | | | | | | XMLRPCServer.single_use attribute was always set to False. This caused xmlrpc server to keep running after build is done as BitBakeServerCommands.removeClient only shuts down server if its single_use attribute is set to True. (Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Set process names to be meaninfulRichard Purdie2016-01-301-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | This means that when you view the process tree, the processes have meaningful names, aiding debugging: $ pstree -p 30021 bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593) ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634) └─{ProcessEQueue}(115591) $ pstree -p 30021 bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335) ├─PRServ(117332)───{PRServ Handler}(117333) ├─Parser-1:2(117336) └─{ProcessEQueue}(117331) Applies to parse threads, PR Server, cooker, the workers and execution threads, working within the 16 character limit as best we can. Needed to tweak the bitbake-worker magic values to tell the workers apart. (Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: xmplrpc: split connect methodEd Bartosh2016-01-062-2/+5
| | | | | | | | | | | | | | Current code in connect method sets up event queue, which requires registering UI handler. This functionality may not be needed for some operations, e.g. for server termination. Moved functionality of setting up event queue in from 'connect' method to 'setupEventQueue' in BitBakeXMLRPCServerConnection class. (Bitbake rev: 4429871da76d6bd29e023ff42740fe7daa6b40fa) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: uievent: add error to registerEventHandler returnEd Bartosh2016-01-061-2/+2
| | | | | | | | | | | | | | | Current code throws Exception("Could not register UI event handler") if event handler can't be registered. The real reason of this is that cooker is in busy state. Error message lacks information about this. Added error message to the return value of registerEventHandler. Included returned error message into the log message and exception text. (Bitbake rev: 07de1ca7d57dcd0cc37406feae2949da12a3fa7a) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Handle SIGTERM more gracefullyRichard Purdie2015-09-091-0/+9
| | | | | | | | | | | | | | | | | | | | | Currently if you send a SIGTERM to the bitbake UI process, the system basically hangs if tasks are executing. This is because the server process doesn't actually try any kind of shutdown before exiting. This patch trys executing a stateForceShutdown command first, which is enough to stop any active tasks before the system exits. I also noticed that terminate can execute multiple times, once at SIGTERM from the handler and once from the real exit. Double execution leads to stack traces and potential hangs (writes to dead pipes), so ensure the code only can run once. With these fixes, bitbake much more correctly deals with SIGTERM to the UI process. (Bitbake rev: 1032ddddbe3241da02ebb3608a1c40f9123b9e80) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/server: Add _uiready flag to handle missing error messagesRichard Purdie2015-09-042-2/+2
| | | | | | | | | | | | | | | | | | | | | If you start and suspend a bitbake execution so the bitbake lock is held, then try and run "bitbake -w '' X", you will see bitbake return an error exit code but print no message about what happened at all. The reason is that the -w option creates a "UI" which swallows the messages. The code which handles this exit failure mode thinks a UI has printed the messages and therefore doesn't do so. This adds in an extra parameter to the UI registration code so that we can figure out whether its a primary UI or not and base decisions on whether to display information on that instead. This fixes the error shown above and some bizarre failures on the Yocto Project Autobuilder. [YOCTO #8239] (Bitbake rev: d1d60a68c2de40c2984d5040d14251c1be121b0b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Fix default function parameter assignment to a listPaul Eggleton2015-08-192-3/+8
| | | | | | | | | | | | | | With python you should not assign a list as the default value of a function parameter - because a list is mutable, the result will be that the first time a value is passed it will actually modify the default. Reference: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments (Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Don't log BBHandledExceptionRichard Purdie2015-06-231-2/+3
| | | | | | | | | | | | If we see a BBHandledException in the idle handler, the understanding is the system handled it, printing a log and traceback is just confusing. Therefore only print these in the cases where its an unknown/unhandled exception. (Bitbake rev: d88ecc2bc44dce8fd92ca3a2c0fd4124a5e464fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/server: Fix up 100% CPU usage at idleRichard Purdie2015-03-102-0/+6
| | | | | | | | | | | | | | | The recent inotify changes are causing a 100% cpu usage issue in the idle handlers. To avoid this, we update the idle functions to optionally report a float value which is the delay before the function needs to be called again. 1 second is fine for the inotify handler, in reality its more like 0.1s due to the default idle function sleep. This reverts performance regressions of 1.5 minutes on a kernel build and ~5-6 minutes on a image from scratch. (Bitbake rev: 0e0ba408c2dce14a0fabd3fdf61d8465a031495b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-19 09:56:23 +0000