linux/poky.git/bitbake/lib/bb/server, branch 1.5_M4

bitbake: server/xmlrpc: stop server on client exit

2013-09-01T14:51:11+00:00

If the server only expects a single client to connect (i.e. no bind parameter set, so there is no way for the clients to get the server port), stop the server after the first client exits. (Bitbake rev: eb6bae56f62082bf147045311154cbae4bca0f4c) Signed-off-by: Alexandru DAMIAN Signed-off-by: Richard Purdie

bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on fds, not time.sleep()

2013-09-01T14:51:10+00:00

The existing backend server implementations were inefficient since they were sleeping for the full length of the timeouts rather than being woken when there was data ready for them. It was assumed they would wake and perhaps did when we forked processes directory but that is no longer the case. This updates both the process and xmlrpc backends to wait using select(). This does mean we need to pass the file descriptors to wait on from the internals who know which these file descriptors are but this is a logical improvement. Tests of a pathaolgical load on the process server of ~420 rapid tasks executed on a server with BB_NUMBER_THREAD=48 went from a wall clock measurement of the overall command execution time of 75s to a much more reasonable 24s. (Bitbake rev: 9bee497960889d9baa0a4284d79a384b18a8e826) Signed-off-by: Richard Purdie

bitbake: server/xmlrpc: Increase timeout to 60s

2013-08-30T16:42:39+00:00

This is a better value that the earlier infinite timeout yet still allows for servers with high loads. It does mean the bitbake process can hang at exit for the timeout period but that should never happen and only happened for me in some test cases which wouldn't happen in normal use. (Bitbake rev: ab8d926b9bc27c58011e7db9327e031ac76ba34b) Signed-off-by: Richard Purdie

bitbake: bitbake: Add ui event handlers filtering

2013-08-26T10:29:46+00:00

Add functionality to allow UIs to update and change the types of events they recieve. To do this we need to add a new command and also need to be able to obtain the current event hander ID. In the case of xmlrpc, this is straightforward, in the case of the process server we need to save the result in a multiprocessing.Value() so we can retrive it. An excplit command was added to the server API to facilitate this. The same function can also be used to mask or unmask specific log messages, allowing the UI to optionally differ from the standard set of message filtering. Based upon work by Cristiana Voicu (Bitbake rev: ba5a6c88785d9889d4172ec79937ac2a5555327e) Signed-off-by: Richard Purdie

bitbake: server/xmlrpc/prserv: Add sane timeout to default xmlrpc server

2013-08-26T10:29:46+00:00

The standard python socket connect has long timouts which make sense for remote connections but not local things like the PR Service. This adds a timeout parameter to the common xmlrpc server creation function and sets it to a more reasonable 5 seconds. Making the PR server instantly exit is a good way to test the effect of this on bitbake. We can remove the bodged timeout in the PRServer terminate function which has the side effect of affecting global scope. (Bitbake rev: 8c01cff94787abbb64fbdf0c16cd63f8f97a7e03) Signed-off-by: Richard Purdie

bitbake: process: Improve exit handling and hangs

2013-08-26T10:29:46+00:00

It turns out we have a number of different ways the process server termination can hang. If we call cancel_join_thread() on the event queue, it means that it can be left containing partial data. This means the reading of the event queue in the terminate() function can hang, the timeout and block parameters to Queue.get() don't make any difference. Equally, if we don't call cancel_join_thread(), the join_thread in terminate() will hang giving a different deadlock. The best solution I could find is to loop over the process is_alive() after requesting it stops, trying to join the thread and if that fails, try and flush the event queue again. It wasn't clear what difference a force option should make in this case, we're gracefully trying to empty queues and shut down regardless of whether its a SIGTERM so I've simply removed the force option. (Bitbake rev: c5c8f33ca4b81877a0115887849881001b745bf0) Signed-off-by: Richard Purdie

bitbake: knotty, xmlrpc: add observer-only mode

2013-06-17T15:09:10+00:00

I add an observer only mode for the knotty UI and the XMLRPC server that will allow the UI to register a callback with a server in order to receive events. The observer-UI is able to send read-only commands to the server, and also is able to register as an event handler. Read-only commands are the commands that do not change the state of the server and have been marked as such in the command module. The observer can switch to a full client if it calls addClient at any time, and the server has no other client running. (Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2) Signed-off-by: Alexandru DAMIAN Signed-off-by: Richard Purdie

bitbake: compat/server/utils: Jettison pre python 2.7.3 workarounds

2013-06-14T16:26:30+00:00

Now we've moved to require python 2.7.3, we can jettison the compatibility workarounds/hacks for older python versions. (Bitbake rev: a51c402304f2080a76720f9b31d6dfdbed393bba) Signed-off-by: Richard Purdie

bitbake: xmlrpc: removes the event servers controller

2013-06-14T13:34:59+00:00

XMLRPC API ran a XMLRPC server at port+2 in order to provide endpoints to register an event server in which to dump the events. This is no longer used, so we remove it. (Bitbake rev: e171a363913a86e56266f4c9d107110c7f5221e3) Signed-off-by: Alexandru DAMIAN Signed-off-by: Richard Purdie

bitbake: xmlrpc: Allow server to remain memory resident

2013-06-07T13:13:18+00:00

This patch enables the XMLRPC server to remain resident in memory after a task is run, and to accept a new controlling client. To check the server after task completion, do lsof bitbake.lock in the build directory. Kill the server with kill. (Bitbake rev: e823e1f0675ff3794eb39ef0b4df2d7a220f4013) Signed-off-by: Alexandru DAMIAN Signed-off-by: Richard Purdie