linux/poky.git/bitbake/lib, branch 1.5_M4.rc2

bitbake: prserv/serv: Multithread the server

2013-08-28T23:13:22+00:00

This makes the PR server multithreaded and able to handle multiple connections at once which means its no longer a build bottle neck when serving one connection at a time. I've experimented and database connection for each thread seems to cause the least issues, pushing the contention for sqllite to handle itself. This means moving the db/table connection code into the actual function methods. It doesn't abstract well as a function since we need the db object around for the lifetime of the function as well as the table else we lose the connection. (Bitbake rev: bf9be2029b2bded5f532bdda4c38ae3dff5d1cf6) Signed-off-by: Richard Purdie

bitbake: prserv/db: Threading fixes

2013-08-28T23:13:22+00:00

Enabling threading for the PRServer causes a number of issues. Firstly is the obtuse error: sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type which is due to the class not being derived from object. See: http://docs.python.org/2/library/sqlite3.html#registering-an-adapter-callable Secondly, we want to enable multithreadded access to the database so we do this when we open it. This opens the way up to multithreading the PR server. (Bitbake rev: 5709efc2ff1e36529bd28f49cd093ccfa7abff7f) Signed-off-by: Richard Purdie

bitbake: serv.py: Fix regression from 972bc43e6d5b

2013-08-28T08:29:46+00:00

commit 972bc43e6d5b1207b944b3baa8f9805adb35dda7 (serv.py: Fix hang when spawned dynamically with bitbake) introduced a regression, because the wrong patch was submitted. The syntax was incorrect in the original patch. The logger iterator must be used with a call to getLogger(). [YOCTO #5059] (Bitbake rev: 85fed8acc3af3e15bf119db2f51c486a9de3646b) Signed-off-by: Jason Wessel Signed-off-by: Richard Purdie

bitbake: data_smart: allow removal of multiple words at once with _remove

2013-08-27T23:34:13+00:00

FOO = "foo bar baz" FOO_remove = "foo baz" (Bitbake rev: 04127dec207d6dfc0ada56c5cc67ec9ad30517a8) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: data_smart: use a split/filter/rejoin for _remove

2013-08-27T23:34:13+00:00

This is more idiomatic, and from the limited performance testing I did, is faster as well. See https://gist.github.com/kergoth/6360248 for the naive benchmark. (Bitbake rev: 1aa49226d5a2bac911feeb90e3d9f19529bc1a3e) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: serv.py: Fix hang when spawned dynamically with bitbake

2013-08-27T23:25:56+00:00

The PRServer has the possibility to hang indefinitely blocking on a semaphore processing a xmlrpc request to send an event back to the main bitbake instance. This was observed during a "bitbake -e" on a heavily loaded machine and the main bitbake instance and cooker exited before the PRServer emitted its first log. The stack trace is provided below as to show what happens every time a logger.info() is executed in the PRServer. Not only does it write to the stream handler but it also tries to send the event to the main event processor. self._notempty.acquire() self.queue.put(event) _ui_handlers[h].event.send(event) fire_ui_handlers(event, d) fire(record, None) self.emit(record) hdlr.handle(record) self.callHandlers(record) self.handle(record) self._log(INFO, msg, args, **kwargs) (self.dbfile, self.host, self.port, str(os.getpid()))) self.work_forever() pid = self.daemonize() self.prserv.start() singleton.start() self.prhost = prserv.serv.auto_start(self.data) cooker.pre_serve() bb.cooker.server_main(self.cooker, self.main) self.run() code = process_obj._bootstrap() self._popen = Popen(self) self.serverImpl.start() server.detach() server = start_server(servermodule, configParams, configuration) ret = main() It was never intended for the PRServer to send its logs anywhere but its own log file. The event processing is an artifact of how the PRServer was forked and it inherits the event log handlers. The simple fix is to clean up and purge all the log handlers after the fork() but before doing any of the typical PRServer work or logging. (Bitbake rev: 972bc43e6d5b1207b944b3baa8f9805adb35dda7) Signed-off-by: Jason Wessel Signed-off-by: Richard Purdie

bitbake: hob: fixes for image combo box

2013-08-27T10:47:35+00:00

When an image from scratch is selected, and recipes parsing is canceled, the image shown by the combo box isn't correct. [YOCTO #5000] (Bitbake rev: f8166ace0bd9155199166990ce15da24eb2e793b) Signed-off-by: Cristiana Voicu Signed-off-by: Richard Purdie

bitbake: hob: add event handlers filtering in Hob

2013-08-27T10:16:58+00:00

Create the _evt_list for hob; it is longer than the knotty uses because it handles more events. (Bitbake rev: 715aed74f972bb6e9b6a5130ca9ede48d4f79f0a) Signed-off-by: Cristiana Voicu Signed-off-by: Richard Purdie

bitbake: bitbake/event.py: UIhandler filter should work without a mask

2013-08-27T10:16:58+00:00

The default for the mask will be * (all the handlers) (Bitbake rev: 4c95e5f46cf2a656100bbf5a0e5a09d506abf9b9) Signed-off-by: Cristiana Voicu Signed-off-by: Richard Purdie

bitbake: command.py: Call updateCache for all states != running

2013-08-26T12:43:47+00:00

updateCache handles the logic for shutting down the parsing so we need to call it for all cases when we're not running. This fixes hangs if Ctrl+C is pressed during parsing. (Bitbake rev: 552b8935dd2f9f11e8d5c08a597a7e966b891480) Signed-off-by: Richard Purdie