linux/poky.git/bitbake/lib/prserv, branch dizzy-12.0.1

bitbake: prserv: Use WAL mode

2014-12-31T10:10:20+00:00

Ideally, we want the PR service to have minimal influence from queued disk IO. sqlite tends to be paranoid about data loss and locks/fsync calls. There is a "WAL mode" which changes the journalling mechanism and would appear much better suited to our use case. This patch therefore switches the database to use WAL mode. With this change, write overhead appears significantly reduced. (Bitbake rev: 90b05e79764b684b20ce8454e89f05763b02ac97) Signed-off-by: Richard Purdie

bitbake: prserv/serv: Ensure sync happens in the correct thread

2014-12-31T10:10:20+00:00

The sync/commit calls are happening in the submission thread which can race against the handler. The handler may start new transactions which then causes the submission thread to error with "cannot start a transaction within a transaction". The fix is to move the calls to the correct thread. (Bitbake rev: 08cf468ab751f4c6e4ffdab2d8e5d748f7698593) Signed-off-by: Richard Purdie

bitbake: prserv: don't wait until exit to sync

2014-12-31T10:10:20+00:00

In the commit 'prserv: Ensure data is committed', the PR server moved to only committing transactions to the database when the PR server is stopped. This improves performance, but it means that if the machine running the PR server loses power unexpectedly or if the PR server process gets SIGKILL, the uncommitted package revision data is lost. To fix this issue, sync the database periodically, once per 30 seconds by default, if it has been marked as dirty. To be safe, continue to sync the database at exit regardless of its status. (Bitbake rev: 973ac2cc63323ca9c3e916effa4765747db3564c) Signed-off-by: Ben Shelton Signed-off-by: Richard Purdie

bitbake: prserv/serv: Improve error message when prserver cannot bind to supplied host address

2014-09-29T11:08:48+00:00

If localhost resolves to a remote address (due to a misconfigured network), starting the pr server will fail without useful information. To reproduce, add ' localhost' to /etc/hosts and run 'bitbake -p'. The error message will be: ERROR: Timeout while attempting to communicate with bitbake server ERROR: Could not connect to server False: Running 'bitbake-prserv --host=localhost --port=0 --start' will fail with: error: [Errno 99] Cannot assign requested address Since these errors does not show the IP address of the attempted socket binding, this results in a lot of wasted time looking at firewall rules, etc. This patch results in the following error message if the socket binding fails: PR Server unable to bind to :0 (Bitbake rev: fae5914030bcf4c061c22fc61034c40c87b7121a) Signed-off-by: Konrad Scherer Signed-off-by: Richard Purdie

bitbake: lib: fix no newline at end of file

2014-07-04T08:28:36+00:00

Add a '\n' to the last line of the file to fix: No newline at end of file (Bitbake rev: 54f1359ed2e9d47980cd221b7b43ef56543fe06d) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: prserv/db: Avoid fsync() calls

2014-05-03T10:24:48+00:00

If the power were to fail, it doesn't matter to us much if the data makes it to disk or not, we'd have other problems. However an fsync() call on a multi build autobuilder is painful so lets avoid them. (Bitbake rev: 4eb2dc8048e2722d64d589f453df1ce6262c71b8) Signed-off-by: Richard Purdie

bitbake: prserv: Fix exit race issues

2014-03-30T09:10:36+00:00

We shouldn't immediately remove the pid file when stopping the server, if we do, this causes a traceback within the server itself which can then hang. Fix this by removing the stale pid file as the last thing we do. Also: * don't printing a new "waiting" line every 0.5 seconds. * make the loop more granular since the user can 'feel' the 0.5 seconds [YOCTO #5984] (Bitbake rev: 81f41a806aeddcc38992163557672e296bcbc967) Signed-off-by: Richard Purdie

bitbake: serv.py: Give pr-server up to 5 seconds to commit data

2013-11-18T17:19:11+00:00

The default value of 0.5 seconds before sending the pr-server a SIGTERM is not enough to guarantee that sqlite has committed all the pr data to the database. By polling the pid to see if it is still running, this allows the pr-server process to shutdown cleanly and finish the final pr data commit. (Bitbake rev: 22eec978e70794923c85689928c6be0cfe71cdcd) Signed-off-by: Konrad Scherer Signed-off-by: Richard Purdie

bitbake: prserv: Ensure data is committed

2013-09-08T09:51:56+00:00

In exclusive mode, we need to complete the transaction for writes to make it to the database. Therefore add sync calls to ensure this happens. Autocommit mode is significantly (100 times) slower so caching the data is of significant benefit. (Bitbake rev: 4e55f7821786a59c2cd7dbd8bfa2a22f5f196e99) Signed-off-by: Richard Purdie

bitbake: prserv/serv: Settle on two threads for optimal performance

2013-09-01T14:51:11+00:00

Using the threading mixin class resulted in large amounts of memory being used by the PR server for no good reason. Using a receiver thread and a thread to do the actual database operations on a single connection gives the same performance with a much saner memory overhead so switch to this. (Bitbake rev: e08455d5f3b8e96765942b9c3b9767c30650557d) Signed-off-by: Richard Purdie