linux/poky.git/bitbake/lib/bb/tests/runqueue.py, branch zeus-next

bitbake: tests/runqueue: Fix to match recent task migration fixes

2019-11-27T11:03:38+00:00

(Bitbake rev: 8569ccb5e9fbdeaaf96b78bd02a263b26de54059) Signed-off-by: Richard Purdie

bitbake: tests/runqueue: Fix hashserve shutdown race

2019-10-02T13:18:59+00:00

The hashserve can delete its socket whilst the cleanup us happening leading to backtraces and test failures. Add code to avoid this race condition. [YOCTO #13542] (Bitbake rev: efd7b025cee25d0ee668c09476395d08fcf5ae1a) Signed-off-by: Richard Purdie

bitbake: runqueue: Change task migration behaviour for rerunning setscene tasks

2019-09-30T16:23:35+00:00

Currently runqueue will rerun setscene tasks multiple times as hashes change. This has caused numerous problems since a setscene task may become "unavailable" for some future signature combination and the code then can't easily "unskip" tasks its already passed into the execution queue. At least for now, only run setscene once and assume they're equivalent at that point. In practise that has been much more stable in testing. Tweak the test to match the change in behaviour. (Bitbake rev: 4205a3ef23834f317642bba155d67cd772176fb6) Signed-off-by: Richard Purdie

bitbake: runqueue: Fix task migration problems

2019-09-30T16:23:35+00:00

Tasks were not migrating consistently, particularly: * if a task was rehashed which had already run * if a task which was valid became invalid due to a rehash We need to always run the migration code for rehashed tasks and then reprocess them for hash validity. This means rearranging the code. It also means several tests are no longer correct and can't be written correctly to work on all possible workflows so those are removed. (Bitbake rev: 8443989ee41e9b162972935513e437b5c66ea74d) Signed-off-by: Richard Purdie

bitbake: bitbake: Rework hash equivalence

2019-09-18T16:52:01+00:00

Reworks the hash equivalence server to address performance issues that were encountered with the REST mechanism used previously, particularly during the heavy request load encountered during signature generation. Notable changes are: 1) The server protocol is no longer HTTP based. Instead, it uses a simpler JSON over a streaming protocol link. This protocol has much lower overhead than HTTP since it eliminates the HTTP headers. 2) The hash equivalence server can either bind to a TCP port, or a Unix domain socket. Unix domain sockets are more efficient for local communication, and so are preferred if the user enables hash equivalence only for the local build. The arguments to the 'bitbake-hashserve' command have been updated accordingly. 3) The value to which BB_HASHSERVE should be set to enable a local hash equivalence server is changed to "auto" instead of "localhost:0". The latter didn't make sense when the local server was using a Unix domain socket. 4) Clients are expected to keep a persistent connection to the server instead of creating a new connection each time a request is made for optimal performance. 5) Most of the client logic has been moved to the hashserve module in bitbake. This makes it easier to share the client code. 6) A new bitbake command has been added called 'bitbake-hashclient'. This command can be used to query a hash equivalence server, including fetching the statistics and running a performance stress test. 7) The table indexes in the SQLite database have been updated to optimize hash lookups. This change is backward compatible, as the database will delete the old indexes first if they exist. 8) The server has been reworked to use python async to maximize performance with persistently connected clients. This requires Python 3.5 or later. (Bitbake rev: 2124eec3a5830afe8e07ffb6f2a0df6a417ac973) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: tests/runqueue: Fix tests

2019-08-15T07:43:22+00:00

There were paths being accidentally included in some of the hashserv tests. Remove that and update the hashes so the tests work independently of paths. (Bitbake rev: 6ddb9f09cb60c2354fa6a67cce412c4dc1e7dc2d) Signed-off-by: Richard Purdie

bitbake: runqueue: Fix event timing race

2019-08-14T16:28:23+00:00

The event from the task notifiing of hash equivalency should only be processed when the task completes. This can otherwise result in a race where a dependent task may run before the original task completes causing various failures. To make this work reliably, the code had to be restructured quite a bit. (Bitbake rev: 1bf5be46f92f125193638cf41ff207d68f592259) Signed-off-by: Richard Purdie

bitbake: tests/runqueue: Add further hash equivalence tests

2019-08-14T16:28:23+00:00

Add some extra hash equivalence runqueue tests based on recent scenarios that caused problems during testing. (Bitbake rev: 373b085ead992a725b2230ededd992b4c61a1a05) Signed-off-by: Richard Purdie

bitbake: tests/runqueue: Add hashserv+runqueue test

2019-08-06T10:21:32+00:00

Add a test which tests the runqueue adaptations for hash equivalency. (Bitbake rev: 477321d0780df177c1582db119c2bb6795912fc6) Signed-off-by: Richard Purdie

bitbake: runqueue: Fix non setscene tasks targets being lost

2019-07-18T11:16:19+00:00

If you specify both setscene and non-setscene tasks on the commandline, the non-setscene tasks could be missed, e.g. "bitbake X:do_patch X:do_populate_sysroot" and do_patch would fail to run. Fix the problem in runqueue and add a testcase. (Bitbake rev: 75292fdec5d9c0b5b3c554c4b7474a63656f7e12) Signed-off-by: Richard Purdie