linux/poky.git/meta/lib/oeqa/core/utils, branch hardknott

oeqa/concurrencytest: Fix display of test stdout/stderr

2021-04-06T08:30:10+00:00

If oe-selftest is run with -j, the output to stdout/stderr is being lost at present. Capture this and display it upon test failure. We have code that previously tried to enable this but it wasn't functioning correctly. This should give more usable error reports on the autobuilder. This code will mix stdout and stderr as the output is streamed from the test server without markup. This is most in keeping with subunit/testools though and the easiest way to handle the various challenges here as far as I can see. (From OE-Core rev: 6a954ce5834c8026adecff8478c3d827640bc647) Signed-off-by: Richard Purdie

oeqa/concurrencytest: Rename variables to improve the code

2021-04-06T08:30:10+00:00

Each time I look at this code I get confused about what the different variables represent. Rename a few of them to better indicate what they represent. (From OE-Core rev: e39d97c0b191add9281bac463ca059685288c81a) Signed-off-by: Richard Purdie

oeqa/selftest: Clean up separate builddir in success case when non-threaded

2020-06-28T07:36:00+00:00

If oe-selftest is run without -j, the separate build directory "build-st" isn't cleaned up afterwards. Mirror the behaviour of the -j option to handle this the same way, only preserve upon failure. To do this, the remove function needs to be moved to the selftest context module so that it can be accessed without requiring the testtools and subunit modules the -j option requires. A dummy wrapper class is used to wrap the tests and clean up afterwards. [YOCTO #13953] (From OE-Core rev: 1b376ade430d40d3cfe9c18f200c764d622710e5) Signed-off-by: Richard Purdie

oeqa/concurrencytest: don't delete build directory for failed tests

2020-05-24T22:56:03+00:00

(From OE-Core rev: 3d5aa170d2e88b852bd2a4452aab9311a24badef) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

oeqa/selftest: Standardise seperate builddir for concurrent and non-concurrent selftest

2020-02-21T09:39:00+00:00

Currently oe-selftest reuses the current build directory and the concurrent version run with -j does not. Standardise and use a separate new build directory in both cases. This will lead to simpler code and more reliable user run tests. (From OE-Core rev: 50238b0717b04e0a1fa69d618e8c8aa8445a80b0) Signed-off-by: Richard Purdie

concurrencytest.py: add outSideTestaddSkip for subunit

2020-02-18T23:53:55+00:00

see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13663 When running oe-selftest in concurrency mode (e.g. with oe-selftest --run-tests oescripts.OEPybootchartguyTests -j 4), if a skip occurred during setUpClass() rather than within individual tests, the entire suite would show "UNKNOWN" as each test's result. This is because subunit doesn't know how to handle skips outside of individual tests. An example of where this occurs is when running the above call to oe-selftest in concurrency mode on a host machine that does not have python3-cairo installed. Patch subunit inside concurrencytest.py to provide a method called outSideTestaddSkip, which will allow subunit to correctly detect the skip in setUpClass(). (From OE-Core rev: 9b8734b584d6e8d9c32ff2a721b29f3f3e61cca7) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

oeqa/concurrencytest: Optimise for autobuilder/clobberdir

2019-12-06T14:41:28+00:00

We're seeing huge delays on the autobuilder during oe-selftest builddir deletion. For example there is a currently running selftest we could do with the results from and its been going 13 hours, at least 8 of which was in deletion of the builddirs. There are a variety of ways we could solve this problem however the autobuilder has a mechanism for deferred deletion, "clobberdir" which it already uses for this kind of work. Whilst in general hardcoding things like this is horrible, I believe in this case the benefits (and resulting improvements on my sanity if nothing else) mean this is a case where we should do it. If/as/when someone can come up with a better solution that is fine and this can be replaced. (From OE-Core rev: 51a84937d32a85cbdb7d5b1d1ec69a290d0c303b) Signed-off-by: Richard Purdie

oeqa/core/utils/concurrencytest.py: Handle exceptions and details

2019-09-30T15:55:21+00:00

Handle the streaming of exception content with details data. The testtools package allows both 'err' and 'details' kwargs but can only pass one of them to the parent. To handle the passing of exception traceback and details data at the same time, encode the traceback into the details object and remove the 'err' arg from the add* result call. This encodes the traceback similar to how 'err' is handled without any details object. Decoding is already done by testtools when the traceback is encoded in the details object. (From OE-Core rev: 3613451825b251784b7673d89db465b9782c3a31) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

oeqa/concurrenttest: Use ionice to delete build directories

2019-09-19T19:30:35+00:00

Autobuilder type infrastructure can benefit from deletion of certain files as background IO due to the way Linux filesystem priority works. We have problems where build directories as part of oe-selftest being delete starves the running tasks of IO to the point builds take much longer to compelte. Having this option of running the deletion at "idle" helps a lot with that. Use the new option added to bb.utils.prunedir(). (From OE-Core rev: d41e7018be56902b7a1be4590e468cd15e02a3b5) Signed-off-by: Richard Purdie

oeqa/core: Implement proper extra result collection and serialization

2019-09-07T20:56:43+00:00

Implement handling of extra result (e.g. ptestresult) collection with the addition of a "extraresults" extraction function in OETestResult. In order to be able to serialize and deserialize the extraresults data, allow OETestResult add* calls to take a details kwarg. The subunit module can handle cross-process transfer of binary data for the details kwarg. With a TestResult proxy class to sit inbetween to encode and decode to and from json. (From OE-Core rev: b0831d43606415807af80e2aa1d0566d0b8c209c) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie