linux/poky.git/meta/lib/oeqa/core, branch hardknott

meta/lib/oeqa/core/tests/cases/timeout.py: add a testcase for the previous fix

2021-05-04T21:57:51+00:00

This is the sequence that didn't properly operate: - a test case that skips and isn't executed - a second test case that is skipped via a dependency decorator, and sets a timeout - a third test case that takes longer than the timeout from the second test case Without the fix, the timeout is not cleared, and the third test case is erroneously aborted. With the fix, the timeout is cleared and the third test case is able to complete. (From OE-Core rev: 4665008247cd4bd28da8c8b56c8c604e2e24d2cb) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie (cherry picked from commit 54ef07a9aa1af8f41cfb9a4802929c918efc43c8) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

oeqa: tear down oeqa decorators if one of them raises an exception in setup

2021-05-04T21:57:51+00:00

Some of the decorators need proper cleanup, such as OETimeout which sets a signal handler that needs to be cleared via teardown. If this is not done then the signal gets called later with unpredictable effects. This can be seen if there's a test that is skipped via a decorator and sets a timeout at the same time: the timeout isn't cleared, and is invoked later in a completely unrelated context. The test case for this is added in the next commit. (From OE-Core rev: be45a8271c06ffbb5d97afd33bb15b1143b6cf8d) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie (cherry picked from commit f42a08e1aabf1ca57e0c09d69fb69cc717c7f156) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

oeqa/runqemu: Support RUNQEMU_TMPFS_DIR as a location to copy snapshot images to

2021-04-06T08:30:10+00:00

We have a working theory that IO queues on the autobuilder are impacting runtime testing under qemu, particularly async writes which inice does not influence. We already pass the snapshot option to qemu which copies the image and runs out of the copy. Add in the ability to copy the image to a specificed location which can be a tmpfs. This means that writes to the image would no longer be blocked by other writes to disk in the system. Preliminary tests show that this does improve the qemu errors at the expense of sometimes showing qemu startup timeouts as on a loaded system with a large test image, it can take longer than 120s to copy the image to tmpfs. Having a most consistent failure mode for loaded tests is probably desireable though. (From OE-Core rev: fd1c26ab426c3699ffd8082b83d65a84c8eb8bff) Signed-off-by: Richard Purdie

oeqa/concurrencytest: Fix display of test stdout/stderr

2021-04-06T08:30:10+00:00

If oe-selftest is run with -j, the output to stdout/stderr is being lost at present. Capture this and display it upon test failure. We have code that previously tried to enable this but it wasn't functioning correctly. This should give more usable error reports on the autobuilder. This code will mix stdout and stderr as the output is streamed from the test server without markup. This is most in keeping with subunit/testools though and the easiest way to handle the various challenges here as far as I can see. (From OE-Core rev: 6a954ce5834c8026adecff8478c3d827640bc647) Signed-off-by: Richard Purdie

oeqa/concurrencytest: Rename variables to improve the code

2021-04-06T08:30:10+00:00

Each time I look at this code I get confused about what the different variables represent. Rename a few of them to better indicate what they represent. (From OE-Core rev: e39d97c0b191add9281bac463ca059685288c81a) Signed-off-by: Richard Purdie

oeqa/core/context: initialize _run_end_time

2020-11-11T10:08:12+00:00

with _run_start_time as value. For partial results of interrupted runs, this info might be otherwise missing for at least one testcase (From OE-Core rev: 1c5e8baf57fa2a33b9ef507b11d9ea9acaa77238) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oeqa/core/context: expose results as variable

2020-11-11T10:08:12+00:00

register an unittest handler for testresults and expose it as variable result. With this even partial results from an interrupted test suite run can be made available (From OE-Core rev: a97ae47525157871b6c098ffc352293e365a4335) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

target/ssh.py: Add dump_target support

2020-09-30T14:01:51+00:00

This adds the dump_target support when the ssh command fails with a 'No route to host'. This is will provide additional data when a Qemu target fails to respond during autobuilder testing. This does not fix 14002 [0], but may help track down why qemu looses networking [0] https://bugzilla.yoctoproject.org/show_bug.cgi?id=14002 (From OE-Core rev: cef1a2b03b359c018911abc29db1895d0f46814c) Signed-off-by: Saul Wold Signed-off-by: Richard Purdie

testimage: Add testimage_dump_target to kwargs

2020-09-30T14:01:51+00:00

This passes the list of commands to run on the OEQemuTarget when the TargetDumper needs to run in a test context due to a failure on the target. This is added here as a kwargs because the 'd' dictionary is not available in the staticmethod getTarget in the OERuntimeTestContextExecutor class. The OEQemuTarget is different from the QemuTarget which already uses the list of commands from testimage_dump_target from 'd'. The create_dir() is needed to initialize the TargetDumper's dump_dir variable. (From OE-Core rev: a63675fab4d9f638570912b15a07932f549cc4d1) Signed-off-by: Saul Wold Signed-off-by: Richard Purdie

oeqa: write @OETestTag content into json test reports for each case

2020-08-08T08:17:49+00:00

This allows using these tags for classification and filtering of test results according to various organization-specific criteria, such as teams responsible for the test, internal test ids, feature domains and so on. Test name itself meanwhile can stay short and human-readable. (From OE-Core rev: 3801b126eb52cd46efe417111afcd27f05d8f72b) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie