linux/poky.git/meta/classes/testimage.bbclass, branch thud

oeqa/core/runner: dump stdout and stderr of each test case

2019-05-21T23:31:49+00:00

Some CI pipelines might perform further processing of the test output (for instance, to plot some metrics into a chart). However, Since `thud` we switched away from the XML-based jUnit reporting, and at the same time we lost the ability of collecting the stdout and stderr of the various tests. We now restore this functionality by adding `stdout` and `stderr` keys to the JSON reports. This behavior is off by default; in order to enable it, one must set the `TESTREPORT_FULLLOGS` variable in the bitbake configuration. (From OE-Core rev: 68919826fe4d4db4582223569c5a939a9c2f9c72) Signed-off-by: Alberto Mardegan Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

oeqa: make it work for multiple users

2019-02-06T16:38:30+00:00

There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. (From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980) (From OE-Core rev: 310a2b137f2cb249885844fa12830365f94866f0) Signed-off-by: Robert Yang Signed-off-by: Ross Burton Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

meta/classes/testimage.bbclass: Only validate IMAGE_FSTYPES when is QEMU

2019-02-06T16:38:30+00:00

When use simpleremote target the flash/boot process is executed manually, the IMAGE_FSTYPES validation is only needed when execute testimage against qemu. The supported_fstypes comes from oeqa.core.target.qemu module. (From OE-Core rev: e7dc5963adbacc091fe8943119262166977623ad) (From OE-Core rev: 5fedb011f9b2d4f6d14757a7cc9c2d8300fa0223) Signed-off-by: Aníbal Limón Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage.bbclass: remove boot parameter systemd.log_target

2019-02-06T16:38:30+00:00

Boot parameter systemd.log_target=console affects command journalctl's output and causes oe selftest case test_systemd_boot_time fail to pass. | Error at obtaining the boot time from journalctl | RESULTS: | RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time - Testcase -1: SKIPPED (0.74s) systemd.log_target=console was introduced by oe-core commit a0bb649 and work with parameter systemd.log_level to enable systemd debug. systemd.log_level has been removed already, so remove systemd.log_target too to make case test_systemd_boot_time pass. (From OE-Core rev: caa776bdcf8ea34c857f45970370bf771075f4bc) (From OE-Core rev: 51423c3945d55ebabe38dd365d24f226155c1a97) Signed-off-by: Kai Kang Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Add possibility to pass parmeters to qemu

2019-02-06T16:38:30+00:00

Add a variable called TEST_QEMUPARAMS in testimage.bbclass to make it possible to pass parameters to qemu. This can be useful for e.g. increasing the amount of RAM available during testimage runs. (From OE-Core rev: 1a9163f5779d233c884c8fd50e0812eabab4fdf3) (From OE-Core rev: 0469c757e4a48d8ca2e2b7b128a285cf6d50513e) Signed-off-by: Erik Botö Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Add support for slirp

2019-02-06T16:38:30+00:00

Enable testimage to support qemu slirp. Configure "QEMU_USE_SLIRP" & "TEST_SERVER_IP" variables to enable slirp. [YOCTO#10713] (From OE-Core rev: 3df9ee85ce7fe52f0893fd33aea3bf1fcc6ead0a) (From OE-Core rev: ab4989123d8a6c2dc8e058110c495320955bf0de) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Enable autorunning of the package manager testsuites

2019-02-06T16:38:30+00:00

Now that the hangs in httpservice are fixed we can let these tests auto skip as appropriate. (From OE-Core rev: 42a0d70291d551578e21f590fcb85ca72a78ccb5) (From OE-Core rev: 1da4c2e18e0897b3dc24d0d54b5d949bec39c990) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Further cleanup DEFAULT_TEST_SUITES

2019-02-06T16:38:30+00:00

Now the test markup of the development tools tests is complete, this can be further tweaked to auto run the correct tests. (From OE-Core rev: bd4f8d12fe1f9f2643ee9e68fa2bb981134294fb) (From OE-Core rev: 7dce3564534ee67c88feaa14304dbee7161df145) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Simplfy DEFAULT_TEST_SUITES logic

2019-02-06T16:38:30+00:00

Now that the tests have correct markup to automatically determine which images they can run against, clean up the default test suites logic to be simpler and not image specific. Some cleanup of the compiler tests still needs to be completed but this is a good first step. The only downside to this is more noise during testing as we now see many skipped messages for simple images like core-image-minimal. The auto type is being removed since it currently breaks badly due to the socat mandatory inclusion from the meta-selftest layer which is a problem which needs to be addressed seperately. (From OE-Core rev: 4966bc33845752eb0aeae54b72e8ba0146a7ed52) (From OE-Core rev: 16c9002d5b4ab091f49a7bec2527d2a21830ab0c) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

testimage: Remove duplicate dependencies

2019-02-06T16:38:30+00:00

(From OE-Core rev: 16d4368e7b85d5179374adb0a2d1ac215793e104) (From OE-Core rev: bfd72574aa1ef035ecafef3cfde9ac5f8801b93c) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie