linux/poky.git/meta/lib/oeqa/core, branch uninative-2.2

oeqa/decorator: Improve reliability

2018-07-18T09:18:41+00:00

Checking if the dependency had any failure is unreliable, for example if the underlying data doesn't get transferred and the list is empty, success of the dependency is assumed. Since we now have success data available, change the code to use it. (From OE-Core rev: 4abba4c30d5a6163a968a119395a679e5e281ab4) Signed-off-by: Richard Purdie

oeqa: Remove xmlrunner

2018-07-18T09:18:41+00:00

This isn't present on modern distros by default and doesn't work with testtools, needing multiple code paths in the code. Remove it in favour of finding a better replacement for results collection/analysis. (From OE-Core rev: 8001d933a8dc86004db014777f094d718086687d) Signed-off-by: Richard Purdie

oeqa/runner: Simplify code

2018-07-18T09:18:41+00:00

There doesn't appear to be any reason we need this _results indirection any more so remove it. (From OE-Core rev: b618261811c48ff3b98eab1b340a8cd09ef183c6) Signed-off-by: Richard Purdie

oeqa/core/threaded: Remove in favour of using concurrenttests

2018-07-18T09:18:41+00:00

We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. (From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53) Signed-off-by: Richard Purdie

oeqa: Add selftest parallelisation support

2018-07-18T09:18:41+00:00

This allows oe-selftest to take a -j option which specifies how much test parallelisation to use. Currently this is "module" based with each module being split and run in a separate build directory. Further splitting could be done but this seems a good compromise between test setup and parallelism. You need python-testtools and python-subunit installed to use this but only when the -j option is specified. See notes posted to the openedmbedded-architecture list for more details about the design choices here. Some of this functionality may make more sense in the oeqa core ultimately. (From OE-Core rev: 326ababfd620ae5ea29bf486b9d68ba3d60cad30) Signed-off-by: Richard Purdie

oeqa: Default to buffer mode for tests

2018-07-16T12:21:35+00:00

Currently some tests run in buffer mode and some don't. Those that don't can corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent. If there is useful output on stdout/stderr, it will be displayed if the test fails. (From OE-Core rev: 978548c0abde2cb94c2782538552f39bdf2bf630) Signed-off-by: Richard Purdie

oeqa/runner: Ensure we don't print misleading results output

2018-07-16T12:21:34+00:00

The current code assumes if something isn't a failure of some kind, it was a pass. When test case IDs weren't matching, this lead to very confusing output where things would fail, then be listed as passing. This adds code to track successes, ensuring we don't end up in this position again with unmatched entries being listed as UNKNOWN. (From OE-Core rev: 4374c296d8963e4f6a1aa7bef7983ad0a1c2fcff) Signed-off-by: Richard Purdie

oeqa/core/runner: Improve test case comparision

2018-07-16T12:21:34+00:00

We can directly compare the test case IDs rather than representations, then if we're using subunit to split the tests, the comparisions still work as intended. (From OE-Core rev: 72e5f46f75454ba4c445c65c1cbc616a9e72fc6e) Signed-off-by: Richard Purdie

oeqa/core/target/ssh.py: increase maximum read bytes from 1024 to 4096

2018-06-04T14:15:00+00:00

When running testimage task for core-image-sato-sdk, the following error appeared. UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 at position 0: invalid start byte Checking the codes, I found it's caused by setting a 1024 limit for the read method of the StreamReader object. Comments from the manual: """ The chars argument indicates the number of decoded code points or bytes to return. The read() method will never return more data than requested, but it might return less, if there is not enough available. """ When running `systemctl status --full' on target, this error occurs. This patch increase the bytes limit to 4096 to fix the error. (From OE-Core rev: f1fad60ae3be4450aca6058d5665fb10a9148b44) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

oeqa/core/decorator/data.py: fix skipIfNotInDataVar

2018-06-04T14:15:00+00:00

The var might not be set, resulting in unexpected error. RESULTS - multilib.MultilibTest.test_check_multilib_libc - Testcase 1593: ERROR The above error is due to MULTILIBS being not set, which is the default for OE. This patch fixes this problem. Also, the debugging message in skipIfNotInDataVar is currently confusing. Instead of DEBUG: Checking if 'MULTILIBS' value is in 'multilib:lib32' to run the test it should be DEBUG: Checking if 'MULTILIBS' value contains 'multilib:lib32' to run the test This patch also fixes it. (From OE-Core rev: 3f5c678664a2bba43d99508779dc2ce227cf52a2) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie