linux/poky.git/scripts/lib/resulttool, branch 3.1_M1

resulttool/store.py: Enable add extra test environment data

2019-11-14T13:20:59+00:00

Enable the option to add extra test environment data to the configuration of each test result (as optional). Example of optional test environment data include: - custom packages included for runtime test - detail machine specification used as target - detail host environment used for bitbake (From OE-Core rev: 3cc887a72c5726024c3699e6a3deb72fad4992a9) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

scripts/resulttool/report: Add total statistic to test result.

2019-11-14T13:20:59+00:00

Add total passed, failed, and skipped statistic to test result. (From OE-Core rev: 7ad74f99b6105f23bcf930712627aac279c6af10) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

scripts/resulttool/report: Enable output raw test results

2019-11-14T13:20:59+00:00

In case of debugging, report user need to acccess the raw test result. Instead of going back to source file/directory/URL to manually pull out the raw result, provide alternative way to let report showing raw test results by providing the result id (optional). (From OE-Core rev: 8b2485bc0d0fa2f72b37e8bafbfa663d456ea98c) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

scripts/resulttool/report: Enable report to use regression_map

2019-11-14T13:20:59+00:00

By default, report will use the store_map to generate the key to reference each result set. In some situation when using store_map with multiple set of tests sharing similar test configurations, the report will only showing partial result set for results that having identical result_id (use of multiconfig to run tests where it generate identical result_id). Enable report to have the option to use the regression_map (optional) instead of the default store_map, where it will take larger set of configurations to generate the key to reference each result set, this will prevent the report from only showing partial result set. (From OE-Core rev: 8577abce6feb1f529537da554d39e86a5b05ca2d) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

oeqa/core/case.py: Encode binary data of log

2019-09-30T15:55:21+00:00

Do not decode the log content into a string only to re-encode it as binary data again. Some logs might un-intentionally contain bytes that do not decode as utf-8, as such preserve the log file content as it was on disk. Handle the decoding on the resulttool side, but also handle the failure to decode the data. (From OE-Core rev: 20531dc0b8f76a6e37cc856f36cd94077b6aba50) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers

2019-09-16T22:02:43+00:00

Add the OEPTestResultTestCase class as a mix-in class to provide helper functions for interacting with ptestresults within the extraresults object generated by the test case. This class also provides default compression of log text and log files. Also add support to resulttool for decoding/decompressing log files embedded in the test results. (From OE-Core rev: 06cba9883a5964320969301fd05eeb6bec3e786d) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

resulttool: Handle multiple series containing ptestresults

2019-09-16T22:02:43+00:00

Handle multiple results series having ptestresults content. The contents are merged on a per-result basis where duplicates are ignored (with a warning message printed). The 'ptestresults.sections' collection is also merged on a per-suite basis. (From OE-Core rev: 47edd51970ed0c33edbe04fd72abd1cfc6ecd3d1) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

resulttool: Add reproducible log extraction

2019-09-06T13:58:10+00:00

Adds an argument to the log subcommand to extract the raw logs from the reproducible selftest. To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility. [YOCTO #13324] (From OE-Core rev: 7a4ebb361ff1efc22e7dafadfa60c98bc8a79ed4) Signed-off-by: Joshua Watt Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

scripts/lib/resulttool/report.py: Add more result types

2019-08-28T15:44:09+00:00

Add additional result types into the dictionary to handle dejagnu style test results. These include PASS, FAIL, XPASS, XFAIL, UNSUPPORTED, UNTESTED, UNRESOLVED and ERROR. (From OE-Core rev: 684fc36402a23760b203f4761f284043031c799c) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

resulttool: Prevent multiple results for the same test

2019-08-21T20:52:59+00:00

Currently, if a test occurs multiple times over different series, the code will sum these. This can lead to confusion over the desired results. Change the code to report the redundant tests and skip adding an additional values. (From OE-Core rev: caeaa0648dff64c56a33f52e45e82bfab6719e3e) Signed-off-by: Jon Mason Signed-off-by: Ross Burton Signed-off-by: Richard Purdie