linux/poky.git/meta/lib/oeqa/core/context.py, branch kirkstone

oeqa/core/context: initialize _run_end_time

2020-11-11T10:08:12+00:00

with _run_start_time as value. For partial results of interrupted runs, this info might be otherwise missing for at least one testcase (From OE-Core rev: 1c5e8baf57fa2a33b9ef507b11d9ea9acaa77238) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oeqa/core/context: expose results as variable

2020-11-11T10:08:12+00:00

register an unittest handler for testresults and expose it as variable result. With this even partial results from an interrupted test suite run can be made available (From OE-Core rev: a97ae47525157871b6c098ffc352293e365a4335) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oe-selftest: Support verbose log output

2020-06-04T12:27:32+00:00

(From OE-Core rev: 6a607da9a6aa05f86b0e77cccd066b9f286cfada) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

oeqa: default runtime json results for testexport

2020-03-29T11:05:31+00:00

Set the json result output dir in the oeqe runtime context to create testresults.json file by default for exported runtime test runs. Use current datetime for the json result property name (not DATETIME from build) to allow multiple result entries. (From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f) Signed-off-by: Stefan Kral Signed-off-by: Richard Purdie

oeqa: enable testresults.json for testexport

2020-03-12T22:49:28+00:00

Add the option --json-result-dir to oeqa core context to enable testresults.json creation for test runs via testexport. Eg. oe-test runtime --json-result-dir . (From OE-Core rev: 9d8edf33d1f5d89b310923b0aa3cc967317c7c49) Signed-off-by: Stefan Kral Signed-off-by: Richard Purdie

oeqa/selftest: Standardise seperate builddir for concurrent and non-concurrent selftest

2020-02-21T09:39:00+00:00

Currently oe-selftest reuses the current build directory and the concurrent version run with -j does not. Standardise and use a separate new build directory in both cases. This will lead to simpler code and more reliable user run tests. (From OE-Core rev: 50238b0717b04e0a1fa69d618e8c8aa8445a80b0) Signed-off-by: Richard Purdie

oeqa/core: Rework OETestTag and remove unused OETestFilter

2019-09-07T20:56:43+00:00

Rework OETestTag so that it does not rely on the existing decorator code base and instead inserts the tags into an attribute on the decorated target (e.g. class/type or method). This allows the use of OETestTag on classes and method. In order to filter tagged tests rework the loaders filtering code, removing the generic-ness (with validation and attributes/etc.) and replace it with a "tags_filter" parameter which is a function that filters a test based on the tags it has. This allows the loader user to filter on tags in more specific ways (e.g. include all untagged tests and any tests tagged with foo). Plumb all this through the context code and testing code. Update the associated tests to pass correctly with the changes. (From OE-Core rev: b8a4a4c2de68110d74607cb9807c9e741ca9441c) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

oeqa: avoid class setup method to run when skipping the whole class

2019-06-21T14:32:36+00:00

For now, even if we have specified to skip the whole module/class via command line, e.g., `oe-selftest -R gotoolchain', the class setup method is still run. This at least results in unnecessary builds, and at worst results in ERROR, if the setup method fails. So improve the skipping mechanism to avoid class setup method to run when specified to skip. (From OE-Core rev: b0b79bf65f5e5e65958090a4a88622b42df896bf) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

context.py: avoid skipping tests by meaningless command argument

2019-06-21T14:32:36+00:00

Currently `oe-selftest -R a' will skip 'archiver' tests. This is not expected. Fix it so that the '-R' should be followed by actual module/class/test names. (From OE-Core rev: de3b070fc2ddd0b63a324679ec5adbe30142fc22) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

context.py: fix skipping function

2019-06-18T10:23:48+00:00

The current codes to skip test cases are logically correct, but they do not work correctly in reality. It does skip the tests as the command line argument specifies, but the related information is wrong. e.g. $ oe-selftest -R archiver bblayers runtime_test 2019-06-17 09:24:53,764 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it 2019-06-17 09:25:06,309 - oe-selftest - INFO - Adding layer libraries: 2019-06-17 09:25:06,310 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta/lib 2019-06-17 09:25:06,310 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta-yocto-bsp/lib 2019-06-17 09:25:06,310 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta-selftest/lib 2019-06-17 09:25:06,312 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable 2019-06-17 09:25:10,521 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea5/chenqi/SWAT/poky/build-selftest/conf/local.conf 2019-06-17 09:25:10,521 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2019-06-17 09:25:10,522 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver) 2019-06-17 09:25:10,522 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "runtime_test"' 2019-06-17 09:25:10,522 - oe-selftest - INFO - Skip by the command line argument "runtime_test" 2019-06-17 09:25:10,523 - oe-selftest - INFO - test_archiver_filters_by_type (archiver.Archiver) 2019-06-17 09:25:10,523 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "runtime_test"' 2019-06-17 09:25:10,523 - oe-selftest - INFO - Skip by the command line argument "runtime_test" The archiver.Archiver.xxx tests should be skipped by 'archiver' command line argument, not 'runtime_test'. Change to use a function generator to achieve the desired effect. After the change, the effect is as follows. $ oe-selftest -R archiver bblayers runtime_test 2019-06-17 09:19:06,223 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it 2019-06-17 09:19:19,598 - oe-selftest - INFO - Adding layer libraries: 2019-06-17 09:19:19,599 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta/lib 2019-06-17 09:19:19,599 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta-yocto-bsp/lib 2019-06-17 09:19:19,599 - oe-selftest - INFO - /buildarea5/chenqi/SWAT/poky/meta-selftest/lib 2019-06-17 09:19:19,602 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable 2019-06-17 09:19:24,368 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea5/chenqi/SWAT/poky/build-selftest/conf/local.conf 2019-06-17 09:19:24,368 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2019-06-17 09:19:24,369 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver) 2019-06-17 09:19:24,369 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "archiver"' 2019-06-17 09:19:24,369 - oe-selftest - INFO - Skip by the command line argument "archiver" 2019-06-17 09:19:24,369 - oe-selftest - INFO - test_archiver_filters_by_type (archiver.Archiver) 2019-06-17 09:19:24,370 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "archiver"' 2019-06-17 09:19:24,370 - oe-selftest - INFO - Skip by the command line argument "archiver" 2019-06-17 09:19:24,370 - oe-selftest - INFO - test_archiver_filters_by_type_and_name (archiver.Archiver) 2019-06-17 09:19:24,370 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "archiver"' 2019-06-17 09:19:24,371 - oe-selftest - INFO - Skip by the command line argument "archiver" 2019-06-17 09:19:24,371 - oe-selftest - INFO - test_archiver_srpm_mode (archiver.Archiver) 2019-06-17 09:19:24,371 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "archiver"' 2019-06-17 09:19:24,372 - oe-selftest - INFO - Skip by the command line argument "archiver" 2019-06-17 09:19:24,372 - oe-selftest - INFO - test_bitbakelayers_add_remove (bblayers.BitbakeLayers) 2019-06-17 09:19:24,373 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "bblayers"' 2019-06-17 09:19:24,373 - oe-selftest - INFO - Skip by the command line argument "bblayers" 2019-06-17 09:19:24,373 - oe-selftest - INFO - test_bitbakelayers_createlayer (bblayers.BitbakeLayers) 2019-06-17 09:19:24,373 - oe-selftest - INFO - ... skipped 'Skip by the command line argument "bblayers"' 2019-06-17 09:19:24,374 - oe-selftest - INFO - Skip by the command line argument "bblayers" [snip] (From OE-Core rev: 50004f431a71c71af4acf0f25403fee5a8447eab) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie