linux/poky.git/meta/lib/oeqa/core/context.py, branch nanbield

oeqa: loader.py: show warning when skipping selected module and abort if all are skipped

2023-03-22T13:53:29+00:00

* skipped modules were triggering an ERROR before: poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers imagefeatures.ImageFeatures.test_image_gen_debugfs -K -B /OE/build/poky/build-eSDK 2023-03-13 15:07:53,430 - oe-selftest - ERROR - Not found eSDK.oeSDKExtSelfTest.test_install_libraries_headers in loaded test cases * but didn't show the reason why it wasn't loaded and more importantly -r was ignored when all selected modules were silently skipped * add a warning when skipping some module and abort if some modules were selected, but all ended being skipped: poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK 2023-03-13 15:11:51,028 - oe-selftest - WARNING - module 'eSDK.oeSDKExtSelfTest.test_install_libraries_headers' was skipped from selected modules, because it doesn't match with module name assumptions: package and module names do not contain upper case characters, whereas class names do 2023-03-13 15:11:51,028 - oe-selftest - ERROR - All selected modules were skipped, this would trigger selftest with all tests and -r ignored. * I was hit by this in oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers which is skipped due to upper case characters in module name and selftest started to run all tests (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name as first from 529) poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK 2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers'] 2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build .. 2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name) I'll rename eSDK to esdk in next commit to avoid this. * also fix small typo in context I've noticed when debugging this (From OE-Core rev: 15229c9abaf4cc398c31d97d171b9fac57141873) Signed-off-by: Martin Jansa Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

oeqa/core/context: initialize _run_end_time

2020-11-11T10:08:12+00:00

with _run_start_time as value. For partial results of interrupted runs, this info might be otherwise missing for at least one testcase (From OE-Core rev: 1c5e8baf57fa2a33b9ef507b11d9ea9acaa77238) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oeqa/core/context: expose results as variable

2020-11-11T10:08:12+00:00

register an unittest handler for testresults and expose it as variable result. With this even partial results from an interrupted test suite run can be made available (From OE-Core rev: a97ae47525157871b6c098ffc352293e365a4335) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oe-selftest: Support verbose log output

2020-06-04T12:27:32+00:00

(From OE-Core rev: 6a607da9a6aa05f86b0e77cccd066b9f286cfada) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

oeqa: default runtime json results for testexport

2020-03-29T11:05:31+00:00

Set the json result output dir in the oeqe runtime context to create testresults.json file by default for exported runtime test runs. Use current datetime for the json result property name (not DATETIME from build) to allow multiple result entries. (From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f) Signed-off-by: Stefan Kral Signed-off-by: Richard Purdie

oeqa: enable testresults.json for testexport

2020-03-12T22:49:28+00:00

Add the option --json-result-dir to oeqa core context to enable testresults.json creation for test runs via testexport. Eg. oe-test runtime --json-result-dir . (From OE-Core rev: 9d8edf33d1f5d89b310923b0aa3cc967317c7c49) Signed-off-by: Stefan Kral Signed-off-by: Richard Purdie

oeqa/selftest: Standardise seperate builddir for concurrent and non-concurrent selftest

2020-02-21T09:39:00+00:00

Currently oe-selftest reuses the current build directory and the concurrent version run with -j does not. Standardise and use a separate new build directory in both cases. This will lead to simpler code and more reliable user run tests. (From OE-Core rev: 50238b0717b04e0a1fa69d618e8c8aa8445a80b0) Signed-off-by: Richard Purdie

oeqa/core: Rework OETestTag and remove unused OETestFilter

2019-09-07T20:56:43+00:00

Rework OETestTag so that it does not rely on the existing decorator code base and instead inserts the tags into an attribute on the decorated target (e.g. class/type or method). This allows the use of OETestTag on classes and method. In order to filter tagged tests rework the loaders filtering code, removing the generic-ness (with validation and attributes/etc.) and replace it with a "tags_filter" parameter which is a function that filters a test based on the tags it has. This allows the loader user to filter on tags in more specific ways (e.g. include all untagged tests and any tests tagged with foo). Plumb all this through the context code and testing code. Update the associated tests to pass correctly with the changes. (From OE-Core rev: b8a4a4c2de68110d74607cb9807c9e741ca9441c) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

oeqa: avoid class setup method to run when skipping the whole class

2019-06-21T14:32:36+00:00

For now, even if we have specified to skip the whole module/class via command line, e.g., `oe-selftest -R gotoolchain', the class setup method is still run. This at least results in unnecessary builds, and at worst results in ERROR, if the setup method fails. So improve the skipping mechanism to avoid class setup method to run when specified to skip. (From OE-Core rev: b0b79bf65f5e5e65958090a4a88622b42df896bf) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

context.py: avoid skipping tests by meaningless command argument

2019-06-21T14:32:36+00:00

Currently `oe-selftest -R a' will skip 'archiver' tests. This is not expected. Fix it so that the '-R' should be followed by actual module/class/test names. (From OE-Core rev: de3b070fc2ddd0b63a324679ec5adbe30142fc22) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie