summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest/context.py
Commit message (Collapse)AuthorAgeFilesLines
* selftest/context: Avoid tracebacks from tests using multiprocessingRichard Purdie2020-08-041-1/+5
| | | | | | | | | | | | | | | | | We can see tracebacks where the SIGTERM handler catches things it shouldn't. Avoid exit(1) unless we're the process that it was intended for. [YOCTO #13664] (From OE-Core rev: d9c62ffac611310efd47ed6397d31dccb72fe868) (From OE-Core rev: 45b4bd7b4d30d81bdff0d471e8d97c2322ed2f75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dba8c1d5ef0b574b7772d59e5992bfad8b7cca13) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context.py: For -t/-T use append argparse actionNathan Rossi2019-09-111-6/+6
| | | | | | | | | | | | | Use the 'append' action of argparse instead of nargs. This changes the behaviour of the option from "-t foo bar -r" to "-t foo -t bar -r". Additionally rename the long form options to be consistent with behaviour, such that they specifying a single tag at a time. (From OE-Core rev: ffe9e4303fa9799d2e8af9188853a262e15af226) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context.py: Change -t/-T args to be optionalNathan Rossi2019-09-071-11/+19
| | | | | | | | | | | Change the -t/-T args to be optional so that they can be used together with the existing -r/-a/... args to run a more flexible filtering of test tags. (From OE-Core rev: 55ee27bb07113a45da18711b5509764f62be4d75) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Add test run filtering based on test tagsNathan Rossi2019-09-071-0/+11
| | | | | | | | | | | Add '--run-only-tags' for running tests which match any of the provided tags, and '--run-exclude-tags' for running all tests except those that have any of the provided tags. (From OE-Core rev: 0dc3caf21c4519ef16c2ac99b93c03e23aab61d9) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-091-1/+4
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: ensure log directory existsChen Qi2018-12-081-0/+1
| | | | | | | | | | | Ensure log directory exists to avoid the following error. FileNotFoundError: [Errno 2] No such file or directory: '/.../build-selftest/tmp/log/oe-selftest-results-20181207043431.log' (From OE-Core rev: 6c41de48db76087fee596d9440d8f05346ab1094) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: Improve log file handlingRichard Purdie2018-12-011-3/+8
| | | | | | | | | | | | | | | | The existing logfile is simply placed in the current directory. Since the test changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid directory. We also see trackbacks if the symlink is invalid. Improve things by: * Placing logs in LOG_DIR (or BUILDDIR if unset). * Using a full path to the log meaning the log and link are placed in the same directory. * Using lexists instead of exists so invalid symlinks are handled correctly. (From OE-Core rev: 750ece11bed0e62a11e0003d1d16a81f7c219761) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation ↵Richard Purdie2018-11-161-1/+2
| | | | | | | | | | warnings Use our own lsb function instead as used elsewhere by the codebase. (From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: Replace deprecated imp module usageRichard Purdie2018-11-161-2/+2
| | | | | | | | | | | | | Avoid the warning: meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp In this case importlib is a direct replacement. (From OE-Core rev: db7a60c36a2d3eefc61ae6e1ede01680dc932035) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Standardize json logging output directoryYeoh Ee Peng2018-11-091-1/+1
| | | | | | | | | | | | Currently sdk & sdkext will output json file to LOG_DIR, while selftest will output json file to TOPDIR/log. Standardize selftest json output file to LOG_DIR. (From OE-Core rev: 2012d6b076fc833864a0254d56d78536314a6799) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Improvements to the json loggingRichard Purdie2018-10-291-14/+11
| | | | | | | | | | | | | Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes (From OE-Core rev: 173f59acf9722e2ef27fdd49c20f7d3d664917eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: write testresult to json filesYeoh Ee Peng2018-10-291-3/+31
| | | | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the oe-selftest log directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. (From OE-Core rev: 10697165c832e3dbb2913b6215164ea75e23ec23) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: Replace bitbake -p with bitbake -eRichard Purdie2018-08-151-2/+2
| | | | | | | | | | | | | Parsing all the recipes is annoying when trying to re-execute oe-selftest and also unnecessary as its really just a sanity check. When the tests were originally being developed the guard was useful but less so now. Replace it with bitbake -e which is fast and checks the basic configuration is valid. (From OE-Core rev: acec5180b8d2731002979179e08439b615631e70) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: check if rm_work is enabledAnuj Mittal2018-07-301-0/+4
| | | | | | | | | | | | | | rm_work if enabled leads to some tests failing that rely on artifacts being present. Check if rm_work.bbclass is included and show an error and exit if it is. Fixes [YOCTO #12694] (From OE-Core rev: dde7e2f590834aa8034f1371954f9b4fbc7a60b7) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logging: use warning instead warnChen Qi2018-07-261-1/+1
| | | | | | | | | | | | | | | | The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ (From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Add selftest parallelisation supportRichard Purdie2018-07-181-2/+6
| | | | | | | | | | | | | | | | | | | This allows oe-selftest to take a -j option which specifies how much test parallelisation to use. Currently this is "module" based with each module being split and run in a separate build directory. Further splitting could be done but this seems a good compromise between test setup and parallelism. You need python-testtools and python-subunit installed to use this but only when the -j option is specified. See notes posted to the openedmbedded-architecture list for more details about the design choices here. Some of this functionality may make more sense in the oeqa core ultimately. (From OE-Core rev: 326ababfd620ae5ea29bf486b9d68ba3d60cad30) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* context: Include a command line argument to run all except certain testsLeonardo Sandoval2017-08-131-3/+8
| | | | | | | | | | | | | | | | | | | | | A new command line argument (-R, which is the oposite of current -r) that allows to run all test cases except the ones indicated through the command line. Some command line examples: * Run all except the distro test case: $ oe-selftest -R distrodata * Run all except the archiver test case and a single bblayers unit test $ oe-selftest -R archiver bblayers.BitbakeLayers.test_bitbakelayers_add_remove [YOCTO #11847] (From OE-Core rev: e40eeaa790b95d9c25832405c0b0d5b3a0d0292b) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/{core,selftest}: Add support to validate if a specified test case isn't ↵Aníbal Limón2017-07-301-2/+6
| | | | | | | | | | | | | | | | | found If some test module/case is specified to run and isn't found the OEQA framework didn't notice it, so complete the implementation using modules_required and validate for the test case prescense. Raise an exception when the test module/case required isn't found. [YOCTO #11645] (From OE-Core rev: e50b415aaaa1581473f85f0a8afa278b5f95129b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and SIGTERMAníbal Limón2017-07-211-31/+76
| | | | | | | | | | | | | In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration. [YOCTO #11650] (From OE-Core rev: 9419c81e69d2facc82e39c846466670c09e6b444) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: Reload testlayer_path when meta-selftest isn't addedAníbal Limón2017-06-231-0/+1
| | | | | | | | | | | When add meta-selftest by the script the testlayer_path needs to be reloaded to avoid None value. (From OE-Core rev: d1120849e12c46aa46b02905a3f43c0853bd11a0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Change the order to logDetails and logSummaryAníbal Limón2017-06-091-1/+1
| | | | | | | | | | | | | Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] (From OE-Core rev: 4e3ab36e8c90abc740cce1ba31faf6595116e1e2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: Migrate to new framework into oeqa.selftest.contextLeonardo Sandoval2017-06-061-0/+224
The new OEQA framework aims to re-use code into the different Test components. The previous oe-selftest implements it-self loading, run, and list test cases in a non-standard way (unittest base) and other functionalities like logging that is now on oeqa core. This ends on a compact oe-selftest script. All needed command line options was migrated but there are some of them pending of implementation and others deprecated. Deprecated options: list-tags: The tag functionality into the old oeqa framework isn't work, the selftest doesn't has tag decorators. {run, list}-tests-by: Ambiguos options it accepts all the posibilites module, class, name, id or tag. Remaining to implement: coverage: It enables covrage reports over a test run, currently isn't on on use and some bugs [1], i filed a bug to add support to OEQA core module in this way other Test components could enable it. repository: It push XML results into a git repository and isn't in use, i filed a bug to implement this into OEQA core module. [2] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0 [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0 (From OE-Core rev: 3b2a20eee4a39f40287bf67545839eaa09fc892d) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-08 06:37:41 +0000