summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest/context.py
Commit message (Collapse)AuthorAgeFilesLines
* selftest: Replace bitbake -p with bitbake -eRichard Purdie2018-08-151-2/+2
| | | | | | | | | | | | | Parsing all the recipes is annoying when trying to re-execute oe-selftest and also unnecessary as its really just a sanity check. When the tests were originally being developed the guard was useful but less so now. Replace it with bitbake -e which is fast and checks the basic configuration is valid. (From OE-Core rev: acec5180b8d2731002979179e08439b615631e70) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: check if rm_work is enabledAnuj Mittal2018-07-301-0/+4
| | | | | | | | | | | | | | rm_work if enabled leads to some tests failing that rely on artifacts being present. Check if rm_work.bbclass is included and show an error and exit if it is. Fixes [YOCTO #12694] (From OE-Core rev: dde7e2f590834aa8034f1371954f9b4fbc7a60b7) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logging: use warning instead warnChen Qi2018-07-261-1/+1
| | | | | | | | | | | | | | | | The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ (From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Add selftest parallelisation supportRichard Purdie2018-07-181-2/+6
| | | | | | | | | | | | | | | | | | | This allows oe-selftest to take a -j option which specifies how much test parallelisation to use. Currently this is "module" based with each module being split and run in a separate build directory. Further splitting could be done but this seems a good compromise between test setup and parallelism. You need python-testtools and python-subunit installed to use this but only when the -j option is specified. See notes posted to the openedmbedded-architecture list for more details about the design choices here. Some of this functionality may make more sense in the oeqa core ultimately. (From OE-Core rev: 326ababfd620ae5ea29bf486b9d68ba3d60cad30) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* context: Include a command line argument to run all except certain testsLeonardo Sandoval2017-08-131-3/+8
| | | | | | | | | | | | | | | | | | | | | A new command line argument (-R, which is the oposite of current -r) that allows to run all test cases except the ones indicated through the command line. Some command line examples: * Run all except the distro test case: $ oe-selftest -R distrodata * Run all except the archiver test case and a single bblayers unit test $ oe-selftest -R archiver bblayers.BitbakeLayers.test_bitbakelayers_add_remove [YOCTO #11847] (From OE-Core rev: e40eeaa790b95d9c25832405c0b0d5b3a0d0292b) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/{core,selftest}: Add support to validate if a specified test case isn't ↵Aníbal Limón2017-07-301-2/+6
| | | | | | | | | | | | | | | | | found If some test module/case is specified to run and isn't found the OEQA framework didn't notice it, so complete the implementation using modules_required and validate for the test case prescense. Raise an exception when the test module/case required isn't found. [YOCTO #11645] (From OE-Core rev: e50b415aaaa1581473f85f0a8afa278b5f95129b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and SIGTERMAníbal Limón2017-07-211-31/+76
| | | | | | | | | | | | | In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration. [YOCTO #11650] (From OE-Core rev: 9419c81e69d2facc82e39c846466670c09e6b444) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: Reload testlayer_path when meta-selftest isn't addedAníbal Limón2017-06-231-0/+1
| | | | | | | | | | | When add meta-selftest by the script the testlayer_path needs to be reloaded to avoid None value. (From OE-Core rev: d1120849e12c46aa46b02905a3f43c0853bd11a0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Change the order to logDetails and logSummaryAníbal Limón2017-06-091-1/+1
| | | | | | | | | | | | | Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] (From OE-Core rev: 4e3ab36e8c90abc740cce1ba31faf6595116e1e2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: Migrate to new framework into oeqa.selftest.contextLeonardo Sandoval2017-06-061-0/+224
The new OEQA framework aims to re-use code into the different Test components. The previous oe-selftest implements it-self loading, run, and list test cases in a non-standard way (unittest base) and other functionalities like logging that is now on oeqa core. This ends on a compact oe-selftest script. All needed command line options was migrated but there are some of them pending of implementation and others deprecated. Deprecated options: list-tags: The tag functionality into the old oeqa framework isn't work, the selftest doesn't has tag decorators. {run, list}-tests-by: Ambiguos options it accepts all the posibilites module, class, name, id or tag. Remaining to implement: coverage: It enables covrage reports over a test run, currently isn't on on use and some bugs [1], i filed a bug to add support to OEQA core module in this way other Test components could enable it. repository: It push XML results into a git repository and isn't in use, i filed a bug to implement this into OEQA core module. [2] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0 [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0 (From OE-Core rev: 3b2a20eee4a39f40287bf67545839eaa09fc892d) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>