summaryrefslogtreecommitdiffstats
path: root/scripts/oe-selftest
Commit message (Collapse)AuthorAgeFilesLines
* oe-selftest: Enable code coverage on unit testsLeonardo Sandoval2015-12-011-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enable code coverage through the library 'python coverage'. In case the environment variable COVERAGE_PROCESS_START is present (one of the requisites for measuring sub-processes; the second one is including some coverage statements into the python sitecustomize.py file) it will be taken into account, otherwise it is exported with value '.coveragerc'. The latter value is a configuration file (also automatically created) with some default settings. Once tests are executed, a coverage report is shown on the log and the coverage output data is stored with name '.coverage.<args>' where '<args>' is the name of the unit tests executed or 'all_tests' when running with --run-all-tests. This output data can be latter used for better reporting using the same tool (coverage). As briefly indicate before, measuring sub-process implies setting the env variable COVERAGE_PROCESS_START (done automatically by the oe-selftest code with this patch if not already set) and creating a sitecustomize.py as explained on [1]. If either one of these is missing, complete coverage will be incomplete. Current measurements for 'oe-selftest --run-all-tests' indicate that current coverage is around 42 % taking into account BBLAYERS, bitbake and scripts folders. More details on [2], indicating the coverage per file/module. This tasks has been done together with Humberto Ibarra <humberto.ibarra.lopez@linux.intel.com> [YOCTO #8679] [1] http://coverage.readthedocs.org/en/latest/subprocess.html [2] https://bugzilla.yoctoproject.org/attachment.cgi?id=2854 (From OE-Core rev: b3feee2cefbbd98b66dc395b651f47c5028c80a0) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: add scripts/lib and bitbake/lib to pathPaul Eggleton2015-07-271-1/+4
| | | | | | | | | | In particular, this allows us to use code from bitbake's bb module (such as tinfoil). (From OE-Core rev: 5ca91cd0b56eb2da80c781fb46ae0b600c3f1eb9) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: add libdirs from BBPATH to sys.pathChristopher Larson2015-07-271-0/+7
| | | | | | | | | | | | This ensures that oeqa.selftest.* from layers are found. [YOCTO #7625] (From OE-Core rev: 182b8ed9f26ed8b6a8eb6dcaec82db9aca14b010) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: obey oeqa.selftest.__path__Christopher Larson2015-07-271-5/+6
| | | | | | | | | | | | | This ensures that all paths that hold selftest tests will be checked (oeqa.selftest is a namespace package). [YOCTO #7625] (From OE-Core rev: 3c60cbced7b101ee52ce4a0a0bce542fd38f1821) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: Added mechanism for including/removing bblayers.incDaniel Istrate2015-07-081-1/+15
| | | | | | | | | | | | When oe-selftest starts it includes bblayers.inc into bblayers.conf When oe-selftest ends it deletes bblayers.inc and the included line from bblayers.conf (From OE-Core rev: cf7bf27f565b34fdcd2caa25aaef068c0970965e) Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: timestamp the test runsRoss Burton2015-06-271-1/+12
| | | | | | | | | | The selftest can take a couple of hours to run, so add a custom result class to timestamp the output to make it easy to spot any slow tests. (From OE-Core rev: cce89c44948ee66ad0abb491be57e270038270e4) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: add command line option to list test classesAlexandru DAMIAN2015-02-081-0/+20
| | | | | | | | | | | | | | | While trying to discover what tests are available, I felt the need to be able to list all individual tests so I can run specific tests. This patch adds the "--list-classes" command line option that lists the unit test classes and methods available. (From OE-Core rev: f0ba400a398a172f9ebf953bb3a26e5d911a17d6) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: fix startup backtraceAlexandru DAMIAN2015-02-081-1/+3
| | | | | | | | | | | Fixing a backtrace dump that happens if the script is started without sourcing the oe-init-build-env first. (From OE-Core rev: 0d769abcab272f41d74ed4d7915d26c7c309253a) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: add command-line parsing and optionsCorneliu Stoicescu2014-07-191-28/+53
| | | | | | | | | [YOCTO #6453] (From OE-Core rev: d9291390d56cae9c9d0f5e44d5e7293260dad077) Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: return based on the test resultsStefan Stanacar2014-02-091-2/+4
| | | | | | | | | | Regardless if the tests passed or not the script returned 0, which isn't what one would expect. (From OE-Core rev: c38f943c7fbb1fc077c875099dce8f73f41043b9) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/selftest/base, scripts/oe-selftest: fix wrong remove path and do a ↵Stefan Stanacar2013-12-051-0/+12
| | | | | | | | | | | | | | | | complete cleanup at the end The script should clean-up all the .inc files that might have been created by tests regardless of the outcome or if the script is interrupted. (currently the last test will leave a conf/selftest.inc around, even if it's not included anywhere) Also fix delete_recipeinc to actually delete what's supposed to. (From OE-Core rev: 6008745c56800e0f5f01a756be0701cebd9de4ae) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-selftest: script to run builds as unittest against bitbake or ↵Stefan Stanacar2013-12-031-0/+148
various scripts The purpose of oe-selftest is to run unittest modules added from meta/lib/oeqa/selftest, which are tests against bitbake tools. Right now the script it's useful for simple tests like: - "bitbake --someoption, change some metadata, bitbake X, check something" type scenarios (PR service, error output, etc) - or "bitbake-layers <...>" type scripts and yocto-bsp tools. This commit also adds some helper modules that the tests will use and a base class. Also, most of the tests will have a dependency on a meta-selftest layer which contains specially modified recipes/bbappends/include files for the purpose of the tests. The tests themselves will usually write to ".inc" files from the layer or in conf/selftest.inc (which is added as an include in local.conf at the start and removed at the end) It's a simple matter or sourcing the enviroment, adding the meta-selftest layer to bblayers.conf and running: oe-selftest to get some results. It would finish faster if at least a core-image-minimal was built before. [ YOCTO #4740 ] (From OE-Core rev: 41a4f8fb005328d3a631a9036ceb6dcf75754410) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>