summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/core/loader.py
Commit message (Collapse)AuthorAgeFilesLines
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-06-181-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: 3248a9e3c5a197321b1c4417509b9309cc3bae97) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/loader: Fix deprecation warningRichard Purdie2018-11-141-1/+1
| | | | | | | | | | Clean up the warning: meta/lib/oeqa/core/loader.py:27: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec() _failed_test_args = inspect.getargspec(unittest.loader._make_failed_test).args (From OE-Core rev: d2deb66830be2d44532fea3d5db763b57778252a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/loader: Ensure invalid test names don't trigger a tracebackRichard Purdie2018-07-261-0/+2
| | | | | | | | | | | oe-selftest -r <invalid test name> currently triggers a traceback. Ensure this doesn't happen and the user gets a sensible error message. (From OE-Core rev: f2042bf3638ed4edfb167e7f7d4be6da60997ead) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader.py: support the 'auto' keywordChen Qi2018-06-041-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In previous OEQA, having 'auto' in TEST_SUITES results in executing as many test cases as possible. This behaviour is broken for now. From the codes in core/loader.py, I can see that it tries to use another keyword 'all'. But in fact, it does not work. I've checked the current manual. The manual says using 'auto'. Below is the current information in manual. """ Alternatively, you can provide the "auto" option to have all applicable tests run against the image. TEST_SUITES_append = " auto" """ So we should restore this behaviour. This patch does so. Also, output warning message is some module is named as 'auto', as this is a reserved keyword. (From OE-Core rev: a65460a063a958cc887c756db5f7ab18e3f5a8c1) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* core/loader.py: fix regex to include all available test casesLeonardo Sandoval2018-01-041-1/+1
| | | | | | | | | | | | Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all. [YOCTO #12385] (From OE-Core rev: 1ecf48fd286a77078451b67879a44f9c9dc7a894) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Make _built_modules_dict() support packages correctlyPeter Kjellerstedt2017-11-081-36/+14
| | | | | | | | | | | | | | | | For test modules in a package, e.g., oelib.license, running `oe-selftest -r oelib.license` or `oe-selftest -r oelib.license.TestSimpleCombinations` would fail with a message that the specified test cases could not be found. This was due to the parsing in _built_modules_dict(), which failed to distinguish between <package>.<module>.<class> and <module>.<class>.<testcase> and treated both cases as the latter. (From OE-Core rev: 80db3d999ae26d298d9d5418a32b11a4f27af9d5) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/{core,selftest}: Add support to validate if a specified test case isn't ↵Aníbal Limón2017-07-301-0/+26
| | | | | | | | | | | | | | | | | found If some test module/case is specified to run and isn't found the OEQA framework didn't notice it, so complete the implementation using modules_required and validate for the test case prescense. Raise an exception when the test module/case required isn't found. [YOCTO #11645] (From OE-Core rev: e50b415aaaa1581473f85f0a8afa278b5f95129b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Generate function _make_failed_test dynamicallyAníbal Limón2017-07-211-13/+12
| | | | | | | | | | | | | | | | | | | | | | Python versions has different features from branches 3.4.x, 3.5.x and 3.6.x, i expected in wrong mode that was incremental for example changes in 3.4.4 be in 3.5.x but that's not true. The _make_failed_test internal method differs and is only available in certain versions >= 3.4.4 and in 3.5.x and 3.6.x branches but not realeses have been made including it. So to avoid futher problems inspect the _make_failed_test and generates function definition according what parameters are needed, the unique supossition is that exception argument is always passed. Related to, http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d8380d098a290510b442a7abd2dd5a50cabf5844 (From OE-Core rev: b4740d2b325a80bcecc5e56dff9add9081fcd31b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader.py: Fix _make_failed_test for python >= 3.4.4Aníbal Limón2017-06-231-9/+13
| | | | | | | | | | | Python unittest change the signature of the _make_failed_test after python 3.4.4 don't pass the method name. (From OE-Core rev: 767b68e6ca22512ff80e6fbc42154f3f0c2206c0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Use full and small module name on filteringAníbal Limón2017-06-131-9/+19
| | | | | | | | | | | | | The small module name was added to support run a whole suite that has more that 3 levels in the test case name, but this broke the behaviour for use a full test case name. [YOCTO #11632] (From OE-Core rev: 9ab20ceb5801bee8dd8b218b3928720da5e1d403) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Allow unittest.TestCase's to be executedAníbal Limón2017-06-121-5/+7
| | | | | | | | | | | | | | | Currently there was a restriction to only execute tests that's inherits from OETestCase but in some circunstancies the features from the OEQA framework isn't needed so we need to support basic unittests. [YOCTO #10828] (From OE-Core rev: baac26f1b36e89e07637b738dd31ec7356f05a02) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Fix filtering on test modules with submodulesAníbal Limón2017-06-121-36/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our filtering allows to specify which tests to run using, <module_name>.[test_class].[test_name] But the module name logic was restricted to only accept one level, for example: runtime_test vs oelib.types, to support multiple submodules use only the first part for filtering. This allows to run the whole tests in a module with more than tree levels. Due to the ambiguity on the test filtering options with test cases with more than tree levels the supported sytnax is, <module> or <module>.[submoduleN].[test_class].[test_name] [YOCTO #11632] (From OE-Core rev: cf2ee12b007e5570959ccfbb643159b21d90426e) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader: Call parent init method in OETestLoaderAnibal Limon2017-06-061-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After Python 3.5 the init method contains initialization of _loading_packages set to avoid infinite loops in recursive loading. Fix, Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 70, in <module> ret = main() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 57, in main results = args.func(logger, args) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 214, in run rc = self._internal_run(logger, args) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 175, in _internal_run self.tc.loadTests(self.module_paths, **self.tc_kwargs['load']) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/context.py", line 51, in loadTests self.suites = self.loader.discover() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/loader.py", line 268, in discover pattern='*.py', top_level_dir=path) File "/usr/lib/python3.5/unittest/loader.py", line 341, in discover tests = list(self._find_tests(start_dir, pattern)) File "/usr/lib/python3.5/unittest/loader.py", line 398, in _find_tests full_path, pattern, namespace) File "/usr/lib/python3.5/unittest/loader.py", line 473, in _find_test_path self._loading_packages.add(name) AttributeError: 'OETestLoader' object has no attribute '_loading_packages' (From OE-Core rev: 17b832ba05d640570a773adbff2fa4e9ff83fce3) Signed-off-by: Anibal Limon <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core: Improve grammar in README and loader commentsRandy MacLeod2017-06-051-4/+4
| | | | | | | (From OE-Core rev: 48cd7ff7ae61ca5772a42ece9fce51976dd0d790) Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core: Add support for run tests by module, class and nameAníbal Limón2017-06-021-1/+65
| | | | | | | | | | This will enable only run certain module tests and filter by class and test name. (From OE-Core rev: 0f7531803aa6ffef2b7c5be9acee3efe7d2974ef) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader.py: Do not import underscore modules by defaultMariano Lopez2017-03-281-4/+30
| | | | | | | | | | | | | | | | Underscore modules are meant to be run only when manually added to the test suite, so far another mechanisms are in place to make this happen with runtime, sdk, and esdk (mostly in test* bbclasses). This will add such functionality in the core framework so other specific frameworks can take use this without adding something else. [YOCTO #10980] (From OE-Core rev: 2c6eac774768aa610a8b3784483b9e90fb629c2d) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader.py: Avoid importing tests with built-ins nameMariano Lopez2017-03-011-0/+11
| | | | | | | | | | | | | | | If importing a test with the same name as a built-in module, it will silently import the built-in and check for tests in built-in module. This happened with syslog module in debian based machines, so add a raise to avoid this behavior. [YOCTO #10978] (From OE-Core rev: d9548f981448307b042807373e469f0d0b110bfe) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/loader.py: Give meaningful error when failed to load classesMariano Lopez2017-03-011-4/+4
| | | | | | | | | | | With this we get the class that is actually having the problem, not just a TypeError with an unknown class causing the error. (From OE-Core rev: d6ff4891376417504018af27e8e729a412feeeea) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/loader.py: Avoid error in TypeError exceptionJuro Bystricky2017-02-151-1/+1
| | | | | | | | | Without this an exception would occur (From OE-Core rev: d1bda068d4bc0ad3ff81ab1f104dc12cedd20b3f) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core: Add loader, context and decorator modulesAníbal Limón2017-01-231-0/+235
loader: Implements OETestLoader handling OETestDecorator and filtering support when load tests. The OETestLoader is responsible to set custom methods, attrs of the OEQA frameowork. [YOCTO #10231] [YOCTO #10317] [YOCTO #10353] decorator: Add base class OETestDecorator to provide a common way to define decorators to be used over OETestCase's, every decorator has a method to be called when loading tests and before test execution starts. Special decorators could be implemented for filter tests on loading phase. context: Provides HIGH level API for loadTests and runTests of certain test component (i.e. runtime, sdk, selftest). [YOCTO #10230] (From OE-Core rev: 275ef03b77ef5f23b75cb01c55206d1ab0261342) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-09 08:06:59 +0000