summaryrefslogtreecommitdiffstats
path: root/meta/classes/testsdk.bbclass
Commit message (Collapse)AuthorAgeFilesLines
* testsdk: Enable multiprocess executionRichard Purdie2018-07-181-1/+6
| | | | | | | | | This uses the new concurrenttest code to enable parallel test execution if specified. (From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/threaded: Remove in favour of using concurrenttestsRichard Purdie2018-07-181-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. (From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: set PREMIRRORS for kernel to speed up testChen Qi2018-07-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently if we do a testsdkext task for the image, it would take very long time to finish. The time is mostly spent fetching kernel source via network. We have done some configuration in auto.conf, trying to make use of own-mirrors.bbclass to avoid fetching kernel via network. However, the solution normally does not work. Below is some log from log.do_fetch. DEBUG: Fetcher failure: Unable to find file \ file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \ anywhere. The tar.gz file is not available. It is generated only if BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users need choose to set it to "1" only if they are trying to make a source mirror. So generally, this var's value is "0". Anyway, we do need to avoid fetching kernel source from network when doing testsdkext. So set PREMIRRORS in auto.conf to achieve this. After this change, the time reduces from 4209.131s to 1399.436s on my local machine. [YOCTO #12729] (From OE-Core rev: fd18ddb0664f69d77681590774b0123251a98728) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image/testimage: Rework auto image test executionRichard Purdie2018-06-211-0/+5
| | | | | | | | | | | | | | | | | | | | | | The TEST_IMAGE interface has never particularly worked and image testing currently gets configured manually. This reworks the interface to better serve the needs of its users, replacing it with TESTIMAGE_AUTO. This does away with the need for the separate class due to better bitbake APIs for changing tasks. TESTIMAGE_AUTO will automatically boot under qemu any image that is built. It also adds in dependencies so that any SDK for which testing is requested will automatically be built first. The code in bitbake.conf was error prone (e.g. testsdk wasn't considered), this improves it to standardise on IMAGE_CLASSES as the standard configuration mechanism. (From OE-Core rev: b34d44f3dfea8254826a46701a4fe3769a900434) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: add a newline after own-mirrorsRobert Yang2018-01-061-1/+1
| | | | | | | | | | Otherwise it would generate lines like the following when multilib: INHERIT += "own-mirrors"SSTATE_MIRRORS += " \n file://.* file:///path/to/../share/sstate-cache/PATH" (From OE-Core rev: 65b2a0e1ba7e176f3e405ed8968665660fc414d3) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* staging.bbclass: make subprocess.check_output() capture stderrRobert Yang2018-01-061-1/+2
| | | | | | | | | This is very useful for debugging. The similar to testsdk.bbclass. (From OE-Core rev: de30b61b87047f61de4629f8e5bd87598de7fd0c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Change the order to logDetails and logSummaryAníbal Limón2017-06-091-2/+2
| | | | | | | | | | | | | Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] (From OE-Core rev: 4e3ab36e8c90abc740cce1ba31faf6595116e1e2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: Enable usage of OEQA thread modeAníbal Limón2017-06-021-0/+4
| | | | | | | | | | | | | oeqa/sdk/context.py - Use OETestContextThreaded. classes/testsdk.bbclass - Enable bb event thread mode to avoid corrupt the PIPE when multiple threads writes. [YOCTO #11450] (From OE-Core rev: 598c6579932c2ca1dbdb022c8bec8af2e6c21e6b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core: Move OETestContext.log{Summary, Details} into OETestResultAníbal Limón2017-06-021-4/+4
| | | | | | | | | | | | | Those methods are used to write in the log the results so it makes sense to have defined at OETestResult because is a format of the result itself. [YOCTO #11450] (From OE-Core rev: 33a783f59ed4e232f41f8b09dfa7955f2ddc2f80) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: Remove unused import of OEStreamLoggerAníbal Limón2017-06-021-1/+0
| | | | | | | (From OE-Core rev: ab434125b9121e3d7c463a4b35ae60a4395f8392) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk: Fix lockfile patchRichard Purdie2017-03-031-3/+0
| | | | | | | | | In the previous lock removal patch I mixed up the two tasks and broke sdk testing. Fix this by removing locks for both sdk and extsdk. (From OE-Core rev: 20c6e876ed6b8c8ac007848e2b36e9a8a56bb703) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk: Drop lockfile usageRichard Purdie2017-03-031-3/+1
| | | | | | | | | | | | We should be able to test sdks in parallel. As far as I can tell, this restriction crept in for no good reason, based on locks that testimage used for the non-qemu case. Therefore remove the lock and allow sdks to test in parallel. (From OE-Core rev: 54477664dc13c6aa15b3246f328b66591a340659) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk: Use auto.conf instead of local.conf to set SSTATE_MIRRORRichard Purdie2017-03-021-2/+3
| | | | | | | | | | | | The local.conf file can get overritten and changed by the SDK so (ab)use auto.conf instead to ensure our configuration tweaks get applied. Also ensure we append to any other configured SSTATE_MIRROR variable that already exists. (From OE-Core rev: 444a71b1c2d8f86fc4c35c8ff8e92bcbbe646b75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdkext/devtool: use finally instead of repeating cleanupRoss Burton2017-03-011-1/+3
| | | | | | | | | | | | | Use the finally: block to always to cleanup. Now that the test harness in testsdk.bbclass has monkey-patched CalledProcessException to display the output we don't need to do that in the test case. (From OE-Core rev: 9f0f6326083ee76b72b431fbfcbe12c1ab2793b2) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk: Handle minimal eSDK and avoid download costsRichard Purdie2017-03-011-0/+11
| | | | | | | | | | | | | | | | | | | | | When using a minimal eSDK, testing currently fails as the sdk isn't populated. We therefore setup the eSDK under test to point at local sstate and execute a command to ensure the toolchain is populated since most of the tests depend on this being present. At the same time, add in a link to DL_DIR through own-mirrors so that tests which fetch source (e.g. the kernel module one) can use the local stash. This cuts test execution of the kernel module test from 2000s to 120s. We did try using DL_DIR directly but that causes uninative issues requiring other workarounds so own-mirrors is neater. Together these fixes unbreak eSDK testing on the autobuilder. (From OE-Core rev: 866b55905eabf93f45c10a08d5b53c459ac4c056) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: remove remaining True option to getVar callsMing Liu2017-01-311-2/+2
| | | | | | | | | | | | | | | | This is a complementary fix to commit 7c552996: [ meta: remove True option to getVar calls ] it intended to remove all True option to getVar calls, but there are still some remaining. Search made with the following regex: getVar ?\((.*), True\) (From OE-Core rev: 87d03ffe03d6f01e360bfd51714be96e62506e0a) Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Migrate to use the new OESDKExtTestContextAníbal Limón2017-01-231-53/+45
| | | | | | | (From OE-Core rev: b254822dad850ce74563c83b7a9e31463501baa7) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Remove the need of TEST_LOG_DIR variableAníbal Limón2017-01-231-6/+0
| | | | | | | | | | | | | The TEST_LOG_DIR was used for store sdk_target_log this log contains the output of the run of build commands now that information could be found also on log.do_testsdk under WORKDIR. The log will continue to store into SDK_DIR instead of TEST_LOG_DIR. (From OE-Core rev: 1c9ba4b698bab916d42b58255692a7bf3d773bbc) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContextAníbal Limón2017-01-231-15/+60
| | | | | | | | | | The functionality provided is the same with imporvements on code reuse and better interfaces. (From OE-Core rev: 7a1ae3149965b162fb2c71fc7067e07a7a189249) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: remove True option to getVar callsJoshua Lock2016-12-161-6/+6
| | | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: print which file is not foundRobert Yang2016-11-231-3/+3
| | | | | | | | | | This is helpful when debug. (From OE-Core rev: eef945787b5717bbb5d3bbddc446d4da86e3eb62) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: Clean up comments, clarify image choices.Robert P. J. Day2016-11-061-6/+8
| | | | | | | | | | Make it clear that SDK testing can use any valid image. (From OE-Core rev: d190c69347921a626665a53469dcf99b3c86994b) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk.bbclass: Use bb.fatal() instead of raising FuncFailedUlf Magnusson2016-10-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | This sets a good example and avoids unnecessarily contributing to perceived complexity and cargo culting. Motivating quote below: < kergoth> the *original* intent was for the function/task to error via whatever appropriate means, bb.fatal, whatever, and funcfailed was what you'd catch if you were calling exec_func/exec_task. that is, it's what those functions raise, not what metadata functions should be raising < kergoth> it didn't end up being used that way < kergoth> but there's really never a reason to raise it yourself FuncFailed.__init__ takes a 'name' argument rather than a 'msg' argument, which also shows that the original purpose got lost. (From OE-Core rev: 086240468265dc15c5b4cdb2594d5aa7c3114dda) Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/populate_sdk_ext: eliminate double execution on installPaul Eggleton2016-07-081-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new BB_SETSCENE_ENFORCE functionality to avoid having to run bitbake twice on installing the extensible SDK - we can now do it all in one invocation which not only takes less time, but we should also get more meaningful errors for some types of failure, in particular where downloading from an sstate mirror fails. One result of this change is that you get the errors printed on the console during normal output rather than this going to the preparing_build_system.log file first. In OE-Core revision 227d2cbf9e0b8c35fa6644e3d72e0699db9607fa, we changed to always print the contents of preparing_build_system.log on failure, but now at least the error contents of that log is duplicated. Besides, I intentionally didn't print out the contents of that log during normal usage because it's quite verbose - the bug that we were attempting to fix was about not getting this information when seeing failures in the automated tests, thus I've moved printing the log to the test handling code instead. Part of the implementation of [YOCTO #9367]. (From OE-Core rev: e1390c1ef85862b91b067ab24f3c06ca506155ad) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: print output correctly on failurePaul Eggleton2016-07-081-2/+2
| | | | | | | | | | | With Python 3 we get a bytes object from the command output and not a string, which gives some ugly formatting for error messages unless you decode it first. (From OE-Core rev: 798bec6fe43116b51247284eb4e415337b2e8e04) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Pass tcname to SDK and SDKExt contextsAníbal Limón2016-02-281-1/+1
| | | | | | | | | | | tcname is needed for eSDK update testcase will be used for publish it and then try to update (From OE-Core rev: a75944a63482597be88ff0f3ce55025647b78e2c) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Move the removal of bitbake PATH to eSDK context onlyAníbal Limón2016-02-281-5/+4
| | | | | | | | | | | | | | The removal of bitbake and scripts PATH is only needed by eSDK tests so move to eSDK context only. This also it's a support for eSDK update test because it needs to execute oe-publish-sdk from scripts. (From OE-Core rev: 4ae0c84568f39661722cbceba8ddab22cffe5003) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Move code for avoid PATHs to oeqa.utilsAníbal Limón2016-02-281-14/+4
| | | | | | | | | | Due to the neeed to use in other modules. (From OE-Core rev: a25aef2bdefae54c8b3eb2bd4afec5a86110ddc7) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: do_testsdkext avoid STAGING_DIR/BASE_WORKDIR in PATHAníbal Limón2016-02-111-1/+10
| | | | | | | | | | | The inclusion of STAGING_DIR/BASE_WORKDIR in PATH is contaminating the environment, i.e. when try to sanity check perl (check_perl_modules) it takes perl from STAGING_DIR causing eSDK install to fail. (From OE-Core rev: 32611395e1da21f6f7f7916fb8077e35ee81bb23) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add help information on how to run tests.Aníbal Limón2016-02-101-0/+10
| | | | | | | | (From OE-Core rev: fcf0dc253f914418e15e2c6afc6f649b9270824a) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/oetest: Fix compatibility SDK tests using eSDK.Aníbal Limón2016-02-101-3/+3
| | | | | | | | | | | | | | | | | When run SDK tests over eSDK we need to use SDKExtTestContext instead of SDKTestContext because if we use SDKTestContext search for SDK manifest and depends on the SDK manifest generation so populate_sdk needs to be executed. Adds a compatibility mode flag to SDKExtTestContext for search tests over sdk module instead of sdkext module and change testsdk calls to comply with this new param. (From OE-Core rev: ebe743235b383b17225553d84bf8e6f80d364dcd) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdkext: Add skeleton for support Extensible SDK tests.Aníbal Limón2016-02-101-1/+9
| | | | | | | | | | | | | | oeqa/sdkext: Add module and __init__.py will contain eSDK tests. classes/testsdk: Add support for run eSDK tests. oeqa/oetest: Create oeSDKExtTest for now only inherit oeSDKTest, modified SDKExtTestContext now inherit SDKTestContext and set sdkext filesdir for store data fixtures. (From OE-Core rev: f3781544a5c077610498a6b7dc5244ee4c5bc6df) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add compatibility SDK testsuite to eSDKAníbal Limón2016-02-101-5/+19
| | | | | | | | | | | Extensible SDK is compatible with SDK test suite so it need to execute the same tests over it. (From OE-Core rev: fb9db1698039ca51ae52ea623d2ca845753f8260) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add function run_test_contextAníbal Limón2016-02-101-30/+32
| | | | | | | | | | | This helper functions will be serve as well to run extensible sdk tests so generalize it to get function context as arg. (From OE-Core rev: 51782e5f77cb3f32a31a221c6e0f9b33cf3a33c9) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oetest.py/TestContext: Move loadTests and runTests inside it.Aníbal Limón2016-02-101-3/+3
| | | | | | | | | | | | Method's for loadTests and runTests make sense to define inside TestContext because it can be different around Image, SDK, SDKExt. (From OE-Core rev: 03af7b99e3ce36ce3e29dc31e33d2cc74eb14849) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/testsdk: Move get test suites routine inside TestContext.Aníbal Limón2016-02-101-17/+2
| | | | | | | | | | | In order to provide better abstraction move functions to get the test suite inside the TestContext. (From OE-Core rev: 552285688441330440bd1bc138aeaea87549024a) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/testsdk: Modularize TestContext.Aníbal Limón2016-02-101-26/+3
| | | | | | | | | | | | Move anonymous duplicated class TestContext from testimage/testsdk to oeqa/oetest now we have two new classes ImageTestContext and SDKTestContext with common code in TestContext class. (From OE-Core rev: 593f2fdf6ee94c5f91761a669048da3598cbe3fa) Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add call to export_proxies on testsdkext.Aníbal Limón2016-02-101-0/+3
| | | | | | | | | | | Extensible SDK needs to use network and some networks requires proxies then export it. (From OE-Core rev: 1c79fce34a02fee696bccb085a09957d71d87d56) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add testsdkext task only install.Aníbal Limón2016-02-101-0/+45
| | | | | | | | | | Add task for test extensible sdk for now only install the SDK. (From OE-Core rev: d5061e07c4fd6ea37a0ceacedb83c7878f5bfd6d) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testsdk: Add new class testsdk.Aníbal Limón2016-02-101-0/+100
Moves all the testsdk code from testimage in order to have it's own class because new tests will be added for extensible SDK. The old paths for store logs "${WORKDIR}/testimage" and sdk "${WORKDIR}/testimage-sdk" was maintained for compatibility may be change to point testsdk after review the codebase. The dependency of QEMU was removed because isn't needed. (From OE-Core rev: b30edc18866865ec757b3fd86eb84de530720acf) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-20 07:23:49 +0000