linux/poky.git/meta/classes/testsdk.bbclass, branch master-uninative

classes: Only allow network in existing network accessing code

2022-01-12T21:10:24+00:00

Use the newly added network task flag against tasks where network access is expected. This is do_fetch, do_checkuri, do_testimage, do_testsdk and do_testsdkext. We can't disable networking in sstate tasks due to sstate downloads and also so we can report hash equivalence to the server so network access is enabled in sstate tasks. Access within build-appliance do_image is also allowed due to the use of pip, this is a poor example made rather obvious now and needs to be reworked. Network access anywhere else in any other task isn't allowed. (From OE-Core rev: 7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4) Signed-off-by: Richard Purdie

classes/testsdk: Split implementation into classes

2018-11-21T11:54:14+00:00

Splits the SDK test implementation into configurable Python classes. The classes used for the normal and extensible SDKs are ${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively. This allows SDK machines to override the classes used to implement the tests. For the traditional SDK, a common "run()" function is provided by the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions that can be overridden in child classes, making it easier to have consistent behavior. The extensible SDK class (oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but no hooks have yet been added as there is not currently a known use case for create derived classes. These changes should be purely organizational; no functional changes have been made to either the standard SDK or extensible SDK tests. [YOCTO #13020] (From OE-Core rev: a06d53928b22d5f88276023c4d57b206db2f27f9) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation warnings

2018-11-16T11:46:07+00:00

Use our own lsb function instead as used elsewhere by the codebase. (From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3) Signed-off-by: Richard Purdie

testsdk: Improvements to the json logging

2018-10-29T17:26:47+00:00

Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Add MACHINE to the identifier * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes (From OE-Core rev: c272079d662237b1b9f87940c88cbadaeda9a9d1) Signed-off-by: Richard Purdie

testsdk.bbclass: write testresult to json files

2018-10-29T17:26:47+00:00

As part of the solution to replace Testopia to store testresult, OEQA sdk and sdkext need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the "WORKDIR" directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. (From OE-Core rev: cdbedba7da9c3f97a642d79a5e8ec363ba938e35) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

testsdk.bbclass: check python module testtools and subunit

2018-07-30T11:44:35+00:00

The testtools or subunit is not part of python's standard library, so check them before use. Fixed when they are not installed on host: $ bitbake core-image-minimal -ctestsdk Exception: ImportError: No module named 'testtools' Now it can run with a warning: WARNING: core-image-minimal-1.0-r0 do_testsdk: Failed to import testtools or subunit, the testcases will run serially (From OE-Core rev: ae53461608c4e71533378369b3f623b4c9002e39) Signed-off-by: Robert Yang Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

testsdk: Enable multiprocess execution

2018-07-18T09:18:41+00:00

This uses the new concurrenttest code to enable parallel test execution if specified. (From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47) Signed-off-by: Richard Purdie

oeqa/core/threaded: Remove in favour of using concurrenttests

2018-07-18T09:18:41+00:00

We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. (From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53) Signed-off-by: Richard Purdie

testsdk.bbclass: set PREMIRRORS for kernel to speed up test

2018-07-06T21:55:02+00:00

Currently if we do a testsdkext task for the image, it would take very long time to finish. The time is mostly spent fetching kernel source via network. We have done some configuration in auto.conf, trying to make use of own-mirrors.bbclass to avoid fetching kernel via network. However, the solution normally does not work. Below is some log from log.do_fetch. DEBUG: Fetcher failure: Unable to find file \ file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \ anywhere. The tar.gz file is not available. It is generated only if BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users need choose to set it to "1" only if they are trying to make a source mirror. So generally, this var's value is "0". Anyway, we do need to avoid fetching kernel source from network when doing testsdkext. So set PREMIRRORS in auto.conf to achieve this. After this change, the time reduces from 4209.131s to 1399.436s on my local machine. [YOCTO #12729] (From OE-Core rev: fd18ddb0664f69d77681590774b0123251a98728) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

image/testimage: Rework auto image test execution

2018-06-21T08:34:40+00:00

The TEST_IMAGE interface has never particularly worked and image testing currently gets configured manually. This reworks the interface to better serve the needs of its users, replacing it with TESTIMAGE_AUTO. This does away with the need for the separate class due to better bitbake APIs for changing tasks. TESTIMAGE_AUTO will automatically boot under qemu any image that is built. It also adds in dependencies so that any SDK for which testing is requested will automatically be built first. The code in bitbake.conf was error prone (e.g. testsdk wasn't considered), this improves it to standardise on IMAGE_CLASSES as the standard configuration mechanism. (From OE-Core rev: b34d44f3dfea8254826a46701a4fe3769a900434) Signed-off-by: Richard Purdie