summaryrefslogtreecommitdiffstats
path: root/meta/classes/testimage.bbclass
Commit message (Collapse)AuthorAgeFilesLines
* testimage: print results for interrupted runsKonrad Weihmann2020-11-111-8/+11
| | | | | | | | | | | | When a run is ended by overall timeout, print the already executed testcases, to provide some hints which testcase might made the test suite reach global timeout. Nonetheless make the testrun exit with an error (From OE-Core rev: 2bcc643195a3b3c66d698fac8b7af037c08545ac) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Add testimage_dump_target to kwargsSaul Wold2020-09-301-0/+1
| | | | | | | | | | | | | | | | | | This passes the list of commands to run on the OEQemuTarget when the TargetDumper needs to run in a test context due to a failure on the target. This is added here as a kwargs because the 'd' dictionary is not available in the staticmethod getTarget in the OERuntimeTestContextExecutor class. The OEQemuTarget is different from the QemuTarget which already uses the list of commands from testimage_dump_target from 'd'. The create_dir() is needed to initialize the TargetDumper's dump_dir variable. (From OE-Core rev: a63675fab4d9f638570912b15a07932f549cc4d1) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image-artifact-names: introduce new bbclass and move some variables into itMartin Jansa2020-09-171-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * similar to kernel-artifact-names for other recipes/bbclasses which need to use some deployed artifacts * bitbake.conf: move IMAGE_BASENAME, IMAGE_VERSION_SUFFIX, IMAGE_NAME, IMAGE_LINK_NAME variables * image_types.bbclass: move IMAGE_NAME_SUFFIX variable * currently IMAGE_NAME_SUFFIX is used only by image.bbclass, image_types.bbclass and meta/recipes-core/images/build-appliance-image_15.0.0.bb but if it's needed by some recipe which isn't itself an image, then it's useful in bitbake.conf, e.g. we have a recipe for creating VirtualBox appliances which combines .wic.vmdk with .ovf file to create .zip with appliance, but for that we need the filename of .wic.vmdk which now contains IMAGE_NAME_SUFFIX https://github.com/webOS-ports/meta-webos-ports/blob/4980ce52a43ac6897657602810313af359f0b839/meta-luneos/recipes-core/images/luneos-emulator-appliance.inc#L24 * we were hardcoding .rootfs suffix where needed, but for quite long time it's configurable with IMAGE_NAME_SUFFIX since: commit 380ee36811939d947024bf78de907e3c071b834f Author: Patrick Ohly <patrick.ohly@intel.com> Date: Mon Mar 7 18:07:52 2016 +0100 image creation: allow overriding .rootfs suffix and might not match with hardcoded .rootfs, so make it easier to use IMAGE_NAME_SUFFIX where needed even without inheritting whole image_types.bbclass [YOCTO #12937] (From OE-Core rev: 456b700d51a5052a285a8477304f902c335223be) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: add an overall timeout settingAlexander Kanavin2020-04-261-0/+11
| | | | | | | | | | | This is useful when tests misbehave and get stuck, or when a significant increase in testing time is undesirable and needs to be caught automatically. (From OE-Core rev: d77546e910ad9048f0057f4465716d417b810065) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: correctly process SIGTERMAlexander Kanavin2020-04-261-4/+4
| | | | | | | | | | | | | | | | | | | Python's unittest will not propagate exceptions upside of itself, but rather will just catch and print them. The working way to make it stop is to send a SIGINT (e.g. simulate a ctrl-c press), which will make it exit with a KeyboardInterrupt exception. This also makes pressing ctrl-c twice from bitbake work again (previously hanging instances of bitbake and qemu were left around, and bitbake would no longer start until they were killed manually). (From OE-Core rev: 06568a06ca22ee279d1829d26d4c38738233c06a) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: enable ovmf supportKonrad Weihmann2020-04-241-1/+5
| | | | | | | | | | Add support for running wic images with EFI as testimage. Introduces a variable called QEMU_USE_OVMF for configuration. (From OE-Core rev: 3af8aaff68ed332d812ea7dc184d392700ad7882) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fix various, innocuous typosRobert P. J. Day2020-02-181-1/+1
| | | | | | | | | | | | | | Corrections: - environment - accommodate - conversion - compatible (From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* weston: add a basic runtime testAlexander Kanavin2020-02-181-1/+1
| | | | | | | | | The test is checking that weston is able to start. (From OE-Core rev: 57700767f90eea8f2b78187c42581aca42d50bbf) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Extend runtime testing infrastructure to allow unconventional ↵Alejandro Hernandez Samaniego2020-02-131-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | booting processes to be tested The current runtime infrastructure contains hardcoded values which Ill refer to as patterns, these patterns are either searched through or sent via the serial terminal to communicate between HOST and TARGET. These patterns are required since they allow us to check when a device has finished booting, to log in, and to check whether a command sent from our tests has returned, this way we are able to check both the status of the commands that were sent along with its output. The testing process goes somewhat as follows: 1. Launch QEMU and start booting. 2. Check when the device has booted by looking for the pattern login:. 3. Log in as the root user (default for our images). 4. Check that we were able to log in succesfully. 5. Start running the runtime test cases defined by TEST_SUITES. 6. One of such test cases could send a command to the QEMU target. 7. Check whether that command returned. 8. Check its output and status, return whether the test case passed or failed. This patch allows this set of patterns to be defined instead of being hardcoded, but it also automatically sets the defaults that we have been using in the past if they have not been manually defined, for this reason, the patch is less invasive and should not affect in any way how tests are currently being run. Cases that can be enabled with this patch: - A customized image that does not use the root user (or maybe we want to check what happens if we dont use the root user). - An image where the PS1 env variable has been modified, and the prompt pattern wouldnt match the default. - Baremetal applications, which do not follow the conventional way of booting Linux and would probably not show a prompt for a user to log in, same applies for testing bootloaders. - poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny from meta-intel, which boots directly to RAM, and does not show a log in prompt since it does not contain a conventional init process. The code itself contains comments that should be self explanatory but here is an example on how these patterns can be defined in a hypothetical case where we want to run test cases as the webserver user instead: TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded" TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n" TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#" The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when used to communicate with the target when booting, anyone familiar with the PACKAGECONFIG syntax should have no trouble setting these. Other patterns would still be set up as default, e.g. search_reached_prompt would still be login: The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following: search_reached_prompt, send_login_user, search_login_succeeded, search_cmd_finished. They are prefixed with either search/send, to differentiate if the pattern is meant to be sent or searched to/from the target terminal. A working example of this code that falls under the baremetal case mentioned above along with a test case is present on the meta-freertos layer, which tests an RTOS image built with OpenEmbedded and automatically runs a test case on it after booting such image: As usual, INHERIT += "testimage" needs to be present on local.conf $ bitbake freertos-demo -c testimage RESULTS: RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s) SUMMARY: freertos-demo () - Ran 1 test in 2.006s freertos-demo - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) (From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831) Signed-off-by: Alejandro Hernandez Samaniego <aehs29@gmail.com> Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Allow testing on QEMU machines with a single serial portAlejandro Hernandez Samaniego2020-02-131-0/+1
| | | | | | | | | | | | | | | | | commmit 6cde67d0a84 enables the use of qemurunner on machines that only have a single serial port, but still sets the default value as serial_ports=2 if not provided. The testimage class does not call qemurunner with a serial_ports argument, hence always defaulting to two. Pass the serial_ports argument from the testimage class to allow tests to run on QEMU machines with a single serial port. (From OE-Core rev: a50d0163770f0b405a8de8a8a9cccd48c1de4112) Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: enable ssh agent forwardingAndré Draszik2019-10-191-0/+12
| | | | | | | | | | | | | | | | | | | Some targets might use ssh to do their power- or serial- control. In that case, ssh might need access to the ssh agent, or otherwise won't work. So export it into the environment. Note that the (old) oeqa/controllers/masterimage.py tries to do that as well by exporting all of BB_ORIGENV into the test environment. Here in testimage.bbclass we are a bit more strict and only pass the ssh related environment variables. (From OE-Core rev: d99617b8f2a9354b7357524fcd2eee16af0677b7) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: support hardware-controlled targetsAndré Draszik2019-10-191-0/+6
| | | | | | | | | | | | | | | | | | | | | | Since the introduction of the new runtime framework for target testing in commit 2aa5a4954d76 ("testimage.bbclass: Migrate class to use new runtime framework") commit 3857e5c91da6 in poky.git, target controllers have no access to the global datastore 'd' anymore. This makes it impossible for a specific OEQA (hardware) controller to access documented properties like TEST_POWERCONTROL_CMD, TEST_SERIALCONTROL_CMD, etc, meaning it's impossible for those controllers to actually control the hardware. To solve this, simply add those documented variables into the target_kwargs[]. (From OE-Core rev: 0ab7e3b573a58cc3a869ef33fd8737ca7fe04550) Signed-off-by: André Draszik <andre.draszik@jci.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: consider QB_DEFAULT_FSTYPEAdrian Freihofer2019-06-081-1/+5
| | | | | | | | | | | | testimage.bbclass starts qemu with the first image type found in the IMAGE_FSTYPES list. It's weird: this ['wic', 'tar'] works but this ['tar'. 'wic'] does not. If QB_DEFAULT_FSTYPE is defined, this fstype is booted. (From OE-Core rev: aedb6bf9b6ccf37f69372642bc4c5dbbca92d0d9) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/runner: dump stdout and stderr of each test caseMardegan, Alberto2019-04-261-1/+2
| | | | | | | | | | | | | | | | | | Some CI pipelines might perform further processing of the test output (for instance, to plot some metrics into a chart). However, Since `thud` we switched away from the XML-based jUnit reporting, and at the same time we lost the ability of collecting the stdout and stderr of the various tests. We now restore this functionality by adding `stdout` and `stderr` keys to the JSON reports. This behavior is off by default; in order to enable it, one must set the `TESTREPORT_FULLLOGS` variable in the bitbake configuration. (From OE-Core rev: fd0048630ece5b21efb3a79e97046be0ab2a1514) Signed-off-by: Alberto Mardegan <amardegan@luxoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: add support for passing runqemu paramsAlexander Kanavin2019-01-261-1/+3
| | | | | | | | | This is particularly useful when setting up GL tests. (From OE-Core rev: 167a46775059b782c6f82ce8c5a47b27262e95d4) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: make it work for multiple usersRobert Yang2019-01-111-1/+2
| | | | | | | | | | | | | | | | | | | | | There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. (From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Fix for QEMU_USE_KVMRobert Yang2019-01-081-7/+1
| | | | | | | | | | | | | | | | | | | | | | Fixed: MACHINE = "qemux86" QEMU_USE_KVM = "qemux86" IMAGE_CLASSES += "testimage" $ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs [snip] File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean raise ValueError("Invalid boolean value '%s'" % value) ValueError: Invalid boolean value 'qemux86' Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm will be enabled if target_arch == build_arch or both of them are x86 archs. (From OE-Core rev: 7c1a8a624cad8d967635c6cb5f99cf655bde3d44) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/classes/testimage.bbclass: Only validate IMAGE_FSTYPES when is QEMUAníbal Limón2019-01-031-6/+7
| | | | | | | | | | | | | When use simpleremote target the flash/boot process is executed manually, the IMAGE_FSTYPES validation is only needed when execute testimage against qemu. The supported_fstypes comes from oeqa.core.target.qemu module. (From OE-Core rev: e7dc5963adbacc091fe8943119262166977623ad) Signed-off-by: Aníbal Limón <anibal.limon@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: remove boot parameter systemd.log_targetKai Kang2018-12-271-7/+1
| | | | | | | | | | | | | | | | | | | Boot parameter systemd.log_target=console affects command journalctl's output and causes oe selftest case test_systemd_boot_time fail to pass. | Error at obtaining the boot time from journalctl | RESULTS: | RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time - Testcase -1: SKIPPED (0.74s) systemd.log_target=console was introduced by oe-core commit a0bb649 and work with parameter systemd.log_level to enable systemd debug. systemd.log_level has been removed already, so remove systemd.log_target too to make case test_systemd_boot_time pass. (From OE-Core rev: caa776bdcf8ea34c857f45970370bf771075f4bc) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Add possibility to pass parmeters to qemuErik Botö2018-12-151-1/+4
| | | | | | | | | | | Add a variable called TEST_QEMUPARAMS in testimage.bbclass to make it possible to pass parameters to qemu. This can be useful for e.g. increasing the amount of RAM available during testimage runs. (From OE-Core rev: 1a9163f5779d233c884c8fd50e0812eabab4fdf3) Signed-off-by: Erik Botö <erik.boto@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Add support for slirpYeoh Ee Peng2018-11-231-0/+5
| | | | | | | | | | | | Enable testimage to support qemu slirp. Configure "QEMU_USE_SLIRP" & "TEST_SERVER_IP" variables to enable slirp. [YOCTO#10713] (From OE-Core rev: 3df9ee85ce7fe52f0893fd33aea3bf1fcc6ead0a) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Enable autorunning of the package manager testsuitesRichard Purdie2018-11-161-3/+1
| | | | | | | | | Now that the hangs in httpservice are fixed we can let these tests auto skip as appropriate. (From OE-Core rev: 42a0d70291d551578e21f590fcb85ca72a78ccb5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation ↵Richard Purdie2018-11-161-1/+1
| | | | | | | | | | warnings Use our own lsb function instead as used elsewhere by the codebase. (From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Further cleanup DEFAULT_TEST_SUITESRichard Purdie2018-11-141-3/+1
| | | | | | | | | Now the test markup of the development tools tests is complete, this can be further tweaked to auto run the correct tests. (From OE-Core rev: bd4f8d12fe1f9f2643ee9e68fa2bb981134294fb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Simplfy DEFAULT_TEST_SUITES logicRichard Purdie2018-11-141-23/+9
| | | | | | | | | | | | | | | | | | | | Now that the tests have correct markup to automatically determine which images they can run against, clean up the default test suites logic to be simpler and not image specific. Some cleanup of the compiler tests still needs to be completed but this is a good first step. The only downside to this is more noise during testing as we now see many skipped messages for simple images like core-image-minimal. The auto type is being removed since it currently breaks badly due to the socat mandatory inclusion from the meta-selftest layer which is a problem which needs to be addressed seperately. (From OE-Core rev: 4966bc33845752eb0aeae54b72e8ba0146a7ed52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Remove duplicate dependenciesRichard Purdie2018-11-141-3/+1
| | | | | | (From OE-Core rev: 16d4368e7b85d5179374adb0a2d1ac215793e104) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Improvements to the json loggingRichard Purdie2018-10-291-14/+14
| | | | | | | | | | | | | | Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Log DISTRO * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes (From OE-Core rev: dda94cfa708cf3c6e5408d6dbcece360b4e8cd27) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: write testresult to json filesYeoh Ee Peng2018-10-291-2/+29
| | | | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia to store testresult, OEQA testimage need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the "WORKDIR" directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. (From OE-Core rev: 5d135d4769b6bb60d575eb6ed196367f9e077cc4) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: define ptest suite variableArmin Kuster2018-10-251-0/+1
| | | | | | | | | | | Ptest has dependencies on other tests so contain them in a variable. This allows layers to not worry about maintaining the depends themselves. (From OE-Core rev: c5c5ad9030ec7ddaa6387e2ba52e9e37b9ea76a2) Signed-off-by: Armin Kuster <akuster@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Ensure opkg/apt test suites are runRichard Purdie2018-10-121-6/+10
| | | | | | | | | | Currently only the dnf package manager tests are automatically added to the list of tests to run. Improve the code to handle automatic addition of the apt and opkg tests too. (From OE-Core rev: c9169e4d1fa67289d5beb0e1f8fb5c54153c4242) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Ensure package-index is triggered for opkg/apt testsRichard Purdie2018-10-111-2/+2
| | | | | | | | | If the opkg/apt tests are run without a package-index they will fail. Trigger this here for now as a dependency until the code can be improved. (From OE-Core rev: 7805eec0f522da2d2b6a08c8a07a9243e26c014a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Ensure TESTIMAGEDEPENDS is appended to for qemuall overrideRichard Purdie2018-10-111-3/+3
| | | | | | | | | The current qemuall override overwrites other dependencies unintentionally. Tweak the code to avoid this by appending to the variable. (From OE-Core rev: fc8df45e6b27dd7a26b7bf88a06ffa831fcf075c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: fix qemu_use_kvm handlingEmmanuel Roullit2018-09-131-2/+2
| | | | | | | | | | | | | | | | QEMU_USE_KVM can either be a boolean or a whitespace separated list of kvm supported machines. For the 'intel-corei7-64' machine, defined in meta-intel, kvm could not be used as the 'x86' substring is not part of its machine name. By changing the order of this 'or' statement and setting the 'QEMU_USE_KVM' variable to 'intel-corei7-64', it is possible to run the 'testimage' task with kvm support successfully. (From OE-Core rev: a22789253aa653dc50fb159b40910248c2f98dd4) Signed-off-by: Emmanuel Roullit <emmanuel.roullit@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake.conf/testimage: Add in compatiblity hacksRichard Purdie2018-06-211-0/+2
| | | | | | | | | | The testimage changes break the autobuilder. The plan is to revert these hacks but due to various changes happening with the autobuilder, we need these for now to keep things working until we can move to the new autobuilder codebase. (From OE-Core rev: 309a02931779f32d1139cc1169a039cbe4638706) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image/testimage: Rework auto image test executionRichard Purdie2018-06-211-1/+9
| | | | | | | | | | | | | | | | | | | | | | The TEST_IMAGE interface has never particularly worked and image testing currently gets configured manually. This reworks the interface to better serve the needs of its users, replacing it with TESTIMAGE_AUTO. This does away with the need for the separate class due to better bitbake APIs for changing tasks. TESTIMAGE_AUTO will automatically boot under qemu any image that is built. It also adds in dependencies so that any SDK for which testing is requested will automatically be built first. The code in bitbake.conf was error prone (e.g. testsdk wasn't considered), this improves it to standardise on IMAGE_CLASSES as the standard configuration mechanism. (From OE-Core rev: b34d44f3dfea8254826a46701a4fe3769a900434) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: move codes into testimage_mainChen Qi2018-06-041-7/+6
| | | | | | | | | | | | | | | | | | | | | | testimage-auto is expected to run testimage task's codes automatically. But in fact, it's currently missing some codes, including testimage_sanity and create_rpm_index. This leads to the problem of unexpected runtime failure of test_dnf_makecache. The error message is as below. RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR This error is caused by the fact that create_rpm_index is not executed before running the tests. There's no reason why such codes should not be in testimage_main, so move them into it. (From OE-Core rev: fa7ba486ded13907f63f9300f66350ba2835a3f7) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: also check 'auto' to create rpm indexChen Qi2018-06-041-1/+1
| | | | | | | | | | | | | | Having 'auto' in TEST_SUITES will also run the 'dnf' test cases, so also check it to determine whether to create rpm index or not. This is to fix the following error when TEST_SUITES = "auto". RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR (From OE-Core rev: c1f4177848d25a9121f2a85da655ee414cd424b1) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: fix behavior of empty TEST_SUITESChen Qi2018-06-041-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | The current behaviour of TEST_SUITES is very confusing. setting: TEST_SUITES = "" result: Execute all test cases. setting: TEST_SUITES = "some_case_not_exist" result: Error out with 'Empty test suite' message. The expected behaviour of TEST_SUITES should be: 1. when 'auto' is in it, execute as many test cases as possible 2. when no valid test case is specified, error out and give user message The first one is implemented by a previous patch. The second one is fixed in this patch. Also add debug message to show test cases to be executed. This is for easier debugging. (From OE-Core rev: 909568821fbad8a6a7034b10a2dc349a210fdfc6) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime/dnf: Add new dnf test casesJose Perez Carranza2018-05-151-3/+8
| | | | | | | | | | | | | Add test cases to test “exclude” and “installroot“ options, also modify the logic of filtering packages on the feed to have all the packages needed by the tests. [YOCTO #10744] (From OE-Core rev: 1121806603c6f621d084b692216f3f616a0768dc) Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: run systemtap test on SDK imagesRoss Burton2018-04-181-2/+4
| | | | | | | (From OE-Core rev: f7f26bd0fe7085515bdebf23107ed8647a0c98fe) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: enable gi test suiteRoss Burton2018-01-291-4/+4
| | | | | | | | (From OE-Core rev: d8243c4588d4f1bb057fd917bfea130c4907e24c) (From OE-Core rev: 1c896b34d2d9fdf941941bd29257a4714795da5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: add ptest to the list of runtime tests whenever possibleAlexander Kanavin2018-01-051-6/+6
| | | | | | | | | | | If no ptest packages are installed in the image, the test does nothing; if ptest packages are installed in the image, then they should be run without user having to enable that manually. (From OE-Core rev: f57feab2727dca916744deb64825f3beaf07961d) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* targetcontrol.py: use oe.types.boolean for QEMU_USE_KVMRobert Yang2017-12-181-1/+1
| | | | | | | | | | | | So that both QEMU_USE_KVM = "True" and "1" will work. [YOCTO #12343] (From OE-Core rev: f28890ae2eb8a9cb2da39588e696a4b0c501bdf0) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Ensure full logs are shown for failuresRichard Purdie2017-11-101-2/+2
| | | | | | | | | | Currently, the fact an error message is shown means the rest of the task logs are suppressed. In this case we don't want that as it hides the real errors and useful information. Therefore override this behaviour. (From OE-Core rev: 7058ac23c5ce32fbdc4be6c05f85550e1fd9fe77) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Pass the logger into OERuntimeTestContextExecutor.getTarget()Richard Purdie2017-11-091-1/+1
| | | | | | | | | | I have no idea why we didn't do this but it means the code has nowhere to log to unless we do this. This means we can then use the logger to log data to the task logs. (From OE-Core rev: e241fb128840889acff79dbf3fae7bfaafcb5357) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: update commentsRobert Yang2017-08-231-2/+2
| | | | | | | | | | | | | It's very important to add IMAGE_CLASSES += "testimage" in local.conf firstly, otherwise the var like TEST_LOG_DIR (defined in testimage.bbclass) will not be in testdata.json. [YOCTO #11547] (From OE-Core rev: 5663ed989f0af5b1c61c74288ec421cbca2261e7) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Use the renamed buildlzipKhem Raj2017-07-081-3/+3
| | | | | | | | | | buildiptables has been replaced with buildlzip (From OE-Core rev: 57cd682e05f82b63090b0cbb4a1d8feb3bfbff23) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: Correct the comment to state right dir for test casesKhem Raj2017-06-121-1/+1
| | | | | | | | (From OE-Core rev: b464e338b30fe038e081e0dcae9f50c120d2f81c) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Change the order to logDetails and logSummaryAníbal Limón2017-06-091-1/+1
| | | | | | | | | | | | | Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] (From OE-Core rev: 4e3ab36e8c90abc740cce1ba31faf6595116e1e2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/test{export,images}: Change modules to listAníbal Limón2017-06-021-1/+1
| | | | | | | | | | With the new filters by <module>.[class].[testname] the modules arg is expected to be a list so use split into TEST_SUITES variable. (From OE-Core rev: b588c5c65e55a31d61c86553158eef409cd71086) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 18:45:39 +0000