summaryrefslogtreecommitdiffstats
path: root/meta/classes/testimage.bbclass
Commit message (Collapse)AuthorAgeFilesLines
...
* testimage: filter proper test cases by tagszjh2015-09-041-0/+3
| | | | | | | | | | | If a test case is decorate by oeqa.utils.decorators.tag, this case will by add a tag, testrunner will filter these tags by TEST_SUITES_TAGS [YOCTO #7849] (From OE-Core rev: 085589b1018ba4d950baf7bcfb499be02c1b29fc) Signed-off-by: zjh <junhuix.zhang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runexported: Make it compatible with host dumpMariano Lopez2015-09-031-1/+5
| | | | | | | | | | | | | | Currently it is not possible to run a exported test, but this patch will allow to use the HosDumper class when running a exported test, otherwise the HostDumper class will break runexpored test. [YOCTO #8118] (From OE-Core rev: 25814dbdd12db0499e4d81876873158e1b8a4fcc) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* dump: allow to have datastore vars on dump commandsMariano Lopez2015-08-301-2/+5
| | | | | | | | | | | | | | | This allows to have datastore variables in the dump commands and will get the data when a new instance it's created. Also this remove special cases from the commands. [YOCTO #8118] (From OE-Core rev: 384927eb8d52bc5f14c63c8421aa62ee859587f0) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* dump: Created new classes for dump host and targetMariano Lopez2015-08-301-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | It makes sense to separate the dump commands from the oeRuntimeTest class, this way it can be used in all the test context. These are the changes included in this patch: - Created classes: BaseDumper, HostDumper, TargetDumper - Create an instance of HostDumper in imagetest.bbclass and add it to TestContext class, this way any class that have access to the TestContext would be able to dump logs from the host - Create an instance of TargetDumper in QemuTarget class after get the runner, this way it is accessible during the tests. [YOCTO #8118] (From OE-Core rev: ad10af6be343b5425fde43055263b0744c161cb3) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Run commands in target and host when test failsMariano Lopez2015-08-301-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch modify three files altought two of them are minimal modifications. This version includes the changes proposed by Paul. testimage.bbclass: Create new vars for easy modification of the dump directory and commands to be run on host and target when a test fails TESTIMAGE_DUMP_DIR: Directory to save the dumps testimage_dump_target: Commands to run on target testimage_dump_host: Commands to run on host oetest.py: - Allow to use the vars defined in testimage class - Now able to run commands in the host and dump the results - Fix an issue with the condition where to run the dump commands (Before it run the commands every test after a failure, now it runs the commands only in tests that failed) - Fix the output to stdout [YOCTO #8118] (From OE-Core rev: 26fe645457633f90bb5ddbb12f5f7b9ca4a06cc5) (From OE-Core rev: 7b4fbbf979ed22434b8e3f83ae145139bb0d9fc7) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: testsdk fixes/improvementsRichard Purdie2015-08-011-34/+33
| | | | | | | | | | | | | | | | | | The "bitbake meta-toolchain" comment is dated and incorrect, fix to point at -c populate_sdk. If a toolchain contains multiple environment files, iterate them and run the sdk test suite on each on in turn rather than giving a fatal error and giving up. Also improve the debug output so that rather than PN, it also show the toolchain tarball under test, and the environment file name its using. Also enable the accidentally disabled cleanup code. (From OE-Core rev: 44c8b1bd58397db85e6f7bb9a57f0d57d2c69ad5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Add support for test suites manifest fileszjh2015-07-091-2/+24
| | | | | | | | | | | | Allow the list of TEST_SUITES to be read from a list of manifest files in the TEST_SUITES_MANIFEST variable. [YOCTO #7848] (From OE-Core rev: c270148b3f0e87bbaafbcffbca4ee7b965be6569) Signed-off-by: zjh <junhuix.zhang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Support full TC path such oeqa.runtime.pnp.get_memory_size in ↵zjh2015-07-091-0/+3
| | | | | | | | | | | | | | | | | | | | TEST_SUITES If a TC name start with "oeqa", it is thought as a full TC path. if not, follow original logic (assume under oeqa.runtime) For example: TEST_SUITES = "oeqa.runtime.pnp.get_memory_size oeqa.runtime.sanity.reboot" will run these two case when bitbake -c testimage [YOCTO #7834] (From OE-Core rev: 2c3fe7351edd954230b14443f82914a3f70e8a40) Signed-off-by: zjh <junhuix.zhang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Don't test xorg/vnc on qemuarm64Richard Purdie2015-06-271-0/+4
| | | | | | | | | | | The qemuarm64 machine doesn't have graphics so don't test xorg/vnc as they won't work. [YOCTO #7103] (From OE-Core rev: 27d17c662aa103dc728f14e78cac772546a9e5dc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Handle ipk/deb packaging format tests correctlyRichard Purdie2015-06-271-2/+4
| | | | | | | | | | | The default test list only works for rpm packaging. This fixes it for deb and ipk too. (From OE-Core rev: 210c8926405fcf695ec00f5768f29ba198320d6a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Improve sdk handling of TEST_SUITESRichard Purdie2015-04-291-1/+3
| | | | | | | | | | Currently TEST_SUITES is used for both target image and sdk versions which can be confusing. This introduces TEST_SUITES_SDK for the sdk version of the code so that the different test sets can be specified independently. (From OE-Core rev: ffd84177c68a6c86e654a9ba2512c299b40ec5e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Fix sdk test manifest handlingRichard Purdie2015-04-291-1/+7
| | | | | | | | | Fix SDK_MANIFEST -> SDK_TARGET_MANIFEST and add support for host version too which is useful in SDK QA tests. (From OE-Core rev: df91dd8d064dc3e59c7f057d3f869500a233a76f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: sort modules based on dependenciesPatrick Ohly2015-04-101-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | TEST_SUITES="auto" is useful to run all suitable tests without having to hard-code the list. However, it did not take test dependencies into account, which can be an issue for tests which really depend on some other test to run first. To fix this, modules get loaded in the order determined by TESTS_SUITES, but then get re-ordered based on dependencies derived from @skipUnlessPassed before running them. The original order is used to break ties when there are no dependencies, so reordering only occurs when really necessary. @skipUnlessPassed gets extended such that it makes the test name a method depends on available for inspection by the test loader in oetest.py. Unfortunately Python's unittest offers no API to inspect tests in a TestSuite, so the code has to rely on implementation details to find all tests. The worst that can happen when the implementation changes is that tests are not found and reordering does not happen. (From OE-Core rev: 6e4543a35836c572b23b9f8162b19d1e038d3ed2) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Add parselogs to the default tests for sato imagesRichard Purdie2014-08-291-2/+2
| | | | | | (From OE-Core rev: d373d4332a03eb71c1112b8dd51052e56ca4fa59) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage.bbclass: add more fields to the sdk TestContextCorneliu Stoicescu2014-08-271-0/+8
| | | | | | | | | | In order to use hasFeature and hasPackage methods in sdk test modules, we need specific fields to be set in the TestContext object. Adding pkgmanifest, imagefeatures and distrofeatures to the TestContext. (From OE-Core rev: cd342b399b2d78724032cdd7042968d3238cd548) Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/classes/testimage.bbclass: add testsdk task and enable functionality ↵Corneliu Stoicescu2014-08-181-7/+90
| | | | | | | | | | | | | | | for it. - add new testsdk task for meta-toolchain testing. - enable the get_tests_list method to work with sdk tests. - add default TEST_SUITES value for meta-toolchain package NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> (From OE-Core rev: b78bc50904d53d5091729de481b99cc3ac4aaa1e) Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: if start fails, don't try to stopPaul Eggleton2014-04-301-1/+1
| | | | | | | | | | | | If we couldn't start the target, it doesn't make sense to try and stop it here since logically it shouldn't now be in any kind of "started" state. (It's the start function's job to clean up after itself if it fails - to that end, fix up the QemuTarget class so that it does.) (From OE-Core rev: 819ebddae6b78120e5e082423793ff988419b5c4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: add task level lockStefan Stanacar2014-03-051-0/+4
| | | | | | | | | | For machines other than qemu it will not be okay to run multiple testimage tasks in parallel. (From OE-Core rev: 9560305a9e28316438cb57421afc7877890c4b76) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: fail if no package manifest is foundStefan Stanacar2014-02-111-2/+5
| | | | | | | | | | | | | Sometimes we may forget to actually build the image we want to test (when testimage task is called manually). Instead of an ugly traceback we should fail nicely. The manifest is written after the rootfs so this ensures the image was actually built. (From OE-Core rev: 85c8dd4170a88a5d7f3d9ca181e75720302727c5) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: add ability to export testsStefan Stanacar2014-02-091-15/+92
| | | | | | | | | | | | | | | | | | | | | Add the ability to export the tests so that they can run independently of the build system, as is required if you want to be able to hand the test execution off to a scheduler. Booting/deployment of the target is still handled by the build system, as before, only the execution of the tests happens outside of the build system. Tests exported are the ones defined in TEST_SUITES. No tests have been changed as interesting parts of the data store have been exported and tests can continue to query them as before. Small adjustments were made for a couple of oeqa modules though. [YOCTO #5613] (From OE-Core rev: 155dd52e0f707e06f50756584a50f744ba6b7844) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/runtime: add test for pythonTing Wang2014-01-061-2/+2
| | | | | | | | | | | Run a python script on the target 1)checks the output. 2)Call os.system method create a testfile (From OE-Core rev: 4465c9368b0c37a3a2c41b68f65de08690a8179b) Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: include traceback when loading a test failsStefan Stanacar2013-12-201-1/+2
| | | | | | | | | | Makes it much easier to figure out where a syntax error is. (From OE-Core rev: 0c30a25c3d5f7fb1087dff45e01595329620235f) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: use the new targetcontrol.py module for running testsStefan Stanacar2013-12-031-68/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes the necessary changes for using the targetcontrol.py module so that one can run the same tests on a qemu instance or a remote machine based on the value of TEST_TARGET variable: "qemu" or "simpleremote". The default value is "qemu" which starts a qemu instance and it's the with what we currently have. With "simpleremote", the remote machine must be up with network and ssh and you need to set TEST_TARGET_IP with the IP address of the remote machine (it can still be a qemu instance that was manually started). Basically testimage.bbclass now does something along the lines of: - load tests -> deploy (prepare) / start target -> run tests. There were a couple of changes necessary for tests and also some cleanups/renames that were needed to adjust this change. (use ip everywhere when refering to target and server_ip when refering to host/build machine) Also two unnecessary and unsed methods were dropped from sshcontrol. [ YOCTO #5554 ] (From OE-Core rev: a7820350fa3271d78ed7476e02f4aef593be1125) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/runtime: build kernel module on target testStefan Stanacar2013-10-181-1/+1
| | | | | | | | | | Builds a simple Hello World module on target. Added to the defaults for core-image-sato-sdk. (From OE-Core rev: c61c3dee162aa1f5bf31b2a09d8b916dc1712056) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Exclude BB_ORIGENV variableRichard Purdie2013-09-261-0/+2
| | | | | | | | | | The BB_ORIGENV variable isn't picklable and causes failures when generating the task signature for the autorun image mode. We don't want to depend on its contents anyway so lets exclude it. (From OE-Core rev: 3f5f9d2681d41e44dade5d3a5a4bff3fd38f5506) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: add informational messages about tests and do some ↵Stefan Stanacar2013-09-241-16/+24
| | | | | | | | | | | | | cleanup/formatting Adds info about tests run even if they passed (in case of errors log will be printed anyway). Also some style change for qemu object. (From OE-Core rev: 4adcf960be9c7477e314788d42b80753c2aaf572) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: remove odd characters in commentsPaul Eggleton2013-09-221-3/+3
| | | | | | | (From OE-Core rev: 5d46c3df578f44597d8ce70add423e0c15982705) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake.conf: include machine name in DEPLOY_DIR_IMAGEPaul Eggleton2013-09-141-0/+1
| | | | | | | | | | | | | | | | | | | This allows a clean seperation between image outputs from different machines, and makes it possible to have convenience symlinks to make the output ready to deploy. This did require some surgery in runqemu; if explicit paths to the image and kernel are not supplied then DEPLOY_DIR_IMAGE needs to be determined from bitbake or set in the environment. However the script does try to avoid requiring it unless it really is needed. Corresponding changes were made in the automated testing code as well. Based on an RFC patch by Koen Kooi <koen@dominion.thruhere.net> (From OE-Core rev: 7e90261aec61f79680b5eaeaf5b18c7b795412a4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: add support for finding tests in other layers1.5_M4.rc1Stefan Stanacar2013-08-281-12/+43
| | | | | | | | | | | | | A layer can add tests in lib/oeqa/runtime (provided it extends BBPATH as normal) and enable them with TEST_SUITES_append = " testname". Test module names shouldn't collide though. (From OE-Core rev: e1e347a2d509303e1c566450b0f2b485d3d6629f) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: increase default boot timeoutStefan Stanacar2013-08-281-1/+1
| | | | | | | | | | | | While I can't reproduce on local builds, sometimes images fail to boot on AB (which runs many builds at once). Assuming there isn't something weird going on, let's just give it more time. (From OE-Core rev: db38e10701cd2392a57e559573b715fd6daf6e2a) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classess/testimage: change default test suitesStefan Stanacar2013-08-261-2/+2
| | | | | | | | | | | | | | Some new tests were added, safe to have them in the defaults for sato-sdk and sato. Not all of the new tests are here though, either because they aren't applicable to default images or take too long. (like build iptables/cvs/sudoky ones, they can be enabled in local.conf and a special target on AB setups.). Also reorder them a bit. (From OE-Core rev: caa18a99ec002e4e87e32cae8a2d28bb0e32c5a6) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage: create a link for qemu boot logStefan Stanacar2013-08-131-6/+14
| | | | | | | | | | | The full name for the log is confusing when there are multiple files. Also move the ssh log path stuff where it's needed. (From OE-Core rev: 1dbef61a0776ec6c9ac9209442bb4c346e706d7d) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass, lib/oeqa: add headers and commentsStefan Stanacar2013-07-161-0/+24
| | | | | | | | | | | Adds some comments to testimage.bbclass and the files it calls, just to give an ideea of what it does. (From OE-Core rev: 8bbb7116cf02466dfc59a17dc7bb51287aeea55b) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage.bbclass: use a copy of rootfs for testsStefan Stanacar2013-07-161-1/+8
| | | | | | | | | | | Make a copy of the rootfs and test that. We can now drop the snapshot option. (From OE-Core rev: ba58f1fe8fb7a0e3ff9320dfc108235d484da6a1) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/runtime: add tests for syslog and dfAlexandru Palalau2013-07-161-2/+2
| | | | | | | | | | | | | | | Add tests for free space and syslog. Changed in v2: - limit df's output to / - syslog: fix restart in case of systemd (From OE-Core rev: 1b39d57e7b5c9b69d565cf4d188ebc2f14e66ae6) Signed-off-by: Alexandru Palalau <alexandru.palalau@intel.com> Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass, lib/oeqa/utils/qemurunner.py: make boot timeout ↵Stefan Stanacar2013-07-161-0/+6
| | | | | | | | | | | | | | configurable and increase it On a loaded host (such as an AB setup where multiple builds run) the 200 seconds timeout might not be enough to reach the login prompt. Also make it configurable so we can set it from local.conf/auto.conf (From OE-Core rev: e828571683333e6f18ed56a84e114b5e54975bc7) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/testimage.bbclass: new class for image testsStefan Stanacar2013-07-091-0/+92
Replacement class for imagetest-qemu.bbclass. It launches a qemu instance and runs test modules defined in TEST_SUITES. (From OE-Core rev: e0e32b978e5af128d7ff4ee2686777b49f919e27) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Radu Moisan <radu.moisan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 01:51:11 +0000