From eb36aaa07f07ecdcd9a6fd6a60b37b352a7cf5d6 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Mon, 23 Sep 2013 16:35:26 -0700 Subject: ref-manual, dev-manual: Edits to running automated tests section. Applied a second round of review edits from Paul Eggleton for the new "Performing Automated Runtime Testing" section. I did some reorganization and some minor wording changes. (From yocto-docs rev: fa8f8e5f0f6c1377a4fcafcd3d933af15ac01ff3) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../dev-manual/dev-manual-common-tasks.xml | 103 +++++++++------------ 1 file changed, 46 insertions(+), 57 deletions(-) (limited to 'documentation/dev-manual') diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index 4d1b731fa3..6eadcef022 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -4257,10 +4257,10 @@ The OpenEmbedded build system makes available a series of automated - tests for images. + tests for images to verify runtime functionality. - The current release of Yocto Project supports these tests - for QEMU images only. + Currently, there is only support for running these tests + under QEMU. These tests are written in Python making use of the unittest module, and the majority of them @@ -4321,37 +4321,6 @@ must accept incoming connections from 192.168.7.0/24, which is the default IP range used for tap devices by runqemu. - Have your image built: - The image on which you want to run tests needs to - be built. - For example, the following command would build the - core-image-sato image when - MACHINE is set to "qemu". - - bitbake core-image-sato - - Run the tests: - You can start the tests automatically or manually. - To have the tests start automatically after the - OpenEmbedded build system successfully creates a QEMU - image, set the - TEST_IMAGE - variable to "1" in your - local.conf file in the - Build Directory. - To manually run the tests, globally inherit - testimage.class: - by editing your local.conf file as - follows: - - INHERIT += "testimage" - - After editing your - local.conf file, use BitBake to - run the tests: - - bitbake -c testimage <qemu-image> - @@ -4367,17 +4336,42 @@ Running Tests - After setting up to run the tests, use BitBake to start - the standard suite of tests manually or, if you are running - them automatically, build your image: - - bitbake core-image-sato -c testimage + You can start the tests automatically or manually: + + Automatically Running Tests: + To run the tests automatically after the + OpenEmbedded build system successfully creates an image, + first set the + TEST_IMAGE + variable to "1" in your local.conf + file in the + Build Directory: + + TEST_IMAGE = "1" + + Next, simply build your image. + If the image successfully builds, the tests will be + run: + bitbake core-image-sato - + + Manually Running Tests: + To manually run the tests, first globally inherit + testimage.class + by editing your local.conf file: + + INHERIT += "testimage" + + Next, use BitBake to run the tests: + + bitbake -c testimage <image> + + - Once the tests start, the following happens: + Regardless of how you run the tests, once they start, the + following happens: A copy of the root filesystem is written to ${WORKDIR}/testimage. @@ -4415,11 +4409,10 @@ All test files reside in meta/lib/oeqa/runtime in the Source Directory. - A test name (sometimes referred to as a module) maps to a - method name within a class within a module. - Tests can be used within multiple classes and the tests - are usually grouped together by the area tested (e.g tests for - systemd reside in + A test name maps directly to a Python module. + Each test module may contain a number of individual tests. + Tests are usually grouped together by the area + tested (e.g tests for systemd reside in meta/lib/oeqa/runtime/systemd.py). @@ -4443,12 +4436,13 @@ variable in local.conf. Each name in TEST_SUITES represents a required test for the image. - Module skipping is not allowed even if a test is not suitable - for an image (e.g. running the rpm tests on - an image without rpm). + Test modules named within TEST_SUITES + cannot be skipped even if a test is not suitable for an image + (e.g. running the rpm tests on an image without + rpm). Appending "auto" to TEST_SUITES causes the build system to try to run all tests that are suitable for the - image (i.e. the build system decides whether to run each test). + image (i.e. each test module may elect to skip itself). @@ -4469,13 +4463,8 @@ - The following list summarizes how to run the tests: + Here are some things to keep in mind when running tests: - Run the default set of tests simply by - running the following: - - bitbake <image> -c testimage - The default tests for the image are defined as: @@ -4509,7 +4498,7 @@ long as BBPATH is extended in the layer's - layer.conf file as normal. + layer.conf file as normal). Just remember that filenames need to map directly to test (module) names and that you do not use module names that collide with existing core tests. -- cgit v1.2.3-54-g00ecf