diff options
Diffstat (limited to 'documentation/test-manual/ptest.rst')
-rw-r--r-- | documentation/test-manual/ptest.rst | 128 |
1 files changed, 128 insertions, 0 deletions
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..2c021af515 --- /dev/null +++ b/documentation/test-manual/ptest.rst | |||
@@ -0,0 +1,128 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | *************************** | ||
4 | Testing Packages With ptest | ||
5 | *************************** | ||
6 | |||
7 | A Package Test (ptest) runs tests against packages built by the | ||
8 | OpenEmbedded build system on the target machine. A ptest contains at | ||
9 | least two items: the actual test, and a shell script (``run-ptest``) | ||
10 | that starts the test. The shell script that starts the test must not | ||
11 | contain the actual test --- the script only starts the test. On the other | ||
12 | hand, the test can be anything from a simple shell script that runs a | ||
13 | binary and checks the output to an elaborate system of test binaries and | ||
14 | data files. | ||
15 | |||
16 | The test generates output in the format used by Automake:: | ||
17 | |||
18 | result: testname | ||
19 | |||
20 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
21 | the testname can be any identifying string. | ||
22 | |||
23 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
24 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
25 | |||
26 | .. note:: | ||
27 | |||
28 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
29 | class. | ||
30 | |||
31 | Adding ptest to Your Build | ||
32 | ========================== | ||
33 | |||
34 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
35 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
36 | is found in the :term:`Build Directory`:: | ||
37 | |||
38 | DISTRO_FEATURES:append = " ptest" | ||
39 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
40 | |||
41 | Once your build is complete, the ptest files are installed into the | ||
42 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
43 | is the name of the package. | ||
44 | |||
45 | Running ptest | ||
46 | ============= | ||
47 | |||
48 | The ``ptest-runner`` package installs a shell script that loops through | ||
49 | all installed ptest test suites and runs them in sequence. Consequently, | ||
50 | you might want to add this package to your image. | ||
51 | |||
52 | Getting Your Package Ready | ||
53 | ========================== | ||
54 | |||
55 | In order to enable a recipe to run installed ptests on target hardware, | ||
56 | you need to prepare the recipes that build the packages you want to | ||
57 | test. Here is what you have to do for each recipe: | ||
58 | |||
59 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
60 | Include the following line in each recipe:: | ||
61 | |||
62 | inherit ptest | ||
63 | |||
64 | .. note:: | ||
65 | |||
66 | Classes for common frameworks already exist in :term:`OpenEmbedded-Core | ||
67 | (OE-Core)`, such as: | ||
68 | |||
69 | - :oe_git:`go-ptest </openembedded-core/tree/meta/classes-recipe/go-ptest.bbclass>` | ||
70 | - :ref:`ref-classes-ptest-cargo` | ||
71 | - :ref:`ref-classes-ptest-gnome` | ||
72 | - :oe_git:`ptest-perl </openembedded-core/tree/meta/classes-recipe/ptest-perl.bbclass>` | ||
73 | - :oe_git:`ptest-python-pytest </openembedded-core/tree/meta/classes-recipe/ptest-python-pytest.bbclass>` | ||
74 | |||
75 | Inheriting these classes with the ``inherit`` keyword in your recipe will | ||
76 | make the next steps automatic. | ||
77 | |||
78 | - *Create run-ptest:* This script starts your test. Locate the | ||
79 | script where you will refer to it using | ||
80 | :term:`SRC_URI`. Here is an | ||
81 | example that starts a test for ``dbus``:: | ||
82 | |||
83 | #!/bin/sh | ||
84 | cd test | ||
85 | make -k runtest-TESTS | ||
86 | |||
87 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
88 | dependencies that normally do not exist for the package (such as | ||
89 | requiring "make" to run the test suite), use the | ||
90 | :term:`DEPENDS` and | ||
91 | :term:`RDEPENDS` variables in | ||
92 | your recipe in order for the package to meet the dependencies. Here | ||
93 | is an example where the package has a runtime dependency on "make":: | ||
94 | |||
95 | RDEPENDS:${PN}-ptest += "make" | ||
96 | |||
97 | - *Add a function to build the test suite:* Not many packages support | ||
98 | cross-compilation of their test suites. Consequently, you usually | ||
99 | need to add a cross-compilation function to the package. | ||
100 | |||
101 | Many packages based on Automake compile and run the test suite by | ||
102 | using a single command such as ``make check``. However, the host | ||
103 | ``make check`` builds and runs on the same computer, while | ||
104 | cross-compiling requires that the package is built on the host but | ||
105 | executed for the target architecture (though often, as in the case | ||
106 | for ptest, the execution occurs on the host). The built version of | ||
107 | Automake that ships with the Yocto Project includes a patch that | ||
108 | separates building and execution. Consequently, packages that use the | ||
109 | unaltered, patched version of ``make check`` automatically | ||
110 | cross-compiles. | ||
111 | |||
112 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
113 | build the test suite. Add a function similar to the following to your | ||
114 | recipe:: | ||
115 | |||
116 | do_compile_ptest() { | ||
117 | oe_runmake buildtest-TESTS | ||
118 | } | ||
119 | |||
120 | - *Ensure special configurations are set:* If the package requires | ||
121 | special configurations prior to compiling the test code, you must | ||
122 | insert a ``do_configure_ptest`` function into the recipe. | ||
123 | |||
124 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
125 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
126 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
127 | to create a ``do_install_ptest`` function and make sure it gets | ||
128 | called after the "make install-ptest" completes. | ||