summaryrefslogtreecommitdiffstats
path: root/documentation/test-manual/ptest.rst
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/test-manual/ptest.rst')
-rw-r--r--documentation/test-manual/ptest.rst128
1 files changed, 128 insertions, 0 deletions
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst
new file mode 100644
index 0000000000..2c021af515
--- /dev/null
+++ b/documentation/test-manual/ptest.rst
@@ -0,0 +1,128 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3***************************
4Testing Packages With ptest
5***************************
6
7A Package Test (ptest) runs tests against packages built by the
8OpenEmbedded build system on the target machine. A ptest contains at
9least two items: the actual test, and a shell script (``run-ptest``)
10that starts the test. The shell script that starts the test must not
11contain the actual test --- the script only starts the test. On the other
12hand, the test can be anything from a simple shell script that runs a
13binary and checks the output to an elaborate system of test binaries and
14data files.
15
16The test generates output in the format used by Automake::
17
18 result: testname
19
20where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
21the testname can be any identifying string.
22
23For a list of Yocto Project recipes that are already enabled with ptest,
24see the :yocto_wiki:`Ptest </Ptest>` wiki page.
25
26.. note::
27
28 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
29 class.
30
31Adding ptest to Your Build
32==========================
33
34To add package testing to your build, add the :term:`DISTRO_FEATURES` and
35:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
36is found in the :term:`Build Directory`::
37
38 DISTRO_FEATURES:append = " ptest"
39 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
40
41Once your build is complete, the ptest files are installed into the
42``/usr/lib/package/ptest`` directory within the image, where ``package``
43is the name of the package.
44
45Running ptest
46=============
47
48The ``ptest-runner`` package installs a shell script that loops through
49all installed ptest test suites and runs them in sequence. Consequently,
50you might want to add this package to your image.
51
52Getting Your Package Ready
53==========================
54
55In order to enable a recipe to run installed ptests on target hardware,
56you need to prepare the recipes that build the packages you want to
57test. Here is what you have to do for each recipe:
58
59- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
60 Include the following line in each recipe::
61
62 inherit ptest
63
64 .. note::
65
66 Classes for common frameworks already exist in :term:`OpenEmbedded-Core
67 (OE-Core)`, such as:
68
69 - :oe_git:`go-ptest </openembedded-core/tree/meta/classes-recipe/go-ptest.bbclass>`
70 - :ref:`ref-classes-ptest-cargo`
71 - :ref:`ref-classes-ptest-gnome`
72 - :oe_git:`ptest-perl </openembedded-core/tree/meta/classes-recipe/ptest-perl.bbclass>`
73 - :oe_git:`ptest-python-pytest </openembedded-core/tree/meta/classes-recipe/ptest-python-pytest.bbclass>`
74
75 Inheriting these classes with the ``inherit`` keyword in your recipe will
76 make the next steps automatic.
77
78- *Create run-ptest:* This script starts your test. Locate the
79 script where you will refer to it using
80 :term:`SRC_URI`. Here is an
81 example that starts a test for ``dbus``::
82
83 #!/bin/sh
84 cd test
85 make -k runtest-TESTS
86
87- *Ensure dependencies are met:* If the test adds build or runtime
88 dependencies that normally do not exist for the package (such as
89 requiring "make" to run the test suite), use the
90 :term:`DEPENDS` and
91 :term:`RDEPENDS` variables in
92 your recipe in order for the package to meet the dependencies. Here
93 is an example where the package has a runtime dependency on "make"::
94
95 RDEPENDS:${PN}-ptest += "make"
96
97- *Add a function to build the test suite:* Not many packages support
98 cross-compilation of their test suites. Consequently, you usually
99 need to add a cross-compilation function to the package.
100
101 Many packages based on Automake compile and run the test suite by
102 using a single command such as ``make check``. However, the host
103 ``make check`` builds and runs on the same computer, while
104 cross-compiling requires that the package is built on the host but
105 executed for the target architecture (though often, as in the case
106 for ptest, the execution occurs on the host). The built version of
107 Automake that ships with the Yocto Project includes a patch that
108 separates building and execution. Consequently, packages that use the
109 unaltered, patched version of ``make check`` automatically
110 cross-compiles.
111
112 Regardless, you still must add a ``do_compile_ptest`` function to
113 build the test suite. Add a function similar to the following to your
114 recipe::
115
116 do_compile_ptest() {
117 oe_runmake buildtest-TESTS
118 }
119
120- *Ensure special configurations are set:* If the package requires
121 special configurations prior to compiling the test code, you must
122 insert a ``do_configure_ptest`` function into the recipe.
123
124- *Install the test suite:* The :ref:`ref-classes-ptest` class
125 automatically copies the file ``run-ptest`` to the target and then runs make
126 ``install-ptest`` to run the tests. If this is not enough, you need
127 to create a ``do_install_ptest`` function and make sure it gets
128 called after the "make install-ptest" completes.