summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorAntonin Godard <antonin.godard@bootlin.com>2024-12-26 16:20:16 +0100
committerSteve Sakoman <steve@sakoman.com>2025-01-09 06:25:36 -0800
commit3f782b6e4527d5b02cc98203b0fb79620b6a89df (patch)
tree8eae2524c9aab854a24414d1f9069bf04adf40b7
parentda6fec8a1da6b22e1cd28ba638ff4b9e4777630e (diff)
downloadpoky-3f782b6e4527d5b02cc98203b0fb79620b6a89df.tar.gz
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ] It makes more sense to document ptests in the test-manual. Since ptests are still related to packages, keep a link to ptests from packages.rst to the test-manual. Reported-by: Yoann Congal <yoann.congal@smile.fr> (From yocto-docs rev: 110e15c4407dfc03c7d931e4488eb43dbfad7570) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> (cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
-rw-r--r--documentation/dev-manual/packages.rst112
-rw-r--r--documentation/migration-guides/migration-1.6.rst2
-rw-r--r--documentation/ref-manual/classes.rst4
-rw-r--r--documentation/ref-manual/features.rst2
-rw-r--r--documentation/ref-manual/qa-checks.rst2
-rw-r--r--documentation/ref-manual/release-process.rst2
-rw-r--r--documentation/ref-manual/variables.rst2
-rw-r--r--documentation/test-manual/index.rst1
-rw-r--r--documentation/test-manual/intro.rst2
-rw-r--r--documentation/test-manual/ptest.rst114
10 files changed, 126 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst
index 33cf78747d..4ba2dcae3a 100644
--- a/documentation/dev-manual/packages.rst
+++ b/documentation/dev-manual/packages.rst
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages:
16- :ref:`dev-manual/packages:generating and using signed packages` 16- :ref:`dev-manual/packages:generating and using signed packages`
17 17
18- :ref:`Setting up and running package test 18- :ref:`Setting up and running package test
19 (ptest) <dev-manual/packages:testing packages with ptest>` 19 (ptest) <test-manual/ptest:testing packages with ptest>`
20 20
21- :ref:`dev-manual/packages:creating node package manager (npm) packages` 21- :ref:`dev-manual/packages:creating node package manager (npm) packages`
22 22
@@ -882,114 +882,8 @@ related to signed package feeds are available:
882Testing Packages With ptest 882Testing Packages With ptest
883=========================== 883===========================
884 884
885A Package Test (ptest) runs tests against packages built by the 885See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the
886OpenEmbedded build system on the target machine. A ptest contains at 886Yocto Project Test Environment Manual.
887least two items: the actual test, and a shell script (``run-ptest``)
888that starts the test. The shell script that starts the test must not
889contain the actual test --- the script only starts the test. On the other
890hand, the test can be anything from a simple shell script that runs a
891binary and checks the output to an elaborate system of test binaries and
892data files.
893
894The test generates output in the format used by Automake::
895
896 result: testname
897
898where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
899the testname can be any identifying string.
900
901For a list of Yocto Project recipes that are already enabled with ptest,
902see the :yocto_wiki:`Ptest </Ptest>` wiki page.
903
904.. note::
905
906 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
907 class.
908
909Adding ptest to Your Build
910--------------------------
911
912To add package testing to your build, add the :term:`DISTRO_FEATURES` and
913:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
914is found in the :term:`Build Directory`::
915
916 DISTRO_FEATURES:append = " ptest"
917 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
918
919Once your build is complete, the ptest files are installed into the
920``/usr/lib/package/ptest`` directory within the image, where ``package``
921is the name of the package.
922
923Running ptest
924-------------
925
926The ``ptest-runner`` package installs a shell script that loops through
927all installed ptest test suites and runs them in sequence. Consequently,
928you might want to add this package to your image.
929
930Getting Your Package Ready
931--------------------------
932
933In order to enable a recipe to run installed ptests on target hardware,
934you need to prepare the recipes that build the packages you want to
935test. Here is what you have to do for each recipe:
936
937- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
938 Include the following line in each recipe::
939
940 inherit ptest
941
942- *Create run-ptest:* This script starts your test. Locate the
943 script where you will refer to it using
944 :term:`SRC_URI`. Here is an
945 example that starts a test for ``dbus``::
946
947 #!/bin/sh
948 cd test
949 make -k runtest-TESTS
950
951- *Ensure dependencies are met:* If the test adds build or runtime
952 dependencies that normally do not exist for the package (such as
953 requiring "make" to run the test suite), use the
954 :term:`DEPENDS` and
955 :term:`RDEPENDS` variables in
956 your recipe in order for the package to meet the dependencies. Here
957 is an example where the package has a runtime dependency on "make"::
958
959 RDEPENDS:${PN}-ptest += "make"
960
961- *Add a function to build the test suite:* Not many packages support
962 cross-compilation of their test suites. Consequently, you usually
963 need to add a cross-compilation function to the package.
964
965 Many packages based on Automake compile and run the test suite by
966 using a single command such as ``make check``. However, the host
967 ``make check`` builds and runs on the same computer, while
968 cross-compiling requires that the package is built on the host but
969 executed for the target architecture (though often, as in the case
970 for ptest, the execution occurs on the host). The built version of
971 Automake that ships with the Yocto Project includes a patch that
972 separates building and execution. Consequently, packages that use the
973 unaltered, patched version of ``make check`` automatically
974 cross-compiles.
975
976 Regardless, you still must add a ``do_compile_ptest`` function to
977 build the test suite. Add a function similar to the following to your
978 recipe::
979
980 do_compile_ptest() {
981 oe_runmake buildtest-TESTS
982 }
983
984- *Ensure special configurations are set:* If the package requires
985 special configurations prior to compiling the test code, you must
986 insert a ``do_configure_ptest`` function into the recipe.
987
988- *Install the test suite:* The :ref:`ref-classes-ptest` class
989 automatically copies the file ``run-ptest`` to the target and then runs make
990 ``install-ptest`` to run the tests. If this is not enough, you need
991 to create a ``do_install_ptest`` function and make sure it gets
992 called after the "make install-ptest" completes.
993 887
994Creating Node Package Manager (NPM) Packages 888Creating Node Package Manager (NPM) Packages
995============================================ 889============================================
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst
index 916169e836..b052a43a31 100644
--- a/documentation/migration-guides/migration-1.6.rst
+++ b/documentation/migration-guides/migration-1.6.rst
@@ -221,7 +221,7 @@ Package Test (ptest)
221 221
222Package Tests (ptest) are built but not installed by default. For 222Package Tests (ptest) are built but not installed by default. For
223information on using Package Tests, see the 223information on using Package Tests, see the
224":ref:`dev-manual/packages:testing packages with ptest`" section in the 224":ref:`test-manual/ptest:testing packages with ptest`" section in the
225Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" 225Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`"
226section. 226section.
227 227
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index 8c3cc0140f..f416a6d623 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -2615,7 +2615,7 @@ runtime tests for recipes that build software that provides these tests.
2615This class is intended to be inherited by individual recipes. However, 2615This class is intended to be inherited by individual recipes. However,
2616the class' functionality is largely disabled unless "ptest" appears in 2616the class' functionality is largely disabled unless "ptest" appears in
2617:term:`DISTRO_FEATURES`. See the 2617:term:`DISTRO_FEATURES`. See the
2618":ref:`dev-manual/packages:testing packages with ptest`" 2618":ref:`test-manual/ptest:testing packages with ptest`"
2619section in the Yocto Project Development Tasks Manual for more information 2619section in the Yocto Project Development Tasks Manual for more information
2620on ptest. 2620on ptest.
2621 2621
@@ -2639,7 +2639,7 @@ Enables package tests (ptests) specifically for GNOME packages, which
2639have tests intended to be executed with ``gnome-desktop-testing``. 2639have tests intended to be executed with ``gnome-desktop-testing``.
2640 2640
2641For information on setting up and running ptests, see the 2641For information on setting up and running ptests, see the
2642":ref:`dev-manual/packages:testing packages with ptest`" 2642":ref:`test-manual/ptest:testing packages with ptest`"
2643section in the Yocto Project Development Tasks Manual. 2643section in the Yocto Project Development Tasks Manual.
2644 2644
2645.. _ref-classes-python3-dir: 2645.. _ref-classes-python3-dir:
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst
index b2ba731bb2..face6c1062 100644
--- a/documentation/ref-manual/features.rst
+++ b/documentation/ref-manual/features.rst
@@ -207,7 +207,7 @@ metadata, as extra layers can define their own:
207 207
208- *ptest:* Enables building the package tests where supported by 208- *ptest:* Enables building the package tests where supported by
209 individual recipes. For more information on package tests, see the 209 individual recipes. For more information on package tests, see the
210 ":ref:`dev-manual/packages:testing packages with ptest`" section 210 ":ref:`test-manual/ptest:testing packages with ptest`" section
211 in the Yocto Project Development Tasks Manual. 211 in the Yocto Project Development Tasks Manual.
212 212
213- *pulseaudio:* Include support for 213- *pulseaudio:* Include support for
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst
index 1b83834df1..27d46de3fd 100644
--- a/documentation/ref-manual/qa-checks.rst
+++ b/documentation/ref-manual/qa-checks.rst
@@ -791,7 +791,7 @@ Errors and Warnings
791 791
792 This check will detect if the source of the package contains some 792 This check will detect if the source of the package contains some
793 upstream-provided tests and, if so, that ptests are implemented for this 793 upstream-provided tests and, if so, that ptests are implemented for this
794 recipe. See the ":ref:`dev-manual/packages:testing packages with ptest`" 794 recipe. See the ":ref:`test-manual/ptest:testing packages with ptest`"
795 section in the Yocto Project Development Tasks Manual. See also the 795 section in the Yocto Project Development Tasks Manual. See also the
796 ":ref:`ref-classes-ptest`" section. 796 ":ref:`ref-classes-ptest`" section.
797 797
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst
index 691e7d3ed9..3383a4e1b0 100644
--- a/documentation/ref-manual/release-process.rst
+++ b/documentation/ref-manual/release-process.rst
@@ -175,7 +175,7 @@ consists of the following pieces:
175 operation and functions. However, the test can also use the IP 175 operation and functions. However, the test can also use the IP
176 address of a machine to test. 176 address of a machine to test.
177 177
178- :ref:`ptest <dev-manual/packages:testing packages with ptest>`: 178- :ref:`ptest <test-manual/ptest:testing packages with ptest>`:
179 Runs tests against packages produced during the build for a given 179 Runs tests against packages produced during the build for a given
180 piece of software. The test allows the packages to be run within a 180 piece of software. The test allows the packages to be run within a
181 target image. 181 target image.
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 869100ce34..200def084f 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -6828,7 +6828,7 @@ system and gives an overview of their function and contents.
6828 6828
6829 :term:`PTEST_ENABLED` 6829 :term:`PTEST_ENABLED`
6830 Specifies whether or not :ref:`Package 6830 Specifies whether or not :ref:`Package
6831 Test <dev-manual/packages:testing packages with ptest>` (ptest) 6831 Test <test-manual/ptest:testing packages with ptest>` (ptest)
6832 functionality is enabled when building a recipe. You should not set 6832 functionality is enabled when building a recipe. You should not set
6833 this variable directly. Enabling and disabling building Package Tests 6833 this variable directly. Enabling and disabling building Package Tests
6834 at build time should be done by adding "ptest" to (or removing it 6834 at build time should be done by adding "ptest" to (or removing it
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst
index 86a2f436ea..ad71f37910 100644
--- a/documentation/test-manual/index.rst
+++ b/documentation/test-manual/index.rst
@@ -12,6 +12,7 @@ Yocto Project Test Environment Manual
12 12
13 intro 13 intro
14 test-process 14 test-process
15 ptest
15 understand-autobuilder 16 understand-autobuilder
16 reproducible-builds 17 reproducible-builds
17 yocto-project-compatible 18 yocto-project-compatible
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst
index c31fd11c7a..7bbd11295f 100644
--- a/documentation/test-manual/intro.rst
+++ b/documentation/test-manual/intro.rst
@@ -140,7 +140,7 @@ the following types of tests:
140- *Package Testing:* A Package Test (ptest) runs tests against packages 140- *Package Testing:* A Package Test (ptest) runs tests against packages
141 built by the OpenEmbedded build system on the target machine. See the 141 built by the OpenEmbedded build system on the target machine. See the
142 :ref:`Testing Packages With 142 :ref:`Testing Packages With
143 ptest <dev-manual/packages:Testing Packages With ptest>` section 143 ptest <test-manual/ptest:Testing Packages With ptest>` section
144 in the Yocto Project Development Tasks Manual and the 144 in the Yocto Project Development Tasks Manual and the
145 ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more 145 ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more
146 information on Ptest. 146 information on Ptest.
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst
new file mode 100644
index 0000000000..dea1bad23b
--- /dev/null
+++ b/documentation/test-manual/ptest.rst
@@ -0,0 +1,114 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3***************************
4Testing Packages With ptest
5***************************
6
7A Package Test (ptest) runs tests against packages built by the
8OpenEmbedded build system on the target machine. A ptest contains at
9least two items: the actual test, and a shell script (``run-ptest``)
10that starts the test. The shell script that starts the test must not
11contain the actual test --- the script only starts the test. On the other
12hand, the test can be anything from a simple shell script that runs a
13binary and checks the output to an elaborate system of test binaries and
14data files.
15
16The test generates output in the format used by Automake::
17
18 result: testname
19
20where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
21the testname can be any identifying string.
22
23For a list of Yocto Project recipes that are already enabled with ptest,
24see the :yocto_wiki:`Ptest </Ptest>` wiki page.
25
26.. note::
27
28 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
29 class.
30
31Adding ptest to Your Build
32==========================
33
34To add package testing to your build, add the :term:`DISTRO_FEATURES` and
35:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
36is found in the :term:`Build Directory`::
37
38 DISTRO_FEATURES:append = " ptest"
39 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
40
41Once your build is complete, the ptest files are installed into the
42``/usr/lib/package/ptest`` directory within the image, where ``package``
43is the name of the package.
44
45Running ptest
46=============
47
48The ``ptest-runner`` package installs a shell script that loops through
49all installed ptest test suites and runs them in sequence. Consequently,
50you might want to add this package to your image.
51
52Getting Your Package Ready
53==========================
54
55In order to enable a recipe to run installed ptests on target hardware,
56you need to prepare the recipes that build the packages you want to
57test. Here is what you have to do for each recipe:
58
59- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
60 Include the following line in each recipe::
61
62 inherit ptest
63
64- *Create run-ptest:* This script starts your test. Locate the
65 script where you will refer to it using
66 :term:`SRC_URI`. Here is an
67 example that starts a test for ``dbus``::
68
69 #!/bin/sh
70 cd test
71 make -k runtest-TESTS
72
73- *Ensure dependencies are met:* If the test adds build or runtime
74 dependencies that normally do not exist for the package (such as
75 requiring "make" to run the test suite), use the
76 :term:`DEPENDS` and
77 :term:`RDEPENDS` variables in
78 your recipe in order for the package to meet the dependencies. Here
79 is an example where the package has a runtime dependency on "make"::
80
81 RDEPENDS:${PN}-ptest += "make"
82
83- *Add a function to build the test suite:* Not many packages support
84 cross-compilation of their test suites. Consequently, you usually
85 need to add a cross-compilation function to the package.
86
87 Many packages based on Automake compile and run the test suite by
88 using a single command such as ``make check``. However, the host
89 ``make check`` builds and runs on the same computer, while
90 cross-compiling requires that the package is built on the host but
91 executed for the target architecture (though often, as in the case
92 for ptest, the execution occurs on the host). The built version of
93 Automake that ships with the Yocto Project includes a patch that
94 separates building and execution. Consequently, packages that use the
95 unaltered, patched version of ``make check`` automatically
96 cross-compiles.
97
98 Regardless, you still must add a ``do_compile_ptest`` function to
99 build the test suite. Add a function similar to the following to your
100 recipe::
101
102 do_compile_ptest() {
103 oe_runmake buildtest-TESTS
104 }
105
106- *Ensure special configurations are set:* If the package requires
107 special configurations prior to compiling the test code, you must
108 insert a ``do_configure_ptest`` function into the recipe.
109
110- *Install the test suite:* The :ref:`ref-classes-ptest` class
111 automatically copies the file ``run-ptest`` to the target and then runs make
112 ``install-ptest`` to run the tests. If this is not enough, you need
113 to create a ``do_install_ptest`` function and make sure it gets
114 called after the "make install-ptest" completes.