summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorAntonin Godard <antonin.godard@bootlin.com>2024-12-26 16:20:16 +0100
committerSteve Sakoman <steve@sakoman.com>2025-01-09 08:41:04 -0800
commit2497f29afd1ff66964b84dc79a7cd1126bbc94fa (patch)
tree494d2d1a6b358c3796d870e1e060a949addf37b9
parent1418534d2215890723c281f852b6a6fd08acdbcf (diff)
downloadpoky-2497f29afd1ff66964b84dc79a7cd1126bbc94fa.tar.gz
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ] It makes more sense to document ptests in the test-manual. Since ptests are still related to packages, keep a link to ptests from packages.rst to the test-manual. Reported-by: Yoann Congal <yoann.congal@smile.fr> (From yocto-docs rev: 8b6ada020d595d86c7bbe78a27b7a6301715b039) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> (cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14) Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
-rw-r--r--documentation/dev-manual/packages.rst112
-rw-r--r--documentation/migration-guides/migration-1.6.rst2
-rw-r--r--documentation/ref-manual/classes.rst4
-rw-r--r--documentation/ref-manual/features.rst2
-rw-r--r--documentation/ref-manual/qa-checks.rst1
-rw-r--r--documentation/ref-manual/release-process.rst2
-rw-r--r--documentation/ref-manual/variables.rst2
-rw-r--r--documentation/test-manual/index.rst1
-rw-r--r--documentation/test-manual/intro.rst2
-rw-r--r--documentation/test-manual/ptest.rst114
10 files changed, 125 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst
index e5028fffdc..0c948cdaef 100644
--- a/documentation/dev-manual/packages.rst
+++ b/documentation/dev-manual/packages.rst
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages:
16- :ref:`dev-manual/packages:generating and using signed packages` 16- :ref:`dev-manual/packages:generating and using signed packages`
17 17
18- :ref:`Setting up and running package test 18- :ref:`Setting up and running package test
19 (ptest) <dev-manual/packages:testing packages with ptest>` 19 (ptest) <test-manual/ptest:testing packages with ptest>`
20 20
21- :ref:`dev-manual/packages:creating node package manager (npm) packages` 21- :ref:`dev-manual/packages:creating node package manager (npm) packages`
22 22
@@ -887,114 +887,8 @@ related to signed package feeds are available:
887Testing Packages With ptest 887Testing Packages With ptest
888=========================== 888===========================
889 889
890A Package Test (ptest) runs tests against packages built by the 890See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the
891OpenEmbedded build system on the target machine. A ptest contains at 891Yocto Project Test Environment Manual.
892least two items: the actual test, and a shell script (``run-ptest``)
893that starts the test. The shell script that starts the test must not
894contain the actual test --- the script only starts the test. On the other
895hand, the test can be anything from a simple shell script that runs a
896binary and checks the output to an elaborate system of test binaries and
897data files.
898
899The test generates output in the format used by Automake::
900
901 result: testname
902
903where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
904the testname can be any identifying string.
905
906For a list of Yocto Project recipes that are already enabled with ptest,
907see the :yocto_wiki:`Ptest </Ptest>` wiki page.
908
909.. note::
910
911 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
912 class.
913
914Adding ptest to Your Build
915--------------------------
916
917To add package testing to your build, add the :term:`DISTRO_FEATURES` and
918:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
919is found in the :term:`Build Directory`::
920
921 DISTRO_FEATURES:append = " ptest"
922 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
923
924Once your build is complete, the ptest files are installed into the
925``/usr/lib/package/ptest`` directory within the image, where ``package``
926is the name of the package.
927
928Running ptest
929-------------
930
931The ``ptest-runner`` package installs a shell script that loops through
932all installed ptest test suites and runs them in sequence. Consequently,
933you might want to add this package to your image.
934
935Getting Your Package Ready
936--------------------------
937
938In order to enable a recipe to run installed ptests on target hardware,
939you need to prepare the recipes that build the packages you want to
940test. Here is what you have to do for each recipe:
941
942- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
943 Include the following line in each recipe::
944
945 inherit ptest
946
947- *Create run-ptest:* This script starts your test. Locate the
948 script where you will refer to it using
949 :term:`SRC_URI`. Here is an
950 example that starts a test for ``dbus``::
951
952 #!/bin/sh
953 cd test
954 make -k runtest-TESTS
955
956- *Ensure dependencies are met:* If the test adds build or runtime
957 dependencies that normally do not exist for the package (such as
958 requiring "make" to run the test suite), use the
959 :term:`DEPENDS` and
960 :term:`RDEPENDS` variables in
961 your recipe in order for the package to meet the dependencies. Here
962 is an example where the package has a runtime dependency on "make"::
963
964 RDEPENDS:${PN}-ptest += "make"
965
966- *Add a function to build the test suite:* Not many packages support
967 cross-compilation of their test suites. Consequently, you usually
968 need to add a cross-compilation function to the package.
969
970 Many packages based on Automake compile and run the test suite by
971 using a single command such as ``make check``. However, the host
972 ``make check`` builds and runs on the same computer, while
973 cross-compiling requires that the package is built on the host but
974 executed for the target architecture (though often, as in the case
975 for ptest, the execution occurs on the host). The built version of
976 Automake that ships with the Yocto Project includes a patch that
977 separates building and execution. Consequently, packages that use the
978 unaltered, patched version of ``make check`` automatically
979 cross-compiles.
980
981 Regardless, you still must add a ``do_compile_ptest`` function to
982 build the test suite. Add a function similar to the following to your
983 recipe::
984
985 do_compile_ptest() {
986 oe_runmake buildtest-TESTS
987 }
988
989- *Ensure special configurations are set:* If the package requires
990 special configurations prior to compiling the test code, you must
991 insert a ``do_configure_ptest`` function into the recipe.
992
993- *Install the test suite:* The :ref:`ref-classes-ptest` class
994 automatically copies the file ``run-ptest`` to the target and then runs make
995 ``install-ptest`` to run the tests. If this is not enough, you need
996 to create a ``do_install_ptest`` function and make sure it gets
997 called after the "make install-ptest" completes.
998 892
999Creating Node Package Manager (NPM) Packages 893Creating Node Package Manager (NPM) Packages
1000============================================ 894============================================
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst
index c50786a36d..ecd4f2c1af 100644
--- a/documentation/migration-guides/migration-1.6.rst
+++ b/documentation/migration-guides/migration-1.6.rst
@@ -220,7 +220,7 @@ Package Test (ptest)
220 220
221Package Tests (ptest) are built but not installed by default. For 221Package Tests (ptest) are built but not installed by default. For
222information on using Package Tests, see the 222information on using Package Tests, see the
223":ref:`dev-manual/packages:testing packages with ptest`" 223":ref:`test-manual/ptest:testing packages with ptest`"
224section in the Yocto Project Development Tasks Manual. For information on the 224section in the Yocto Project Development Tasks Manual. For information on the
225``ptest`` class, see the ":ref:`ref-classes-ptest`" section. 225``ptest`` class, see the ":ref:`ref-classes-ptest`" section.
226 226
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index 52a77a11be..b609c3af66 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -2478,7 +2478,7 @@ runtime tests for recipes that build software that provides these tests.
2478This class is intended to be inherited by individual recipes. However, 2478This class is intended to be inherited by individual recipes. However,
2479the class' functionality is largely disabled unless "ptest" appears in 2479the class' functionality is largely disabled unless "ptest" appears in
2480:term:`DISTRO_FEATURES`. See the 2480:term:`DISTRO_FEATURES`. See the
2481":ref:`dev-manual/packages:testing packages with ptest`" 2481":ref:`test-manual/ptest:testing packages with ptest`"
2482section in the Yocto Project Development Tasks Manual for more information 2482section in the Yocto Project Development Tasks Manual for more information
2483on ptest. 2483on ptest.
2484 2484
@@ -2491,7 +2491,7 @@ Enables package tests (ptests) specifically for GNOME packages, which
2491have tests intended to be executed with ``gnome-desktop-testing``. 2491have tests intended to be executed with ``gnome-desktop-testing``.
2492 2492
2493For information on setting up and running ptests, see the 2493For information on setting up and running ptests, see the
2494":ref:`dev-manual/packages:testing packages with ptest`" 2494":ref:`test-manual/ptest:testing packages with ptest`"
2495section in the Yocto Project Development Tasks Manual. 2495section in the Yocto Project Development Tasks Manual.
2496 2496
2497.. _ref-classes-python3-dir: 2497.. _ref-classes-python3-dir:
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst
index bd393abd07..4d5f900ab8 100644
--- a/documentation/ref-manual/features.rst
+++ b/documentation/ref-manual/features.rst
@@ -157,7 +157,7 @@ metadata:
157 157
158- *ptest:* Enables building the package tests where supported by 158- *ptest:* Enables building the package tests where supported by
159 individual recipes. For more information on package tests, see the 159 individual recipes. For more information on package tests, see the
160 ":ref:`dev-manual/packages:testing packages with ptest`" section 160 ":ref:`test-manual/ptest:testing packages with ptest`" section
161 in the Yocto Project Development Tasks Manual. 161 in the Yocto Project Development Tasks Manual.
162 162
163- *smbfs:* Include SMB networks client support (for mounting 163- *smbfs:* Include SMB networks client support (for mounting
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst
index 1e5fe21b02..92d2a42afb 100644
--- a/documentation/ref-manual/qa-checks.rst
+++ b/documentation/ref-manual/qa-checks.rst
@@ -789,7 +789,6 @@ Errors and Warnings
789 use a relative path rather than an absolute one, or to pick up the path from 789 use a relative path rather than an absolute one, or to pick up the path from
790 runtime configuration or environment variables. 790 runtime configuration or environment variables.
791 791
792
793Configuring and Disabling QA Checks 792Configuring and Disabling QA Checks
794=================================== 793===================================
795 794
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst
index 5733dd70ff..89f79812e8 100644
--- a/documentation/ref-manual/release-process.rst
+++ b/documentation/ref-manual/release-process.rst
@@ -175,7 +175,7 @@ consists of the following pieces:
175 operation and functions. However, the test can also use the IP 175 operation and functions. However, the test can also use the IP
176 address of a machine to test. 176 address of a machine to test.
177 177
178- :ref:`ptest <dev-manual/packages:testing packages with ptest>`: 178- :ref:`ptest <test-manual/ptest:testing packages with ptest>`:
179 Runs tests against packages produced during the build for a given 179 Runs tests against packages produced during the build for a given
180 piece of software. The test allows the packages to be run within a 180 piece of software. The test allows the packages to be run within a
181 target image. 181 target image.
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 7f0d09903b..8ab7391f98 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -6434,7 +6434,7 @@ system and gives an overview of their function and contents.
6434 6434
6435 :term:`PTEST_ENABLED` 6435 :term:`PTEST_ENABLED`
6436 Specifies whether or not :ref:`Package 6436 Specifies whether or not :ref:`Package
6437 Test <dev-manual/packages:testing packages with ptest>` (ptest) 6437 Test <test-manual/ptest:testing packages with ptest>` (ptest)
6438 functionality is enabled when building a recipe. You should not set 6438 functionality is enabled when building a recipe. You should not set
6439 this variable directly. Enabling and disabling building Package Tests 6439 this variable directly. Enabling and disabling building Package Tests
6440 at build time should be done by adding "ptest" to (or removing it 6440 at build time should be done by adding "ptest" to (or removing it
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst
index 86a2f436ea..ad71f37910 100644
--- a/documentation/test-manual/index.rst
+++ b/documentation/test-manual/index.rst
@@ -12,6 +12,7 @@ Yocto Project Test Environment Manual
12 12
13 intro 13 intro
14 test-process 14 test-process
15 ptest
15 understand-autobuilder 16 understand-autobuilder
16 reproducible-builds 17 reproducible-builds
17 yocto-project-compatible 18 yocto-project-compatible
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst
index bb267d2d88..ff9b6d3794 100644
--- a/documentation/test-manual/intro.rst
+++ b/documentation/test-manual/intro.rst
@@ -141,7 +141,7 @@ the following types of tests:
141- *Package Testing:* A Package Test (ptest) runs tests against packages 141- *Package Testing:* A Package Test (ptest) runs tests against packages
142 built by the OpenEmbedded build system on the target machine. See the 142 built by the OpenEmbedded build system on the target machine. See the
143 :ref:`Testing Packages With 143 :ref:`Testing Packages With
144 ptest <dev-manual/packages:Testing Packages With ptest>` section 144 ptest <test-manual/ptest:Testing Packages With ptest>` section
145 in the Yocto Project Development Tasks Manual and the 145 in the Yocto Project Development Tasks Manual and the
146 ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more 146 ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more
147 information on Ptest. 147 information on Ptest.
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst
new file mode 100644
index 0000000000..dea1bad23b
--- /dev/null
+++ b/documentation/test-manual/ptest.rst
@@ -0,0 +1,114 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3***************************
4Testing Packages With ptest
5***************************
6
7A Package Test (ptest) runs tests against packages built by the
8OpenEmbedded build system on the target machine. A ptest contains at
9least two items: the actual test, and a shell script (``run-ptest``)
10that starts the test. The shell script that starts the test must not
11contain the actual test --- the script only starts the test. On the other
12hand, the test can be anything from a simple shell script that runs a
13binary and checks the output to an elaborate system of test binaries and
14data files.
15
16The test generates output in the format used by Automake::
17
18 result: testname
19
20where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
21the testname can be any identifying string.
22
23For a list of Yocto Project recipes that are already enabled with ptest,
24see the :yocto_wiki:`Ptest </Ptest>` wiki page.
25
26.. note::
27
28 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
29 class.
30
31Adding ptest to Your Build
32==========================
33
34To add package testing to your build, add the :term:`DISTRO_FEATURES` and
35:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
36is found in the :term:`Build Directory`::
37
38 DISTRO_FEATURES:append = " ptest"
39 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
40
41Once your build is complete, the ptest files are installed into the
42``/usr/lib/package/ptest`` directory within the image, where ``package``
43is the name of the package.
44
45Running ptest
46=============
47
48The ``ptest-runner`` package installs a shell script that loops through
49all installed ptest test suites and runs them in sequence. Consequently,
50you might want to add this package to your image.
51
52Getting Your Package Ready
53==========================
54
55In order to enable a recipe to run installed ptests on target hardware,
56you need to prepare the recipes that build the packages you want to
57test. Here is what you have to do for each recipe:
58
59- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
60 Include the following line in each recipe::
61
62 inherit ptest
63
64- *Create run-ptest:* This script starts your test. Locate the
65 script where you will refer to it using
66 :term:`SRC_URI`. Here is an
67 example that starts a test for ``dbus``::
68
69 #!/bin/sh
70 cd test
71 make -k runtest-TESTS
72
73- *Ensure dependencies are met:* If the test adds build or runtime
74 dependencies that normally do not exist for the package (such as
75 requiring "make" to run the test suite), use the
76 :term:`DEPENDS` and
77 :term:`RDEPENDS` variables in
78 your recipe in order for the package to meet the dependencies. Here
79 is an example where the package has a runtime dependency on "make"::
80
81 RDEPENDS:${PN}-ptest += "make"
82
83- *Add a function to build the test suite:* Not many packages support
84 cross-compilation of their test suites. Consequently, you usually
85 need to add a cross-compilation function to the package.
86
87 Many packages based on Automake compile and run the test suite by
88 using a single command such as ``make check``. However, the host
89 ``make check`` builds and runs on the same computer, while
90 cross-compiling requires that the package is built on the host but
91 executed for the target architecture (though often, as in the case
92 for ptest, the execution occurs on the host). The built version of
93 Automake that ships with the Yocto Project includes a patch that
94 separates building and execution. Consequently, packages that use the
95 unaltered, patched version of ``make check`` automatically
96 cross-compiles.
97
98 Regardless, you still must add a ``do_compile_ptest`` function to
99 build the test suite. Add a function similar to the following to your
100 recipe::
101
102 do_compile_ptest() {
103 oe_runmake buildtest-TESTS
104 }
105
106- *Ensure special configurations are set:* If the package requires
107 special configurations prior to compiling the test code, you must
108 insert a ``do_configure_ptest`` function into the recipe.
109
110- *Install the test suite:* The :ref:`ref-classes-ptest` class
111 automatically copies the file ``run-ptest`` to the target and then runs make
112 ``install-ptest`` to run the tests. If this is not enough, you need
113 to create a ``do_install_ptest`` function and make sure it gets
114 called after the "make install-ptest" completes.