diff options
author | Antonin Godard <antonin.godard@bootlin.com> | 2024-12-26 16:20:16 +0100 |
---|---|---|
committer | Steve Sakoman <steve@sakoman.com> | 2025-01-09 06:25:36 -0800 |
commit | 3f782b6e4527d5b02cc98203b0fb79620b6a89df (patch) | |
tree | 8eae2524c9aab854a24414d1f9069bf04adf40b7 /documentation | |
parent | da6fec8a1da6b22e1cd28ba638ff4b9e4777630e (diff) | |
download | poky-3f782b6e4527d5b02cc98203b0fb79620b6a89df.tar.gz |
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ]
It makes more sense to document ptests in the test-manual. Since ptests
are still related to packages, keep a link to ptests from packages.rst
to the test-manual.
Reported-by: Yoann Congal <yoann.congal@smile.fr>
(From yocto-docs rev: 110e15c4407dfc03c7d931e4488eb43dbfad7570)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Diffstat (limited to 'documentation')
-rw-r--r-- | documentation/dev-manual/packages.rst | 112 | ||||
-rw-r--r-- | documentation/migration-guides/migration-1.6.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/classes.rst | 4 | ||||
-rw-r--r-- | documentation/ref-manual/features.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/qa-checks.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/release-process.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/variables.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/index.rst | 1 | ||||
-rw-r--r-- | documentation/test-manual/intro.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/ptest.rst | 114 |
10 files changed, 126 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index 33cf78747d..4ba2dcae3a 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
17 | 17 | ||
18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
20 | 20 | ||
21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
22 | 22 | ||
@@ -882,114 +882,8 @@ related to signed package feeds are available: | |||
882 | Testing Packages With ptest | 882 | Testing Packages With ptest |
883 | =========================== | 883 | =========================== |
884 | 884 | ||
885 | A Package Test (ptest) runs tests against packages built by the | 885 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
886 | OpenEmbedded build system on the target machine. A ptest contains at | 886 | Yocto Project Test Environment Manual. |
887 | least two items: the actual test, and a shell script (``run-ptest``) | ||
888 | that starts the test. The shell script that starts the test must not | ||
889 | contain the actual test --- the script only starts the test. On the other | ||
890 | hand, the test can be anything from a simple shell script that runs a | ||
891 | binary and checks the output to an elaborate system of test binaries and | ||
892 | data files. | ||
893 | |||
894 | The test generates output in the format used by Automake:: | ||
895 | |||
896 | result: testname | ||
897 | |||
898 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
899 | the testname can be any identifying string. | ||
900 | |||
901 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
902 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
903 | |||
904 | .. note:: | ||
905 | |||
906 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
907 | class. | ||
908 | |||
909 | Adding ptest to Your Build | ||
910 | -------------------------- | ||
911 | |||
912 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
913 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
914 | is found in the :term:`Build Directory`:: | ||
915 | |||
916 | DISTRO_FEATURES:append = " ptest" | ||
917 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
918 | |||
919 | Once your build is complete, the ptest files are installed into the | ||
920 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
921 | is the name of the package. | ||
922 | |||
923 | Running ptest | ||
924 | ------------- | ||
925 | |||
926 | The ``ptest-runner`` package installs a shell script that loops through | ||
927 | all installed ptest test suites and runs them in sequence. Consequently, | ||
928 | you might want to add this package to your image. | ||
929 | |||
930 | Getting Your Package Ready | ||
931 | -------------------------- | ||
932 | |||
933 | In order to enable a recipe to run installed ptests on target hardware, | ||
934 | you need to prepare the recipes that build the packages you want to | ||
935 | test. Here is what you have to do for each recipe: | ||
936 | |||
937 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
938 | Include the following line in each recipe:: | ||
939 | |||
940 | inherit ptest | ||
941 | |||
942 | - *Create run-ptest:* This script starts your test. Locate the | ||
943 | script where you will refer to it using | ||
944 | :term:`SRC_URI`. Here is an | ||
945 | example that starts a test for ``dbus``:: | ||
946 | |||
947 | #!/bin/sh | ||
948 | cd test | ||
949 | make -k runtest-TESTS | ||
950 | |||
951 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
952 | dependencies that normally do not exist for the package (such as | ||
953 | requiring "make" to run the test suite), use the | ||
954 | :term:`DEPENDS` and | ||
955 | :term:`RDEPENDS` variables in | ||
956 | your recipe in order for the package to meet the dependencies. Here | ||
957 | is an example where the package has a runtime dependency on "make":: | ||
958 | |||
959 | RDEPENDS:${PN}-ptest += "make" | ||
960 | |||
961 | - *Add a function to build the test suite:* Not many packages support | ||
962 | cross-compilation of their test suites. Consequently, you usually | ||
963 | need to add a cross-compilation function to the package. | ||
964 | |||
965 | Many packages based on Automake compile and run the test suite by | ||
966 | using a single command such as ``make check``. However, the host | ||
967 | ``make check`` builds and runs on the same computer, while | ||
968 | cross-compiling requires that the package is built on the host but | ||
969 | executed for the target architecture (though often, as in the case | ||
970 | for ptest, the execution occurs on the host). The built version of | ||
971 | Automake that ships with the Yocto Project includes a patch that | ||
972 | separates building and execution. Consequently, packages that use the | ||
973 | unaltered, patched version of ``make check`` automatically | ||
974 | cross-compiles. | ||
975 | |||
976 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
977 | build the test suite. Add a function similar to the following to your | ||
978 | recipe:: | ||
979 | |||
980 | do_compile_ptest() { | ||
981 | oe_runmake buildtest-TESTS | ||
982 | } | ||
983 | |||
984 | - *Ensure special configurations are set:* If the package requires | ||
985 | special configurations prior to compiling the test code, you must | ||
986 | insert a ``do_configure_ptest`` function into the recipe. | ||
987 | |||
988 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
989 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
990 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
991 | to create a ``do_install_ptest`` function and make sure it gets | ||
992 | called after the "make install-ptest" completes. | ||
993 | 887 | ||
994 | Creating Node Package Manager (NPM) Packages | 888 | Creating Node Package Manager (NPM) Packages |
995 | ============================================ | 889 | ============================================ |
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst index 916169e836..b052a43a31 100644 --- a/documentation/migration-guides/migration-1.6.rst +++ b/documentation/migration-guides/migration-1.6.rst | |||
@@ -221,7 +221,7 @@ Package Test (ptest) | |||
221 | 221 | ||
222 | Package Tests (ptest) are built but not installed by default. For | 222 | Package Tests (ptest) are built but not installed by default. For |
223 | information on using Package Tests, see the | 223 | information on using Package Tests, see the |
224 | ":ref:`dev-manual/packages:testing packages with ptest`" section in the | 224 | ":ref:`test-manual/ptest:testing packages with ptest`" section in the |
225 | Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" | 225 | Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" |
226 | section. | 226 | section. |
227 | 227 | ||
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 8c3cc0140f..f416a6d623 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst | |||
@@ -2615,7 +2615,7 @@ runtime tests for recipes that build software that provides these tests. | |||
2615 | This class is intended to be inherited by individual recipes. However, | 2615 | This class is intended to be inherited by individual recipes. However, |
2616 | the class' functionality is largely disabled unless "ptest" appears in | 2616 | the class' functionality is largely disabled unless "ptest" appears in |
2617 | :term:`DISTRO_FEATURES`. See the | 2617 | :term:`DISTRO_FEATURES`. See the |
2618 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2618 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2619 | section in the Yocto Project Development Tasks Manual for more information | 2619 | section in the Yocto Project Development Tasks Manual for more information |
2620 | on ptest. | 2620 | on ptest. |
2621 | 2621 | ||
@@ -2639,7 +2639,7 @@ Enables package tests (ptests) specifically for GNOME packages, which | |||
2639 | have tests intended to be executed with ``gnome-desktop-testing``. | 2639 | have tests intended to be executed with ``gnome-desktop-testing``. |
2640 | 2640 | ||
2641 | For information on setting up and running ptests, see the | 2641 | For information on setting up and running ptests, see the |
2642 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2642 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2643 | section in the Yocto Project Development Tasks Manual. | 2643 | section in the Yocto Project Development Tasks Manual. |
2644 | 2644 | ||
2645 | .. _ref-classes-python3-dir: | 2645 | .. _ref-classes-python3-dir: |
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst index b2ba731bb2..face6c1062 100644 --- a/documentation/ref-manual/features.rst +++ b/documentation/ref-manual/features.rst | |||
@@ -207,7 +207,7 @@ metadata, as extra layers can define their own: | |||
207 | 207 | ||
208 | - *ptest:* Enables building the package tests where supported by | 208 | - *ptest:* Enables building the package tests where supported by |
209 | individual recipes. For more information on package tests, see the | 209 | individual recipes. For more information on package tests, see the |
210 | ":ref:`dev-manual/packages:testing packages with ptest`" section | 210 | ":ref:`test-manual/ptest:testing packages with ptest`" section |
211 | in the Yocto Project Development Tasks Manual. | 211 | in the Yocto Project Development Tasks Manual. |
212 | 212 | ||
213 | - *pulseaudio:* Include support for | 213 | - *pulseaudio:* Include support for |
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst index 1b83834df1..27d46de3fd 100644 --- a/documentation/ref-manual/qa-checks.rst +++ b/documentation/ref-manual/qa-checks.rst | |||
@@ -791,7 +791,7 @@ Errors and Warnings | |||
791 | 791 | ||
792 | This check will detect if the source of the package contains some | 792 | This check will detect if the source of the package contains some |
793 | upstream-provided tests and, if so, that ptests are implemented for this | 793 | upstream-provided tests and, if so, that ptests are implemented for this |
794 | recipe. See the ":ref:`dev-manual/packages:testing packages with ptest`" | 794 | recipe. See the ":ref:`test-manual/ptest:testing packages with ptest`" |
795 | section in the Yocto Project Development Tasks Manual. See also the | 795 | section in the Yocto Project Development Tasks Manual. See also the |
796 | ":ref:`ref-classes-ptest`" section. | 796 | ":ref:`ref-classes-ptest`" section. |
797 | 797 | ||
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 691e7d3ed9..3383a4e1b0 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst | |||
@@ -175,7 +175,7 @@ consists of the following pieces: | |||
175 | operation and functions. However, the test can also use the IP | 175 | operation and functions. However, the test can also use the IP |
176 | address of a machine to test. | 176 | address of a machine to test. |
177 | 177 | ||
178 | - :ref:`ptest <dev-manual/packages:testing packages with ptest>`: | 178 | - :ref:`ptest <test-manual/ptest:testing packages with ptest>`: |
179 | Runs tests against packages produced during the build for a given | 179 | Runs tests against packages produced during the build for a given |
180 | piece of software. The test allows the packages to be run within a | 180 | piece of software. The test allows the packages to be run within a |
181 | target image. | 181 | target image. |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 869100ce34..200def084f 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
@@ -6828,7 +6828,7 @@ system and gives an overview of their function and contents. | |||
6828 | 6828 | ||
6829 | :term:`PTEST_ENABLED` | 6829 | :term:`PTEST_ENABLED` |
6830 | Specifies whether or not :ref:`Package | 6830 | Specifies whether or not :ref:`Package |
6831 | Test <dev-manual/packages:testing packages with ptest>` (ptest) | 6831 | Test <test-manual/ptest:testing packages with ptest>` (ptest) |
6832 | functionality is enabled when building a recipe. You should not set | 6832 | functionality is enabled when building a recipe. You should not set |
6833 | this variable directly. Enabling and disabling building Package Tests | 6833 | this variable directly. Enabling and disabling building Package Tests |
6834 | at build time should be done by adding "ptest" to (or removing it | 6834 | at build time should be done by adding "ptest" to (or removing it |
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea..ad71f37910 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst | |||
@@ -12,6 +12,7 @@ Yocto Project Test Environment Manual | |||
12 | 12 | ||
13 | intro | 13 | intro |
14 | test-process | 14 | test-process |
15 | ptest | ||
15 | understand-autobuilder | 16 | understand-autobuilder |
16 | reproducible-builds | 17 | reproducible-builds |
17 | yocto-project-compatible | 18 | yocto-project-compatible |
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index c31fd11c7a..7bbd11295f 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst | |||
@@ -140,7 +140,7 @@ the following types of tests: | |||
140 | - *Package Testing:* A Package Test (ptest) runs tests against packages | 140 | - *Package Testing:* A Package Test (ptest) runs tests against packages |
141 | built by the OpenEmbedded build system on the target machine. See the | 141 | built by the OpenEmbedded build system on the target machine. See the |
142 | :ref:`Testing Packages With | 142 | :ref:`Testing Packages With |
143 | ptest <dev-manual/packages:Testing Packages With ptest>` section | 143 | ptest <test-manual/ptest:Testing Packages With ptest>` section |
144 | in the Yocto Project Development Tasks Manual and the | 144 | in the Yocto Project Development Tasks Manual and the |
145 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more | 145 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more |
146 | information on Ptest. | 146 | information on Ptest. |
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..dea1bad23b --- /dev/null +++ b/documentation/test-manual/ptest.rst | |||
@@ -0,0 +1,114 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | *************************** | ||
4 | Testing Packages With ptest | ||
5 | *************************** | ||
6 | |||
7 | A Package Test (ptest) runs tests against packages built by the | ||
8 | OpenEmbedded build system on the target machine. A ptest contains at | ||
9 | least two items: the actual test, and a shell script (``run-ptest``) | ||
10 | that starts the test. The shell script that starts the test must not | ||
11 | contain the actual test --- the script only starts the test. On the other | ||
12 | hand, the test can be anything from a simple shell script that runs a | ||
13 | binary and checks the output to an elaborate system of test binaries and | ||
14 | data files. | ||
15 | |||
16 | The test generates output in the format used by Automake:: | ||
17 | |||
18 | result: testname | ||
19 | |||
20 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
21 | the testname can be any identifying string. | ||
22 | |||
23 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
24 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
25 | |||
26 | .. note:: | ||
27 | |||
28 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
29 | class. | ||
30 | |||
31 | Adding ptest to Your Build | ||
32 | ========================== | ||
33 | |||
34 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
35 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
36 | is found in the :term:`Build Directory`:: | ||
37 | |||
38 | DISTRO_FEATURES:append = " ptest" | ||
39 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
40 | |||
41 | Once your build is complete, the ptest files are installed into the | ||
42 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
43 | is the name of the package. | ||
44 | |||
45 | Running ptest | ||
46 | ============= | ||
47 | |||
48 | The ``ptest-runner`` package installs a shell script that loops through | ||
49 | all installed ptest test suites and runs them in sequence. Consequently, | ||
50 | you might want to add this package to your image. | ||
51 | |||
52 | Getting Your Package Ready | ||
53 | ========================== | ||
54 | |||
55 | In order to enable a recipe to run installed ptests on target hardware, | ||
56 | you need to prepare the recipes that build the packages you want to | ||
57 | test. Here is what you have to do for each recipe: | ||
58 | |||
59 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
60 | Include the following line in each recipe:: | ||
61 | |||
62 | inherit ptest | ||
63 | |||
64 | - *Create run-ptest:* This script starts your test. Locate the | ||
65 | script where you will refer to it using | ||
66 | :term:`SRC_URI`. Here is an | ||
67 | example that starts a test for ``dbus``:: | ||
68 | |||
69 | #!/bin/sh | ||
70 | cd test | ||
71 | make -k runtest-TESTS | ||
72 | |||
73 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
74 | dependencies that normally do not exist for the package (such as | ||
75 | requiring "make" to run the test suite), use the | ||
76 | :term:`DEPENDS` and | ||
77 | :term:`RDEPENDS` variables in | ||
78 | your recipe in order for the package to meet the dependencies. Here | ||
79 | is an example where the package has a runtime dependency on "make":: | ||
80 | |||
81 | RDEPENDS:${PN}-ptest += "make" | ||
82 | |||
83 | - *Add a function to build the test suite:* Not many packages support | ||
84 | cross-compilation of their test suites. Consequently, you usually | ||
85 | need to add a cross-compilation function to the package. | ||
86 | |||
87 | Many packages based on Automake compile and run the test suite by | ||
88 | using a single command such as ``make check``. However, the host | ||
89 | ``make check`` builds and runs on the same computer, while | ||
90 | cross-compiling requires that the package is built on the host but | ||
91 | executed for the target architecture (though often, as in the case | ||
92 | for ptest, the execution occurs on the host). The built version of | ||
93 | Automake that ships with the Yocto Project includes a patch that | ||
94 | separates building and execution. Consequently, packages that use the | ||
95 | unaltered, patched version of ``make check`` automatically | ||
96 | cross-compiles. | ||
97 | |||
98 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
99 | build the test suite. Add a function similar to the following to your | ||
100 | recipe:: | ||
101 | |||
102 | do_compile_ptest() { | ||
103 | oe_runmake buildtest-TESTS | ||
104 | } | ||
105 | |||
106 | - *Ensure special configurations are set:* If the package requires | ||
107 | special configurations prior to compiling the test code, you must | ||
108 | insert a ``do_configure_ptest`` function into the recipe. | ||
109 | |||
110 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
111 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
112 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
113 | to create a ``do_install_ptest`` function and make sure it gets | ||
114 | called after the "make install-ptest" completes. | ||