diff options
author | Antonin Godard <antonin.godard@bootlin.com> | 2024-12-26 16:20:16 +0100 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2025-01-09 13:58:17 +0000 |
commit | d720994f023f9c0bf23d6234089b21e23d837dc5 (patch) | |
tree | f556cf36bc93f8036d60ab50d331d0810512dba8 /documentation | |
parent | b301c74baf33caee0ee8560fb019034ea6c9856f (diff) | |
download | poky-d720994f023f9c0bf23d6234089b21e23d837dc5.tar.gz |
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ]
It makes more sense to document ptests in the test-manual. Since ptests
are still related to packages, keep a link to ptests from packages.rst
to the test-manual.
Reported-by: Yoann Congal <yoann.congal@smile.fr>
(From yocto-docs rev: b389c06b709e4791e1cce5e8a5b58f6b0cd03a14)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r-- | documentation/dev-manual/packages.rst | 112 | ||||
-rw-r--r-- | documentation/migration-guides/migration-1.6.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/classes.rst | 4 | ||||
-rw-r--r-- | documentation/ref-manual/features.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/qa-checks.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/release-process.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/variables.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/index.rst | 1 | ||||
-rw-r--r-- | documentation/test-manual/intro.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/ptest.rst | 114 |
10 files changed, 126 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index 33cf78747d..4ba2dcae3a 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
17 | 17 | ||
18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
20 | 20 | ||
21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
22 | 22 | ||
@@ -882,114 +882,8 @@ related to signed package feeds are available: | |||
882 | Testing Packages With ptest | 882 | Testing Packages With ptest |
883 | =========================== | 883 | =========================== |
884 | 884 | ||
885 | A Package Test (ptest) runs tests against packages built by the | 885 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
886 | OpenEmbedded build system on the target machine. A ptest contains at | 886 | Yocto Project Test Environment Manual. |
887 | least two items: the actual test, and a shell script (``run-ptest``) | ||
888 | that starts the test. The shell script that starts the test must not | ||
889 | contain the actual test --- the script only starts the test. On the other | ||
890 | hand, the test can be anything from a simple shell script that runs a | ||
891 | binary and checks the output to an elaborate system of test binaries and | ||
892 | data files. | ||
893 | |||
894 | The test generates output in the format used by Automake:: | ||
895 | |||
896 | result: testname | ||
897 | |||
898 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
899 | the testname can be any identifying string. | ||
900 | |||
901 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
902 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
903 | |||
904 | .. note:: | ||
905 | |||
906 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
907 | class. | ||
908 | |||
909 | Adding ptest to Your Build | ||
910 | -------------------------- | ||
911 | |||
912 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
913 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
914 | is found in the :term:`Build Directory`:: | ||
915 | |||
916 | DISTRO_FEATURES:append = " ptest" | ||
917 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
918 | |||
919 | Once your build is complete, the ptest files are installed into the | ||
920 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
921 | is the name of the package. | ||
922 | |||
923 | Running ptest | ||
924 | ------------- | ||
925 | |||
926 | The ``ptest-runner`` package installs a shell script that loops through | ||
927 | all installed ptest test suites and runs them in sequence. Consequently, | ||
928 | you might want to add this package to your image. | ||
929 | |||
930 | Getting Your Package Ready | ||
931 | -------------------------- | ||
932 | |||
933 | In order to enable a recipe to run installed ptests on target hardware, | ||
934 | you need to prepare the recipes that build the packages you want to | ||
935 | test. Here is what you have to do for each recipe: | ||
936 | |||
937 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
938 | Include the following line in each recipe:: | ||
939 | |||
940 | inherit ptest | ||
941 | |||
942 | - *Create run-ptest:* This script starts your test. Locate the | ||
943 | script where you will refer to it using | ||
944 | :term:`SRC_URI`. Here is an | ||
945 | example that starts a test for ``dbus``:: | ||
946 | |||
947 | #!/bin/sh | ||
948 | cd test | ||
949 | make -k runtest-TESTS | ||
950 | |||
951 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
952 | dependencies that normally do not exist for the package (such as | ||
953 | requiring "make" to run the test suite), use the | ||
954 | :term:`DEPENDS` and | ||
955 | :term:`RDEPENDS` variables in | ||
956 | your recipe in order for the package to meet the dependencies. Here | ||
957 | is an example where the package has a runtime dependency on "make":: | ||
958 | |||
959 | RDEPENDS:${PN}-ptest += "make" | ||
960 | |||
961 | - *Add a function to build the test suite:* Not many packages support | ||
962 | cross-compilation of their test suites. Consequently, you usually | ||
963 | need to add a cross-compilation function to the package. | ||
964 | |||
965 | Many packages based on Automake compile and run the test suite by | ||
966 | using a single command such as ``make check``. However, the host | ||
967 | ``make check`` builds and runs on the same computer, while | ||
968 | cross-compiling requires that the package is built on the host but | ||
969 | executed for the target architecture (though often, as in the case | ||
970 | for ptest, the execution occurs on the host). The built version of | ||
971 | Automake that ships with the Yocto Project includes a patch that | ||
972 | separates building and execution. Consequently, packages that use the | ||
973 | unaltered, patched version of ``make check`` automatically | ||
974 | cross-compiles. | ||
975 | |||
976 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
977 | build the test suite. Add a function similar to the following to your | ||
978 | recipe:: | ||
979 | |||
980 | do_compile_ptest() { | ||
981 | oe_runmake buildtest-TESTS | ||
982 | } | ||
983 | |||
984 | - *Ensure special configurations are set:* If the package requires | ||
985 | special configurations prior to compiling the test code, you must | ||
986 | insert a ``do_configure_ptest`` function into the recipe. | ||
987 | |||
988 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
989 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
990 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
991 | to create a ``do_install_ptest`` function and make sure it gets | ||
992 | called after the "make install-ptest" completes. | ||
993 | 887 | ||
994 | Creating Node Package Manager (NPM) Packages | 888 | Creating Node Package Manager (NPM) Packages |
995 | ============================================ | 889 | ============================================ |
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst index 916169e836..b052a43a31 100644 --- a/documentation/migration-guides/migration-1.6.rst +++ b/documentation/migration-guides/migration-1.6.rst | |||
@@ -221,7 +221,7 @@ Package Test (ptest) | |||
221 | 221 | ||
222 | Package Tests (ptest) are built but not installed by default. For | 222 | Package Tests (ptest) are built but not installed by default. For |
223 | information on using Package Tests, see the | 223 | information on using Package Tests, see the |
224 | ":ref:`dev-manual/packages:testing packages with ptest`" section in the | 224 | ":ref:`test-manual/ptest:testing packages with ptest`" section in the |
225 | Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" | 225 | Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" |
226 | section. | 226 | section. |
227 | 227 | ||
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index aee0d55c30..c853462d6f 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst | |||
@@ -2627,7 +2627,7 @@ runtime tests for recipes that build software that provides these tests. | |||
2627 | This class is intended to be inherited by individual recipes. However, | 2627 | This class is intended to be inherited by individual recipes. However, |
2628 | the class' functionality is largely disabled unless "ptest" appears in | 2628 | the class' functionality is largely disabled unless "ptest" appears in |
2629 | :term:`DISTRO_FEATURES`. See the | 2629 | :term:`DISTRO_FEATURES`. See the |
2630 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2630 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2631 | section in the Yocto Project Development Tasks Manual for more information | 2631 | section in the Yocto Project Development Tasks Manual for more information |
2632 | on ptest. | 2632 | on ptest. |
2633 | 2633 | ||
@@ -2651,7 +2651,7 @@ Enables package tests (ptests) specifically for GNOME packages, which | |||
2651 | have tests intended to be executed with ``gnome-desktop-testing``. | 2651 | have tests intended to be executed with ``gnome-desktop-testing``. |
2652 | 2652 | ||
2653 | For information on setting up and running ptests, see the | 2653 | For information on setting up and running ptests, see the |
2654 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2654 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2655 | section in the Yocto Project Development Tasks Manual. | 2655 | section in the Yocto Project Development Tasks Manual. |
2656 | 2656 | ||
2657 | .. _ref-classes-python3-dir: | 2657 | .. _ref-classes-python3-dir: |
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst index 4d327f1a7d..70e85f7a20 100644 --- a/documentation/ref-manual/features.rst +++ b/documentation/ref-manual/features.rst | |||
@@ -207,7 +207,7 @@ metadata, as extra layers can define their own: | |||
207 | 207 | ||
208 | - *ptest:* Enables building the package tests where supported by | 208 | - *ptest:* Enables building the package tests where supported by |
209 | individual recipes. For more information on package tests, see the | 209 | individual recipes. For more information on package tests, see the |
210 | ":ref:`dev-manual/packages:testing packages with ptest`" section | 210 | ":ref:`test-manual/ptest:testing packages with ptest`" section |
211 | in the Yocto Project Development Tasks Manual. | 211 | in the Yocto Project Development Tasks Manual. |
212 | 212 | ||
213 | - *pulseaudio:* Include support for | 213 | - *pulseaudio:* Include support for |
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst index 1b83834df1..27d46de3fd 100644 --- a/documentation/ref-manual/qa-checks.rst +++ b/documentation/ref-manual/qa-checks.rst | |||
@@ -791,7 +791,7 @@ Errors and Warnings | |||
791 | 791 | ||
792 | This check will detect if the source of the package contains some | 792 | This check will detect if the source of the package contains some |
793 | upstream-provided tests and, if so, that ptests are implemented for this | 793 | upstream-provided tests and, if so, that ptests are implemented for this |
794 | recipe. See the ":ref:`dev-manual/packages:testing packages with ptest`" | 794 | recipe. See the ":ref:`test-manual/ptest:testing packages with ptest`" |
795 | section in the Yocto Project Development Tasks Manual. See also the | 795 | section in the Yocto Project Development Tasks Manual. See also the |
796 | ":ref:`ref-classes-ptest`" section. | 796 | ":ref:`ref-classes-ptest`" section. |
797 | 797 | ||
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 691e7d3ed9..3383a4e1b0 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst | |||
@@ -175,7 +175,7 @@ consists of the following pieces: | |||
175 | operation and functions. However, the test can also use the IP | 175 | operation and functions. However, the test can also use the IP |
176 | address of a machine to test. | 176 | address of a machine to test. |
177 | 177 | ||
178 | - :ref:`ptest <dev-manual/packages:testing packages with ptest>`: | 178 | - :ref:`ptest <test-manual/ptest:testing packages with ptest>`: |
179 | Runs tests against packages produced during the build for a given | 179 | Runs tests against packages produced during the build for a given |
180 | piece of software. The test allows the packages to be run within a | 180 | piece of software. The test allows the packages to be run within a |
181 | target image. | 181 | target image. |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index a45c77209d..f4d50b7918 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
@@ -6947,7 +6947,7 @@ system and gives an overview of their function and contents. | |||
6947 | 6947 | ||
6948 | :term:`PTEST_ENABLED` | 6948 | :term:`PTEST_ENABLED` |
6949 | Specifies whether or not :ref:`Package | 6949 | Specifies whether or not :ref:`Package |
6950 | Test <dev-manual/packages:testing packages with ptest>` (ptest) | 6950 | Test <test-manual/ptest:testing packages with ptest>` (ptest) |
6951 | functionality is enabled when building a recipe. You should not set | 6951 | functionality is enabled when building a recipe. You should not set |
6952 | this variable directly. Enabling and disabling building Package Tests | 6952 | this variable directly. Enabling and disabling building Package Tests |
6953 | at build time should be done by adding "ptest" to (or removing it | 6953 | at build time should be done by adding "ptest" to (or removing it |
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea..ad71f37910 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst | |||
@@ -12,6 +12,7 @@ Yocto Project Test Environment Manual | |||
12 | 12 | ||
13 | intro | 13 | intro |
14 | test-process | 14 | test-process |
15 | ptest | ||
15 | understand-autobuilder | 16 | understand-autobuilder |
16 | reproducible-builds | 17 | reproducible-builds |
17 | yocto-project-compatible | 18 | yocto-project-compatible |
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index 882ca16486..252d3ea731 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst | |||
@@ -140,7 +140,7 @@ the following types of tests: | |||
140 | - *Package Testing:* A Package Test (ptest) runs tests against packages | 140 | - *Package Testing:* A Package Test (ptest) runs tests against packages |
141 | built by the OpenEmbedded build system on the target machine. See the | 141 | built by the OpenEmbedded build system on the target machine. See the |
142 | :ref:`Testing Packages With | 142 | :ref:`Testing Packages With |
143 | ptest <dev-manual/packages:Testing Packages With ptest>` section | 143 | ptest <test-manual/ptest:Testing Packages With ptest>` section |
144 | in the Yocto Project Development Tasks Manual and the | 144 | in the Yocto Project Development Tasks Manual and the |
145 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more | 145 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more |
146 | information on Ptest. | 146 | information on Ptest. |
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..dea1bad23b --- /dev/null +++ b/documentation/test-manual/ptest.rst | |||
@@ -0,0 +1,114 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | *************************** | ||
4 | Testing Packages With ptest | ||
5 | *************************** | ||
6 | |||
7 | A Package Test (ptest) runs tests against packages built by the | ||
8 | OpenEmbedded build system on the target machine. A ptest contains at | ||
9 | least two items: the actual test, and a shell script (``run-ptest``) | ||
10 | that starts the test. The shell script that starts the test must not | ||
11 | contain the actual test --- the script only starts the test. On the other | ||
12 | hand, the test can be anything from a simple shell script that runs a | ||
13 | binary and checks the output to an elaborate system of test binaries and | ||
14 | data files. | ||
15 | |||
16 | The test generates output in the format used by Automake:: | ||
17 | |||
18 | result: testname | ||
19 | |||
20 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
21 | the testname can be any identifying string. | ||
22 | |||
23 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
24 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
25 | |||
26 | .. note:: | ||
27 | |||
28 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
29 | class. | ||
30 | |||
31 | Adding ptest to Your Build | ||
32 | ========================== | ||
33 | |||
34 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
35 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
36 | is found in the :term:`Build Directory`:: | ||
37 | |||
38 | DISTRO_FEATURES:append = " ptest" | ||
39 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
40 | |||
41 | Once your build is complete, the ptest files are installed into the | ||
42 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
43 | is the name of the package. | ||
44 | |||
45 | Running ptest | ||
46 | ============= | ||
47 | |||
48 | The ``ptest-runner`` package installs a shell script that loops through | ||
49 | all installed ptest test suites and runs them in sequence. Consequently, | ||
50 | you might want to add this package to your image. | ||
51 | |||
52 | Getting Your Package Ready | ||
53 | ========================== | ||
54 | |||
55 | In order to enable a recipe to run installed ptests on target hardware, | ||
56 | you need to prepare the recipes that build the packages you want to | ||
57 | test. Here is what you have to do for each recipe: | ||
58 | |||
59 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
60 | Include the following line in each recipe:: | ||
61 | |||
62 | inherit ptest | ||
63 | |||
64 | - *Create run-ptest:* This script starts your test. Locate the | ||
65 | script where you will refer to it using | ||
66 | :term:`SRC_URI`. Here is an | ||
67 | example that starts a test for ``dbus``:: | ||
68 | |||
69 | #!/bin/sh | ||
70 | cd test | ||
71 | make -k runtest-TESTS | ||
72 | |||
73 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
74 | dependencies that normally do not exist for the package (such as | ||
75 | requiring "make" to run the test suite), use the | ||
76 | :term:`DEPENDS` and | ||
77 | :term:`RDEPENDS` variables in | ||
78 | your recipe in order for the package to meet the dependencies. Here | ||
79 | is an example where the package has a runtime dependency on "make":: | ||
80 | |||
81 | RDEPENDS:${PN}-ptest += "make" | ||
82 | |||
83 | - *Add a function to build the test suite:* Not many packages support | ||
84 | cross-compilation of their test suites. Consequently, you usually | ||
85 | need to add a cross-compilation function to the package. | ||
86 | |||
87 | Many packages based on Automake compile and run the test suite by | ||
88 | using a single command such as ``make check``. However, the host | ||
89 | ``make check`` builds and runs on the same computer, while | ||
90 | cross-compiling requires that the package is built on the host but | ||
91 | executed for the target architecture (though often, as in the case | ||
92 | for ptest, the execution occurs on the host). The built version of | ||
93 | Automake that ships with the Yocto Project includes a patch that | ||
94 | separates building and execution. Consequently, packages that use the | ||
95 | unaltered, patched version of ``make check`` automatically | ||
96 | cross-compiles. | ||
97 | |||
98 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
99 | build the test suite. Add a function similar to the following to your | ||
100 | recipe:: | ||
101 | |||
102 | do_compile_ptest() { | ||
103 | oe_runmake buildtest-TESTS | ||
104 | } | ||
105 | |||
106 | - *Ensure special configurations are set:* If the package requires | ||
107 | special configurations prior to compiling the test code, you must | ||
108 | insert a ``do_configure_ptest`` function into the recipe. | ||
109 | |||
110 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
111 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
112 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
113 | to create a ``do_install_ptest`` function and make sure it gets | ||
114 | called after the "make install-ptest" completes. | ||