diff options
author | Antonin Godard <antonin.godard@bootlin.com> | 2024-12-26 16:20:16 +0100 |
---|---|---|
committer | Steve Sakoman <steve@sakoman.com> | 2025-01-09 08:41:04 -0800 |
commit | 2497f29afd1ff66964b84dc79a7cd1126bbc94fa (patch) | |
tree | 494d2d1a6b358c3796d870e1e060a949addf37b9 | |
parent | 1418534d2215890723c281f852b6a6fd08acdbcf (diff) | |
download | poky-2497f29afd1ff66964b84dc79a7cd1126bbc94fa.tar.gz |
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ]
It makes more sense to document ptests in the test-manual. Since ptests
are still related to packages, keep a link to ptests from packages.rst
to the test-manual.
Reported-by: Yoann Congal <yoann.congal@smile.fr>
(From yocto-docs rev: 8b6ada020d595d86c7bbe78a27b7a6301715b039)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
-rw-r--r-- | documentation/dev-manual/packages.rst | 112 | ||||
-rw-r--r-- | documentation/migration-guides/migration-1.6.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/classes.rst | 4 | ||||
-rw-r--r-- | documentation/ref-manual/features.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/qa-checks.rst | 1 | ||||
-rw-r--r-- | documentation/ref-manual/release-process.rst | 2 | ||||
-rw-r--r-- | documentation/ref-manual/variables.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/index.rst | 1 | ||||
-rw-r--r-- | documentation/test-manual/intro.rst | 2 | ||||
-rw-r--r-- | documentation/test-manual/ptest.rst | 114 |
10 files changed, 125 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index e5028fffdc..0c948cdaef 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
17 | 17 | ||
18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
20 | 20 | ||
21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
22 | 22 | ||
@@ -887,114 +887,8 @@ related to signed package feeds are available: | |||
887 | Testing Packages With ptest | 887 | Testing Packages With ptest |
888 | =========================== | 888 | =========================== |
889 | 889 | ||
890 | A Package Test (ptest) runs tests against packages built by the | 890 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
891 | OpenEmbedded build system on the target machine. A ptest contains at | 891 | Yocto Project Test Environment Manual. |
892 | least two items: the actual test, and a shell script (``run-ptest``) | ||
893 | that starts the test. The shell script that starts the test must not | ||
894 | contain the actual test --- the script only starts the test. On the other | ||
895 | hand, the test can be anything from a simple shell script that runs a | ||
896 | binary and checks the output to an elaborate system of test binaries and | ||
897 | data files. | ||
898 | |||
899 | The test generates output in the format used by Automake:: | ||
900 | |||
901 | result: testname | ||
902 | |||
903 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
904 | the testname can be any identifying string. | ||
905 | |||
906 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
907 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
908 | |||
909 | .. note:: | ||
910 | |||
911 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
912 | class. | ||
913 | |||
914 | Adding ptest to Your Build | ||
915 | -------------------------- | ||
916 | |||
917 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
918 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
919 | is found in the :term:`Build Directory`:: | ||
920 | |||
921 | DISTRO_FEATURES:append = " ptest" | ||
922 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
923 | |||
924 | Once your build is complete, the ptest files are installed into the | ||
925 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
926 | is the name of the package. | ||
927 | |||
928 | Running ptest | ||
929 | ------------- | ||
930 | |||
931 | The ``ptest-runner`` package installs a shell script that loops through | ||
932 | all installed ptest test suites and runs them in sequence. Consequently, | ||
933 | you might want to add this package to your image. | ||
934 | |||
935 | Getting Your Package Ready | ||
936 | -------------------------- | ||
937 | |||
938 | In order to enable a recipe to run installed ptests on target hardware, | ||
939 | you need to prepare the recipes that build the packages you want to | ||
940 | test. Here is what you have to do for each recipe: | ||
941 | |||
942 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
943 | Include the following line in each recipe:: | ||
944 | |||
945 | inherit ptest | ||
946 | |||
947 | - *Create run-ptest:* This script starts your test. Locate the | ||
948 | script where you will refer to it using | ||
949 | :term:`SRC_URI`. Here is an | ||
950 | example that starts a test for ``dbus``:: | ||
951 | |||
952 | #!/bin/sh | ||
953 | cd test | ||
954 | make -k runtest-TESTS | ||
955 | |||
956 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
957 | dependencies that normally do not exist for the package (such as | ||
958 | requiring "make" to run the test suite), use the | ||
959 | :term:`DEPENDS` and | ||
960 | :term:`RDEPENDS` variables in | ||
961 | your recipe in order for the package to meet the dependencies. Here | ||
962 | is an example where the package has a runtime dependency on "make":: | ||
963 | |||
964 | RDEPENDS:${PN}-ptest += "make" | ||
965 | |||
966 | - *Add a function to build the test suite:* Not many packages support | ||
967 | cross-compilation of their test suites. Consequently, you usually | ||
968 | need to add a cross-compilation function to the package. | ||
969 | |||
970 | Many packages based on Automake compile and run the test suite by | ||
971 | using a single command such as ``make check``. However, the host | ||
972 | ``make check`` builds and runs on the same computer, while | ||
973 | cross-compiling requires that the package is built on the host but | ||
974 | executed for the target architecture (though often, as in the case | ||
975 | for ptest, the execution occurs on the host). The built version of | ||
976 | Automake that ships with the Yocto Project includes a patch that | ||
977 | separates building and execution. Consequently, packages that use the | ||
978 | unaltered, patched version of ``make check`` automatically | ||
979 | cross-compiles. | ||
980 | |||
981 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
982 | build the test suite. Add a function similar to the following to your | ||
983 | recipe:: | ||
984 | |||
985 | do_compile_ptest() { | ||
986 | oe_runmake buildtest-TESTS | ||
987 | } | ||
988 | |||
989 | - *Ensure special configurations are set:* If the package requires | ||
990 | special configurations prior to compiling the test code, you must | ||
991 | insert a ``do_configure_ptest`` function into the recipe. | ||
992 | |||
993 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
994 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
995 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
996 | to create a ``do_install_ptest`` function and make sure it gets | ||
997 | called after the "make install-ptest" completes. | ||
998 | 892 | ||
999 | Creating Node Package Manager (NPM) Packages | 893 | Creating Node Package Manager (NPM) Packages |
1000 | ============================================ | 894 | ============================================ |
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst index c50786a36d..ecd4f2c1af 100644 --- a/documentation/migration-guides/migration-1.6.rst +++ b/documentation/migration-guides/migration-1.6.rst | |||
@@ -220,7 +220,7 @@ Package Test (ptest) | |||
220 | 220 | ||
221 | Package Tests (ptest) are built but not installed by default. For | 221 | Package Tests (ptest) are built but not installed by default. For |
222 | information on using Package Tests, see the | 222 | information on using Package Tests, see the |
223 | ":ref:`dev-manual/packages:testing packages with ptest`" | 223 | ":ref:`test-manual/ptest:testing packages with ptest`" |
224 | section in the Yocto Project Development Tasks Manual. For information on the | 224 | section in the Yocto Project Development Tasks Manual. For information on the |
225 | ``ptest`` class, see the ":ref:`ref-classes-ptest`" section. | 225 | ``ptest`` class, see the ":ref:`ref-classes-ptest`" section. |
226 | 226 | ||
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 52a77a11be..b609c3af66 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst | |||
@@ -2478,7 +2478,7 @@ runtime tests for recipes that build software that provides these tests. | |||
2478 | This class is intended to be inherited by individual recipes. However, | 2478 | This class is intended to be inherited by individual recipes. However, |
2479 | the class' functionality is largely disabled unless "ptest" appears in | 2479 | the class' functionality is largely disabled unless "ptest" appears in |
2480 | :term:`DISTRO_FEATURES`. See the | 2480 | :term:`DISTRO_FEATURES`. See the |
2481 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2481 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2482 | section in the Yocto Project Development Tasks Manual for more information | 2482 | section in the Yocto Project Development Tasks Manual for more information |
2483 | on ptest. | 2483 | on ptest. |
2484 | 2484 | ||
@@ -2491,7 +2491,7 @@ Enables package tests (ptests) specifically for GNOME packages, which | |||
2491 | have tests intended to be executed with ``gnome-desktop-testing``. | 2491 | have tests intended to be executed with ``gnome-desktop-testing``. |
2492 | 2492 | ||
2493 | For information on setting up and running ptests, see the | 2493 | For information on setting up and running ptests, see the |
2494 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2494 | ":ref:`test-manual/ptest:testing packages with ptest`" |
2495 | section in the Yocto Project Development Tasks Manual. | 2495 | section in the Yocto Project Development Tasks Manual. |
2496 | 2496 | ||
2497 | .. _ref-classes-python3-dir: | 2497 | .. _ref-classes-python3-dir: |
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst index bd393abd07..4d5f900ab8 100644 --- a/documentation/ref-manual/features.rst +++ b/documentation/ref-manual/features.rst | |||
@@ -157,7 +157,7 @@ metadata: | |||
157 | 157 | ||
158 | - *ptest:* Enables building the package tests where supported by | 158 | - *ptest:* Enables building the package tests where supported by |
159 | individual recipes. For more information on package tests, see the | 159 | individual recipes. For more information on package tests, see the |
160 | ":ref:`dev-manual/packages:testing packages with ptest`" section | 160 | ":ref:`test-manual/ptest:testing packages with ptest`" section |
161 | in the Yocto Project Development Tasks Manual. | 161 | in the Yocto Project Development Tasks Manual. |
162 | 162 | ||
163 | - *smbfs:* Include SMB networks client support (for mounting | 163 | - *smbfs:* Include SMB networks client support (for mounting |
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst index 1e5fe21b02..92d2a42afb 100644 --- a/documentation/ref-manual/qa-checks.rst +++ b/documentation/ref-manual/qa-checks.rst | |||
@@ -789,7 +789,6 @@ Errors and Warnings | |||
789 | use a relative path rather than an absolute one, or to pick up the path from | 789 | use a relative path rather than an absolute one, or to pick up the path from |
790 | runtime configuration or environment variables. | 790 | runtime configuration or environment variables. |
791 | 791 | ||
792 | |||
793 | Configuring and Disabling QA Checks | 792 | Configuring and Disabling QA Checks |
794 | =================================== | 793 | =================================== |
795 | 794 | ||
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 5733dd70ff..89f79812e8 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst | |||
@@ -175,7 +175,7 @@ consists of the following pieces: | |||
175 | operation and functions. However, the test can also use the IP | 175 | operation and functions. However, the test can also use the IP |
176 | address of a machine to test. | 176 | address of a machine to test. |
177 | 177 | ||
178 | - :ref:`ptest <dev-manual/packages:testing packages with ptest>`: | 178 | - :ref:`ptest <test-manual/ptest:testing packages with ptest>`: |
179 | Runs tests against packages produced during the build for a given | 179 | Runs tests against packages produced during the build for a given |
180 | piece of software. The test allows the packages to be run within a | 180 | piece of software. The test allows the packages to be run within a |
181 | target image. | 181 | target image. |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 7f0d09903b..8ab7391f98 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
@@ -6434,7 +6434,7 @@ system and gives an overview of their function and contents. | |||
6434 | 6434 | ||
6435 | :term:`PTEST_ENABLED` | 6435 | :term:`PTEST_ENABLED` |
6436 | Specifies whether or not :ref:`Package | 6436 | Specifies whether or not :ref:`Package |
6437 | Test <dev-manual/packages:testing packages with ptest>` (ptest) | 6437 | Test <test-manual/ptest:testing packages with ptest>` (ptest) |
6438 | functionality is enabled when building a recipe. You should not set | 6438 | functionality is enabled when building a recipe. You should not set |
6439 | this variable directly. Enabling and disabling building Package Tests | 6439 | this variable directly. Enabling and disabling building Package Tests |
6440 | at build time should be done by adding "ptest" to (or removing it | 6440 | at build time should be done by adding "ptest" to (or removing it |
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea..ad71f37910 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst | |||
@@ -12,6 +12,7 @@ Yocto Project Test Environment Manual | |||
12 | 12 | ||
13 | intro | 13 | intro |
14 | test-process | 14 | test-process |
15 | ptest | ||
15 | understand-autobuilder | 16 | understand-autobuilder |
16 | reproducible-builds | 17 | reproducible-builds |
17 | yocto-project-compatible | 18 | yocto-project-compatible |
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index bb267d2d88..ff9b6d3794 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst | |||
@@ -141,7 +141,7 @@ the following types of tests: | |||
141 | - *Package Testing:* A Package Test (ptest) runs tests against packages | 141 | - *Package Testing:* A Package Test (ptest) runs tests against packages |
142 | built by the OpenEmbedded build system on the target machine. See the | 142 | built by the OpenEmbedded build system on the target machine. See the |
143 | :ref:`Testing Packages With | 143 | :ref:`Testing Packages With |
144 | ptest <dev-manual/packages:Testing Packages With ptest>` section | 144 | ptest <test-manual/ptest:Testing Packages With ptest>` section |
145 | in the Yocto Project Development Tasks Manual and the | 145 | in the Yocto Project Development Tasks Manual and the |
146 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more | 146 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more |
147 | information on Ptest. | 147 | information on Ptest. |
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..dea1bad23b --- /dev/null +++ b/documentation/test-manual/ptest.rst | |||
@@ -0,0 +1,114 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | *************************** | ||
4 | Testing Packages With ptest | ||
5 | *************************** | ||
6 | |||
7 | A Package Test (ptest) runs tests against packages built by the | ||
8 | OpenEmbedded build system on the target machine. A ptest contains at | ||
9 | least two items: the actual test, and a shell script (``run-ptest``) | ||
10 | that starts the test. The shell script that starts the test must not | ||
11 | contain the actual test --- the script only starts the test. On the other | ||
12 | hand, the test can be anything from a simple shell script that runs a | ||
13 | binary and checks the output to an elaborate system of test binaries and | ||
14 | data files. | ||
15 | |||
16 | The test generates output in the format used by Automake:: | ||
17 | |||
18 | result: testname | ||
19 | |||
20 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
21 | the testname can be any identifying string. | ||
22 | |||
23 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
24 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
25 | |||
26 | .. note:: | ||
27 | |||
28 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
29 | class. | ||
30 | |||
31 | Adding ptest to Your Build | ||
32 | ========================== | ||
33 | |||
34 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
35 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
36 | is found in the :term:`Build Directory`:: | ||
37 | |||
38 | DISTRO_FEATURES:append = " ptest" | ||
39 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
40 | |||
41 | Once your build is complete, the ptest files are installed into the | ||
42 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
43 | is the name of the package. | ||
44 | |||
45 | Running ptest | ||
46 | ============= | ||
47 | |||
48 | The ``ptest-runner`` package installs a shell script that loops through | ||
49 | all installed ptest test suites and runs them in sequence. Consequently, | ||
50 | you might want to add this package to your image. | ||
51 | |||
52 | Getting Your Package Ready | ||
53 | ========================== | ||
54 | |||
55 | In order to enable a recipe to run installed ptests on target hardware, | ||
56 | you need to prepare the recipes that build the packages you want to | ||
57 | test. Here is what you have to do for each recipe: | ||
58 | |||
59 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
60 | Include the following line in each recipe:: | ||
61 | |||
62 | inherit ptest | ||
63 | |||
64 | - *Create run-ptest:* This script starts your test. Locate the | ||
65 | script where you will refer to it using | ||
66 | :term:`SRC_URI`. Here is an | ||
67 | example that starts a test for ``dbus``:: | ||
68 | |||
69 | #!/bin/sh | ||
70 | cd test | ||
71 | make -k runtest-TESTS | ||
72 | |||
73 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
74 | dependencies that normally do not exist for the package (such as | ||
75 | requiring "make" to run the test suite), use the | ||
76 | :term:`DEPENDS` and | ||
77 | :term:`RDEPENDS` variables in | ||
78 | your recipe in order for the package to meet the dependencies. Here | ||
79 | is an example where the package has a runtime dependency on "make":: | ||
80 | |||
81 | RDEPENDS:${PN}-ptest += "make" | ||
82 | |||
83 | - *Add a function to build the test suite:* Not many packages support | ||
84 | cross-compilation of their test suites. Consequently, you usually | ||
85 | need to add a cross-compilation function to the package. | ||
86 | |||
87 | Many packages based on Automake compile and run the test suite by | ||
88 | using a single command such as ``make check``. However, the host | ||
89 | ``make check`` builds and runs on the same computer, while | ||
90 | cross-compiling requires that the package is built on the host but | ||
91 | executed for the target architecture (though often, as in the case | ||
92 | for ptest, the execution occurs on the host). The built version of | ||
93 | Automake that ships with the Yocto Project includes a patch that | ||
94 | separates building and execution. Consequently, packages that use the | ||
95 | unaltered, patched version of ``make check`` automatically | ||
96 | cross-compiles. | ||
97 | |||
98 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
99 | build the test suite. Add a function similar to the following to your | ||
100 | recipe:: | ||
101 | |||
102 | do_compile_ptest() { | ||
103 | oe_runmake buildtest-TESTS | ||
104 | } | ||
105 | |||
106 | - *Ensure special configurations are set:* If the package requires | ||
107 | special configurations prior to compiling the test code, you must | ||
108 | insert a ``do_configure_ptest`` function into the recipe. | ||
109 | |||
110 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
111 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
112 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
113 | to create a ``do_install_ptest`` function and make sure it gets | ||
114 | called after the "make install-ptest" completes. | ||