diff options
| author | Antonin Godard <antonin.godard@bootlin.com> | 2024-12-26 16:20:16 +0100 |
|---|---|---|
| committer | Steve Sakoman <steve@sakoman.com> | 2025-01-09 08:41:04 -0800 |
| commit | 2497f29afd1ff66964b84dc79a7cd1126bbc94fa (patch) | |
| tree | 494d2d1a6b358c3796d870e1e060a949addf37b9 | |
| parent | 1418534d2215890723c281f852b6a6fd08acdbcf (diff) | |
| download | poky-2497f29afd1ff66964b84dc79a7cd1126bbc94fa.tar.gz | |
ref-manual/packages: move ptest section to the test-manual
[ YOCTO #15106 ]
It makes more sense to document ptests in the test-manual. Since ptests
are still related to packages, keep a link to ptests from packages.rst
to the test-manual.
Reported-by: Yoann Congal <yoann.congal@smile.fr>
(From yocto-docs rev: 8b6ada020d595d86c7bbe78a27b7a6301715b039)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
| -rw-r--r-- | documentation/dev-manual/packages.rst | 112 | ||||
| -rw-r--r-- | documentation/migration-guides/migration-1.6.rst | 2 | ||||
| -rw-r--r-- | documentation/ref-manual/classes.rst | 4 | ||||
| -rw-r--r-- | documentation/ref-manual/features.rst | 2 | ||||
| -rw-r--r-- | documentation/ref-manual/qa-checks.rst | 1 | ||||
| -rw-r--r-- | documentation/ref-manual/release-process.rst | 2 | ||||
| -rw-r--r-- | documentation/ref-manual/variables.rst | 2 | ||||
| -rw-r--r-- | documentation/test-manual/index.rst | 1 | ||||
| -rw-r--r-- | documentation/test-manual/intro.rst | 2 | ||||
| -rw-r--r-- | documentation/test-manual/ptest.rst | 114 |
10 files changed, 125 insertions, 117 deletions
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index e5028fffdc..0c948cdaef 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
| @@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
| 16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
| 17 | 17 | ||
| 18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
| 19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
| 20 | 20 | ||
| 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
| 22 | 22 | ||
| @@ -887,114 +887,8 @@ related to signed package feeds are available: | |||
| 887 | Testing Packages With ptest | 887 | Testing Packages With ptest |
| 888 | =========================== | 888 | =========================== |
| 889 | 889 | ||
| 890 | A Package Test (ptest) runs tests against packages built by the | 890 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
| 891 | OpenEmbedded build system on the target machine. A ptest contains at | 891 | Yocto Project Test Environment Manual. |
| 892 | least two items: the actual test, and a shell script (``run-ptest``) | ||
| 893 | that starts the test. The shell script that starts the test must not | ||
| 894 | contain the actual test --- the script only starts the test. On the other | ||
| 895 | hand, the test can be anything from a simple shell script that runs a | ||
| 896 | binary and checks the output to an elaborate system of test binaries and | ||
| 897 | data files. | ||
| 898 | |||
| 899 | The test generates output in the format used by Automake:: | ||
| 900 | |||
| 901 | result: testname | ||
| 902 | |||
| 903 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
| 904 | the testname can be any identifying string. | ||
| 905 | |||
| 906 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
| 907 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
| 908 | |||
| 909 | .. note:: | ||
| 910 | |||
| 911 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
| 912 | class. | ||
| 913 | |||
| 914 | Adding ptest to Your Build | ||
| 915 | -------------------------- | ||
| 916 | |||
| 917 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
| 918 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
| 919 | is found in the :term:`Build Directory`:: | ||
| 920 | |||
| 921 | DISTRO_FEATURES:append = " ptest" | ||
| 922 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
| 923 | |||
| 924 | Once your build is complete, the ptest files are installed into the | ||
| 925 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
| 926 | is the name of the package. | ||
| 927 | |||
| 928 | Running ptest | ||
| 929 | ------------- | ||
| 930 | |||
| 931 | The ``ptest-runner`` package installs a shell script that loops through | ||
| 932 | all installed ptest test suites and runs them in sequence. Consequently, | ||
| 933 | you might want to add this package to your image. | ||
| 934 | |||
| 935 | Getting Your Package Ready | ||
| 936 | -------------------------- | ||
| 937 | |||
| 938 | In order to enable a recipe to run installed ptests on target hardware, | ||
| 939 | you need to prepare the recipes that build the packages you want to | ||
| 940 | test. Here is what you have to do for each recipe: | ||
| 941 | |||
| 942 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
| 943 | Include the following line in each recipe:: | ||
| 944 | |||
| 945 | inherit ptest | ||
| 946 | |||
| 947 | - *Create run-ptest:* This script starts your test. Locate the | ||
| 948 | script where you will refer to it using | ||
| 949 | :term:`SRC_URI`. Here is an | ||
| 950 | example that starts a test for ``dbus``:: | ||
| 951 | |||
| 952 | #!/bin/sh | ||
| 953 | cd test | ||
| 954 | make -k runtest-TESTS | ||
| 955 | |||
| 956 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
| 957 | dependencies that normally do not exist for the package (such as | ||
| 958 | requiring "make" to run the test suite), use the | ||
| 959 | :term:`DEPENDS` and | ||
| 960 | :term:`RDEPENDS` variables in | ||
| 961 | your recipe in order for the package to meet the dependencies. Here | ||
| 962 | is an example where the package has a runtime dependency on "make":: | ||
| 963 | |||
| 964 | RDEPENDS:${PN}-ptest += "make" | ||
| 965 | |||
| 966 | - *Add a function to build the test suite:* Not many packages support | ||
| 967 | cross-compilation of their test suites. Consequently, you usually | ||
| 968 | need to add a cross-compilation function to the package. | ||
| 969 | |||
| 970 | Many packages based on Automake compile and run the test suite by | ||
| 971 | using a single command such as ``make check``. However, the host | ||
| 972 | ``make check`` builds and runs on the same computer, while | ||
| 973 | cross-compiling requires that the package is built on the host but | ||
| 974 | executed for the target architecture (though often, as in the case | ||
| 975 | for ptest, the execution occurs on the host). The built version of | ||
| 976 | Automake that ships with the Yocto Project includes a patch that | ||
| 977 | separates building and execution. Consequently, packages that use the | ||
| 978 | unaltered, patched version of ``make check`` automatically | ||
| 979 | cross-compiles. | ||
| 980 | |||
| 981 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
| 982 | build the test suite. Add a function similar to the following to your | ||
| 983 | recipe:: | ||
| 984 | |||
| 985 | do_compile_ptest() { | ||
| 986 | oe_runmake buildtest-TESTS | ||
| 987 | } | ||
| 988 | |||
| 989 | - *Ensure special configurations are set:* If the package requires | ||
| 990 | special configurations prior to compiling the test code, you must | ||
| 991 | insert a ``do_configure_ptest`` function into the recipe. | ||
| 992 | |||
| 993 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
| 994 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
| 995 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
| 996 | to create a ``do_install_ptest`` function and make sure it gets | ||
| 997 | called after the "make install-ptest" completes. | ||
| 998 | 892 | ||
| 999 | Creating Node Package Manager (NPM) Packages | 893 | Creating Node Package Manager (NPM) Packages |
| 1000 | ============================================ | 894 | ============================================ |
diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst index c50786a36d..ecd4f2c1af 100644 --- a/documentation/migration-guides/migration-1.6.rst +++ b/documentation/migration-guides/migration-1.6.rst | |||
| @@ -220,7 +220,7 @@ Package Test (ptest) | |||
| 220 | 220 | ||
| 221 | Package Tests (ptest) are built but not installed by default. For | 221 | Package Tests (ptest) are built but not installed by default. For |
| 222 | information on using Package Tests, see the | 222 | information on using Package Tests, see the |
| 223 | ":ref:`dev-manual/packages:testing packages with ptest`" | 223 | ":ref:`test-manual/ptest:testing packages with ptest`" |
| 224 | section in the Yocto Project Development Tasks Manual. For information on the | 224 | section in the Yocto Project Development Tasks Manual. For information on the |
| 225 | ``ptest`` class, see the ":ref:`ref-classes-ptest`" section. | 225 | ``ptest`` class, see the ":ref:`ref-classes-ptest`" section. |
| 226 | 226 | ||
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 52a77a11be..b609c3af66 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst | |||
| @@ -2478,7 +2478,7 @@ runtime tests for recipes that build software that provides these tests. | |||
| 2478 | This class is intended to be inherited by individual recipes. However, | 2478 | This class is intended to be inherited by individual recipes. However, |
| 2479 | the class' functionality is largely disabled unless "ptest" appears in | 2479 | the class' functionality is largely disabled unless "ptest" appears in |
| 2480 | :term:`DISTRO_FEATURES`. See the | 2480 | :term:`DISTRO_FEATURES`. See the |
| 2481 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2481 | ":ref:`test-manual/ptest:testing packages with ptest`" |
| 2482 | section in the Yocto Project Development Tasks Manual for more information | 2482 | section in the Yocto Project Development Tasks Manual for more information |
| 2483 | on ptest. | 2483 | on ptest. |
| 2484 | 2484 | ||
| @@ -2491,7 +2491,7 @@ Enables package tests (ptests) specifically for GNOME packages, which | |||
| 2491 | have tests intended to be executed with ``gnome-desktop-testing``. | 2491 | have tests intended to be executed with ``gnome-desktop-testing``. |
| 2492 | 2492 | ||
| 2493 | For information on setting up and running ptests, see the | 2493 | For information on setting up and running ptests, see the |
| 2494 | ":ref:`dev-manual/packages:testing packages with ptest`" | 2494 | ":ref:`test-manual/ptest:testing packages with ptest`" |
| 2495 | section in the Yocto Project Development Tasks Manual. | 2495 | section in the Yocto Project Development Tasks Manual. |
| 2496 | 2496 | ||
| 2497 | .. _ref-classes-python3-dir: | 2497 | .. _ref-classes-python3-dir: |
diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst index bd393abd07..4d5f900ab8 100644 --- a/documentation/ref-manual/features.rst +++ b/documentation/ref-manual/features.rst | |||
| @@ -157,7 +157,7 @@ metadata: | |||
| 157 | 157 | ||
| 158 | - *ptest:* Enables building the package tests where supported by | 158 | - *ptest:* Enables building the package tests where supported by |
| 159 | individual recipes. For more information on package tests, see the | 159 | individual recipes. For more information on package tests, see the |
| 160 | ":ref:`dev-manual/packages:testing packages with ptest`" section | 160 | ":ref:`test-manual/ptest:testing packages with ptest`" section |
| 161 | in the Yocto Project Development Tasks Manual. | 161 | in the Yocto Project Development Tasks Manual. |
| 162 | 162 | ||
| 163 | - *smbfs:* Include SMB networks client support (for mounting | 163 | - *smbfs:* Include SMB networks client support (for mounting |
diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst index 1e5fe21b02..92d2a42afb 100644 --- a/documentation/ref-manual/qa-checks.rst +++ b/documentation/ref-manual/qa-checks.rst | |||
| @@ -789,7 +789,6 @@ Errors and Warnings | |||
| 789 | use a relative path rather than an absolute one, or to pick up the path from | 789 | use a relative path rather than an absolute one, or to pick up the path from |
| 790 | runtime configuration or environment variables. | 790 | runtime configuration or environment variables. |
| 791 | 791 | ||
| 792 | |||
| 793 | Configuring and Disabling QA Checks | 792 | Configuring and Disabling QA Checks |
| 794 | =================================== | 793 | =================================== |
| 795 | 794 | ||
diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 5733dd70ff..89f79812e8 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst | |||
| @@ -175,7 +175,7 @@ consists of the following pieces: | |||
| 175 | operation and functions. However, the test can also use the IP | 175 | operation and functions. However, the test can also use the IP |
| 176 | address of a machine to test. | 176 | address of a machine to test. |
| 177 | 177 | ||
| 178 | - :ref:`ptest <dev-manual/packages:testing packages with ptest>`: | 178 | - :ref:`ptest <test-manual/ptest:testing packages with ptest>`: |
| 179 | Runs tests against packages produced during the build for a given | 179 | Runs tests against packages produced during the build for a given |
| 180 | piece of software. The test allows the packages to be run within a | 180 | piece of software. The test allows the packages to be run within a |
| 181 | target image. | 181 | target image. |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 7f0d09903b..8ab7391f98 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
| @@ -6434,7 +6434,7 @@ system and gives an overview of their function and contents. | |||
| 6434 | 6434 | ||
| 6435 | :term:`PTEST_ENABLED` | 6435 | :term:`PTEST_ENABLED` |
| 6436 | Specifies whether or not :ref:`Package | 6436 | Specifies whether or not :ref:`Package |
| 6437 | Test <dev-manual/packages:testing packages with ptest>` (ptest) | 6437 | Test <test-manual/ptest:testing packages with ptest>` (ptest) |
| 6438 | functionality is enabled when building a recipe. You should not set | 6438 | functionality is enabled when building a recipe. You should not set |
| 6439 | this variable directly. Enabling and disabling building Package Tests | 6439 | this variable directly. Enabling and disabling building Package Tests |
| 6440 | at build time should be done by adding "ptest" to (or removing it | 6440 | at build time should be done by adding "ptest" to (or removing it |
diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea..ad71f37910 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst | |||
| @@ -12,6 +12,7 @@ Yocto Project Test Environment Manual | |||
| 12 | 12 | ||
| 13 | intro | 13 | intro |
| 14 | test-process | 14 | test-process |
| 15 | ptest | ||
| 15 | understand-autobuilder | 16 | understand-autobuilder |
| 16 | reproducible-builds | 17 | reproducible-builds |
| 17 | yocto-project-compatible | 18 | yocto-project-compatible |
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index bb267d2d88..ff9b6d3794 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst | |||
| @@ -141,7 +141,7 @@ the following types of tests: | |||
| 141 | - *Package Testing:* A Package Test (ptest) runs tests against packages | 141 | - *Package Testing:* A Package Test (ptest) runs tests against packages |
| 142 | built by the OpenEmbedded build system on the target machine. See the | 142 | built by the OpenEmbedded build system on the target machine. See the |
| 143 | :ref:`Testing Packages With | 143 | :ref:`Testing Packages With |
| 144 | ptest <dev-manual/packages:Testing Packages With ptest>` section | 144 | ptest <test-manual/ptest:Testing Packages With ptest>` section |
| 145 | in the Yocto Project Development Tasks Manual and the | 145 | in the Yocto Project Development Tasks Manual and the |
| 146 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more | 146 | ":yocto_wiki:`Ptest </Ptest>`" Wiki page for more |
| 147 | information on Ptest. | 147 | information on Ptest. |
diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..dea1bad23b --- /dev/null +++ b/documentation/test-manual/ptest.rst | |||
| @@ -0,0 +1,114 @@ | |||
| 1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
| 2 | |||
| 3 | *************************** | ||
| 4 | Testing Packages With ptest | ||
| 5 | *************************** | ||
| 6 | |||
| 7 | A Package Test (ptest) runs tests against packages built by the | ||
| 8 | OpenEmbedded build system on the target machine. A ptest contains at | ||
| 9 | least two items: the actual test, and a shell script (``run-ptest``) | ||
| 10 | that starts the test. The shell script that starts the test must not | ||
| 11 | contain the actual test --- the script only starts the test. On the other | ||
| 12 | hand, the test can be anything from a simple shell script that runs a | ||
| 13 | binary and checks the output to an elaborate system of test binaries and | ||
| 14 | data files. | ||
| 15 | |||
| 16 | The test generates output in the format used by Automake:: | ||
| 17 | |||
| 18 | result: testname | ||
| 19 | |||
| 20 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
| 21 | the testname can be any identifying string. | ||
| 22 | |||
| 23 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
| 24 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
| 25 | |||
| 26 | .. note:: | ||
| 27 | |||
| 28 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
| 29 | class. | ||
| 30 | |||
| 31 | Adding ptest to Your Build | ||
| 32 | ========================== | ||
| 33 | |||
| 34 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
| 35 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
| 36 | is found in the :term:`Build Directory`:: | ||
| 37 | |||
| 38 | DISTRO_FEATURES:append = " ptest" | ||
| 39 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
| 40 | |||
| 41 | Once your build is complete, the ptest files are installed into the | ||
| 42 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
| 43 | is the name of the package. | ||
| 44 | |||
| 45 | Running ptest | ||
| 46 | ============= | ||
| 47 | |||
| 48 | The ``ptest-runner`` package installs a shell script that loops through | ||
| 49 | all installed ptest test suites and runs them in sequence. Consequently, | ||
| 50 | you might want to add this package to your image. | ||
| 51 | |||
| 52 | Getting Your Package Ready | ||
| 53 | ========================== | ||
| 54 | |||
| 55 | In order to enable a recipe to run installed ptests on target hardware, | ||
| 56 | you need to prepare the recipes that build the packages you want to | ||
| 57 | test. Here is what you have to do for each recipe: | ||
| 58 | |||
| 59 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
| 60 | Include the following line in each recipe:: | ||
| 61 | |||
| 62 | inherit ptest | ||
| 63 | |||
| 64 | - *Create run-ptest:* This script starts your test. Locate the | ||
| 65 | script where you will refer to it using | ||
| 66 | :term:`SRC_URI`. Here is an | ||
| 67 | example that starts a test for ``dbus``:: | ||
| 68 | |||
| 69 | #!/bin/sh | ||
| 70 | cd test | ||
| 71 | make -k runtest-TESTS | ||
| 72 | |||
| 73 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
| 74 | dependencies that normally do not exist for the package (such as | ||
| 75 | requiring "make" to run the test suite), use the | ||
| 76 | :term:`DEPENDS` and | ||
| 77 | :term:`RDEPENDS` variables in | ||
| 78 | your recipe in order for the package to meet the dependencies. Here | ||
| 79 | is an example where the package has a runtime dependency on "make":: | ||
| 80 | |||
| 81 | RDEPENDS:${PN}-ptest += "make" | ||
| 82 | |||
| 83 | - *Add a function to build the test suite:* Not many packages support | ||
| 84 | cross-compilation of their test suites. Consequently, you usually | ||
| 85 | need to add a cross-compilation function to the package. | ||
| 86 | |||
| 87 | Many packages based on Automake compile and run the test suite by | ||
| 88 | using a single command such as ``make check``. However, the host | ||
| 89 | ``make check`` builds and runs on the same computer, while | ||
| 90 | cross-compiling requires that the package is built on the host but | ||
| 91 | executed for the target architecture (though often, as in the case | ||
| 92 | for ptest, the execution occurs on the host). The built version of | ||
| 93 | Automake that ships with the Yocto Project includes a patch that | ||
| 94 | separates building and execution. Consequently, packages that use the | ||
| 95 | unaltered, patched version of ``make check`` automatically | ||
| 96 | cross-compiles. | ||
| 97 | |||
| 98 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
| 99 | build the test suite. Add a function similar to the following to your | ||
| 100 | recipe:: | ||
| 101 | |||
| 102 | do_compile_ptest() { | ||
| 103 | oe_runmake buildtest-TESTS | ||
| 104 | } | ||
| 105 | |||
| 106 | - *Ensure special configurations are set:* If the package requires | ||
| 107 | special configurations prior to compiling the test code, you must | ||
| 108 | insert a ``do_configure_ptest`` function into the recipe. | ||
| 109 | |||
| 110 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
| 111 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
| 112 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
| 113 | to create a ``do_install_ptest`` function and make sure it gets | ||
| 114 | called after the "make install-ptest" completes. | ||
