summaryrefslogtreecommitdiffstats
path: root/documentation/test-manual/intro.rst
diff options
context:
space:
mode:
authorMichael Opdenacker <michael.opdenacker@bootlin.com>2023-12-06 16:36:00 +0100
committerRichard Purdie <richard.purdie@linuxfoundation.org>2023-12-18 13:18:55 +0000
commit84b242ef8bc103e49d83dcc25d968ab59524e1f3 (patch)
tree0fd504622c840757a2995a158ebc422605a8d5e9 /documentation/test-manual/intro.rst
parentc24e3b9f676ea68c100154f1aa865430ed2c99f1 (diff)
downloadpoky-84b242ef8bc103e49d83dcc25d968ab59524e1f3.tar.gz
test-manual: text and formatting fixes
(From yocto-docs rev: 7c4f616f773bb9071b395e977b2ca9f8ac8f7e2a) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation/test-manual/intro.rst')
-rw-r--r--documentation/test-manual/intro.rst44
1 files changed, 22 insertions, 22 deletions
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst
index aaf64ae017..f27210642c 100644
--- a/documentation/test-manual/intro.rst
+++ b/documentation/test-manual/intro.rst
@@ -14,7 +14,7 @@ release works as intended. All the project's testing infrastructure and
14processes are publicly visible and available so that the community can 14processes are publicly visible and available so that the community can
15see what testing is being performed, how it's being done and the current 15see what testing is being performed, how it's being done and the current
16status of the tests and the project at any given time. It is intended 16status of the tests and the project at any given time. It is intended
17that Other organizations can leverage off the process and testing 17that other organizations can leverage off the process and testing
18environment used by the Yocto Project to create their own automated, 18environment used by the Yocto Project to create their own automated,
19production test environment, building upon the foundations from the 19production test environment, building upon the foundations from the
20project core. 20project core.
@@ -93,8 +93,8 @@ the following types of tests:
93- *Build Testing:* Tests whether specific configurations build by 93- *Build Testing:* Tests whether specific configurations build by
94 varying :term:`MACHINE`, 94 varying :term:`MACHINE`,
95 :term:`DISTRO`, other configuration 95 :term:`DISTRO`, other configuration
96 options, and the specific target images being built (or world). Used 96 options, and the specific target images being built (or ``world``). This is
97 to trigger builds of all the different test configurations on the 97 used to trigger builds of all the different test configurations on the
98 Autobuilder. Builds usually cover many different targets for 98 Autobuilder. Builds usually cover many different targets for
99 different architectures, machines, and distributions, as well as 99 different architectures, machines, and distributions, as well as
100 different configurations, such as different init systems. The 100 different configurations, such as different init systems. The
@@ -120,7 +120,7 @@ the following types of tests:
120 120
121 $ bitbake image -c testsdkext 121 $ bitbake image -c testsdkext
122 122
123 The tests utilize the :ref:`ref-classes-testsdk` class and the 123 The tests use the :ref:`ref-classes-testsdk` class and the
124 ``do_testsdkext`` task. 124 ``do_testsdkext`` task.
125 125
126- *Feature Testing:* Various scenario-based tests are run through the 126- *Feature Testing:* Various scenario-based tests are run through the
@@ -131,7 +131,7 @@ the following types of tests:
131 131
132 $ bitbake image -c testimage 132 $ bitbake image -c testimage
133 133
134 The tests utilize the :ref:`ref-classes-testimage` 134 The tests use the :ref:`ref-classes-testimage`
135 class and the :ref:`ref-tasks-testimage` task. 135 class and the :ref:`ref-tasks-testimage` task.
136 136
137- *Layer Testing:* The Autobuilder has the possibility to test whether 137- *Layer Testing:* The Autobuilder has the possibility to test whether
@@ -151,7 +151,7 @@ the following types of tests:
151 151
152 $ bitbake image -c testsdk 152 $ bitbake image -c testsdk
153 153
154 The tests utilize the :ref:`ref-classes-testsdk` class and 154 The tests use the :ref:`ref-classes-testsdk` class and
155 the ``do_testsdk`` task. 155 the ``do_testsdk`` task.
156 156
157- *Unit Testing:* Unit tests on various components of the system run 157- *Unit Testing:* Unit tests on various components of the system run
@@ -190,7 +190,7 @@ Tests map into the codebase as follows:
190 $ bitbake-selftest -v 190 $ bitbake-selftest -v
191 191
192 To skip tests that access the Internet, use the ``BB_SKIP_NETTESTS`` 192 To skip tests that access the Internet, use the ``BB_SKIP_NETTESTS``
193 variable when running "bitbake-selftest" as follows:: 193 variable when running ``bitbake-selftest`` as follows::
194 194
195 $ BB_SKIP_NETTESTS=yes bitbake-selftest 195 $ BB_SKIP_NETTESTS=yes bitbake-selftest
196 196
@@ -215,7 +215,7 @@ Tests map into the codebase as follows:
215 - These tests use OE to test the workflows, which include testing 215 - These tests use OE to test the workflows, which include testing
216 specific features, behaviors of tasks, and API unit tests. 216 specific features, behaviors of tasks, and API unit tests.
217 217
218 - The tests can take advantage of parallelism through the "-j" 218 - The tests can take advantage of parallelism through the ``-j``
219 option, which can specify a number of threads to spread the tests 219 option, which can specify a number of threads to spread the tests
220 across. Note that all tests from a given class of tests will run 220 across. Note that all tests from a given class of tests will run
221 in the same thread. To parallelize large numbers of tests you can 221 in the same thread. To parallelize large numbers of tests you can
@@ -231,18 +231,18 @@ Tests map into the codebase as follows:
231 $ oe-selftest -a 231 $ oe-selftest -a
232 232
233 - To run a specific test, use the following command form where 233 - To run a specific test, use the following command form where
234 testname is the name of the specific test:: 234 ``testname`` is the name of the specific test::
235 235
236 $ oe-selftest -r <testname> 236 $ oe-selftest -r <testname>
237 237
238 For example, the following command would run the tinfoil 238 For example, the following command would run the ``tinfoil``
239 getVar API test:: 239 ``getVar`` API test::
240 240
241 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar 241 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
242 242
243 It is also possible to run a set 243 It is also possible to run a set
244 of tests. For example the following command will run all of the 244 of tests. For example the following command will run all of the
245 tinfoil tests:: 245 ``tinfoil`` tests::
246 246
247 $ oe-selftest -r tinfoil 247 $ oe-selftest -r tinfoil
248 248
@@ -391,14 +391,14 @@ so tests within a given test class should always run in the same build,
391while tests in different classes or modules may be split into different 391while tests in different classes or modules may be split into different
392builds. There is no data store available for these tests since the tests 392builds. There is no data store available for these tests since the tests
393launch the ``bitbake`` command and exist outside of its context. As a 393launch the ``bitbake`` command and exist outside of its context. As a
394result, common bitbake library functions (bb.\*) are also unavailable. 394result, common BitBake library functions (``bb.\*``) are also unavailable.
395 395
396``testimage`` 396``testimage``
397------------- 397-------------
398 398
399These tests are run once an image is up and running, either on target 399These tests are run once an image is up and running, either on target
400hardware or under QEMU. As a result, they are assumed to be running in a 400hardware or under QEMU. As a result, they are assumed to be running in a
401target image environment, as opposed to a host build environment. A 401target image environment, as opposed to in a host build environment. A
402simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains 402simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains
403the following:: 403the following::
404 404
@@ -413,19 +413,19 @@ the following::
413 413
414In this example, the ``OERuntimeTestCase`` class wraps 414In this example, the ``OERuntimeTestCase`` class wraps
415``unittest.TestCase``. Within the test, ``self.target`` represents the 415``unittest.TestCase``. Within the test, ``self.target`` represents the
416target system, where commands can be run on it using the ``run()`` 416target system, where commands can be run using the ``run()``
417method. 417method.
418 418
419To ensure certain test or package dependencies are met, you can use the 419To ensure certain tests or package dependencies are met, you can use the
420``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test 420``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test
421in this example would only make sense if python3-core is installed in 421in this example would only make sense if ``python3-core`` is installed in
422the image. 422the image.
423 423
424``testsdk_ext`` 424``testsdk_ext``
425--------------- 425---------------
426 426
427These tests are run against built extensible SDKs (eSDKs). The tests can 427These tests are run against built extensible SDKs (eSDKs). The tests can
428assume that the eSDK environment has already been setup. An example from 428assume that the eSDK environment has already been set up. An example from
429``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following:: 429``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following::
430 430
431 class DevtoolTest(OESDKExtTestCase): 431 class DevtoolTest(OESDKExtTestCase):
@@ -472,9 +472,9 @@ following::
472 output = self._run(cmd) 472 output = self._run(cmd)
473 self.assertEqual(output, "Hello, world\n") 473 self.assertEqual(output, "Hello, world\n")
474 474
475In this example, if nativesdk-python3-core has been installed into the SDK, the code runs 475In this example, if ``nativesdk-python3-core`` has been installed into the SDK,
476the python3 interpreter with a basic command to check it is working 476the code runs the ``python3`` interpreter with a basic command to check it is
477correctly. The test would only run if Python3 is installed in the SDK. 477working correctly. The test would only run if Python3 is installed in the SDK.
478 478
479``oe-build-perf-test`` 479``oe-build-perf-test``
480---------------------- 480----------------------
@@ -520,7 +520,7 @@ an isolated directory.
520 520
521This can delete files from :term:`SSTATE_DIR` which would potentially break 521This can delete files from :term:`SSTATE_DIR` which would potentially break
522other builds running in parallel. If this is required, :term:`SSTATE_DIR` must 522other builds running in parallel. If this is required, :term:`SSTATE_DIR` must
523be set to an isolated directory. Alternatively, you can use the "-f" 523be set to an isolated directory. Alternatively, you can use the ``-f``
524option with the ``bitbake`` command to "taint" tasks by changing the 524option with the ``bitbake`` command to "taint" tasks by changing the
525sstate checksums to ensure sstate cache items will not be reused. 525sstate checksums to ensure sstate cache items will not be reused.
526 526