diff options
Diffstat (limited to 'documentation/test-manual/intro.rst')
| -rw-r--r-- | documentation/test-manual/intro.rst | 44 |
1 files changed, 22 insertions, 22 deletions
diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index aaf64ae017..f27210642c 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst | |||
| @@ -14,7 +14,7 @@ release works as intended. All the project's testing infrastructure and | |||
| 14 | processes are publicly visible and available so that the community can | 14 | processes are publicly visible and available so that the community can |
| 15 | see what testing is being performed, how it's being done and the current | 15 | see what testing is being performed, how it's being done and the current |
| 16 | status of the tests and the project at any given time. It is intended | 16 | status of the tests and the project at any given time. It is intended |
| 17 | that Other organizations can leverage off the process and testing | 17 | that other organizations can leverage off the process and testing |
| 18 | environment used by the Yocto Project to create their own automated, | 18 | environment used by the Yocto Project to create their own automated, |
| 19 | production test environment, building upon the foundations from the | 19 | production test environment, building upon the foundations from the |
| 20 | project core. | 20 | project core. |
| @@ -93,8 +93,8 @@ the following types of tests: | |||
| 93 | - *Build Testing:* Tests whether specific configurations build by | 93 | - *Build Testing:* Tests whether specific configurations build by |
| 94 | varying :term:`MACHINE`, | 94 | varying :term:`MACHINE`, |
| 95 | :term:`DISTRO`, other configuration | 95 | :term:`DISTRO`, other configuration |
| 96 | options, and the specific target images being built (or world). Used | 96 | options, and the specific target images being built (or ``world``). This is |
| 97 | to trigger builds of all the different test configurations on the | 97 | used to trigger builds of all the different test configurations on the |
| 98 | Autobuilder. Builds usually cover many different targets for | 98 | Autobuilder. Builds usually cover many different targets for |
| 99 | different architectures, machines, and distributions, as well as | 99 | different architectures, machines, and distributions, as well as |
| 100 | different configurations, such as different init systems. The | 100 | different configurations, such as different init systems. The |
| @@ -120,7 +120,7 @@ the following types of tests: | |||
| 120 | 120 | ||
| 121 | $ bitbake image -c testsdkext | 121 | $ bitbake image -c testsdkext |
| 122 | 122 | ||
| 123 | The tests utilize the :ref:`ref-classes-testsdk` class and the | 123 | The tests use the :ref:`ref-classes-testsdk` class and the |
| 124 | ``do_testsdkext`` task. | 124 | ``do_testsdkext`` task. |
| 125 | 125 | ||
| 126 | - *Feature Testing:* Various scenario-based tests are run through the | 126 | - *Feature Testing:* Various scenario-based tests are run through the |
| @@ -131,7 +131,7 @@ the following types of tests: | |||
| 131 | 131 | ||
| 132 | $ bitbake image -c testimage | 132 | $ bitbake image -c testimage |
| 133 | 133 | ||
| 134 | The tests utilize the :ref:`ref-classes-testimage` | 134 | The tests use the :ref:`ref-classes-testimage` |
| 135 | class and the :ref:`ref-tasks-testimage` task. | 135 | class and the :ref:`ref-tasks-testimage` task. |
| 136 | 136 | ||
| 137 | - *Layer Testing:* The Autobuilder has the possibility to test whether | 137 | - *Layer Testing:* The Autobuilder has the possibility to test whether |
| @@ -151,7 +151,7 @@ the following types of tests: | |||
| 151 | 151 | ||
| 152 | $ bitbake image -c testsdk | 152 | $ bitbake image -c testsdk |
| 153 | 153 | ||
| 154 | The tests utilize the :ref:`ref-classes-testsdk` class and | 154 | The tests use the :ref:`ref-classes-testsdk` class and |
| 155 | the ``do_testsdk`` task. | 155 | the ``do_testsdk`` task. |
| 156 | 156 | ||
| 157 | - *Unit Testing:* Unit tests on various components of the system run | 157 | - *Unit Testing:* Unit tests on various components of the system run |
| @@ -190,7 +190,7 @@ Tests map into the codebase as follows: | |||
| 190 | $ bitbake-selftest -v | 190 | $ bitbake-selftest -v |
| 191 | 191 | ||
| 192 | To skip tests that access the Internet, use the ``BB_SKIP_NETTESTS`` | 192 | To skip tests that access the Internet, use the ``BB_SKIP_NETTESTS`` |
| 193 | variable when running "bitbake-selftest" as follows:: | 193 | variable when running ``bitbake-selftest`` as follows:: |
| 194 | 194 | ||
| 195 | $ BB_SKIP_NETTESTS=yes bitbake-selftest | 195 | $ BB_SKIP_NETTESTS=yes bitbake-selftest |
| 196 | 196 | ||
| @@ -215,7 +215,7 @@ Tests map into the codebase as follows: | |||
| 215 | - These tests use OE to test the workflows, which include testing | 215 | - These tests use OE to test the workflows, which include testing |
| 216 | specific features, behaviors of tasks, and API unit tests. | 216 | specific features, behaviors of tasks, and API unit tests. |
| 217 | 217 | ||
| 218 | - The tests can take advantage of parallelism through the "-j" | 218 | - The tests can take advantage of parallelism through the ``-j`` |
| 219 | option, which can specify a number of threads to spread the tests | 219 | option, which can specify a number of threads to spread the tests |
| 220 | across. Note that all tests from a given class of tests will run | 220 | across. Note that all tests from a given class of tests will run |
| 221 | in the same thread. To parallelize large numbers of tests you can | 221 | in the same thread. To parallelize large numbers of tests you can |
| @@ -231,18 +231,18 @@ Tests map into the codebase as follows: | |||
| 231 | $ oe-selftest -a | 231 | $ oe-selftest -a |
| 232 | 232 | ||
| 233 | - To run a specific test, use the following command form where | 233 | - To run a specific test, use the following command form where |
| 234 | testname is the name of the specific test:: | 234 | ``testname`` is the name of the specific test:: |
| 235 | 235 | ||
| 236 | $ oe-selftest -r <testname> | 236 | $ oe-selftest -r <testname> |
| 237 | 237 | ||
| 238 | For example, the following command would run the tinfoil | 238 | For example, the following command would run the ``tinfoil`` |
| 239 | getVar API test:: | 239 | ``getVar`` API test:: |
| 240 | 240 | ||
| 241 | $ oe-selftest -r tinfoil.TinfoilTests.test_getvar | 241 | $ oe-selftest -r tinfoil.TinfoilTests.test_getvar |
| 242 | 242 | ||
| 243 | It is also possible to run a set | 243 | It is also possible to run a set |
| 244 | of tests. For example the following command will run all of the | 244 | of tests. For example the following command will run all of the |
| 245 | tinfoil tests:: | 245 | ``tinfoil`` tests:: |
| 246 | 246 | ||
| 247 | $ oe-selftest -r tinfoil | 247 | $ oe-selftest -r tinfoil |
| 248 | 248 | ||
| @@ -391,14 +391,14 @@ so tests within a given test class should always run in the same build, | |||
| 391 | while tests in different classes or modules may be split into different | 391 | while tests in different classes or modules may be split into different |
| 392 | builds. There is no data store available for these tests since the tests | 392 | builds. There is no data store available for these tests since the tests |
| 393 | launch the ``bitbake`` command and exist outside of its context. As a | 393 | launch the ``bitbake`` command and exist outside of its context. As a |
| 394 | result, common bitbake library functions (bb.\*) are also unavailable. | 394 | result, common BitBake library functions (``bb.\*``) are also unavailable. |
| 395 | 395 | ||
| 396 | ``testimage`` | 396 | ``testimage`` |
| 397 | ------------- | 397 | ------------- |
| 398 | 398 | ||
| 399 | These tests are run once an image is up and running, either on target | 399 | These tests are run once an image is up and running, either on target |
| 400 | hardware or under QEMU. As a result, they are assumed to be running in a | 400 | hardware or under QEMU. As a result, they are assumed to be running in a |
| 401 | target image environment, as opposed to a host build environment. A | 401 | target image environment, as opposed to in a host build environment. A |
| 402 | simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains | 402 | simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains |
| 403 | the following:: | 403 | the following:: |
| 404 | 404 | ||
| @@ -413,19 +413,19 @@ the following:: | |||
| 413 | 413 | ||
| 414 | In this example, the ``OERuntimeTestCase`` class wraps | 414 | In this example, the ``OERuntimeTestCase`` class wraps |
| 415 | ``unittest.TestCase``. Within the test, ``self.target`` represents the | 415 | ``unittest.TestCase``. Within the test, ``self.target`` represents the |
| 416 | target system, where commands can be run on it using the ``run()`` | 416 | target system, where commands can be run using the ``run()`` |
| 417 | method. | 417 | method. |
| 418 | 418 | ||
| 419 | To ensure certain test or package dependencies are met, you can use the | 419 | To ensure certain tests or package dependencies are met, you can use the |
| 420 | ``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test | 420 | ``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test |
| 421 | in this example would only make sense if python3-core is installed in | 421 | in this example would only make sense if ``python3-core`` is installed in |
| 422 | the image. | 422 | the image. |
| 423 | 423 | ||
| 424 | ``testsdk_ext`` | 424 | ``testsdk_ext`` |
| 425 | --------------- | 425 | --------------- |
| 426 | 426 | ||
| 427 | These tests are run against built extensible SDKs (eSDKs). The tests can | 427 | These tests are run against built extensible SDKs (eSDKs). The tests can |
| 428 | assume that the eSDK environment has already been setup. An example from | 428 | assume that the eSDK environment has already been set up. An example from |
| 429 | ``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following:: | 429 | ``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following:: |
| 430 | 430 | ||
| 431 | class DevtoolTest(OESDKExtTestCase): | 431 | class DevtoolTest(OESDKExtTestCase): |
| @@ -472,9 +472,9 @@ following:: | |||
| 472 | output = self._run(cmd) | 472 | output = self._run(cmd) |
| 473 | self.assertEqual(output, "Hello, world\n") | 473 | self.assertEqual(output, "Hello, world\n") |
| 474 | 474 | ||
| 475 | In this example, if nativesdk-python3-core has been installed into the SDK, the code runs | 475 | In this example, if ``nativesdk-python3-core`` has been installed into the SDK, |
| 476 | the python3 interpreter with a basic command to check it is working | 476 | the code runs the ``python3`` interpreter with a basic command to check it is |
| 477 | correctly. The test would only run if Python3 is installed in the SDK. | 477 | working correctly. The test would only run if Python3 is installed in the SDK. |
| 478 | 478 | ||
| 479 | ``oe-build-perf-test`` | 479 | ``oe-build-perf-test`` |
| 480 | ---------------------- | 480 | ---------------------- |
| @@ -520,7 +520,7 @@ an isolated directory. | |||
| 520 | 520 | ||
| 521 | This can delete files from :term:`SSTATE_DIR` which would potentially break | 521 | This can delete files from :term:`SSTATE_DIR` which would potentially break |
| 522 | other builds running in parallel. If this is required, :term:`SSTATE_DIR` must | 522 | other builds running in parallel. If this is required, :term:`SSTATE_DIR` must |
| 523 | be set to an isolated directory. Alternatively, you can use the "-f" | 523 | be set to an isolated directory. Alternatively, you can use the ``-f`` |
| 524 | option with the ``bitbake`` command to "taint" tasks by changing the | 524 | option with the ``bitbake`` command to "taint" tasks by changing the |
| 525 | sstate checksums to ensure sstate cache items will not be reused. | 525 | sstate checksums to ensure sstate cache items will not be reused. |
| 526 | 526 | ||
