diff options
Diffstat (limited to 'documentation/dev-manual')
-rw-r--r-- | documentation/dev-manual/bblock.rst | 129 | ||||
-rw-r--r-- | documentation/dev-manual/bmaptool.rst | 29 | ||||
-rw-r--r-- | documentation/dev-manual/building.rst | 84 | ||||
-rw-r--r-- | documentation/dev-manual/customizing-images.rst | 17 | ||||
-rw-r--r-- | documentation/dev-manual/debugging.rst | 2 | ||||
-rw-r--r-- | documentation/dev-manual/external-scm.rst | 9 | ||||
-rw-r--r-- | documentation/dev-manual/index.rst | 2 | ||||
-rw-r--r-- | documentation/dev-manual/layers.rst | 90 | ||||
-rw-r--r-- | documentation/dev-manual/new-recipe.rst | 12 | ||||
-rw-r--r-- | documentation/dev-manual/packages.rst | 125 | ||||
-rw-r--r-- | documentation/dev-manual/qemu.rst | 13 | ||||
-rw-r--r-- | documentation/dev-manual/runtime-testing.rst | 594 | ||||
-rw-r--r-- | documentation/dev-manual/sbom.rst | 15 | ||||
-rw-r--r-- | documentation/dev-manual/start.rst | 2 | ||||
-rw-r--r-- | documentation/dev-manual/upgrading-recipes.rst | 2 | ||||
-rw-r--r-- | documentation/dev-manual/vulnerabilities.rst | 96 | ||||
-rw-r--r-- | documentation/dev-manual/wic.rst | 4 |
17 files changed, 427 insertions, 798 deletions
diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst new file mode 100644 index 0000000000..605bb75655 --- /dev/null +++ b/documentation/dev-manual/bblock.rst | |||
@@ -0,0 +1,129 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Locking and Unlocking Recipes Using ``bblock`` | ||
4 | ********************************************** | ||
5 | |||
6 | By design, the OpenEmbedded build system builds everything from scratch | ||
7 | unless BitBake determines that specific tasks do not require rebuilding. | ||
8 | At startup, it computes a signature for all tasks, based on the task's input. | ||
9 | Then, it compares these signatures with the ones from the sstate cache (if they | ||
10 | exist). Any changes cause the task to rerun. | ||
11 | |||
12 | During development, changes might trigger BitBake to rebuild certain | ||
13 | recipes, even when we know they do not require rebuilding at that stage. | ||
14 | For example, modifying a recipe can lead to rebuilding its native | ||
15 | counterpart, which might prove unnecessary. Editing the ``python3`` recipe, | ||
16 | for instance, can prompt BitBake to rebuild ``python3-native`` along with any | ||
17 | recipes that depend on it. | ||
18 | |||
19 | To prevent this, use ``bblock`` to lock specific tasks or recipes to | ||
20 | specific signatures, forcing BitBake to use the sstate cache for them. | ||
21 | |||
22 | .. warning:: | ||
23 | |||
24 | Use ``bblock`` only during the development phase. | ||
25 | |||
26 | Forcing BitBake to use the sstate cache, regardless of input changes, means | ||
27 | the recipe metadata no longer directly reflect the output. Use this feature | ||
28 | with caution. If you do not understand why signatures change, see the section | ||
29 | on :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`. | ||
30 | |||
31 | |||
32 | Locking tasks and recipes | ||
33 | ------------------------- | ||
34 | |||
35 | To lock a recipe, use:: | ||
36 | |||
37 | $ bblock recipe | ||
38 | |||
39 | You can also use a space-separated list of recipes to lock multiple recipes:: | ||
40 | |||
41 | $ bblock recipe1 recipe2 | ||
42 | |||
43 | Locking a recipe means locking all tasks of the recipe. If you need to | ||
44 | lock only particular tasks, use the `-t` option with a comma-separated | ||
45 | list of tasks:: | ||
46 | |||
47 | $ bblock -t task1,task2 recipe | ||
48 | |||
49 | |||
50 | Unlocking tasks and recipes | ||
51 | --------------------------- | ||
52 | |||
53 | To unlock a recipe, use the ``-r`` option:: | ||
54 | |||
55 | $ bblock -r recipe | ||
56 | |||
57 | You can also use a space-separated list of recipes to unlock multiple recipes:: | ||
58 | |||
59 | $ bblock -r recipe1 recipe2 | ||
60 | |||
61 | Unlocking a recipe means unlocking all tasks of the recipe. If you need to | ||
62 | unlock only particular tasks use the ``-t`` option with a comma-separated | ||
63 | list of tasks:: | ||
64 | |||
65 | $ bblock -r -t task1,task2 recipe | ||
66 | |||
67 | To unlock all recipes, do not specify any recipe:: | ||
68 | |||
69 | $ bblock -r | ||
70 | |||
71 | |||
72 | Configuration file | ||
73 | ------------------ | ||
74 | |||
75 | ``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, | ||
76 | included by default in :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`. | ||
77 | |||
78 | To dump the file, use the ``-d`` option:: | ||
79 | |||
80 | $ bblock -d | ||
81 | |||
82 | |||
83 | Locking mechanism | ||
84 | ----------------- | ||
85 | |||
86 | ``bblock`` computes the signature(s) of the task(s) and sets the 3 following | ||
87 | variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` | ||
88 | and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. | ||
89 | |||
90 | In particular, ``bblock`` sets:: | ||
91 | |||
92 | SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" | ||
93 | SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" | ||
94 | |||
95 | SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" | ||
96 | |||
97 | This produces architecture specific locks and reminds user that some tasks | ||
98 | have locked signatures. | ||
99 | |||
100 | Example | ||
101 | ------- | ||
102 | |||
103 | When working on the ``python3`` recipe, we can lock ``python3-native`` with | ||
104 | the following:: | ||
105 | |||
106 | $ bblock python3-native | ||
107 | $ bblock -d | ||
108 | # Generated by bblock | ||
109 | SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" | ||
110 | SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" | ||
111 | |||
112 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" | ||
113 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" | ||
114 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" | ||
115 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" | ||
116 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" | ||
117 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" | ||
118 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" | ||
119 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" | ||
120 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" | ||
121 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" | ||
122 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" | ||
123 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" | ||
124 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" | ||
125 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" | ||
126 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" | ||
127 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" | ||
128 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" | ||
129 | |||
diff --git a/documentation/dev-manual/bmaptool.rst b/documentation/dev-manual/bmaptool.rst index f6f0e6afaf..87162a49c9 100644 --- a/documentation/dev-manual/bmaptool.rst +++ b/documentation/dev-manual/bmaptool.rst | |||
@@ -1,13 +1,13 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | 1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK |
2 | 2 | ||
3 | Flashing Images Using ``bmaptool`` | 3 | Flashing Images Using `bmaptool` |
4 | ********************************** | 4 | ******************************** |
5 | 5 | ||
6 | A fast and easy way to flash an image to a bootable device is to use | 6 | A fast and easy way to flash an image to a bootable device is to use |
7 | bmaptool, which is integrated into the OpenEmbedded build system. | 7 | `bmaptool`, which is integrated into the OpenEmbedded build system. |
8 | bmaptool is a generic tool that creates a file's block map (bmap) and | 8 | `bmaptool` is a generic tool that creates a file's block map (bmap) and |
9 | then uses that map to copy the file. As compared to traditional tools | 9 | then uses that map to copy the file. As compared to traditional tools |
10 | such as dd or cp, bmaptool can copy (or flash) large files like raw | 10 | such as `dd` or `cp`, `bmaptool` can copy (or flash) large files like raw |
11 | system image files much faster. | 11 | system image files much faster. |
12 | 12 | ||
13 | .. note:: | 13 | .. note:: |
@@ -20,13 +20,13 @@ system image files much faster. | |||
20 | $ sudo apt install bmap-tools | 20 | $ sudo apt install bmap-tools |
21 | 21 | ||
22 | - If you are unable to install the ``bmap-tools`` package, you will | 22 | - If you are unable to install the ``bmap-tools`` package, you will |
23 | need to build bmaptool before using it. Use the following command:: | 23 | need to build `bmaptool` before using it. Use the following command:: |
24 | 24 | ||
25 | $ bitbake bmaptool-native | 25 | $ bitbake bmaptool-native -caddto_recipe_sysroot |
26 | 26 | ||
27 | Following, is an example that shows how to flash a Wic image. Realize | 27 | Following, is an example that shows how to flash a Wic image. Realize |
28 | that while this example uses a Wic image, you can use bmaptool to flash | 28 | that while this example uses a Wic image, you can use `bmaptool` to flash |
29 | any type of image. Use these steps to flash an image using bmaptool: | 29 | any type of image. Use these steps to flash an image using `bmaptool`: |
30 | 30 | ||
31 | #. *Update your local.conf File:* You need to have the following set | 31 | #. *Update your local.conf File:* You need to have the following set |
32 | in your ``local.conf`` file before building your image:: | 32 | in your ``local.conf`` file before building your image:: |
@@ -39,18 +39,17 @@ any type of image. Use these steps to flash an image using bmaptool: | |||
39 | 39 | ||
40 | $ bitbake image | 40 | $ bitbake image |
41 | 41 | ||
42 | #. *Flash the Device:* Flash the device with the image by using bmaptool | 42 | #. *Flash the Device:* Flash the device with the image by using `bmaptool` |
43 | depending on your particular setup. The following commands assume the | 43 | depending on your particular setup. The following commands assume the |
44 | image resides in the :term:`Build Directory`'s ``deploy/images/`` area: | 44 | image resides in the :term:`Build Directory`'s ``deploy/images/`` area: |
45 | 45 | ||
46 | - If you have write access to the media, use this command form:: | 46 | - If you installed the package for `bmaptool`, you can directly run:: |
47 | 47 | ||
48 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | 48 | $ sudo bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX |
49 | 49 | ||
50 | - If you do not have write access to the media, set your permissions | 50 | - Otherwise, if you built `bmaptool` with BitBake, run:: |
51 | first and then use the same command form:: | ||
52 | 51 | ||
53 | $ sudo chmod 666 /dev/sdX | 52 | $ sudo chmod a+w /dev/sdX # get write access to the media, needed only once after booting |
54 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | 53 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX |
55 | 54 | ||
56 | For help on the ``bmaptool`` command, use the following command:: | 55 | For help on the ``bmaptool`` command, use the following command:: |
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst index fe502690dd..4770a5a184 100644 --- a/documentation/dev-manual/building.rst +++ b/documentation/dev-manual/building.rst | |||
@@ -280,7 +280,9 @@ Follow these steps to create an :term:`Initramfs` image: | |||
280 | #. *Create the Initramfs Image Recipe:* You can reference the | 280 | #. *Create the Initramfs Image Recipe:* You can reference the |
281 | ``core-image-minimal-initramfs.bb`` recipe found in the | 281 | ``core-image-minimal-initramfs.bb`` recipe found in the |
282 | ``meta/recipes-core`` directory of the :term:`Source Directory` | 282 | ``meta/recipes-core`` directory of the :term:`Source Directory` |
283 | as an example from which to work. | 283 | as an example from which to work. The ``core-image-minimal-initramfs`` recipe |
284 | is based on the :ref:`initramfs-framework <dev-manual/building:Customizing an | ||
285 | Initramfs using \`\`initramfs-framework\`\`>` recipe described below. | ||
284 | 286 | ||
285 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel | 287 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel |
286 | Image:* If you want the :term:`Initramfs` image that is built to be bundled | 288 | Image:* If you want the :term:`Initramfs` image that is built to be bundled |
@@ -308,6 +310,86 @@ Follow these steps to create an :term:`Initramfs` image: | |||
308 | and bundled with the kernel image if you used the | 310 | and bundled with the kernel image if you used the |
309 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. | 311 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. |
310 | 312 | ||
313 | Customizing an Initramfs using ``initramfs-framework`` | ||
314 | ------------------------------------------------------ | ||
315 | |||
316 | The ``core-image-minimal-initramfs.bb`` recipe found in | ||
317 | :oe_git:`meta/recipes-core/images | ||
318 | </openembedded-core/tree/meta/recipes-core/images>` uses the | ||
319 | :oe_git:`initramfs-framework_1.0.bb | ||
320 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb>` | ||
321 | recipe as its base component. The goal of the ``initramfs-framework`` recipe is | ||
322 | to provide the building blocks to build a customized :term:`Initramfs`. | ||
323 | |||
324 | The ``initramfs-framework`` recipe relies on shell initialization scripts | ||
325 | defined in :oe_git:`meta/recipes-core/initrdscripts/initramfs-framework | ||
326 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework>`. Since some of | ||
327 | these scripts do not apply for all use cases, the ``initramfs-framework`` recipe | ||
328 | defines different packages: | ||
329 | |||
330 | - ``initramfs-framework-base``: this package installs the basic components of | ||
331 | an :term:`Initramfs`, such as the ``init`` script or the ``/dev/console`` | ||
332 | character special file. As this package is a runtime dependency of all | ||
333 | modules listed below, it is automatically pulled in when one of the modules | ||
334 | is installed in the image. | ||
335 | - ``initramfs-module-exec``: support for execution of applications. | ||
336 | - ``initramfs-module-mdev``: support for `mdev | ||
337 | <https://wiki.gentoo.org/wiki/Mdev>`__. | ||
338 | - ``initramfs-module-udev``: support for :wikipedia:`Udev <Udev>`. | ||
339 | - ``initramfs-module-e2fs``: support for :wikipedia:`ext4/ext3/ext2 | ||
340 | <Extended_file_system>` filesystems. | ||
341 | - ``initramfs-module-nfsrootfs``: support for locating and mounting the root | ||
342 | partition via :wikipedia:`NFS <Network_File_System>`. | ||
343 | - ``initramfs-module-rootfs``: support for locating and mounting the root | ||
344 | partition. | ||
345 | - ``initramfs-module-debug``: dynamic debug support. | ||
346 | - ``initramfs-module-lvm``: :wikipedia:`LVM <Logical_volume_management>` rootfs support. | ||
347 | - ``initramfs-module-overlayroot``: support for mounting a read-write overlay | ||
348 | on top of a read-only root filesystem. | ||
349 | |||
350 | In addition to the packages defined by the ``initramfs-framework`` recipe | ||
351 | itself, the following packages are defined by the recipes present in | ||
352 | :oe_git:`meta/recipes-core/initrdscripts </openembedded-core/tree/meta/recipes-core/initrdscripts>`: | ||
353 | |||
354 | - ``initramfs-module-install``: module to create and install a partition layout | ||
355 | on a selected block device. | ||
356 | - ``initramfs-module-install-efi``: module to create and install an EFI | ||
357 | partition layout on a selected block device. | ||
358 | - ``initramfs-module-setup-live``: module to start a shell in the | ||
359 | :term:`Initramfs` if ``root=/dev/ram0`` in passed in the `Kernel command-line | ||
360 | <https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html>`__ | ||
361 | or the ``root=`` parameter was not passed. | ||
362 | |||
363 | To customize the :term:`Initramfs`, you can add or remove packages listed | ||
364 | earlier from the :term:`PACKAGE_INSTALL` variable with a :ref:`bbappend | ||
365 | <dev-manual/layers:Appending Other Layers Metadata With Your Layer>` on the | ||
366 | ``core-image-minimal-initramfs`` recipe, or create a custom recipe for the | ||
367 | :term:`Initramfs` taking ``core-image-minimal-initramfs`` as example. | ||
368 | |||
369 | Custom scripts can be added to the :term:`Initramfs` by writing your own | ||
370 | recipes. The recipes are conventionally named ``initramfs-module-<module name>`` | ||
371 | where ``<module name>`` is the name of the module. The recipe should set its | ||
372 | :term:`RDEPENDS` package-specific variables to include | ||
373 | ``initramfs-framework-base`` and the other packages on which the module depends | ||
374 | at runtime. | ||
375 | |||
376 | The recipe must install shell initialization scripts in :term:`${D} <D>`\ | ||
377 | ``/init.d`` and must follow the ``<number>-<script name>`` naming scheme where: | ||
378 | |||
379 | - ``<number>`` is a *two-digit* number that affects the execution order of the | ||
380 | script compared to others. For example, the script ``80-setup-live`` would be | ||
381 | executed after ``01-udev`` because 80 is greater than 01. | ||
382 | |||
383 | This number being two-digits is important here as the scripts are executed | ||
384 | alphabetically. For example, the script ``10-script`` would be executed | ||
385 | before the script ``8-script``, because ``1`` is inferior to ``8``. | ||
386 | Therefore, the script should be named ``08-script``. | ||
387 | |||
388 | - ``<script name>`` is the script name which you can choose freely. | ||
389 | |||
390 | If two script use the same ``<number>``, they are sorted alphabetically based | ||
391 | on ``<script name>``. | ||
392 | |||
311 | Bundling an Initramfs Image From a Separate Multiconfig | 393 | Bundling an Initramfs Image From a Separate Multiconfig |
312 | ------------------------------------------------------- | 394 | ------------------------------------------------------- |
313 | 395 | ||
diff --git a/documentation/dev-manual/customizing-images.rst b/documentation/dev-manual/customizing-images.rst index 5b18958ade..53cad9c79c 100644 --- a/documentation/dev-manual/customizing-images.rst +++ b/documentation/dev-manual/customizing-images.rst | |||
@@ -80,15 +80,14 @@ recipe that are enabled with :term:`IMAGE_FEATURES`. The value of | |||
80 | :term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within | 80 | :term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within |
81 | ``meta/conf/bitbake.conf``. | 81 | ``meta/conf/bitbake.conf``. |
82 | 82 | ||
83 | To illustrate how you can use these variables to modify your image, | 83 | To illustrate how you can use these variables to modify your image, consider an |
84 | consider an example that selects the SSH server. The Yocto Project ships | 84 | example that selects the SSH server. The Yocto Project ships with two SSH |
85 | with two SSH servers you can use with your images: Dropbear and OpenSSH. | 85 | servers you can use with your images: Dropbear and OpenSSH. Dropbear is a |
86 | Dropbear is a minimal SSH server appropriate for resource-constrained | 86 | minimal SSH server appropriate for resource-constrained environments, while |
87 | environments, while OpenSSH is a well-known standard SSH server | 87 | OpenSSH is a well-known standard SSH server implementation. By default, the |
88 | implementation. By default, the ``core-image-sato`` image is configured | 88 | ``core-image-sato`` image is configured to use Dropbear. The |
89 | to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb`` | 89 | ``core-image-full-cmdline`` image includes OpenSSH. The ``core-image-minimal`` |
90 | images both include OpenSSH. The ``core-image-minimal`` image does not | 90 | image does not contain an SSH server. |
91 | contain an SSH server. | ||
92 | 91 | ||
93 | You can customize your image and change these defaults. Edit the | 92 | You can customize your image and change these defaults. Edit the |
94 | :term:`IMAGE_FEATURES` variable in your recipe or use the | 93 | :term:`IMAGE_FEATURES` variable in your recipe or use the |
diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst index 92458a0c37..8552b26aea 100644 --- a/documentation/dev-manual/debugging.rst +++ b/documentation/dev-manual/debugging.rst | |||
@@ -36,7 +36,7 @@ section: | |||
36 | use the BitBake ``-e`` option to examine variable values after a | 36 | use the BitBake ``-e`` option to examine variable values after a |
37 | recipe has been parsed. | 37 | recipe has been parsed. |
38 | 38 | ||
39 | - ":ref:`dev-manual/debugging:viewing package information with \`\`oe-pkgdata-util\`\``" | 39 | - ":ref:`dev-manual/debugging:viewing package information with ``oe-pkgdata-util```" |
40 | describes how to use the ``oe-pkgdata-util`` utility to query | 40 | describes how to use the ``oe-pkgdata-util`` utility to query |
41 | :term:`PKGDATA_DIR` and | 41 | :term:`PKGDATA_DIR` and |
42 | display package-related information for built packages. | 42 | display package-related information for built packages. |
diff --git a/documentation/dev-manual/external-scm.rst b/documentation/dev-manual/external-scm.rst index 97a7e63e36..896b1b5ac7 100644 --- a/documentation/dev-manual/external-scm.rst +++ b/documentation/dev-manual/external-scm.rst | |||
@@ -12,10 +12,13 @@ revision number for changes. Currently, you can do this with Apache | |||
12 | Subversion (SVN), Git, and Bazaar (BZR) repositories. | 12 | Subversion (SVN), Git, and Bazaar (BZR) repositories. |
13 | 13 | ||
14 | To enable this behavior, the :term:`PV` of | 14 | To enable this behavior, the :term:`PV` of |
15 | the recipe needs to reference | 15 | the recipe needs to include a ``+`` sign in its assignment. |
16 | :term:`SRCPV`. Here is an example:: | 16 | Here is an example:: |
17 | 17 | ||
18 | PV = "1.2.3+git${SRCPV}" | 18 | PV = "1.2.3+git" |
19 | |||
20 | :term:`Bitbake` later includes the source control information in :term:`PKGV` | ||
21 | during the packaging phase. | ||
19 | 22 | ||
20 | Then, you can add the following to your | 23 | Then, you can add the following to your |
21 | ``local.conf``:: | 24 | ``local.conf``:: |
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst index 9ccf60f701..8243c0f4cb 100644 --- a/documentation/dev-manual/index.rst +++ b/documentation/dev-manual/index.rst | |||
@@ -39,7 +39,6 @@ Yocto Project Development Tasks Manual | |||
39 | external-scm | 39 | external-scm |
40 | read-only-rootfs | 40 | read-only-rootfs |
41 | build-quality | 41 | build-quality |
42 | runtime-testing | ||
43 | debugging | 42 | debugging |
44 | licenses | 43 | licenses |
45 | security-subjects | 44 | security-subjects |
@@ -48,5 +47,6 @@ Yocto Project Development Tasks Manual | |||
48 | error-reporting-tool | 47 | error-reporting-tool |
49 | wayland | 48 | wayland |
50 | qemu | 49 | qemu |
50 | bblock | ||
51 | 51 | ||
52 | .. include:: /boilerplate.rst | 52 | .. include:: /boilerplate.rst |
diff --git a/documentation/dev-manual/layers.rst b/documentation/dev-manual/layers.rst index 91889bd0ae..89c8466933 100644 --- a/documentation/dev-manual/layers.rst +++ b/documentation/dev-manual/layers.rst | |||
@@ -644,6 +644,96 @@ variable and append the layer's root name:: | |||
644 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake | 644 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake |
645 | might address this. | 645 | might address this. |
646 | 646 | ||
647 | Providing Global-level Configurations With Your Layer | ||
648 | ----------------------------------------------------- | ||
649 | |||
650 | When creating a layer, you may need to define configurations that should take | ||
651 | effect globally in your build environment when the layer is part of the build. | ||
652 | The ``layer.conf`` file is a :term:`configuration file` that affects the build | ||
653 | system globally, so it is a candidate for this use-case. | ||
654 | |||
655 | .. warning:: | ||
656 | |||
657 | Providing unconditional global level configuration from the ``layer.conf`` | ||
658 | file is *not* a good practice, and should be avoided. For this reason, the | ||
659 | section :ref:`ref-conditional-layer-confs` below shows how the ``layer.conf`` | ||
660 | file can be used to provide configurations only if a certain condition is | ||
661 | met. | ||
662 | |||
663 | For example, if your layer provides a Linux kernel recipe named | ||
664 | ``linux-custom``, you may want to make :term:`PREFERRED_PROVIDER_virtual/kernel | ||
665 | <PREFERRED_PROVIDER>` point to ``linux-custom``:: | ||
666 | |||
667 | PREFERRED_PROVIDER_virtual/kernel = "linux-custom" | ||
668 | |||
669 | This can be defined in the ``layer.conf`` file. If your layer is at the last | ||
670 | position in the :term:`BBLAYERS` list, it will take precedence over previous | ||
671 | ``PREFERRED_PROVIDER_virtual/kernel`` assignments (unless one is set from a | ||
672 | :term:`configuration file` that is parsed later, such as machine or distro | ||
673 | configuration files). | ||
674 | |||
675 | .. _ref-conditional-layer-confs: | ||
676 | |||
677 | Conditionally Provide Global-level Configurations With Your Layer | ||
678 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
679 | |||
680 | In some cases, your layer may provide global configurations only if some | ||
681 | features it provides are enabled. Since the ``layer.conf`` file is parsed at an | ||
682 | earlier stage in the parsing process, the :term:`DISTRO_FEATURES` and | ||
683 | :term:`MACHINE_FEATURES` variables are not yet available to ``layer.conf``, and | ||
684 | declaring conditional assignments based on these variables is not possible. The | ||
685 | following technique shows a way to bypass this limitation by using the | ||
686 | :term:`USER_CLASSES` variable and a conditional ``require`` command. | ||
687 | |||
688 | In the following steps, let's assume our layer is named ``meta-mylayer`` and | ||
689 | that this layer defines a custom :ref:`distro feature <ref-features-distro>` | ||
690 | named ``mylayer-kernel``. We will set the :term:`PREFERRED_PROVIDER` variable | ||
691 | for the kernel only if our feature ``mylayer-kernel`` is part of the | ||
692 | :term:`DISTRO_FEATURES`: | ||
693 | |||
694 | #. Create an include file in the directory | ||
695 | ``meta-mylayer/conf/distro/include/``, for example a file named | ||
696 | ``mylayer-kernel-provider.inc`` that sets the kernel provider to | ||
697 | ``linux-custom``:: | ||
698 | |||
699 | PREFERRED_PROVIDER_virtual/kernel = "linux-custom" | ||
700 | |||
701 | #. Provide a path to this include file in your ``layer.conf``:: | ||
702 | |||
703 | META_MYLAYER_KERNEL_PROVIDER_PATH = "${LAYERDIR}/conf/distro/include/mylayer-kernel-provider.inc" | ||
704 | |||
705 | #. Create a new class in ``meta-mylayer/classes-global/``, for example a class | ||
706 | ``meta-mylayer-cfg.bbclass``. Make it conditionally require the file | ||
707 | ``mylayer-kernel-provider.inc`` defined above, using the variable | ||
708 | ``META_MYLAYER_KERNEL_PROVIDER_PATH`` defined in ``layer.conf``:: | ||
709 | |||
710 | require ${@bb.utils.contains('DISTRO_FEATURES', 'mylayer-kernel', '${META_MYLAYER_KERNEL_PROVIDER_PATH}', '', d)} | ||
711 | |||
712 | For details on the ``bb.utils.contains`` function, see its definition in | ||
713 | :bitbake_git:`lib/bb/utils.py </tree/lib/bb/utils.py>`. | ||
714 | |||
715 | .. note:: | ||
716 | |||
717 | The ``require`` command is designed to not fail if the function | ||
718 | ``bb.utils.contains`` returns an empty string. | ||
719 | |||
720 | #. Back to your ``layer.conf`` file, add the class ``meta-mylayer-cfg`` class to | ||
721 | the :term:`USER_CLASSES` variable:: | ||
722 | |||
723 | USER_CLASSES:append = " meta-mylayer-cfg" | ||
724 | |||
725 | This will add the class ``meta-mylayer-cfg`` to the list of classes to | ||
726 | globally inherit. Since the ``require`` command is conditional in | ||
727 | ``meta-mylayer-cfg.bbclass``, even though inherited the class will have no | ||
728 | effect unless the feature ``mylayer-kernel`` is enabled through | ||
729 | :term:`DISTRO_FEATURES`. | ||
730 | |||
731 | This technique can also be used for :ref:`Machine features | ||
732 | <ref-features-machine>` by following the same steps. Though not mandatory, it is | ||
733 | recommended to put include files for :term:`DISTRO_FEATURES` in your layer's | ||
734 | ``conf/distro/include`` and the ones for :term:`MACHINE_FEATURES` in your | ||
735 | layer's ``conf/machine/include``. | ||
736 | |||
647 | Managing Layers | 737 | Managing Layers |
648 | =============== | 738 | =============== |
649 | 739 | ||
diff --git a/documentation/dev-manual/new-recipe.rst b/documentation/dev-manual/new-recipe.rst index 61fc2eb122..af88db937b 100644 --- a/documentation/dev-manual/new-recipe.rst +++ b/documentation/dev-manual/new-recipe.rst | |||
@@ -56,7 +56,7 @@ necessary when adding a recipe to build a new piece of software to be | |||
56 | included in a build. | 56 | included in a build. |
57 | 57 | ||
58 | You can find a complete description of the ``devtool add`` command in | 58 | You can find a complete description of the ``devtool add`` command in |
59 | the ":ref:`sdk-manual/extensible:a closer look at \`\`devtool add\`\``" section | 59 | the ":ref:`sdk-manual/extensible:a closer look at ``devtool add```" section |
60 | in the Yocto Project Application Development and the Extensible Software | 60 | in the Yocto Project Application Development and the Extensible Software |
61 | Development Kit (eSDK) manual. | 61 | Development Kit (eSDK) manual. |
62 | 62 | ||
@@ -291,13 +291,13 @@ another example that specifies these types of files, see the | |||
291 | 291 | ||
292 | Another way of specifying source is from an SCM. For Git repositories, | 292 | Another way of specifying source is from an SCM. For Git repositories, |
293 | you must specify :term:`SRCREV` and you should specify :term:`PV` to include | 293 | you must specify :term:`SRCREV` and you should specify :term:`PV` to include |
294 | the revision with :term:`SRCPV`. Here is an example from the recipe | 294 | a ``+`` sign in its definition. Here is an example from the recipe |
295 | ``meta/recipes-core/musl/gcompat_git.bb``:: | 295 | :oe_git:`meta/recipes-sato/l3afpad/l3afpad_git.bb </openembedded-core/tree/meta/recipes-sato/l3afpad/l3afpad_git.bb>`:: |
296 | 296 | ||
297 | SRC_URI = "git://git.adelielinux.org/adelie/gcompat.git;protocol=https;branch=current" | 297 | SRC_URI = "git://github.com/stevenhoneyman/l3afpad.git;branch=master;protocol=https" |
298 | 298 | ||
299 | PV = "1.0.0+1.1+git${SRCPV}" | 299 | PV = "0.8.18.1.11+git" |
300 | SRCREV = "af5a49e489fdc04b9cf02547650d7aeaccd43793" | 300 | SRCREV ="3cdccdc9505643e50f8208171d9eee5de11a42ff" |
301 | 301 | ||
302 | If your :term:`SRC_URI` statement includes URLs pointing to individual files | 302 | If your :term:`SRC_URI` statement includes URLs pointing to individual files |
303 | fetched from a remote server other than a version control system, | 303 | fetched from a remote server other than a version control system, |
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index e5028fffdc..4ba2dcae3a 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
17 | 17 | ||
18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
20 | 20 | ||
21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
22 | 22 | ||
@@ -84,10 +84,6 @@ the following: | |||
84 | 84 | ||
85 | - :term:`PR`: The recipe revision. | 85 | - :term:`PR`: The recipe revision. |
86 | 86 | ||
87 | - :term:`SRCPV`: The OpenEmbedded | ||
88 | build system uses this string to help define the value of :term:`PV` when | ||
89 | the source code revision needs to be included in it. | ||
90 | |||
91 | - :yocto_wiki:`PR Service </PR_Service>`: A | 87 | - :yocto_wiki:`PR Service </PR_Service>`: A |
92 | network-based service that helps automate keeping package feeds | 88 | network-based service that helps automate keeping package feeds |
93 | compatible with existing package manager applications such as RPM, | 89 | compatible with existing package manager applications such as RPM, |
@@ -256,15 +252,14 @@ the software:: | |||
256 | 252 | ||
257 | SRCREV = "${AUTOREV}" | 253 | SRCREV = "${AUTOREV}" |
258 | 254 | ||
259 | Furthermore, you need to reference :term:`SRCPV` in :term:`PV` in order to | 255 | Furthermore, you need to include a ``+`` sign in :term:`PV` in order to |
260 | automatically update the version whenever the revision of the source | 256 | automatically update the version whenever the revision of the source |
261 | code changes. Here is an example:: | 257 | code changes. Here is an example:: |
262 | 258 | ||
263 | PV = "1.0+git${SRCPV}" | 259 | PV = "1.0+git" |
264 | |||
265 | The OpenEmbedded build system substitutes :term:`SRCPV` with the following: | ||
266 | 260 | ||
267 | .. code-block:: none | 261 | The OpenEmbedded build system will automatically add the source control |
262 | information to the end of the variable :term:`PKGV`, in this format:: | ||
268 | 263 | ||
269 | AUTOINC+source_code_revision | 264 | AUTOINC+source_code_revision |
270 | 265 | ||
@@ -887,114 +882,8 @@ related to signed package feeds are available: | |||
887 | Testing Packages With ptest | 882 | Testing Packages With ptest |
888 | =========================== | 883 | =========================== |
889 | 884 | ||
890 | A Package Test (ptest) runs tests against packages built by the | 885 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
891 | OpenEmbedded build system on the target machine. A ptest contains at | 886 | Yocto Project Test Environment Manual. |
892 | least two items: the actual test, and a shell script (``run-ptest``) | ||
893 | that starts the test. The shell script that starts the test must not | ||
894 | contain the actual test --- the script only starts the test. On the other | ||
895 | hand, the test can be anything from a simple shell script that runs a | ||
896 | binary and checks the output to an elaborate system of test binaries and | ||
897 | data files. | ||
898 | |||
899 | The test generates output in the format used by Automake:: | ||
900 | |||
901 | result: testname | ||
902 | |||
903 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
904 | the testname can be any identifying string. | ||
905 | |||
906 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
907 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
908 | |||
909 | .. note:: | ||
910 | |||
911 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
912 | class. | ||
913 | |||
914 | Adding ptest to Your Build | ||
915 | -------------------------- | ||
916 | |||
917 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
918 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
919 | is found in the :term:`Build Directory`:: | ||
920 | |||
921 | DISTRO_FEATURES:append = " ptest" | ||
922 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
923 | |||
924 | Once your build is complete, the ptest files are installed into the | ||
925 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
926 | is the name of the package. | ||
927 | |||
928 | Running ptest | ||
929 | ------------- | ||
930 | |||
931 | The ``ptest-runner`` package installs a shell script that loops through | ||
932 | all installed ptest test suites and runs them in sequence. Consequently, | ||
933 | you might want to add this package to your image. | ||
934 | |||
935 | Getting Your Package Ready | ||
936 | -------------------------- | ||
937 | |||
938 | In order to enable a recipe to run installed ptests on target hardware, | ||
939 | you need to prepare the recipes that build the packages you want to | ||
940 | test. Here is what you have to do for each recipe: | ||
941 | |||
942 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
943 | Include the following line in each recipe:: | ||
944 | |||
945 | inherit ptest | ||
946 | |||
947 | - *Create run-ptest:* This script starts your test. Locate the | ||
948 | script where you will refer to it using | ||
949 | :term:`SRC_URI`. Here is an | ||
950 | example that starts a test for ``dbus``:: | ||
951 | |||
952 | #!/bin/sh | ||
953 | cd test | ||
954 | make -k runtest-TESTS | ||
955 | |||
956 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
957 | dependencies that normally do not exist for the package (such as | ||
958 | requiring "make" to run the test suite), use the | ||
959 | :term:`DEPENDS` and | ||
960 | :term:`RDEPENDS` variables in | ||
961 | your recipe in order for the package to meet the dependencies. Here | ||
962 | is an example where the package has a runtime dependency on "make":: | ||
963 | |||
964 | RDEPENDS:${PN}-ptest += "make" | ||
965 | |||
966 | - *Add a function to build the test suite:* Not many packages support | ||
967 | cross-compilation of their test suites. Consequently, you usually | ||
968 | need to add a cross-compilation function to the package. | ||
969 | |||
970 | Many packages based on Automake compile and run the test suite by | ||
971 | using a single command such as ``make check``. However, the host | ||
972 | ``make check`` builds and runs on the same computer, while | ||
973 | cross-compiling requires that the package is built on the host but | ||
974 | executed for the target architecture (though often, as in the case | ||
975 | for ptest, the execution occurs on the host). The built version of | ||
976 | Automake that ships with the Yocto Project includes a patch that | ||
977 | separates building and execution. Consequently, packages that use the | ||
978 | unaltered, patched version of ``make check`` automatically | ||
979 | cross-compiles. | ||
980 | |||
981 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
982 | build the test suite. Add a function similar to the following to your | ||
983 | recipe:: | ||
984 | |||
985 | do_compile_ptest() { | ||
986 | oe_runmake buildtest-TESTS | ||
987 | } | ||
988 | |||
989 | - *Ensure special configurations are set:* If the package requires | ||
990 | special configurations prior to compiling the test code, you must | ||
991 | insert a ``do_configure_ptest`` function into the recipe. | ||
992 | |||
993 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
994 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
995 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
996 | to create a ``do_install_ptest`` function and make sure it gets | ||
997 | called after the "make install-ptest" completes. | ||
998 | 887 | ||
999 | Creating Node Package Manager (NPM) Packages | 888 | Creating Node Package Manager (NPM) Packages |
1000 | ============================================ | 889 | ============================================ |
diff --git a/documentation/dev-manual/qemu.rst b/documentation/dev-manual/qemu.rst index 19f3e40d63..253aff9977 100644 --- a/documentation/dev-manual/qemu.rst +++ b/documentation/dev-manual/qemu.rst | |||
@@ -75,7 +75,7 @@ available. Follow these general steps to run QEMU: | |||
75 | your :term:`Build Directory`. | 75 | your :term:`Build Directory`. |
76 | 76 | ||
77 | - If you have not built an image, you can go to the | 77 | - If you have not built an image, you can go to the |
78 | :yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu/>` area and download a | 78 | :yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu/>` area and download a |
79 | pre-built image that matches your architecture and can be run on | 79 | pre-built image that matches your architecture and can be run on |
80 | QEMU. | 80 | QEMU. |
81 | 81 | ||
@@ -280,12 +280,11 @@ present, the toolchain is also automatically used. | |||
280 | networking. | 280 | networking. |
281 | 281 | ||
282 | - SSH servers are available in some QEMU images. The ``core-image-sato`` | 282 | - SSH servers are available in some QEMU images. The ``core-image-sato`` |
283 | QEMU image has a Dropbear secure shell (SSH) server that runs with | 283 | QEMU image has a Dropbear secure shell (SSH) server that runs with the |
284 | the root password disabled. The ``core-image-full-cmdline`` and | 284 | root password disabled. The ``core-image-full-cmdline`` QEMU image has |
285 | ``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear. | 285 | OpenSSH instead of Dropbear. Including these SSH servers allow you to use |
286 | Including these SSH servers allow you to use standard ``ssh`` and | 286 | standard ``ssh`` and ``scp`` commands. The ``core-image-minimal`` QEMU |
287 | ``scp`` commands. The ``core-image-minimal`` QEMU image, however, | 287 | image, however, contains no SSH server. |
288 | contains no SSH server. | ||
289 | 288 | ||
290 | - You can use a provided, user-space NFS server to boot the QEMU | 289 | - You can use a provided, user-space NFS server to boot the QEMU |
291 | session using a local copy of the root filesystem on the host. In | 290 | session using a local copy of the root filesystem on the host. In |
diff --git a/documentation/dev-manual/runtime-testing.rst b/documentation/dev-manual/runtime-testing.rst deleted file mode 100644 index 7a2b42f25a..0000000000 --- a/documentation/dev-manual/runtime-testing.rst +++ /dev/null | |||
@@ -1,594 +0,0 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Performing Automated Runtime Testing | ||
4 | ************************************ | ||
5 | |||
6 | The OpenEmbedded build system makes available a series of automated | ||
7 | tests for images to verify runtime functionality. You can run these | ||
8 | tests on either QEMU or actual target hardware. Tests are written in | ||
9 | Python making use of the ``unittest`` module, and the majority of them | ||
10 | run commands on the target system over SSH. This section describes how | ||
11 | you set up the environment to use these tests, run available tests, and | ||
12 | write and add your own tests. | ||
13 | |||
14 | For information on the test and QA infrastructure available within the | ||
15 | Yocto Project, see the ":ref:`ref-manual/release-process:testing and quality assurance`" | ||
16 | section in the Yocto Project Reference Manual. | ||
17 | |||
18 | Enabling Tests | ||
19 | ============== | ||
20 | |||
21 | Depending on whether you are planning to run tests using QEMU or on the | ||
22 | hardware, you have to take different steps to enable the tests. See the | ||
23 | following subsections for information on how to enable both types of | ||
24 | tests. | ||
25 | |||
26 | Enabling Runtime Tests on QEMU | ||
27 | ------------------------------ | ||
28 | |||
29 | In order to run tests, you need to do the following: | ||
30 | |||
31 | - *Set up to avoid interaction with sudo for networking:* To | ||
32 | accomplish this, you must do one of the following: | ||
33 | |||
34 | - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all | ||
35 | commands or just for ``runqemu-ifup``. You must provide the full | ||
36 | path as that can change if you are using multiple clones of the | ||
37 | source repository. | ||
38 | |||
39 | .. note:: | ||
40 | |||
41 | On some distributions, you also need to comment out "Defaults | ||
42 | requiretty" in ``/etc/sudoers``. | ||
43 | |||
44 | - Manually configure a tap interface for your system. | ||
45 | |||
46 | - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which | ||
47 | should generate a list of tap devices. This is the option | ||
48 | typically chosen for Autobuilder-type environments. | ||
49 | |||
50 | .. note:: | ||
51 | |||
52 | - Be sure to use an absolute path when calling this script | ||
53 | with sudo. | ||
54 | |||
55 | - Ensure that your host has the package ``iptables`` installed. | ||
56 | |||
57 | - The package recipe ``qemu-helper-native`` is required to run | ||
58 | this script. Build the package using the following command:: | ||
59 | |||
60 | $ bitbake qemu-helper-native | ||
61 | |||
62 | - *Set the DISPLAY variable:* You need to set this variable so that | ||
63 | you have an X server available (e.g. start ``vncserver`` for a | ||
64 | headless machine). | ||
65 | |||
66 | - *Be sure your host's firewall accepts incoming connections from | ||
67 | 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an | ||
68 | HTTP server on a random high number port, which is used to serve | ||
69 | files to the target. The DNF module serves | ||
70 | ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands. | ||
71 | That means your host's firewall must accept incoming connections from | ||
72 | 192.168.7.0/24, which is the default IP range used for tap devices by | ||
73 | ``runqemu``. | ||
74 | |||
75 | - *Be sure your host has the correct packages installed:* Depending | ||
76 | your host's distribution, you need to have the following packages | ||
77 | installed: | ||
78 | |||
79 | - Ubuntu and Debian: ``sysstat`` and ``iproute2`` | ||
80 | |||
81 | - openSUSE: ``sysstat`` and ``iproute2`` | ||
82 | |||
83 | - Fedora: ``sysstat`` and ``iproute`` | ||
84 | |||
85 | - CentOS: ``sysstat`` and ``iproute`` | ||
86 | |||
87 | Once you start running the tests, the following happens: | ||
88 | |||
89 | #. A copy of the root filesystem is written to ``${WORKDIR}/testimage``. | ||
90 | |||
91 | #. The image is booted under QEMU using the standard ``runqemu`` script. | ||
92 | |||
93 | #. A default timeout of 500 seconds occurs to allow for the boot process | ||
94 | to reach the login prompt. You can change the timeout period by | ||
95 | setting | ||
96 | :term:`TEST_QEMUBOOT_TIMEOUT` | ||
97 | in the ``local.conf`` file. | ||
98 | |||
99 | #. Once the boot process is reached and the login prompt appears, the | ||
100 | tests run. The full boot log is written to | ||
101 | ``${WORKDIR}/testimage/qemu_boot_log``. | ||
102 | |||
103 | #. Each test module loads in the order found in :term:`TEST_SUITES`. You can | ||
104 | find the full output of the commands run over SSH in | ||
105 | ``${WORKDIR}/testimgage/ssh_target_log``. | ||
106 | |||
107 | #. If no failures occur, the task running the tests ends successfully. | ||
108 | You can find the output from the ``unittest`` in the task log at | ||
109 | ``${WORKDIR}/temp/log.do_testimage``. | ||
110 | |||
111 | Enabling Runtime Tests on Hardware | ||
112 | ---------------------------------- | ||
113 | |||
114 | The OpenEmbedded build system can run tests on real hardware, and for | ||
115 | certain devices it can also deploy the image to be tested onto the | ||
116 | device beforehand. | ||
117 | |||
118 | For automated deployment, a "controller image" is installed onto the | ||
119 | hardware once as part of setup. Then, each time tests are to be run, the | ||
120 | following occurs: | ||
121 | |||
122 | #. The controller image is booted into and used to write the image to be | ||
123 | tested to a second partition. | ||
124 | |||
125 | #. The device is then rebooted using an external script that you need to | ||
126 | provide. | ||
127 | |||
128 | #. The device boots into the image to be tested. | ||
129 | |||
130 | When running tests (independent of whether the image has been deployed | ||
131 | automatically or not), the device is expected to be connected to a | ||
132 | network on a pre-determined IP address. You can either use static IP | ||
133 | addresses written into the image, or set the image to use DHCP and have | ||
134 | your DHCP server on the test network assign a known IP address based on | ||
135 | the MAC address of the device. | ||
136 | |||
137 | In order to run tests on hardware, you need to set :term:`TEST_TARGET` to an | ||
138 | appropriate value. For QEMU, you do not have to change anything, the | ||
139 | default value is "qemu". For running tests on hardware, the following | ||
140 | options are available: | ||
141 | |||
142 | - *"simpleremote":* Choose "simpleremote" if you are going to run tests | ||
143 | on a target system that is already running the image to be tested and | ||
144 | is available on the network. You can use "simpleremote" in | ||
145 | conjunction with either real hardware or an image running within a | ||
146 | separately started QEMU or any other virtual machine manager. | ||
147 | |||
148 | - *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is | ||
149 | an EFI-based machine with ``systemd-boot`` as bootloader and | ||
150 | ``core-image-testmaster`` (or something similar) is installed. Also, | ||
151 | your hardware under test must be in a DHCP-enabled network that gives | ||
152 | it the same IP address for each reboot. | ||
153 | |||
154 | If you choose "SystemdbootTarget", there are additional requirements | ||
155 | and considerations. See the | ||
156 | ":ref:`dev-manual/runtime-testing:selecting systemdboottarget`" section, which | ||
157 | follows, for more information. | ||
158 | |||
159 | - *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying | ||
160 | images and running tests on the BeagleBone "Black" or original | ||
161 | "White" hardware. For information on how to use these tests, see the | ||
162 | comments at the top of the BeagleBoneTarget | ||
163 | ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file. | ||
164 | |||
165 | - *"GrubTarget":* Choose "GrubTarget" if you are deploying images and running | ||
166 | tests on any generic PC that boots using GRUB. For information on how | ||
167 | to use these tests, see the comments at the top of the GrubTarget | ||
168 | ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file. | ||
169 | |||
170 | - *"your-target":* Create your own custom target if you want to run | ||
171 | tests when you are deploying images and running tests on a custom | ||
172 | machine within your BSP layer. To do this, you need to add a Python | ||
173 | unit that defines the target class under ``lib/oeqa/controllers/`` | ||
174 | within your layer. You must also provide an empty ``__init__.py``. | ||
175 | For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``. | ||
176 | |||
177 | Selecting SystemdbootTarget | ||
178 | --------------------------- | ||
179 | |||
180 | If you did not set :term:`TEST_TARGET` to "SystemdbootTarget", then you do | ||
181 | not need any information in this section. You can skip down to the | ||
182 | ":ref:`dev-manual/runtime-testing:running tests`" section. | ||
183 | |||
184 | If you did set :term:`TEST_TARGET` to "SystemdbootTarget", you also need to | ||
185 | perform a one-time setup of your controller image by doing the following: | ||
186 | |||
187 | #. *Set EFI_PROVIDER:* Be sure that :term:`EFI_PROVIDER` is as follows:: | ||
188 | |||
189 | EFI_PROVIDER = "systemd-boot" | ||
190 | |||
191 | #. *Build the controller image:* Build the ``core-image-testmaster`` image. | ||
192 | The ``core-image-testmaster`` recipe is provided as an example for a | ||
193 | "controller" image and you can customize the image recipe as you would | ||
194 | any other recipe. | ||
195 | |||
196 | Image recipe requirements are: | ||
197 | |||
198 | - Inherits ``core-image`` so that kernel modules are installed. | ||
199 | |||
200 | - Installs normal linux utilities not BusyBox ones (e.g. ``bash``, | ||
201 | ``coreutils``, ``tar``, ``gzip``, and ``kmod``). | ||
202 | |||
203 | - Uses a custom :term:`Initramfs` image with a custom | ||
204 | installer. A normal image that you can install usually creates a | ||
205 | single root filesystem partition. This image uses another installer that | ||
206 | creates a specific partition layout. Not all Board Support | ||
207 | Packages (BSPs) can use an installer. For such cases, you need to | ||
208 | manually create the following partition layout on the target: | ||
209 | |||
210 | - First partition mounted under ``/boot``, labeled "boot". | ||
211 | |||
212 | - The main root filesystem partition where this image gets installed, | ||
213 | which is mounted under ``/``. | ||
214 | |||
215 | - Another partition labeled "testrootfs" where test images get | ||
216 | deployed. | ||
217 | |||
218 | #. *Install image:* Install the image that you just built on the target | ||
219 | system. | ||
220 | |||
221 | The final thing you need to do when setting :term:`TEST_TARGET` to | ||
222 | "SystemdbootTarget" is to set up the test image: | ||
223 | |||
224 | #. *Set up your local.conf file:* Make sure you have the following | ||
225 | statements in your ``local.conf`` file:: | ||
226 | |||
227 | IMAGE_FSTYPES += "tar.gz" | ||
228 | IMAGE_CLASSES += "testimage" | ||
229 | TEST_TARGET = "SystemdbootTarget" | ||
230 | TEST_TARGET_IP = "192.168.2.3" | ||
231 | |||
232 | #. *Build your test image:* Use BitBake to build the image:: | ||
233 | |||
234 | $ bitbake core-image-sato | ||
235 | |||
236 | Power Control | ||
237 | ------------- | ||
238 | |||
239 | For most hardware targets other than "simpleremote", you can control | ||
240 | power: | ||
241 | |||
242 | - You can use :term:`TEST_POWERCONTROL_CMD` together with | ||
243 | :term:`TEST_POWERCONTROL_EXTRA_ARGS` as a command that runs on the host | ||
244 | and does power cycling. The test code passes one argument to that | ||
245 | command: off, on or cycle (off then on). Here is an example that | ||
246 | could appear in your ``local.conf`` file:: | ||
247 | |||
248 | TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" | ||
249 | |||
250 | In this example, the expect | ||
251 | script does the following: | ||
252 | |||
253 | .. code-block:: shell | ||
254 | |||
255 | ssh test@10.11.12.1 "pyctl nuc1 arg" | ||
256 | |||
257 | It then runs a Python script that controls power for a label called | ||
258 | ``nuc1``. | ||
259 | |||
260 | .. note:: | ||
261 | |||
262 | You need to customize :term:`TEST_POWERCONTROL_CMD` and | ||
263 | :term:`TEST_POWERCONTROL_EXTRA_ARGS` for your own setup. The one requirement | ||
264 | is that it accepts "on", "off", and "cycle" as the last argument. | ||
265 | |||
266 | - When no command is defined, it connects to the device over SSH and | ||
267 | uses the classic reboot command to reboot the device. Classic reboot | ||
268 | is fine as long as the machine actually reboots (i.e. the SSH test | ||
269 | has not failed). It is useful for scenarios where you have a simple | ||
270 | setup, typically with a single board, and where some manual | ||
271 | interaction is okay from time to time. | ||
272 | |||
273 | If you have no hardware to automatically perform power control but still | ||
274 | wish to experiment with automated hardware testing, you can use the | ||
275 | ``dialog-power-control`` script that shows a dialog prompting you to perform | ||
276 | the required power action. This script requires either KDialog or Zenity | ||
277 | to be installed. To use this script, set the | ||
278 | :term:`TEST_POWERCONTROL_CMD` | ||
279 | variable as follows:: | ||
280 | |||
281 | TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" | ||
282 | |||
283 | Serial Console Connection | ||
284 | ------------------------- | ||
285 | |||
286 | For test target classes requiring a serial console to interact with the | ||
287 | bootloader (e.g. BeagleBoneTarget and GrubTarget), | ||
288 | you need to specify a command to use to connect to the serial console of | ||
289 | the target machine by using the | ||
290 | :term:`TEST_SERIALCONTROL_CMD` | ||
291 | variable and optionally the | ||
292 | :term:`TEST_SERIALCONTROL_EXTRA_ARGS` | ||
293 | variable. | ||
294 | |||
295 | These cases could be a serial terminal program if the machine is | ||
296 | connected to a local serial port, or a ``telnet`` or ``ssh`` command | ||
297 | connecting to a remote console server. Regardless of the case, the | ||
298 | command simply needs to connect to the serial console and forward that | ||
299 | connection to standard input and output as any normal terminal program | ||
300 | does. For example, to use the picocom terminal program on serial device | ||
301 | ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:: | ||
302 | |||
303 | TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" | ||
304 | |||
305 | For local | ||
306 | devices where the serial port device disappears when the device reboots, | ||
307 | an additional "serdevtry" wrapper script is provided. To use this | ||
308 | wrapper, simply prefix the terminal command with | ||
309 | ``${COREBASE}/scripts/contrib/serdevtry``:: | ||
310 | |||
311 | TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0" | ||
312 | |||
313 | Running Tests | ||
314 | ============= | ||
315 | |||
316 | You can start the tests automatically or manually: | ||
317 | |||
318 | - *Automatically running tests:* To run the tests automatically after the | ||
319 | OpenEmbedded build system successfully creates an image, first set the | ||
320 | :term:`TESTIMAGE_AUTO` variable to "1" in your ``local.conf`` file in the | ||
321 | :term:`Build Directory`:: | ||
322 | |||
323 | TESTIMAGE_AUTO = "1" | ||
324 | |||
325 | Next, build your image. If the image successfully builds, the | ||
326 | tests run:: | ||
327 | |||
328 | bitbake core-image-sato | ||
329 | |||
330 | - *Manually running tests:* To manually run the tests, first globally | ||
331 | inherit the :ref:`ref-classes-testimage` class by editing your | ||
332 | ``local.conf`` file:: | ||
333 | |||
334 | IMAGE_CLASSES += "testimage" | ||
335 | |||
336 | Next, use BitBake to run the tests:: | ||
337 | |||
338 | bitbake -c testimage image | ||
339 | |||
340 | All test files reside in ``meta/lib/oeqa/runtime/cases`` in the | ||
341 | :term:`Source Directory`. A test name maps | ||
342 | directly to a Python module. Each test module may contain a number of | ||
343 | individual tests. Tests are usually grouped together by the area tested | ||
344 | (e.g tests for systemd reside in ``meta/lib/oeqa/runtime/cases/systemd.py``). | ||
345 | |||
346 | You can add tests to any layer provided you place them in the proper | ||
347 | area and you extend :term:`BBPATH` in | ||
348 | the ``local.conf`` file as normal. Be sure that tests reside in | ||
349 | ``layer/lib/oeqa/runtime/cases``. | ||
350 | |||
351 | .. note:: | ||
352 | |||
353 | Be sure that module names do not collide with module names used in | ||
354 | the default set of test modules in ``meta/lib/oeqa/runtime/cases``. | ||
355 | |||
356 | You can change the set of tests run by appending or overriding | ||
357 | :term:`TEST_SUITES` variable in | ||
358 | ``local.conf``. Each name in :term:`TEST_SUITES` represents a required test | ||
359 | for the image. Test modules named within :term:`TEST_SUITES` cannot be | ||
360 | skipped even if a test is not suitable for an image (e.g. running the | ||
361 | RPM tests on an image without ``rpm``). Appending "auto" to | ||
362 | :term:`TEST_SUITES` causes the build system to try to run all tests that are | ||
363 | suitable for the image (i.e. each test module may elect to skip itself). | ||
364 | |||
365 | The order you list tests in :term:`TEST_SUITES` is important and influences | ||
366 | test dependencies. Consequently, tests that depend on other tests should | ||
367 | be added after the test on which they depend. For example, since the | ||
368 | ``ssh`` test depends on the ``ping`` test, "ssh" needs to come after | ||
369 | "ping" in the list. The test class provides no re-ordering or dependency | ||
370 | handling. | ||
371 | |||
372 | .. note:: | ||
373 | |||
374 | Each module can have multiple classes with multiple test methods. | ||
375 | And, Python ``unittest`` rules apply. | ||
376 | |||
377 | Here are some things to keep in mind when running tests: | ||
378 | |||
379 | - The default tests for the image are defined as:: | ||
380 | |||
381 | DEFAULT_TEST_SUITES:pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg" | ||
382 | |||
383 | - Add your own test to the list of the by using the following:: | ||
384 | |||
385 | TEST_SUITES:append = " mytest" | ||
386 | |||
387 | - Run a specific list of tests as follows:: | ||
388 | |||
389 | TEST_SUITES = "test1 test2 test3" | ||
390 | |||
391 | Remember, order is important. Be sure to place a test that is | ||
392 | dependent on another test later in the order. | ||
393 | |||
394 | Exporting Tests | ||
395 | =============== | ||
396 | |||
397 | You can export tests so that they can run independently of the build | ||
398 | system. Exporting tests is required if you want to be able to hand the | ||
399 | test execution off to a scheduler. You can only export tests that are | ||
400 | defined in :term:`TEST_SUITES`. | ||
401 | |||
402 | If your image is already built, make sure the following are set in your | ||
403 | ``local.conf`` file:: | ||
404 | |||
405 | INHERIT += "testexport" | ||
406 | TEST_TARGET_IP = "IP-address-for-the-test-target" | ||
407 | TEST_SERVER_IP = "IP-address-for-the-test-server" | ||
408 | |||
409 | You can then export the tests with the | ||
410 | following BitBake command form:: | ||
411 | |||
412 | $ bitbake image -c testexport | ||
413 | |||
414 | Exporting the tests places them in the :term:`Build Directory` in | ||
415 | ``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` | ||
416 | variable. | ||
417 | |||
418 | You can now run the tests outside of the build environment:: | ||
419 | |||
420 | $ cd tmp/testexport/image | ||
421 | $ ./runexported.py testdata.json | ||
422 | |||
423 | Here is a complete example that shows IP addresses and uses the | ||
424 | ``core-image-sato`` image:: | ||
425 | |||
426 | INHERIT += "testexport" | ||
427 | TEST_TARGET_IP = "192.168.7.2" | ||
428 | TEST_SERVER_IP = "192.168.7.1" | ||
429 | |||
430 | Use BitBake to export the tests:: | ||
431 | |||
432 | $ bitbake core-image-sato -c testexport | ||
433 | |||
434 | Run the tests outside of | ||
435 | the build environment using the following:: | ||
436 | |||
437 | $ cd tmp/testexport/core-image-sato | ||
438 | $ ./runexported.py testdata.json | ||
439 | |||
440 | Writing New Tests | ||
441 | ================= | ||
442 | |||
443 | As mentioned previously, all new test files need to be in the proper | ||
444 | place for the build system to find them. New tests for additional | ||
445 | functionality outside of the core should be added to the layer that adds | ||
446 | the functionality, in ``layer/lib/oeqa/runtime/cases`` (as long as | ||
447 | :term:`BBPATH` is extended in the | ||
448 | layer's ``layer.conf`` file as normal). Just remember the following: | ||
449 | |||
450 | - Filenames need to map directly to test (module) names. | ||
451 | |||
452 | - Do not use module names that collide with existing core tests. | ||
453 | |||
454 | - Minimally, an empty ``__init__.py`` file must be present in the runtime | ||
455 | directory. | ||
456 | |||
457 | To create a new test, start by copying an existing module (e.g. | ||
458 | ``oe_syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use | ||
459 | code from ``meta/lib/oeqa/utils``, which are helper classes. | ||
460 | |||
461 | .. note:: | ||
462 | |||
463 | Structure shell commands such that you rely on them and they return a | ||
464 | single code for success. Be aware that sometimes you will need to | ||
465 | parse the output. See the ``df.py`` and ``date.py`` modules for examples. | ||
466 | |||
467 | You will notice that all test classes inherit ``oeRuntimeTest``, which | ||
468 | is found in ``meta/lib/oetest.py``. This base class offers some helper | ||
469 | attributes, which are described in the following sections: | ||
470 | |||
471 | Class Methods | ||
472 | ------------- | ||
473 | |||
474 | Class methods are as follows: | ||
475 | |||
476 | - *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed | ||
477 | package list of the image, which is based on the manifest file that | ||
478 | is generated during the :ref:`ref-tasks-rootfs` task. | ||
479 | |||
480 | - *hasFeature(feature):* Returns "True" if the feature is in | ||
481 | :term:`IMAGE_FEATURES` or | ||
482 | :term:`DISTRO_FEATURES`. | ||
483 | |||
484 | Class Attributes | ||
485 | ---------------- | ||
486 | |||
487 | Class attributes are as follows: | ||
488 | |||
489 | - *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image. | ||
490 | Otherwise, ``pscmd`` equals "ps" (busybox). | ||
491 | |||
492 | - *tc:* The called test context, which gives access to the | ||
493 | following attributes: | ||
494 | |||
495 | - *d:* The BitBake datastore, which allows you to use stuff such | ||
496 | as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``. | ||
497 | |||
498 | - *testslist and testsrequired:* Used internally. The tests | ||
499 | do not need these. | ||
500 | |||
501 | - *filesdir:* The absolute path to | ||
502 | ``meta/lib/oeqa/runtime/files``, which contains helper files for | ||
503 | tests meant for copying on the target such as small files written | ||
504 | in C for compilation. | ||
505 | |||
506 | - *target:* The target controller object used to deploy and | ||
507 | start an image on a particular target (e.g. Qemu, SimpleRemote, | ||
508 | and SystemdbootTarget). Tests usually use the following: | ||
509 | |||
510 | - *ip:* The target's IP address. | ||
511 | |||
512 | - *server_ip:* The host's IP address, which is usually used | ||
513 | by the DNF test suite. | ||
514 | |||
515 | - *run(cmd, timeout=None):* The single, most used method. | ||
516 | This command is a wrapper for: ``ssh root@host "cmd"``. The | ||
517 | command returns a tuple: (status, output), which are what their | ||
518 | names imply - the return code of "cmd" and whatever output it | ||
519 | produces. The optional timeout argument represents the number | ||
520 | of seconds the test should wait for "cmd" to return. If the | ||
521 | argument is "None", the test uses the default instance's | ||
522 | timeout period, which is 300 seconds. If the argument is "0", | ||
523 | the test runs until the command returns. | ||
524 | |||
525 | - *copy_to(localpath, remotepath):* | ||
526 | ``scp localpath root@ip:remotepath``. | ||
527 | |||
528 | - *copy_from(remotepath, localpath):* | ||
529 | ``scp root@host:remotepath localpath``. | ||
530 | |||
531 | Instance Attributes | ||
532 | ------------------- | ||
533 | |||
534 | There is a single instance attribute, which is ``target``. The ``target`` | ||
535 | instance attribute is identical to the class attribute of the same name, | ||
536 | which is described in the previous section. This attribute exists as | ||
537 | both an instance and class attribute so tests can use | ||
538 | ``self.target.run(cmd)`` in instance methods instead of | ||
539 | ``oeRuntimeTest.tc.target.run(cmd)``. | ||
540 | |||
541 | Installing Packages in the DUT Without the Package Manager | ||
542 | ========================================================== | ||
543 | |||
544 | When a test requires a package built by BitBake, it is possible to | ||
545 | install that package. Installing the package does not require a package | ||
546 | manager be installed in the device under test (DUT). It does, however, | ||
547 | require an SSH connection and the target must be using the | ||
548 | ``sshcontrol`` class. | ||
549 | |||
550 | .. note:: | ||
551 | |||
552 | This method uses ``scp`` to copy files from the host to the target, which | ||
553 | causes permissions and special attributes to be lost. | ||
554 | |||
555 | A JSON file is used to define the packages needed by a test. This file | ||
556 | must be in the same path as the file used to define the tests. | ||
557 | Furthermore, the filename must map directly to the test module name with | ||
558 | a ``.json`` extension. | ||
559 | |||
560 | The JSON file must include an object with the test name as keys of an | ||
561 | object or an array. This object (or array of objects) uses the following | ||
562 | data: | ||
563 | |||
564 | - "pkg" --- a mandatory string that is the name of the package to be | ||
565 | installed. | ||
566 | |||
567 | - "rm" --- an optional boolean, which defaults to "false", that specifies | ||
568 | to remove the package after the test. | ||
569 | |||
570 | - "extract" --- an optional boolean, which defaults to "false", that | ||
571 | specifies if the package must be extracted from the package format. | ||
572 | When set to "true", the package is not automatically installed into | ||
573 | the DUT. | ||
574 | |||
575 | Here is an example JSON file that handles test "foo" installing | ||
576 | package "bar" and test "foobar" installing packages "foo" and "bar". | ||
577 | Once the test is complete, the packages are removed from the DUT:: | ||
578 | |||
579 | { | ||
580 | "foo": { | ||
581 | "pkg": "bar" | ||
582 | }, | ||
583 | "foobar": [ | ||
584 | { | ||
585 | "pkg": "foo", | ||
586 | "rm": true | ||
587 | }, | ||
588 | { | ||
589 | "pkg": "bar", | ||
590 | "rm": true | ||
591 | } | ||
592 | ] | ||
593 | } | ||
594 | |||
diff --git a/documentation/dev-manual/sbom.rst b/documentation/dev-manual/sbom.rst index b72bad1554..7c4b5804fb 100644 --- a/documentation/dev-manual/sbom.rst +++ b/documentation/dev-manual/sbom.rst | |||
@@ -30,16 +30,9 @@ To make this happen, you must inherit the | |||
30 | 30 | ||
31 | INHERIT += "create-spdx" | 31 | INHERIT += "create-spdx" |
32 | 32 | ||
33 | Upon building an image, you will then get: | 33 | Upon building an image, you will then get the compressed archive |
34 | 34 | ``IMAGE-MACHINE.spdx.tar.zst`` contains the index and the files for the single | |
35 | - :term:`SPDX` output in JSON format as an ``IMAGE-MACHINE.spdx.json`` file in | 35 | recipes. |
36 | ``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`. | ||
37 | |||
38 | - This toplevel file is accompanied by an ``IMAGE-MACHINE.spdx.index.json`` | ||
39 | containing an index of JSON :term:`SPDX` files for individual recipes. | ||
40 | |||
41 | - The compressed archive ``IMAGE-MACHINE.spdx.tar.zst`` contains the index | ||
42 | and the files for the single recipes. | ||
43 | 36 | ||
44 | The :ref:`ref-classes-create-spdx` class offers options to include | 37 | The :ref:`ref-classes-create-spdx` class offers options to include |
45 | more information in the output :term:`SPDX` data: | 38 | more information in the output :term:`SPDX` data: |
@@ -56,7 +49,7 @@ more information in the output :term:`SPDX` data: | |||
56 | 49 | ||
57 | Though the toplevel :term:`SPDX` output is available in | 50 | Though the toplevel :term:`SPDX` output is available in |
58 | ``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary | 51 | ``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary |
59 | generated files are available in ``tmp/deploy/spdx/MACHINE`` too, such as: | 52 | generated files are available in ``tmp/deploy/spdx`` too, such as: |
60 | 53 | ||
61 | - The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst`` | 54 | - The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst`` |
62 | archive. | 55 | archive. |
diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst index 386e5f5d29..f4da61b53f 100644 --- a/documentation/dev-manual/start.rst +++ b/documentation/dev-manual/start.rst | |||
@@ -615,7 +615,7 @@ Accessing Source Archives | |||
615 | The Yocto Project also provides source archives of its releases, which | 615 | The Yocto Project also provides source archives of its releases, which |
616 | are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory | 616 | are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory |
617 | containing the release you wish to use, for example | 617 | containing the release you wish to use, for example |
618 | :yocto_dl:`yocto-&DISTRO; </releases/yocto/yocto-&DISTRO;/>`. | 618 | :yocto_dl:`&DISTRO_REL_LATEST_TAG; </releases/yocto/&DISTRO_REL_LATEST_TAG;/>`. |
619 | 619 | ||
620 | You will find there source archives of individual components (if you wish | 620 | You will find there source archives of individual components (if you wish |
621 | to use them individually), and of the corresponding Poky release bundling | 621 | to use them individually), and of the corresponding Poky release bundling |
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst index 4fac78bdfb..a38fd7837c 100644 --- a/documentation/dev-manual/upgrading-recipes.rst +++ b/documentation/dev-manual/upgrading-recipes.rst | |||
@@ -333,7 +333,7 @@ Manually Upgrading a Recipe | |||
333 | 333 | ||
334 | If for some reason you choose not to upgrade recipes using | 334 | If for some reason you choose not to upgrade recipes using |
335 | :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or | 335 | :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or |
336 | by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, | 336 | by :ref:`dev-manual/upgrading-recipes:Using ``devtool upgrade```, |
337 | you can manually edit the recipe files to upgrade the versions. | 337 | you can manually edit the recipe files to upgrade the versions. |
338 | 338 | ||
339 | .. note:: | 339 | .. note:: |
diff --git a/documentation/dev-manual/vulnerabilities.rst b/documentation/dev-manual/vulnerabilities.rst index 1bc2a85929..f5f9fe3a0c 100644 --- a/documentation/dev-manual/vulnerabilities.rst +++ b/documentation/dev-manual/vulnerabilities.rst | |||
@@ -62,37 +62,77 @@ found in ``build/tmp/deploy/cve``. | |||
62 | 62 | ||
63 | For example the CVE check report for the ``flex-native`` recipe looks like:: | 63 | For example the CVE check report for the ``flex-native`` recipe looks like:: |
64 | 64 | ||
65 | $ cat poky/build/tmp/deploy/cve/flex-native | 65 | $ cat ./tmp/deploy/cve/flex-native_cve.json |
66 | LAYER: meta | 66 | { |
67 | PACKAGE NAME: flex-native | 67 | "version": "1", |
68 | PACKAGE VERSION: 2.6.4 | 68 | "package": [ |
69 | CVE: CVE-2016-6354 | 69 | { |
70 | CVE STATUS: Patched | 70 | "name": "flex-native", |
71 | CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read. | 71 | "layer": "meta", |
72 | CVSS v2 BASE SCORE: 7.5 | 72 | "version": "2.6.4", |
73 | CVSS v3 BASE SCORE: 9.8 | 73 | "products": [ |
74 | VECTOR: NETWORK | 74 | { |
75 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354 | 75 | "product": "flex", |
76 | 76 | "cvesInRecord": "No" | |
77 | LAYER: meta | 77 | }, |
78 | PACKAGE NAME: flex-native | 78 | { |
79 | PACKAGE VERSION: 2.6.4 | 79 | "product": "flex", |
80 | CVE: CVE-2019-6293 | 80 | "cvesInRecord": "Yes" |
81 | CVE STATUS: Ignored | 81 | } |
82 | CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service. | 82 | ], |
83 | CVSS v2 BASE SCORE: 4.3 | 83 | "issue": [ |
84 | CVSS v3 BASE SCORE: 5.5 | 84 | { |
85 | VECTOR: NETWORK | 85 | "id": "CVE-2006-0459", |
86 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293 | 86 | "status": "Patched", |
87 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2006-0459", | ||
88 | "summary": "flex.skl in Will Estes and John Millaway Fast Lexical Analyzer Generator (flex) before 2.5.33 does not allocate enough memory for grammars containing (1) REJECT statements or (2) trailing context rules, which causes flex to generate code that contains a buffer overflow that might allow context-dependent attackers to execute arbitrary code.", | ||
89 | "scorev2": "7.5", | ||
90 | "scorev3": "0.0", | ||
91 | "scorev4": "0.0", | ||
92 | "modified": "2024-11-21T00:06Z", | ||
93 | "vector": "NETWORK", | ||
94 | "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P", | ||
95 | "detail": "version-not-in-range" | ||
96 | }, | ||
97 | { | ||
98 | "id": "CVE-2016-6354", | ||
99 | "status": "Patched", | ||
100 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2016-6354", | ||
101 | "summary": "Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.", | ||
102 | "scorev2": "7.5", | ||
103 | "scorev3": "9.8", | ||
104 | "scorev4": "0.0", | ||
105 | "modified": "2024-11-21T02:55Z", | ||
106 | "vector": "NETWORK", | ||
107 | "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P", | ||
108 | "detail": "version-not-in-range" | ||
109 | }, | ||
110 | { | ||
111 | "id": "CVE-2019-6293", | ||
112 | "status": "Ignored", | ||
113 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2019-6293", | ||
114 | "summary": "An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.", | ||
115 | "scorev2": "4.3", | ||
116 | "scorev3": "5.5", | ||
117 | "scorev4": "0.0", | ||
118 | "modified": "2024-11-21T04:46Z", | ||
119 | "vector": "NETWORK", | ||
120 | "vectorString": "AV:N/AC:M/Au:N/C:N/I:N/A:P", | ||
121 | "detail": "upstream-wontfix", | ||
122 | "description": "there is stack exhaustion but no bug and it is building the parser, not running it, effectively similar to a compiler ICE. Upstream no plans to address this." | ||
123 | } | ||
124 | ] | ||
125 | } | ||
126 | ] | ||
127 | } | ||
87 | 128 | ||
88 | For images, a summary of all recipes included in the image and their CVEs is also | 129 | For images, a summary of all recipes included in the image and their CVEs is also |
89 | generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found | 130 | generated in the JSON format. These ``.json`` reports can be found |
90 | in the ``tmp/deploy/images`` directory for each compiled image. | 131 | in the ``tmp/deploy/images`` directory for each compiled image. |
91 | 132 | ||
92 | At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: | 133 | At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: |
93 | 134 | ||
94 | WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log | 135 | WARNING: qemu-native-9.2.0-r0 do_cve_check: Found unpatched CVE (CVE-2023-1386) |
95 | WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log | ||
96 | 136 | ||
97 | It is also possible to check the CVE status of individual packages as follows:: | 137 | It is also possible to check the CVE status of individual packages as follows:: |
98 | 138 | ||
@@ -111,10 +151,10 @@ upstream `NIST CVE database <https://nvd.nist.gov/>`__. | |||
111 | 151 | ||
112 | The variable supports using vendor and product names like this:: | 152 | The variable supports using vendor and product names like this:: |
113 | 153 | ||
114 | CVE_PRODUCT = "flex_project:flex" | 154 | CVE_PRODUCT = "flex_project:flex westes:flex" |
115 | 155 | ||
116 | In this example the vendor name used in the CVE database is ``flex_project`` and the | 156 | In this example we have two possible vendors names, ``flex_project`` and ``westes``, |
117 | product is ``flex``. With this setting the ``flex`` recipe only maps to this specific | 157 | with the product name ``flex``. With this setting the ``flex`` recipe only maps to this specific |
118 | product and not products from other vendors with same name ``flex``. | 158 | product and not products from other vendors with same name ``flex``. |
119 | 159 | ||
120 | Similarly, when the recipe version :term:`PV` is not compatible with software versions used by | 160 | Similarly, when the recipe version :term:`PV` is not compatible with software versions used by |
diff --git a/documentation/dev-manual/wic.rst b/documentation/dev-manual/wic.rst index a3880f3a1c..fced0e170c 100644 --- a/documentation/dev-manual/wic.rst +++ b/documentation/dev-manual/wic.rst | |||
@@ -513,7 +513,7 @@ or :: | |||
513 | 513 | ||
514 | For more information on how to use the ``bmaptool`` | 514 | For more information on how to use the ``bmaptool`` |
515 | to flash a device with an image, see the | 515 | to flash a device with an image, see the |
516 | ":ref:`dev-manual/bmaptool:flashing images using \`\`bmaptool\`\``" | 516 | ":ref:`dev-manual/bmaptool:flashing images using \`bmaptool\``" |
517 | section. | 517 | section. |
518 | 518 | ||
519 | Using a Modified Kickstart File | 519 | Using a Modified Kickstart File |
@@ -721,7 +721,7 @@ the existing kernel, and then inserts a new kernel: | |||
721 | 721 | ||
722 | Once the new kernel is added back into the image, you can use the | 722 | Once the new kernel is added back into the image, you can use the |
723 | ``dd`` command or :ref:`bmaptool | 723 | ``dd`` command or :ref:`bmaptool |
724 | <dev-manual/bmaptool:flashing images using \`\`bmaptool\`\`>` | 724 | <dev-manual/bmaptool:flashing images using \`bmaptool\`>` commands |
725 | to flash your wic image onto an SD card or USB stick and test your | 725 | to flash your wic image onto an SD card or USB stick and test your |
726 | target. | 726 | target. |
727 | 727 | ||