summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual')
-rw-r--r--documentation/dev-manual/bblock.rst129
-rw-r--r--documentation/dev-manual/bmaptool.rst29
-rw-r--r--documentation/dev-manual/build-quality.rst2
-rw-r--r--documentation/dev-manual/building.rst246
-rw-r--r--documentation/dev-manual/custom-distribution.rst34
-rw-r--r--documentation/dev-manual/customizing-images.rst20
-rw-r--r--documentation/dev-manual/debugging.rst18
-rw-r--r--documentation/dev-manual/devtool.rst1322
-rw-r--r--documentation/dev-manual/disk-space.rst4
-rw-r--r--documentation/dev-manual/efficiently-fetching-sources.rst4
-rw-r--r--documentation/dev-manual/external-scm.rst9
-rw-r--r--documentation/dev-manual/figures/devtool-add-flow.pngbin0 -> 181699 bytes
-rw-r--r--documentation/dev-manual/figures/devtool-modify-flow.pngbin0 -> 171676 bytes
-rw-r--r--documentation/dev-manual/figures/devtool-upgrade-flow.pngbin0 -> 138917 bytes
-rw-r--r--documentation/dev-manual/index.rst4
-rw-r--r--documentation/dev-manual/init-manager.rst4
-rw-r--r--documentation/dev-manual/layers.rst144
-rw-r--r--documentation/dev-manual/libraries.rst32
-rw-r--r--documentation/dev-manual/licenses.rst4
-rw-r--r--documentation/dev-manual/multiconfig.rst312
-rw-r--r--documentation/dev-manual/new-recipe.rst65
-rw-r--r--documentation/dev-manual/packages.rst143
-rw-r--r--documentation/dev-manual/prebuilt-libraries.rst6
-rw-r--r--documentation/dev-manual/qemu.rst16
-rw-r--r--documentation/dev-manual/runtime-testing.rst594
-rw-r--r--documentation/dev-manual/sbom.rst17
-rw-r--r--documentation/dev-manual/securing-images.rst12
-rw-r--r--documentation/dev-manual/start.rst33
-rw-r--r--documentation/dev-manual/temporary-source-code.rst15
-rw-r--r--documentation/dev-manual/upgrading-recipes.rst4
-rw-r--r--documentation/dev-manual/vulnerabilities.rst102
-rw-r--r--documentation/dev-manual/wic.rst60
32 files changed, 2285 insertions, 1099 deletions
diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
new file mode 100644
index 0000000000..605bb75655
--- /dev/null
+++ b/documentation/dev-manual/bblock.rst
@@ -0,0 +1,129 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Locking and Unlocking Recipes Using ``bblock``
4**********************************************
5
6By design, the OpenEmbedded build system builds everything from scratch
7unless BitBake determines that specific tasks do not require rebuilding.
8At startup, it computes a signature for all tasks, based on the task's input.
9Then, it compares these signatures with the ones from the sstate cache (if they
10exist). Any changes cause the task to rerun.
11
12During development, changes might trigger BitBake to rebuild certain
13recipes, even when we know they do not require rebuilding at that stage.
14For example, modifying a recipe can lead to rebuilding its native
15counterpart, which might prove unnecessary. Editing the ``python3`` recipe,
16for instance, can prompt BitBake to rebuild ``python3-native`` along with any
17recipes that depend on it.
18
19To prevent this, use ``bblock`` to lock specific tasks or recipes to
20specific signatures, forcing BitBake to use the sstate cache for them.
21
22.. warning::
23
24 Use ``bblock`` only during the development phase.
25
26 Forcing BitBake to use the sstate cache, regardless of input changes, means
27 the recipe metadata no longer directly reflect the output. Use this feature
28 with caution. If you do not understand why signatures change, see the section
29 on :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`.
30
31
32Locking tasks and recipes
33-------------------------
34
35To lock a recipe, use::
36
37 $ bblock recipe
38
39You can also use a space-separated list of recipes to lock multiple recipes::
40
41 $ bblock recipe1 recipe2
42
43Locking a recipe means locking all tasks of the recipe. If you need to
44lock only particular tasks, use the `-t` option with a comma-separated
45list of tasks::
46
47 $ bblock -t task1,task2 recipe
48
49
50Unlocking tasks and recipes
51---------------------------
52
53To unlock a recipe, use the ``-r`` option::
54
55 $ bblock -r recipe
56
57You can also use a space-separated list of recipes to unlock multiple recipes::
58
59 $ bblock -r recipe1 recipe2
60
61Unlocking a recipe means unlocking all tasks of the recipe. If you need to
62unlock only particular tasks use the ``-t`` option with a comma-separated
63list of tasks::
64
65 $ bblock -r -t task1,task2 recipe
66
67To unlock all recipes, do not specify any recipe::
68
69 $ bblock -r
70
71
72Configuration file
73------------------
74
75``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
76included by default in :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`.
77
78To dump the file, use the ``-d`` option::
79
80 $ bblock -d
81
82
83Locking mechanism
84-----------------
85
86``bblock`` computes the signature(s) of the task(s) and sets the 3 following
87variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
88and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
89
90In particular, ``bblock`` sets::
91
92 SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
93 SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
94
95 SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
96
97This produces architecture specific locks and reminds user that some tasks
98have locked signatures.
99
100Example
101-------
102
103When working on the ``python3`` recipe, we can lock ``python3-native`` with
104the following::
105
106 $ bblock python3-native
107 $ bblock -d
108 # Generated by bblock
109 SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
110 SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
111
112 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
113 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
114 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
115 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
116 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
117 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
118 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
119 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
120 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
121 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
122 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
123 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
124 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
125 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
126 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
127 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
128 SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
129
diff --git a/documentation/dev-manual/bmaptool.rst b/documentation/dev-manual/bmaptool.rst
index f6f0e6afaf..87162a49c9 100644
--- a/documentation/dev-manual/bmaptool.rst
+++ b/documentation/dev-manual/bmaptool.rst
@@ -1,13 +1,13 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2 2
3Flashing Images Using ``bmaptool`` 3Flashing Images Using `bmaptool`
4********************************** 4********************************
5 5
6A fast and easy way to flash an image to a bootable device is to use 6A fast and easy way to flash an image to a bootable device is to use
7bmaptool, which is integrated into the OpenEmbedded build system. 7`bmaptool`, which is integrated into the OpenEmbedded build system.
8bmaptool is a generic tool that creates a file's block map (bmap) and 8`bmaptool` is a generic tool that creates a file's block map (bmap) and
9then uses that map to copy the file. As compared to traditional tools 9then uses that map to copy the file. As compared to traditional tools
10such as dd or cp, bmaptool can copy (or flash) large files like raw 10such as `dd` or `cp`, `bmaptool` can copy (or flash) large files like raw
11system image files much faster. 11system image files much faster.
12 12
13.. note:: 13.. note::
@@ -20,13 +20,13 @@ system image files much faster.
20 $ sudo apt install bmap-tools 20 $ sudo apt install bmap-tools
21 21
22 - If you are unable to install the ``bmap-tools`` package, you will 22 - If you are unable to install the ``bmap-tools`` package, you will
23 need to build bmaptool before using it. Use the following command:: 23 need to build `bmaptool` before using it. Use the following command::
24 24
25 $ bitbake bmaptool-native 25 $ bitbake bmaptool-native -caddto_recipe_sysroot
26 26
27Following, is an example that shows how to flash a Wic image. Realize 27Following, is an example that shows how to flash a Wic image. Realize
28that while this example uses a Wic image, you can use bmaptool to flash 28that while this example uses a Wic image, you can use `bmaptool` to flash
29any type of image. Use these steps to flash an image using bmaptool: 29any type of image. Use these steps to flash an image using `bmaptool`:
30 30
31#. *Update your local.conf File:* You need to have the following set 31#. *Update your local.conf File:* You need to have the following set
32 in your ``local.conf`` file before building your image:: 32 in your ``local.conf`` file before building your image::
@@ -39,18 +39,17 @@ any type of image. Use these steps to flash an image using bmaptool:
39 39
40 $ bitbake image 40 $ bitbake image
41 41
42#. *Flash the Device:* Flash the device with the image by using bmaptool 42#. *Flash the Device:* Flash the device with the image by using `bmaptool`
43 depending on your particular setup. The following commands assume the 43 depending on your particular setup. The following commands assume the
44 image resides in the :term:`Build Directory`'s ``deploy/images/`` area: 44 image resides in the :term:`Build Directory`'s ``deploy/images/`` area:
45 45
46 - If you have write access to the media, use this command form:: 46 - If you installed the package for `bmaptool`, you can directly run::
47 47
48 $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX 48 $ sudo bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
49 49
50 - If you do not have write access to the media, set your permissions 50 - Otherwise, if you built `bmaptool` with BitBake, run::
51 first and then use the same command form::
52 51
53 $ sudo chmod 666 /dev/sdX 52 $ sudo chmod a+w /dev/sdX # get write access to the media, needed only once after booting
54 $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX 53 $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
55 54
56For help on the ``bmaptool`` command, use the following command:: 55For help on the ``bmaptool`` command, use the following command::
diff --git a/documentation/dev-manual/build-quality.rst b/documentation/dev-manual/build-quality.rst
index 713ea3a48e..fbe5fb6f0b 100644
--- a/documentation/dev-manual/build-quality.rst
+++ b/documentation/dev-manual/build-quality.rst
@@ -236,7 +236,7 @@ Here is an example of ``image-info.txt``:
236 DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be 236 DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be
237 USER_CLASSES = buildstats 237 USER_CLASSES = buildstats
238 IMAGE_CLASSES = qemuboot qemuboot license_image 238 IMAGE_CLASSES = qemuboot qemuboot license_image
239 IMAGE_FEATURES = debug-tweaks 239 IMAGE_FEATURES = allow-empty-password empty-root-password allow-root-login post-install-logging
240 IMAGE_LINGUAS = 240 IMAGE_LINGUAS =
241 IMAGE_INSTALL = packagegroup-core-boot speex speexdsp 241 IMAGE_INSTALL = packagegroup-core-boot speex speexdsp
242 BAD_RECOMMENDATIONS = 242 BAD_RECOMMENDATIONS =
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst
index fe502690dd..32c7aa5da0 100644
--- a/documentation/dev-manual/building.rst
+++ b/documentation/dev-manual/building.rst
@@ -48,7 +48,7 @@ The following figure and list overviews the build process:
48 :width: 100% 48 :width: 100%
49 49
50#. *Set up Your Host Development System to Support Development Using the 50#. *Set up Your Host Development System to Support Development Using the
51 Yocto Project*: See the ":doc:`start`" section for options on how to get a 51 Yocto Project*: See the ":doc:`/dev-manual/start`" section for options on how to get a
52 build host ready to use the Yocto Project. 52 build host ready to use the Yocto Project.
53 53
54#. *Initialize the Build Environment:* Initialize the build environment 54#. *Initialize the Build Environment:* Initialize the build environment
@@ -113,160 +113,8 @@ The following figure and list overviews the build process:
113Building Images for Multiple Targets Using Multiple Configurations 113Building Images for Multiple Targets Using Multiple Configurations
114================================================================== 114==================================================================
115 115
116You can use a single ``bitbake`` command to build multiple images or 116See the :doc:`/dev-manual/multiconfig` section of the Yocto Project Development Tasks
117packages for different targets where each image or package requires a 117Manual.
118different configuration (multiple configuration builds). The builds, in
119this scenario, are sometimes referred to as "multiconfigs", and this
120section uses that term throughout.
121
122This section describes how to set up for multiple configuration builds
123and how to account for cross-build dependencies between the
124multiconfigs.
125
126Setting Up and Running a Multiple Configuration Build
127-----------------------------------------------------
128
129To accomplish a multiple configuration build, you must define each
130target's configuration separately using a parallel configuration file in
131the :term:`Build Directory` or configuration directory within a layer, and you
132must follow a required file hierarchy. Additionally, you must enable the
133multiple configuration builds in your ``local.conf`` file.
134
135Follow these steps to set up and execute multiple configuration builds:
136
137- *Create Separate Configuration Files*: You need to create a single
138 configuration file for each build target (each multiconfig).
139 The configuration definitions are implementation dependent but often
140 each configuration file will define the machine and the
141 temporary directory BitBake uses for the build. Whether the same
142 temporary directory (:term:`TMPDIR`) can be shared will depend on what is
143 similar and what is different between the configurations. Multiple MACHINE
144 targets can share the same (:term:`TMPDIR`) as long as the rest of the
145 configuration is the same, multiple :term:`DISTRO` settings would need separate
146 (:term:`TMPDIR`) directories.
147
148 For example, consider a scenario with two different multiconfigs for the same
149 :term:`MACHINE`: "qemux86" built
150 for two distributions such as "poky" and "poky-lsb". In this case,
151 you would need to use the different :term:`TMPDIR`.
152
153 Here is an example showing the minimal statements needed in a
154 configuration file for a "qemux86" target whose temporary build
155 directory is ``tmpmultix86``::
156
157 MACHINE = "qemux86"
158 TMPDIR = "${TOPDIR}/tmpmultix86"
159
160 The location for these multiconfig configuration files is specific.
161 They must reside in the current :term:`Build Directory` in a sub-directory of
162 ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory
163 under a directory named ``multiconfig``. Here is an example that defines
164 two configuration files for the "x86" and "arm" multiconfigs:
165
166 .. image:: figures/multiconfig_files.png
167 :align: center
168 :width: 50%
169
170 The usual :term:`BBPATH` search path is used to locate multiconfig files in
171 a similar way to other conf files.
172
173- *Add the BitBake Multi-configuration Variable to the Local
174 Configuration File*: Use the
175 :term:`BBMULTICONFIG`
176 variable in your ``conf/local.conf`` configuration file to specify
177 each multiconfig. Continuing with the example from the previous
178 figure, the :term:`BBMULTICONFIG` variable needs to enable two
179 multiconfigs: "x86" and "arm" by specifying each configuration file::
180
181 BBMULTICONFIG = "x86 arm"
182
183 .. note::
184
185 A "default" configuration already exists by definition. This
186 configuration is named: "" (i.e. empty string) and is defined by
187 the variables coming from your ``local.conf``
188 file. Consequently, the previous example actually adds two
189 additional configurations to your build: "arm" and "x86" along
190 with "".
191
192- *Launch BitBake*: Use the following BitBake command form to launch
193 the multiple configuration build::
194
195 $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ]
196
197 For the example in this section, the following command applies::
198
199 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
200
201 The previous BitBake command builds a ``core-image-minimal`` image
202 that is configured through the ``x86.conf`` configuration file, a
203 ``core-image-sato`` image that is configured through the ``arm.conf``
204 configuration file and a ``core-image-base`` that is configured
205 through your ``local.conf`` configuration file.
206
207.. note::
208
209 Support for multiple configuration builds in the Yocto Project &DISTRO;
210 (&DISTRO_NAME;) Release does not include Shared State (sstate)
211 optimizations. Consequently, if a build uses the same object twice
212 in, for example, two different :term:`TMPDIR`
213 directories, the build either loads from an existing sstate cache for
214 that build at the start or builds the object fresh.
215
216Enabling Multiple Configuration Build Dependencies
217--------------------------------------------------
218
219Sometimes dependencies can exist between targets (multiconfigs) in a
220multiple configuration build. For example, suppose that in order to
221build a ``core-image-sato`` image for an "x86" multiconfig, the root
222filesystem of an "arm" multiconfig must exist. This dependency is
223essentially that the
224:ref:`ref-tasks-image` task in the
225``core-image-sato`` recipe depends on the completion of the
226:ref:`ref-tasks-rootfs` task of the
227``core-image-minimal`` recipe.
228
229To enable dependencies in a multiple configuration build, you must
230declare the dependencies in the recipe using the following statement
231form::
232
233 task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
234
235To better show how to use this statement, consider the example scenario
236from the first paragraph of this section. The following statement needs
237to be added to the recipe that builds the ``core-image-sato`` image::
238
239 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
240
241In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The
242task on which the :ref:`ref-tasks-image` task in the recipe depends is the
243:ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated
244with the "arm" multiconfig.
245
246Once you set up this dependency, you can build the "x86" multiconfig
247using a BitBake command as follows::
248
249 $ bitbake mc:x86:core-image-sato
250
251This command executes all the tasks needed to create the
252``core-image-sato`` image for the "x86" multiconfig. Because of the
253dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the
254"arm" multiconfig build.
255
256Having a recipe depend on the root filesystem of another build might not
257seem that useful. Consider this change to the statement in the
258``core-image-sato`` recipe::
259
260 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
261
262In this case, BitBake must
263create the ``core-image-minimal`` image for the "arm" build since the
264"x86" build depends on it.
265
266Because "x86" and "arm" are enabled for multiple configuration builds
267and have separate configuration files, BitBake places the artifacts for
268each build in the respective temporary build directories (i.e.
269:term:`TMPDIR`).
270 118
271Building an Initial RAM Filesystem (Initramfs) Image 119Building an Initial RAM Filesystem (Initramfs) Image
272==================================================== 120====================================================
@@ -280,7 +128,9 @@ Follow these steps to create an :term:`Initramfs` image:
280#. *Create the Initramfs Image Recipe:* You can reference the 128#. *Create the Initramfs Image Recipe:* You can reference the
281 ``core-image-minimal-initramfs.bb`` recipe found in the 129 ``core-image-minimal-initramfs.bb`` recipe found in the
282 ``meta/recipes-core`` directory of the :term:`Source Directory` 130 ``meta/recipes-core`` directory of the :term:`Source Directory`
283 as an example from which to work. 131 as an example from which to work. The ``core-image-minimal-initramfs`` recipe
132 is based on the :ref:`initramfs-framework <dev-manual/building:Customizing an
133 Initramfs using \`\`initramfs-framework\`\`>` recipe described below.
284 134
285#. *Decide if You Need to Bundle the Initramfs Image Into the Kernel 135#. *Decide if You Need to Bundle the Initramfs Image Into the Kernel
286 Image:* If you want the :term:`Initramfs` image that is built to be bundled 136 Image:* If you want the :term:`Initramfs` image that is built to be bundled
@@ -308,6 +158,86 @@ Follow these steps to create an :term:`Initramfs` image:
308 and bundled with the kernel image if you used the 158 and bundled with the kernel image if you used the
309 :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. 159 :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier.
310 160
161Customizing an Initramfs using ``initramfs-framework``
162------------------------------------------------------
163
164The ``core-image-minimal-initramfs.bb`` recipe found in
165:oe_git:`meta/recipes-core/images
166</openembedded-core/tree/meta/recipes-core/images>` uses the
167:oe_git:`initramfs-framework_1.0.bb
168</openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb>`
169recipe as its base component. The goal of the ``initramfs-framework`` recipe is
170to provide the building blocks to build a customized :term:`Initramfs`.
171
172The ``initramfs-framework`` recipe relies on shell initialization scripts
173defined in :oe_git:`meta/recipes-core/initrdscripts/initramfs-framework
174</openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework>`. Since some of
175these scripts do not apply for all use cases, the ``initramfs-framework`` recipe
176defines different packages:
177
178- ``initramfs-framework-base``: this package installs the basic components of
179 an :term:`Initramfs`, such as the ``init`` script or the ``/dev/console``
180 character special file. As this package is a runtime dependency of all
181 modules listed below, it is automatically pulled in when one of the modules
182 is installed in the image.
183- ``initramfs-module-exec``: support for execution of applications.
184- ``initramfs-module-mdev``: support for `mdev
185 <https://wiki.gentoo.org/wiki/Mdev>`__.
186- ``initramfs-module-udev``: support for :wikipedia:`Udev <Udev>`.
187- ``initramfs-module-e2fs``: support for :wikipedia:`ext4/ext3/ext2
188 <Extended_file_system>` filesystems.
189- ``initramfs-module-nfsrootfs``: support for locating and mounting the root
190 partition via :wikipedia:`NFS <Network_File_System>`.
191- ``initramfs-module-rootfs``: support for locating and mounting the root
192 partition.
193- ``initramfs-module-debug``: dynamic debug support.
194- ``initramfs-module-lvm``: :wikipedia:`LVM <Logical_volume_management>` rootfs support.
195- ``initramfs-module-overlayroot``: support for mounting a read-write overlay
196 on top of a read-only root filesystem.
197
198In addition to the packages defined by the ``initramfs-framework`` recipe
199itself, the following packages are defined by the recipes present in
200:oe_git:`meta/recipes-core/initrdscripts </openembedded-core/tree/meta/recipes-core/initrdscripts>`:
201
202- ``initramfs-module-install``: module to create and install a partition layout
203 on a selected block device.
204- ``initramfs-module-install-efi``: module to create and install an EFI
205 partition layout on a selected block device.
206- ``initramfs-module-setup-live``: module to start a shell in the
207 :term:`Initramfs` if ``root=/dev/ram0`` in passed in the `Kernel command-line
208 <https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html>`__
209 or the ``root=`` parameter was not passed.
210
211To customize the :term:`Initramfs`, you can add or remove packages listed
212earlier from the :term:`PACKAGE_INSTALL` variable with a :ref:`bbappend
213<dev-manual/layers:Appending Other Layers Metadata With Your Layer>` on the
214``core-image-minimal-initramfs`` recipe, or create a custom recipe for the
215:term:`Initramfs` taking ``core-image-minimal-initramfs`` as example.
216
217Custom scripts can be added to the :term:`Initramfs` by writing your own
218recipes. The recipes are conventionally named ``initramfs-module-<module name>``
219where ``<module name>`` is the name of the module. The recipe should set its
220:term:`RDEPENDS` package-specific variables to include
221``initramfs-framework-base`` and the other packages on which the module depends
222at runtime.
223
224The recipe must install shell initialization scripts in :term:`${D} <D>`\
225``/init.d`` and must follow the ``<number>-<script name>`` naming scheme where:
226
227- ``<number>`` is a *two-digit* number that affects the execution order of the
228 script compared to others. For example, the script ``80-setup-live`` would be
229 executed after ``01-udev`` because 80 is greater than 01.
230
231 This number being two-digits is important here as the scripts are executed
232 alphabetically. For example, the script ``10-script`` would be executed
233 before the script ``8-script``, because ``1`` is inferior to ``8``.
234 Therefore, the script should be named ``08-script``.
235
236- ``<script name>`` is the script name which you can choose freely.
237
238 If two script use the same ``<number>``, they are sorted alphabetically based
239 on ``<script name>``.
240
311Bundling an Initramfs Image From a Separate Multiconfig 241Bundling an Initramfs Image From a Separate Multiconfig
312------------------------------------------------------- 242-------------------------------------------------------
313 243
@@ -661,7 +591,7 @@ If build speed and package feed maintenance are considerations, you
661should consider the points in this section that can help you optimize 591should consider the points in this section that can help you optimize
662your tunings to best consider build times and package feed maintenance. 592your tunings to best consider build times and package feed maintenance.
663 593
664- *Share the :term:`Build Directory`:* If at all possible, share the 594- *Share the* :term:`Build Directory` *:* If at all possible, share the
665 :term:`TMPDIR` across builds. The Yocto Project supports switching between 595 :term:`TMPDIR` across builds. The Yocto Project supports switching between
666 different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice 596 different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice
667 is well supported and regularly used by developers when building for 597 is well supported and regularly used by developers when building for
@@ -883,7 +813,7 @@ directory:
883 813
884#. *Using Local Files Only:* Inside your ``local.conf`` file, add the 814#. *Using Local Files Only:* Inside your ``local.conf`` file, add the
885 :term:`SOURCE_MIRROR_URL` variable, inherit the 815 :term:`SOURCE_MIRROR_URL` variable, inherit the
886 :ref:`ref-classes-own-mirrors` class, and use the 816 :ref:`ref-classes-own-mirrors` class, and add the
887 :term:`BB_NO_NETWORK` variable to your ``local.conf``:: 817 :term:`BB_NO_NETWORK` variable to your ``local.conf``::
888 818
889 SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" 819 SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/"
diff --git a/documentation/dev-manual/custom-distribution.rst b/documentation/dev-manual/custom-distribution.rst
index 47faed0d04..0bc386d606 100644
--- a/documentation/dev-manual/custom-distribution.rst
+++ b/documentation/dev-manual/custom-distribution.rst
@@ -4,10 +4,16 @@ Creating Your Own Distribution
4****************************** 4******************************
5 5
6When you build an image using the Yocto Project and do not alter any 6When you build an image using the Yocto Project and do not alter any
7distribution :term:`Metadata`, you are 7distribution :term:`Metadata`, you are using the Poky distribution.
8creating a Poky distribution. If you wish to gain more control over 8Poky is explicitly a *reference* distribution for testing and
9package alternative selections, compile-time options, and other 9development purposes. It enables most hardware and software features
10low-level configurations, you can create your own distribution. 10so that they can be tested, but this also means that from a security
11point of view the attack surface is very large. Additionally, at some
12point it is likely that you will want to gain more control over package
13alternative selections, compile-time options, and other low-level
14configurations. For both of these reasons, if you are using the Yocto
15Project for production use then you are strongly encouraged to create
16your own distribution.
11 17
12To create your own distribution, the basic steps consist of creating 18To create your own distribution, the basic steps consist of creating
13your own distribution layer, creating your own distribution 19your own distribution layer, creating your own distribution
@@ -107,3 +113,23 @@ layer. The following steps provide some more detail:
107 For information on append files, see the 113 For information on append files, see the
108 ":ref:`dev-manual/layers:appending other layers metadata with your layer`" 114 ":ref:`dev-manual/layers:appending other layers metadata with your layer`"
109 section. 115 section.
116
117Copying and modifying the Poky distribution
118===========================================
119
120Instead of creating a custom distribution from scratch as per above, you may
121wish to start your custom distribution configuration by copying the Poky
122distribution provided within the ``meta-poky`` layer and then modifying it.
123This is fine, however if you do this you should keep the following in mind:
124
125- Every reference to Poky needs to be updated in your copy so that it
126 will still apply. This includes override usage within files (e.g. ``:poky``)
127 and in directory names. This is a good opportunity to evaluate each one of
128 these customizations to see if they are needed for your use case.
129
130- Unless you also intend to use them, the ``poky-tiny``, ``poky-altcfg`` and
131 ``poky-bleeding`` variants and any references to them can be removed.
132
133- More generally, the Poky distribution configuration enables a lot more
134 than you likely need for your production use case. You should evaluate *every*
135 configuration choice made in your copy to determine if it is needed.
diff --git a/documentation/dev-manual/customizing-images.rst b/documentation/dev-manual/customizing-images.rst
index 5b18958ade..6eed9ef56a 100644
--- a/documentation/dev-manual/customizing-images.rst
+++ b/documentation/dev-manual/customizing-images.rst
@@ -62,8 +62,7 @@ To understand how these features work, the best reference is
62:ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`. 62:ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`.
63This class lists out the available 63This class lists out the available
64:term:`IMAGE_FEATURES` of which most map to package groups while some, such 64:term:`IMAGE_FEATURES` of which most map to package groups while some, such
65as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general 65as ``read-only-rootfs``, resolve as general configuration settings.
66configuration settings.
67 66
68In summary, the file looks at the contents of the :term:`IMAGE_FEATURES` 67In summary, the file looks at the contents of the :term:`IMAGE_FEATURES`
69variable and then maps or configures the feature accordingly. Based on 68variable and then maps or configures the feature accordingly. Based on
@@ -80,15 +79,14 @@ recipe that are enabled with :term:`IMAGE_FEATURES`. The value of
80:term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within 79:term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within
81``meta/conf/bitbake.conf``. 80``meta/conf/bitbake.conf``.
82 81
83To illustrate how you can use these variables to modify your image, 82To illustrate how you can use these variables to modify your image, consider an
84consider an example that selects the SSH server. The Yocto Project ships 83example that selects the SSH server. The Yocto Project ships with two SSH
85with two SSH servers you can use with your images: Dropbear and OpenSSH. 84servers you can use with your images: Dropbear and OpenSSH. Dropbear is a
86Dropbear is a minimal SSH server appropriate for resource-constrained 85minimal SSH server appropriate for resource-constrained environments, while
87environments, while OpenSSH is a well-known standard SSH server 86OpenSSH is a well-known standard SSH server implementation. By default, the
88implementation. By default, the ``core-image-sato`` image is configured 87``core-image-sato`` image is configured to use Dropbear. The
89to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb`` 88``core-image-full-cmdline`` image includes OpenSSH. The ``core-image-minimal``
90images both include OpenSSH. The ``core-image-minimal`` image does not 89image does not contain an SSH server.
91contain an SSH server.
92 90
93You can customize your image and change these defaults. Edit the 91You can customize your image and change these defaults. Edit the
94:term:`IMAGE_FEATURES` variable in your recipe or use the 92:term:`IMAGE_FEATURES` variable in your recipe or use the
diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst
index e20637e1c6..6c45ccf652 100644
--- a/documentation/dev-manual/debugging.rst
+++ b/documentation/dev-manual/debugging.rst
@@ -36,7 +36,7 @@ section:
36 use the BitBake ``-e`` option to examine variable values after a 36 use the BitBake ``-e`` option to examine variable values after a
37 recipe has been parsed. 37 recipe has been parsed.
38 38
39- ":ref:`dev-manual/debugging:viewing package information with \`\`oe-pkgdata-util\`\``" 39- ":ref:`dev-manual/debugging:viewing package information with ``oe-pkgdata-util```"
40 describes how to use the ``oe-pkgdata-util`` utility to query 40 describes how to use the ``oe-pkgdata-util`` utility to query
41 :term:`PKGDATA_DIR` and 41 :term:`PKGDATA_DIR` and
42 display package-related information for built packages. 42 display package-related information for built packages.
@@ -270,13 +270,17 @@ format and can be converted to images (e.g. using the ``dot`` tool from
270 displays paths between graph nodes. 270 displays paths between graph nodes.
271 271
272You can use a different method to view dependency information by using 272You can use a different method to view dependency information by using
273the following command:: 273either::
274 274
275 $ bitbake -g -u taskexp recipename 275 $ bitbake -g -u taskexp recipename
276 276
277This command 277or::
278displays a GUI window from which you can view build-time and runtime 278
279dependencies for the recipes involved in building recipename. 279 $ bitbake -g -u taskexp_ncurses recipename
280
281The ``-u taskdep`` option GUI window from which you can view build-time and
282runtime dependencies for the recipes involved in building recipename. The
283``-u taskexp_ncurses`` option uses ncurses instead of GTK to render the UI.
280 284
281Viewing Task Variable Dependencies 285Viewing Task Variable Dependencies
282================================== 286==================================
@@ -886,7 +890,7 @@ The build should work without issue.
886As with all solved problems, if they originated upstream, you need to 890As with all solved problems, if they originated upstream, you need to
887submit the fix for the recipe in OE-Core and upstream so that the 891submit the fix for the recipe in OE-Core and upstream so that the
888problem is taken care of at its source. See the 892problem is taken care of at its source. See the
889":doc:`../contributor-guide/submit-changes`" section for more information. 893":doc:`/contributor-guide/submit-changes`" section for more information.
890 894
891Debugging With the GNU Project Debugger (GDB) Remotely 895Debugging With the GNU Project Debugger (GDB) Remotely
892====================================================== 896======================================================
@@ -1257,7 +1261,7 @@ Here are some other tips that you might find useful:
1257 :yocto_bugs:`Bugzilla <>`. For information on 1261 :yocto_bugs:`Bugzilla <>`. For information on
1258 how to submit a bug against the Yocto Project, see the Yocto Project 1262 how to submit a bug against the Yocto Project, see the Yocto Project
1259 Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` 1263 Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>`
1260 and the ":doc:`../contributor-guide/report-defect`" section. 1264 and the ":doc:`/contributor-guide/report-defect`" section.
1261 1265
1262 .. note:: 1266 .. note::
1263 1267
diff --git a/documentation/dev-manual/devtool.rst b/documentation/dev-manual/devtool.rst
new file mode 100644
index 0000000000..c82dc9c333
--- /dev/null
+++ b/documentation/dev-manual/devtool.rst
@@ -0,0 +1,1322 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Using the ``devtool`` command-line tool
4***************************************
5
6The ``devtool`` command-line tool provides a number of features that
7help you build, test, and package software. This command is available
8alongside the ``bitbake`` command. Additionally, the ``devtool`` command
9is a key part of the :term:`Extensible Software Development Kit (eSDK)`.
10
11Use ``devtool add`` to Add an Application
12=========================================
13
14The ``devtool add`` command generates a new recipe based on existing
15source code. This command takes advantage of the
16:ref:`devtool-the-workspace-layer-structure`
17layer that many ``devtool`` commands use. The command is flexible enough
18to allow you to extract source code into both the workspace or a
19separate local Git repository and to use existing code that does not
20need to be extracted.
21
22Depending on your particular scenario, the arguments and options you use
23with ``devtool add`` form different combinations. The following diagram
24shows common development flows you would use with the ``devtool add``
25command:
26
27.. image:: figures/devtool-add-flow.png
28 :width: 100%
29
30#. *Generating the New Recipe*: The top part of the flow shows three
31 scenarios by which you could use ``devtool add`` to generate a recipe
32 based on existing source code.
33
34 In a shared development environment, it is typical for other
35 developers to be responsible for various areas of source code. As a
36 developer, you are probably interested in using that source code as
37 part of your development within the Yocto Project. All you need is
38 access to the code, a recipe, and a controlled area in which to do
39 your work.
40
41 Within the diagram, three possible scenarios feed into the
42 ``devtool add`` workflow:
43
44 - *Left*: The left scenario in the figure represents a common
45 situation where the source code does not exist locally and needs
46 to be extracted. In this situation, the source code is extracted
47 to the default workspace --- you do not want the files in some
48 specific location outside of the workspace. Thus, everything you
49 need will be located in the workspace::
50
51 $ devtool add recipe fetchuri
52
53 With this command, ``devtool`` extracts the upstream
54 source files into a local Git repository within the ``sources``
55 folder. The command then creates a recipe named recipe and a
56 corresponding append file in the workspace. If you do not provide
57 recipe, the command makes an attempt to determine the recipe name.
58
59 - *Middle*: The middle scenario in the figure also represents a
60 situation where the source code does not exist locally. In this
61 case, the code is again upstream and needs to be extracted to some
62 local area --- this time outside of the default workspace.
63
64 .. note::
65
66 If required, ``devtool`` always creates a Git repository locally
67 during the extraction.
68
69 Furthermore, the first positional argument ``srctree`` in this case
70 identifies where the ``devtool add`` command will locate the
71 extracted code outside of the workspace. You need to specify an
72 empty directory::
73
74 $ devtool add recipe srctree fetchuri
75
76 In summary, the source code is pulled from fetchuri and extracted into the
77 location defined by ``srctree`` as a local Git repository.
78
79 Within workspace, ``devtool`` creates a recipe named recipe along
80 with an associated append file.
81
82 - *Right*: The right scenario in the figure represents a situation
83 where the ``srctree`` has been previously prepared outside of the
84 ``devtool`` workspace.
85
86 The following command provides a new recipe name and identifies
87 the existing source tree location::
88
89 $ devtool add recipe srctree
90
91 The command examines the source code and creates a recipe named
92 recipe for the code and places the recipe into the workspace.
93
94 Because the extracted source code already exists, ``devtool`` does
95 not try to relocate the source code into the workspace --- only the
96 new recipe is placed in the workspace.
97
98 Aside from a recipe folder, the command also creates an associated
99 append folder and places an initial ``*.bbappend`` file within.
100
101#. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
102 editor as defined by the ``$EDITOR`` environment variable and modify
103 the file::
104
105 $ devtool edit-recipe recipe
106
107 From within the editor, you can make modifications to the recipe that
108 take effect when you build it later.
109
110#. *Build the Recipe or Rebuild the Image*: The next step you take
111 depends on what you are going to do with the new code.
112
113 If you need to eventually move the build output to the target
114 hardware, use the following ``devtool`` command::
115
116 $ devtool build recipe
117
118 On the other hand, if you want an image to contain the recipe's
119 packages from the workspace for immediate deployment onto a device
120 (e.g. for testing purposes), you can use the ``devtool build-image``
121 command::
122
123 $ devtool build-image image
124
125#. *Deploy the Build Output*: When you use the ``devtool build`` command
126 to build out your recipe, you probably want to see if the resulting
127 build output works as expected on the target hardware.
128
129 .. note::
130
131 This step assumes you have a previously built image that is
132 already either running in QEMU or is running on actual hardware.
133 Also, it is assumed that for deployment of the image to the
134 target, SSH is installed in the image and, if the image is running
135 on real hardware, you have network access to and from your
136 development machine.
137
138 You can deploy your build output to that target hardware by using the
139 ``devtool deploy-target`` command::
140
141 $ devtool deploy-target recipe target
142
143 The target is a live target machine running as an SSH server.
144
145 You can, of course, also deploy the image you build to actual
146 hardware by using the ``devtool build-image`` command. However,
147 ``devtool`` does not provide a specific command that allows you to
148 deploy the image to actual hardware.
149
150#. *Finish Your Work With the Recipe*: The ``devtool finish`` command
151 creates any patches corresponding to commits in the local Git
152 repository, moves the new recipe to a more permanent layer, and then
153 resets the recipe so that the recipe is built normally rather than
154 from the workspace::
155
156 $ devtool finish recipe layer
157
158 .. note::
159
160 Any changes you want to turn into patches must be committed to the
161 Git repository in the source tree.
162
163 As mentioned, the ``devtool finish`` command moves the final recipe
164 to its permanent layer.
165
166 As a final process of the ``devtool finish`` command, the state of
167 the standard layers and the upstream source is restored so that you
168 can build the recipe from those areas rather than the workspace.
169
170 .. note::
171
172 You can use the ``devtool reset`` command to put things back should you
173 decide you do not want to proceed with your work. If you do use this
174 command, realize that the source tree is preserved.
175
176Use ``devtool modify`` to Modify the Source of an Existing Component
177====================================================================
178
179The ``devtool modify`` command prepares the way to work on existing code
180that already has a local recipe in place that is used to build the
181software. The command is flexible enough to allow you to extract code
182from an upstream source, specify the existing recipe, and keep track of
183and gather any patch files from other developers that are associated
184with the code.
185
186Depending on your particular scenario, the arguments and options you use
187with ``devtool modify`` form different combinations. The following
188diagram shows common development flows for the ``devtool modify``
189command:
190
191.. image:: figures/devtool-modify-flow.png
192 :width: 100%
193
194#. *Preparing to Modify the Code*: The top part of the flow shows three
195 scenarios by which you could use ``devtool modify`` to prepare to
196 work on source files. Each scenario assumes the following:
197
198 - The recipe exists locally in a layer external to the ``devtool``
199 workspace.
200
201 - The source files exist either upstream in an un-extracted state or
202 locally in a previously extracted state.
203
204 The typical situation is where another developer has created a layer
205 for use with the Yocto Project and their recipe already resides in
206 that layer. Furthermore, their source code is readily available
207 either upstream or locally.
208
209 - *Left*: The left scenario in the figure represents a common
210 situation where the source code does not exist locally and it
211 needs to be extracted from an upstream source. In this situation,
212 the source is extracted into the default ``devtool`` workspace
213 location. The recipe, in this scenario, is in its own layer
214 outside the workspace (i.e. ``meta-``\ layername).
215
216 The following command identifies the recipe and, by default,
217 extracts the source files::
218
219 $ devtool modify recipe
220
221 Once ``devtool`` locates the recipe, ``devtool`` uses the recipe's
222 :term:`SRC_URI` statements to locate the source code and any local
223 patch files from other developers.
224
225 With this scenario, there is no ``srctree`` argument. Consequently, the
226 default behavior of the ``devtool modify`` command is to extract
227 the source files pointed to by the :term:`SRC_URI` statements into a
228 local Git structure. Furthermore, the location for the extracted
229 source is the default area within the ``devtool`` workspace. The
230 result is that the command sets up both the source code and an
231 append file within the workspace while the recipe remains in its
232 original location.
233
234 Additionally, if you have any non-patch local files (i.e. files
235 referred to with ``file://`` entries in :term:`SRC_URI` statement
236 excluding ``*.patch/`` or ``*.diff``), these files are copied to
237 an ``oe-local-files`` folder under the newly created source tree.
238 Copying the files here gives you a convenient area from which you
239 can modify the files. Any changes or additions you make to those
240 files are incorporated into the build the next time you build the
241 software just as are other changes you might have made to the
242 source.
243
244 - *Middle*: The middle scenario in the figure represents a situation
245 where the source code also does not exist locally. In this case,
246 the code is again upstream and needs to be extracted to some local
247 area as a Git repository. The recipe, in this scenario, is again
248 local and in its own layer outside the workspace.
249
250 The following command tells ``devtool`` the recipe with which to
251 work and, in this case, identifies a local area for the extracted
252 source files that exists outside of the default ``devtool``
253 workspace::
254
255 $ devtool modify recipe srctree
256
257 .. note::
258
259 You cannot provide a URL for ``srctree`` using the ``devtool`` command.
260
261 As with all extractions, the command uses the recipe's :term:`SRC_URI`
262 statements to locate the source files and any associated patch
263 files. Non-patch files are copied to an ``oe-local-files`` folder
264 under the newly created source tree.
265
266 Once the files are located, the command by default extracts them
267 into ``srctree``.
268
269 Within workspace, ``devtool`` creates an append file for the
270 recipe. The recipe remains in its original location but the source
271 files are extracted to the location you provide with ``srctree``.
272
273 - *Right*: The right scenario in the figure represents a situation
274 where the source tree (``srctree``) already exists locally as a
275 previously extracted Git structure outside of the ``devtool``
276 workspace. In this example, the recipe also exists elsewhere
277 locally in its own layer.
278
279 The following command tells ``devtool`` the recipe with which to
280 work, uses the "-n" option to indicate source does not need to be
281 extracted, and uses ``srctree`` to point to the previously extracted
282 source files::
283
284 $ devtool modify -n recipe srctree
285
286 If an ``oe-local-files`` subdirectory happens to exist and it
287 contains non-patch files, the files are used. However, if the
288 subdirectory does not exist and you run the ``devtool finish``
289 command, any non-patch files that might exist next to the recipe
290 are removed because it appears to ``devtool`` that you have
291 deleted those files.
292
293 Once the ``devtool modify`` command finishes, it creates only an
294 append file for the recipe in the ``devtool`` workspace. The
295 recipe and the source code remain in their original locations.
296
297#. *Edit the Source*: Once you have used the ``devtool modify`` command,
298 you are free to make changes to the source files. You can use any
299 editor you like to make and save your source code modifications.
300
301#. *Build the Recipe or Rebuild the Image*: The next step you take
302 depends on what you are going to do with the new code.
303
304 If you need to eventually move the build output to the target
305 hardware, use the following ``devtool`` command::
306
307 $ devtool build recipe
308
309 On the other hand, if you want an image to contain the recipe's
310 packages from the workspace for immediate deployment onto a device
311 (e.g. for testing purposes), you can use the ``devtool build-image``
312 command::
313
314 $ devtool build-image image
315
316#. *Deploy the Build Output*: When you use the ``devtool build`` command
317 to build out your recipe, you probably want to see if the resulting
318 build output works as expected on target hardware.
319
320 .. note::
321
322 This step assumes you have a previously built image that is
323 already either running in QEMU or running on actual hardware.
324 Also, it is assumed that for deployment of the image to the
325 target, SSH is installed in the image and if the image is running
326 on real hardware that you have network access to and from your
327 development machine.
328
329 You can deploy your build output to that target hardware by using the
330 ``devtool deploy-target`` command::
331
332 $ devtool deploy-target recipe target
333
334 The target is a live target machine running as an SSH server.
335
336 You can, of course, use other methods to deploy the image you built
337 using the ``devtool build-image`` command to actual hardware.
338 ``devtool`` does not provide a specific command to deploy the image
339 to actual hardware.
340
341#. *Finish Your Work With the Recipe*: The ``devtool finish`` command
342 creates any patches corresponding to commits in the local Git
343 repository, updates the recipe to point to them (or creates a
344 ``.bbappend`` file to do so, depending on the specified destination
345 layer), and then resets the recipe so that the recipe is built
346 normally rather than from the workspace::
347
348 $ devtool finish recipe layer
349
350 .. note::
351
352 Any changes you want to turn into patches must be staged and
353 committed within the local Git repository before you use the
354 ``devtool finish`` command.
355
356 Because there is no need to move the recipe, ``devtool finish``
357 either updates the original recipe in the original layer or the
358 command creates a ``.bbappend`` file in a different layer as provided
359 by layer. Any work you did in the ``oe-local-files`` directory is
360 preserved in the original files next to the recipe during the
361 ``devtool finish`` command.
362
363 As a final process of the ``devtool finish`` command, the state of
364 the standard layers and the upstream source is restored so that you
365 can build the recipe from those areas rather than from the workspace.
366
367 .. note::
368
369 You can use the ``devtool reset`` command to put things back should you
370 decide you do not want to proceed with your work. If you do use this
371 command, realize that the source tree is preserved.
372
373``devtool ide-sdk`` configures IDEs and bootstraps SDKs
374=======================================================
375
376The ``devtool ide-sdk`` command can provide an IDE configuration for IDEs when
377working on the source code of one or more recipes.
378Depending on the programming language, and the build system used by the recipe,
379the tools required for cross-development and remote debugging are different.
380For example:
381
382- A C/C++ project usually uses CMake or Meson.
383
384- A Python project uses setuptools or one of its successors.
385
386- A Rust project uses Cargo.
387
388Also, the IDE plugins needed for the integration of a build system with the
389IDE and the corresponding settings are usually specific to these build-systems.
390To hide all these details from the user, ``devtool ide-sdk`` does two things:
391
392- It generates any kind of SDK needed for cross-development and remote
393 debugging of the specified recipes.
394
395- It generates the configuration for the IDE (and the IDE plugins) for using
396 the cross-toolchain and remote debugging tools provided by the SDK directly
397 from the IDE.
398
399For supported build systems the configurations generated by ``devtool ide-sdk``
400combine the advantages of the ``devtool modify`` based workflow
401(see :ref:`using_devtool`) with the advantages of the simple Environment Setup
402script based workflow (see :ref:`running_the_ext_sdk_env`) provided by Yocto's
403SDK or eSDK:
404
405- The source code of the recipe is in the workspace created by
406 ``devtool modify`` or ``devtool add``.
407 Using ``devtool build``, ``devtool build-image``,
408 ``devtool deploy-target`` or ``bitbake`` is possible.
409 Also ``devtool ide-sdk`` can be used to update the SDK and the IDE
410 configuration at any time.
411
412- ``devtool ide-sdk`` aims to support multiple programming languages and
413 multiple IDEs natively. "Natively" means that the IDE is configured to call
414 the build tool (e.g. ``cmake`` or ``meson``) directly. This has several
415 advantages.
416 First of all, it is usually much faster to call for example ``cmake`` than
417 ``devtool build``.
418 It also allows to benefit from the very good integration that IDEs like
419 VSCode offer for tools like CMake or GDB.
420
421 However, supporting many programming languages and multiple
422 IDEs is quite an elaborate and constantly evolving thing. Support for IDEs
423 is therefore implemented as plugins. Plugins can also be provided by
424 optional layers.
425
426So much about the introduction to the default mode of ``devtool sdk-ide`` which
427is called the "modified" mode because it uses the workspace created by
428``devtool modify`` and the per recipe :term:`Sysroots <Sysroot>` of BitBake.
429
430For some recipes and use cases, this default behavior of ``devtool ide-sdk``
431with full ``devtool`` and ``bitbake`` integration might not be suitable.
432To offer full feature parity with the SDK and the eSDK, ``devtool ide-sdk`` has
433a second mode called "shared" mode.
434If ``devtool ide-sdk`` is called with the ``--mode=shared`` option, it
435bootstraps an SDK directly from the BitBake environment, which offers the same
436Environment Setup script as described in :ref:`running_the_ext_sdk_env`.
437In addition to the (e)SDK installer-based setup, the IDE gets configured
438to use the shared :term:`Sysroots <Sysroot>` and the tools from the SDK.
439``devtool ide-sdk --mode=shared`` is basically a wrapper for the setup of the
440extensible SDK as described in :ref:`setting_up_ext_sdk_in_build`.
441
442The use of ``devtool ide-sdk`` is an alternative to using one of the SDK
443installers.
444``devtool ide-sdk`` allows the creation of SDKs that offer all the
445functionality of the SDK and the eSDK installers. Compared to the installers,
446however, the SDK created with ``devtool ide-sdk`` is much more flexible.
447For example, it is very easy to change the :term:`MACHINE` in the
448``local.conf`` file, update the layer meta data and then regenerate the SDK.
449
450Let's take a look at an example of how to use ``devtool ide-sdk`` in each of
451the two modes:
452
453#. *Modified mode*:
454
455 In order to use the ``devtool ide-sdk``, a few settings are needed. As a
456 starting example, the following lines of code can be added to the
457 ``local.conf`` file::
458
459 # Build the companion debug file system
460 IMAGE_GEN_DEBUGFS = "1"
461 # Optimize build time: with devtool ide-sdk the dbg tar is not needed
462 IMAGE_FSTYPES_DEBUGFS = ""
463 # Without copying the binaries into roofs-dbg, GDB does not find all source files.
464 IMAGE_CLASSES += "image-combined-dbg"
465
466 # SSH is mandatory, no password simplifies the usage
467 EXTRA_IMAGE_FEATURES += "\
468 ssh-server-openssh \
469 allow-empty-password \
470 allow-root-login \
471 empty-root-password \
472 "
473
474 # Remote debugging needs gdbserver on the target device
475 IMAGE_INSTALL:append = " gdbserver"
476
477 # Add the recipes which should be modified to the image
478 # Otherwise some dependencies might be missing.
479 IMAGE_INSTALL:append = " my-recipe"
480
481 Assuming the BitBake environment is set up correctly and a workspace has
482 been created for the recipe using ``devtool modify my-recipe`` or probably
483 even better by using ``devtool modify my-recipe --debug-build``, the
484 following command can create the SDK and the configuration for VSCode in
485 the recipe workspace::
486
487 $ devtool ide-sdk my-recipe core-image-minimal --target root@192.168.7.2
488
489 The command requires an image recipe (``core-image-minimal`` for this
490 example) that is used to create the SDK.
491 This firmware image should also be installed on the target device.
492 It is possible to pass multiple package recipes::
493
494 $ devtool ide-sdk my-recipe-1 my-recipe-2 core-image-minimal --target root@192.168.7.2
495
496 ``devtool ide-sdk`` tries to create an IDE configuration for all package
497 recipes.
498
499 What this command does exactly depends on the recipe, more precisely on the
500 build tool used by the recipe. The basic idea is to configure the IDE so
501 that it calls the build tool exactly as ``bitbake`` does.
502
503 For example, a CMake preset is created for a recipe that inherits
504 :ref:`ref-classes-cmake`. In the case of VSCode, CMake presets are supported
505 by the CMake Tools plugin. This is an example of how the build configuration
506 used by ``bitbake`` is exported to an IDE configuration that gives exactly
507 the same build results.
508
509 Support for remote debugging with seamless integration into the IDE is
510 important for a cross-SDK. ``devtool ide-sdk`` automatically generates the
511 necessary helper scripts for deploying the compiled artifacts to the target
512 device as well as the necessary configuration for the debugger and the IDE.
513
514 .. note::
515
516 To ensure that the debug symbols on the build machine match the binaries
517 running on the target device, it is essential that the image built by
518 ``devtool ide-sdk`` is running on the target device.
519
520 The default IDE is VSCode. Some hints about using VSCode:
521
522 - VSCode can be used to work on the BitBake recipes or the application
523 source code.
524 Usually there is one instance of VSCode running in the folder where the
525 BitBake recipes are. This instance has the
526 `Yocto Project BitBake plugin <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`_
527 running.
528
529 .. warning::
530
531 Some VSCode plugins (Python, BitBake and others) need a reasonable
532 configuration to work as expected. Otherwise, some plugins try to
533 index the build directory of BitBake, which keeps your system quite
534 busy until an out of memory exception stops this nonsense.
535 Other plugins, such as the BitBake plugin, do not behave as expected.
536
537 To work around such issues, the ``oe-init-build-env`` script creates
538 an initial ``.vscode/settings.json`` file if ``code`` can be found
539 and the ``.vscode`` folder does not yet exist.
540 It is best to run ``oe-init-build-env`` once before starting VSCode.
541 An alternative approach is to use a build folder outside the layers,
542 e.g. ``oe-init-build-env ../build``.
543
544 The BitBake plugin also offers to create devtool workspaces and run
545 ``devtool ide-sdk`` with a few mouse clicks.
546 Of course, issuing commands in the terminal works as well.
547
548 - To work on the source code of a recipe another instance of VSCode is
549 started in the recipe's workspace. Example::
550
551 code build/workspace/sources/my-recipe
552
553 This instance of VSCode uses plugins that are useful for the development
554 of the application. ``devtool ide-sdk`` generates the necessary
555 ``extensions.json``, ``settings.json``, ``tasks.json`` and ``launch.json``
556 configuration files for all the involved plugins.
557
558 When the source code folder present in the workspace folder is opened in
559 VSCode for the first time, a pop-up message recommends installing the
560 required plugins.
561 After accepting the installation of the plugins, working with the source
562 code or some debugging tasks should work as usual with VSCode.
563
564 Starting the VSCode instances in the recipe workspace folders can also be
565 done by a mouse click on the recipe workspaces in the first VSCode
566 instance.
567
568 - To work with CMake press ``Ctrl + Shift + p``, type ``cmake``. This will
569 show some possible commands like selecting a CMake preset, compiling or
570 running CTest.
571
572 For recipes inheriting :ref:`ref-classes-cmake-qemu` rather than
573 :ref:`ref-classes-cmake`, executing cross-compiled unit tests on the host
574 can be supported transparently with QEMU user-mode.
575
576 - To work with Meson press ``Ctrl + Shift + p``, type ``meson``. This will
577 show some possible commands like compiling or executing the unit tests.
578
579 A note on running cross-compiled unit tests on the host: Meson enables
580 support for QEMU user mode by default. It is expected that the execution
581 of the unit tests from the IDE will work without any additional steps,
582 given that the code is suitable for the execution on the host machine.
583
584 - For the deployment to the target device, just press ``Ctrl + Shift + p``,
585 type ``task``. Select ``install && deploy-target``.
586
587 - For remote debugging, switch to the debugging view by pressing the "play"
588 button with the ``bug icon`` on the left side. This will provide a green
589 play button with a drop-down list where a debug configuration can be
590 selected. After selecting one of the generated configurations, press the
591 "play" button.
592
593 Starting a remote debugging session automatically initiates the
594 deployment to the target device. If this is not desired, the
595 ``"dependsOn": ["install && deploy-target...]`` parameter of the tasks
596 with ``"label": "gdbserver start...`` can be removed from the
597 ``tasks.json`` file.
598
599 VSCode supports GDB with many different setups and configurations for
600 many different use cases. However, most of these setups have some
601 limitations when it comes to cross-development, support only a few target
602 architectures or require a high performance target device. Therefore
603 ``devtool ide-sdk`` supports the classic, generic setup with GDB on the
604 development host and gdbserver on the target device.
605
606 Roughly summarized, this means:
607
608 - The binaries are copied via SSH to the remote target device by a
609 script referred by ``tasks.json``.
610
611 - gdbserver is started on the remote target device via SSH by a script
612 referred by ``tasks.json``.
613
614 Changing the parameters that are passed to the debugging executable
615 requires modifying the generated script. The script is located at
616 ``oe-scripts/gdbserver_*``. Defining the parameters in the ``args``
617 field in the ``launch.json`` file does not work.
618
619 - VSCode connects to gdbserver as documented in
620 `Remote debugging or debugging with a local debugger server
621 <https://code.visualstudio.com/docs/cpp/launch-json-reference#_remote-debugging-or-debugging-with-a-local-debugger-server>`__.
622
623 Additionally ``--ide=none`` is supported. With the ``none`` IDE parameter,
624 some generic configuration files like ``gdbinit`` files and some helper
625 scripts starting gdbserver remotely on the target device as well as the GDB
626 client on the host are generated.
627
628 Here is a usage example for the ``cmake-example`` recipe from the
629 ``meta-selftest`` layer which inherits :ref:`ref-classes-cmake-qemu`:
630
631 .. code-block:: sh
632
633 # Create the SDK
634 devtool modify cmake-example --debug-build
635 devtool ide-sdk cmake-example core-image-minimal -c --ide=none
636
637 # Install the firmware on a target device or start QEMU
638 runqemu
639
640 # From exploring the workspace of cmake-example
641 cd build/workspace/sources/cmake-example
642
643 # Find cmake-native and save the path into a variable
644 # Note: using just cmake instead of $CMAKE_NATIVE would work in many cases
645 CMAKE_NATIVE="$(jq -r '.configurePresets[0] | "\(.cmakeExecutable)"' CMakeUserPresets.json)"
646
647 # List available CMake presets
648 "$CMAKE_NATIVE" --list-presets
649 Available configure presets:
650
651 "cmake-example-cortexa57" - cmake-example: cortexa57
652
653 # Re-compile the already compiled sources
654 "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57
655 ninja: no work to do.
656 # Do a clean re-build
657 "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target clean
658 [1/1] Cleaning all built files...
659 Cleaning... 8 files.
660 "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target all
661 [7/7] Linking CXX executable cmake-example
662
663 # Run the cross-compiled unit tests with QEMU user-mode
664 "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target test
665 [0/1] Running tests...
666 Test project .../build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0
667 Start 1: test-cmake-example
668 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec
669
670 100% tests passed, 0 tests failed out of 1
671
672 Total Test time (real) = 0.03 sec
673
674 # Using CTest directly is possible as well
675 CTEST_NATIVE="$(dirname "$CMAKE_NATIVE")/ctest"
676
677 # List available CMake presets
678 "$CTEST_NATIVE" --list-presets
679 Available test presets:
680
681 "cmake-example-cortexa57" - cmake-example: cortexa57
682
683 # Run the cross-compiled unit tests with QEMU user-mode
684 "$CTEST_NATIVE" --preset "cmake-example-cortexa57"
685 Test project ...build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0
686 Start 1: test-cmake-example
687 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec
688
689 100% tests passed, 0 tests failed out of 1
690
691 Total Test time (real) = 0.03 sec
692
693 # Deploying the new build to the target device (default is QEUM at 192.168.7.2)
694 oe-scripts/install_and_deploy_cmake-example-cortexa57
695
696 # Start a remote debugging session with gdbserver on the target and GDB on the host
697 oe-scripts/gdbserver_1234_usr-bin-cmake-example_m
698 oe-scripts/gdb_1234_usr-bin-cmake-example
699 break main
700 run
701 step
702 stepi
703 continue
704 quit
705
706 # Stop gdbserver on the target device
707 oe-scripts/gdbserver_1234_usr-bin-cmake-example_m stop
708
709#. *Shared sysroots mode*
710
711 Creating an SDK with shared :term:`Sysroots <Sysroot>` that contains all the
712 dependencies needed to work with ``my-recipe`` is possible with the following
713 example command::
714
715 $ devtool ide-sdk --mode=shared my-recipe
716
717 For VSCode the cross-toolchain is exposed as a CMake kit. CMake kits are
718 defined in ``~/.local/share/CMakeTools/cmake-tools-kits.json``.
719 The following example shows how the cross-toolchain can be selected in
720 VSCode. First of all we need a folder containing a CMake project.
721 For this example, let's create a CMake project and start VSCode::
722
723 mkdir kit-test
724 echo "project(foo VERSION 1.0)" > kit-test/CMakeLists.txt
725 code kit-test
726
727 If there is a CMake project in the workspace, cross-compilation is
728 supported:
729
730 - Press ``Ctrl + Shift + P``, type ``CMake: Scan for Kits``
731 - Press ``Ctrl + Shift + P``, type ``CMake: Select a Kit``
732
733 Finally most of the features provided by CMake and the IDE should be
734 available.
735
736 Other IDEs than VSCode are supported as well. However,
737 ``devtool ide-sdk --mode=shared --ide=none my-recipe`` is currently
738 just a simple wrapper for the setup of the extensible SDK, as described in
739 :ref:`setting_up_ext_sdk_in_build`.
740
741Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
742=======================================================================================================
743
744The ``devtool upgrade`` command upgrades an existing recipe to that of a
745more up-to-date version found upstream. Throughout the life of software,
746recipes continually undergo version upgrades by their upstream
747publishers. You can use the ``devtool upgrade`` workflow to make sure
748your recipes you are using for builds are up-to-date with their upstream
749counterparts.
750
751.. note::
752
753 Several methods exist by which you can upgrade recipes ---
754 ``devtool upgrade`` happens to be one. You can read about all the methods by
755 which you can upgrade recipes in the
756 :ref:`dev-manual/upgrading-recipes:upgrading recipes` section of the Yocto
757 Project Development Tasks Manual.
758
759The ``devtool upgrade`` command is flexible enough to allow you to specify
760source code revision and versioning schemes, extract code into or out of the
761``devtool`` :ref:`devtool-the-workspace-layer-structure`, and work with any
762source file forms that the
763:ref:`bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support.
764
765The following diagram shows the common development flow used with the
766``devtool upgrade`` command:
767
768.. image:: figures/devtool-upgrade-flow.png
769 :width: 100%
770
771#. *Initiate the Upgrade*: The top part of the flow shows the typical
772 scenario by which you use the ``devtool upgrade`` command. The
773 following conditions exist:
774
775 - The recipe exists in a local layer external to the ``devtool``
776 workspace.
777
778 - The source files for the new release exist in the same location
779 pointed to by :term:`SRC_URI`
780 in the recipe (e.g. a tarball with the new version number in the
781 name, or as a different revision in the upstream Git repository).
782
783 A common situation is where third-party software has undergone a
784 revision so that it has been upgraded. The recipe you have access to
785 is likely in your own layer. Thus, you need to upgrade the recipe to
786 use the newer version of the software::
787
788 $ devtool upgrade -V version recipe
789
790 By default, the ``devtool upgrade`` command extracts source
791 code into the ``sources`` directory in the
792 :ref:`devtool-the-workspace-layer-structure`.
793 If you want the code extracted to any other location, you need to
794 provide the ``srctree`` positional argument with the command as follows::
795
796 $ devtool upgrade -V version recipe srctree
797
798 .. note::
799
800 In this example, the "-V" option specifies the new version. If you
801 don't use "-V", the command upgrades the recipe to the latest
802 version.
803
804 If the source files pointed to by the :term:`SRC_URI` statement in the
805 recipe are in a Git repository, you must provide the "-S" option and
806 specify a revision for the software.
807
808 Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable
809 to locate the source code and any local patch files from other
810 developers. The result is that the command sets up the source code,
811 the new version of the recipe, and an append file all within the
812 workspace.
813
814 Additionally, if you have any non-patch local files (i.e. files
815 referred to with ``file://`` entries in :term:`SRC_URI` statement
816 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
817 ``oe-local-files`` folder under the newly created source tree.
818 Copying the files here gives you a convenient area from which you can
819 modify the files. Any changes or additions you make to those files
820 are incorporated into the build the next time you build the software
821 just as are other changes you might have made to the source.
822
823#. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen
824 after upgrading the software to a new version. Conflicts occur
825 if your recipe specifies some patch files in :term:`SRC_URI` that
826 conflict with changes made in the new version of the software. For
827 such cases, you need to resolve the conflicts by editing the source
828 and following the normal ``git rebase`` conflict resolution process.
829
830 Before moving onto the next step, be sure to resolve any such
831 conflicts created through use of a newer or different version of the
832 software.
833
834#. *Build the Recipe or Rebuild the Image*: The next step you take
835 depends on what you are going to do with the new code.
836
837 If you need to eventually move the build output to the target
838 hardware, use the following ``devtool`` command::
839
840 $ devtool build recipe
841
842 On the other hand, if you want an image to contain the recipe's
843 packages from the workspace for immediate deployment onto a device
844 (e.g. for testing purposes), you can use the ``devtool build-image``
845 command::
846
847 $ devtool build-image image
848
849#. *Deploy the Build Output*: When you use the ``devtool build`` command
850 or ``bitbake`` to build your recipe, you probably want to see if the
851 resulting build output works as expected on target hardware.
852
853 .. note::
854
855 This step assumes you have a previously built image that is
856 already either running in QEMU or running on actual hardware.
857 Also, it is assumed that for deployment of the image to the
858 target, SSH is installed in the image and if the image is running
859 on real hardware that you have network access to and from your
860 development machine.
861
862 You can deploy your build output to that target hardware by using the
863 ``devtool deploy-target`` command::
864
865 $ devtool deploy-target recipe target
866
867 The target is a live target machine running as an SSH server.
868
869 You can, of course, also deploy the image you build using the
870 ``devtool build-image`` command to actual hardware. However,
871 ``devtool`` does not provide a specific command that allows you to do
872 this.
873
874#. *Finish Your Work With the Recipe*: The ``devtool finish`` command
875 creates any patches corresponding to commits in the local Git
876 repository, moves the new recipe to a more permanent layer, and then
877 resets the recipe so that the recipe is built normally rather than
878 from the workspace.
879
880 Any work you did in the ``oe-local-files`` directory is preserved in
881 the original files next to the recipe during the ``devtool finish``
882 command.
883
884 If you specify a destination layer that is the same as the original
885 source, then the old version of the recipe and associated files are
886 removed prior to adding the new version::
887
888 $ devtool finish recipe layer
889
890 .. note::
891
892 Any changes you want to turn into patches must be committed to the
893 Git repository in the source tree.
894
895 As a final process of the ``devtool finish`` command, the state of
896 the standard layers and the upstream source is restored so that you
897 can build the recipe from those areas rather than the workspace.
898
899 .. note::
900
901 You can use the ``devtool reset`` command to put things back should you
902 decide you do not want to proceed with your work. If you do use this
903 command, realize that the source tree is preserved.
904
905A Closer Look at ``devtool add``
906================================
907
908The ``devtool add`` command automatically creates a recipe based on the
909source tree you provide with the command. Currently, the command has
910support for the following:
911
912- Autotools (``autoconf`` and ``automake``)
913
914- CMake
915
916- Scons
917
918- ``qmake``
919
920- Plain ``Makefile``
921
922- Out-of-tree kernel module
923
924- Binary package (i.e. "-b" option)
925
926- Node.js module
927
928- Python modules that use ``setuptools`` or ``distutils``
929
930Apart from binary packages, the determination of how a source tree
931should be treated is automatic based on the files present within that
932source tree. For example, if a ``CMakeLists.txt`` file is found, then
933the source tree is assumed to be using CMake and is treated accordingly.
934
935.. note::
936
937 In most cases, you need to edit the automatically generated recipe in
938 order to make it build properly. Typically, you would go through
939 several edit and build cycles until the recipe successfully builds.
940 Once the recipe builds, you could use possible further iterations to
941 test the recipe on the target device.
942
943The remainder of this section covers specifics regarding how parts of
944the recipe are generated.
945
946Name and Version
947----------------
948
949If you do not specify a name and version on the command line,
950``devtool add`` uses various metadata within the source tree in an
951attempt to determine the name and version of the software being built.
952Based on what the tool determines, ``devtool`` sets the name of the
953created recipe file accordingly.
954
955If ``devtool`` cannot determine the name and version, the command prints
956an error. For such cases, you must re-run the command and provide the
957name and version, just the name, or just the version as part of the
958command line.
959
960Sometimes the name or version determined from the source tree might be
961incorrect. For such a case, you must reset the recipe::
962
963 $ devtool reset -n recipename
964
965After running the ``devtool reset`` command, you need to
966run ``devtool add`` again and provide the name or the version.
967
968Dependency Detection and Mapping
969--------------------------------
970
971The ``devtool add`` command attempts to detect build-time dependencies and map
972them to other recipes in the system. During this mapping, the command fills in
973the names of those recipes as part of the :term:`DEPENDS` variable within the
974recipe. If a dependency cannot be mapped, ``devtool`` places a comment
975in the recipe indicating such. The inability to map a dependency can
976result from naming not being recognized or because the dependency simply
977is not available. For cases where the dependency is not available, you
978must use the ``devtool add`` command to add an additional recipe that
979satisfies the dependency. Once you add that recipe, you need to update
980the :term:`DEPENDS` variable in the original recipe to include the new
981recipe.
982
983If you need to add runtime dependencies, you can do so by adding the
984following to your recipe::
985
986 RDEPENDS:${PN} += "dependency1 dependency2 ..."
987
988.. note::
989
990 The ``devtool add`` command often cannot distinguish between mandatory and
991 optional dependencies. Consequently, some of the detected dependencies might
992 in fact be optional. When in doubt, consult the documentation or the
993 configure script for the software the recipe is building for further
994 details. In some cases, you might find you can substitute the
995 dependency with an option that disables the associated functionality
996 passed to the configure script.
997
998License Detection
999-----------------
1000
1001The ``devtool add`` command attempts to determine if the software you are
1002adding is able to be distributed under a common, open-source license. If
1003so, the command sets the :term:`LICENSE` value accordingly.
1004You should double-check the value added by the command against the
1005documentation or source files for the software you are building and, if
1006necessary, update that :term:`LICENSE` value.
1007
1008The ``devtool add`` command also sets the :term:`LIC_FILES_CHKSUM`
1009value to point to all files that appear to be license-related. Realize
1010that license statements often appear in comments at the top of source
1011files or within the documentation. In such cases, the command does not
1012recognize those license statements. Consequently, you might need to
1013amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those
1014comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly
1015important for third-party software. The mechanism attempts to ensure
1016correct licensing should you upgrade the recipe to a newer upstream
1017version in future. Any change in licensing is detected and you receive
1018an error prompting you to check the license text again.
1019
1020If the ``devtool add`` command cannot determine licensing information,
1021``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the
1022:term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue
1023with development even though the settings are unlikely to be correct in
1024all cases. You should check the documentation or source files for the
1025software you are building to determine the actual license.
1026
1027Adding Makefile-Only Software
1028-----------------------------
1029
1030The use of Make by itself is very common in both proprietary and
1031open-source software. Unfortunately, Makefiles are often not written
1032with cross-compilation in mind. Thus, ``devtool add`` often cannot do
1033very much to ensure that these Makefiles build correctly. It is very
1034common, for example, to explicitly call ``gcc`` instead of using the
1035:term:`CC` variable. Usually, in a
1036cross-compilation environment, ``gcc`` is the compiler for the build
1037host and the cross-compiler is named something similar to
1038``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
1039point to the associated sysroot for the target machine).
1040
1041When writing a recipe for Makefile-only software, keep the following in
1042mind:
1043
1044- You probably need to patch the Makefile to use variables instead of
1045 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
1046
1047- The environment in which Make runs is set up with various standard
1048 variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a
1049 similar manner to the environment set up by an :ref:`SDK
1050 <overview-manual/concepts:Application Development SDK>`'s environment
1051 setup script. One easy way to see these variables is to run the
1052 ``devtool build`` command on the recipe and then look in
1053 ``oe-logs/run.do_compile``. Towards the top of this file, there is
1054 a list of environment variables that are set. You can take
1055 advantage of these variables within the Makefile.
1056
1057- If the Makefile sets a default for a variable using "=", that default
1058 overrides the value set in the environment, which is usually not
1059 desirable. For this case, you can either patch the Makefile so it
1060 sets the default using the "?=" operator, or you can alternatively
1061 force the value on the ``make`` command line. To force the value on
1062 the command line, add the variable setting to
1063 :term:`EXTRA_OEMAKE` or
1064 :term:`PACKAGECONFIG_CONFARGS`
1065 within the recipe. Here is an example using :term:`EXTRA_OEMAKE`::
1066
1067 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
1068
1069 In the above example,
1070 single quotes are used around the variable settings as the values are
1071 likely to contain spaces because required default options are passed
1072 to the compiler.
1073
1074- Hardcoding paths inside Makefiles is often problematic in a
1075 cross-compilation environment. This is particularly true because
1076 those hardcoded paths often point to locations on the build host and
1077 thus will either be read-only or will introduce contamination into
1078 the cross-compilation because they are specific to the build host
1079 rather than the target. Patching the Makefile to use prefix variables
1080 or other path variables is usually the way to handle this situation.
1081
1082- Sometimes a Makefile runs target-specific commands such as
1083 ``ldconfig``. For such cases, you might be able to apply patches that
1084 remove these commands from the Makefile.
1085
1086Adding Native Tools
1087-------------------
1088
1089Often, you need to build additional tools that run on the :term:`Build Host`
1090as opposed to the target. You should indicate this requirement by using one of
1091the following methods when you run ``devtool add``:
1092
1093- Specify the name of the recipe such that it ends with "-native".
1094 Specifying the name like this produces a recipe that only builds for
1095 the build host.
1096
1097- Specify the "--also-native" option with the ``devtool add``
1098 command. Specifying this option creates a recipe file that still
1099 builds for the target but also creates a variant with a "-native"
1100 suffix that builds for the build host.
1101
1102.. note::
1103
1104 If you need to add a tool that is shipped as part of a source tree
1105 that builds code for the target, you can typically accomplish this by
1106 building the native and target parts separately rather than within
1107 the same compilation process. Realize though that with the
1108 "--also-native" option, you can add the tool using just one
1109 recipe file.
1110
1111Adding Node.js Modules
1112----------------------
1113
1114You can use the ``devtool add`` command two different ways to add
1115Node.js modules: through ``npm`` or from a repository or local source.
1116
1117Use the following form to add Node.js modules through ``npm``::
1118
1119 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
1120
1121The name and
1122version parameters are mandatory. Lockdown and shrinkwrap files are
1123generated and pointed to by the recipe in order to freeze the version
1124that is fetched for the dependencies according to the first time. This
1125also saves checksums that are verified on future fetches. Together,
1126these behaviors ensure the reproducibility and integrity of the build.
1127
1128.. note::
1129
1130 - You must use quotes around the URL. ``devtool add`` does not
1131 require the quotes, but the shell considers ";" as a splitter
1132 between multiple commands. Thus, without the quotes,
1133 ``devtool add`` does not receive the other parts, which results in
1134 several "command not found" errors.
1135
1136As mentioned earlier, you can also add Node.js modules directly from a
1137repository or local source tree. To add modules this way, use
1138``devtool add`` in the following form::
1139
1140 $ devtool add https://github.com/diversario/node-ssdp
1141
1142In this example, ``devtool`` fetches the specified Git repository, detects the
1143code as Node.js code, fetches dependencies using ``npm``, and sets
1144:term:`SRC_URI` accordingly.
1145
1146Working With Recipes
1147====================
1148
1149When building a recipe using the ``devtool build`` command, the typical
1150build progresses as follows:
1151
1152#. Fetch the source
1153
1154#. Unpack the source
1155
1156#. Configure the source
1157
1158#. Compile the source
1159
1160#. Install the build output
1161
1162#. Package the installed output
1163
1164For recipes in the workspace, fetching and unpacking is disabled as the
1165source tree has already been prepared and is persistent. Each of these
1166build steps is defined as a function (task), usually with a "do\_" prefix
1167(e.g. :ref:`ref-tasks-fetch`,
1168:ref:`ref-tasks-unpack`, and so
1169forth). These functions are typically shell scripts but can instead be
1170written in Python.
1171
1172If you look at the contents of a recipe, you will see that the recipe
1173does not include complete instructions for building the software.
1174Instead, common functionality is encapsulated in classes inherited with
1175the ``inherit`` directive. This technique leaves the recipe to describe
1176just the things that are specific to the software being built. There is
1177a :ref:`ref-classes-base` class that is implicitly inherited by all recipes
1178and provides the functionality that most recipes typically need.
1179
1180The remainder of this section presents information useful when working
1181with recipes.
1182
1183Finding Logs and Work Files
1184---------------------------
1185
1186After the first run of the ``devtool build`` command, recipes that were
1187previously created using the ``devtool add`` command or whose sources
1188were modified using the ``devtool modify`` command contain symbolic
1189links created within the source tree:
1190
1191- ``oe-logs``: This link points to the directory in which log files and
1192 run scripts for each build step are created.
1193
1194- ``oe-workdir``: This link points to the temporary work area for the
1195 recipe. The following locations under ``oe-workdir`` are particularly
1196 useful:
1197
1198 - ``image/``: Contains all of the files installed during the
1199 :ref:`ref-tasks-install` stage.
1200 Within a recipe, this directory is referred to by the expression
1201 ``${``\ :term:`D`\ ``}``.
1202
1203 - ``sysroot-destdir/``: Contains a subset of files installed within
1204 :ref:`ref-tasks-install` that have been put into the shared sysroot. For
1205 more information, see the
1206 ":ref:`dev-manual/new-recipe:sharing files between recipes`" section.
1207
1208 - ``packages-split/``: Contains subdirectories for each package
1209 produced by the recipe. For more information, see the
1210 ":ref:`dev-manual/devtool:packaging`" section.
1211
1212You can use these links to get more information on what is happening at
1213each build step.
1214
1215Setting Configure Arguments
1216---------------------------
1217
1218If the software your recipe is building uses GNU autoconf, then a fixed
1219set of arguments is passed to it to enable cross-compilation plus any
1220extras specified by :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`
1221set within the recipe. If you wish to pass additional options, add them
1222to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build
1223tools have similar variables (e.g. :term:`EXTRA_OECMAKE` for CMake,
1224:term:`EXTRA_OESCONS` for Scons, and so forth). If you need to pass anything on
1225the ``make`` command line, you can use :term:`EXTRA_OEMAKE` or the
1226:term:`PACKAGECONFIG_CONFARGS` variables to do so.
1227
1228You can use the ``devtool configure-help`` command to help you set the
1229arguments listed in the previous paragraph. The command determines the
1230exact options being passed, and shows them to you along with any custom
1231arguments specified through :term:`EXTRA_OECONF` or
1232:term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you
1233the output of the configure script's "--help" option as a
1234reference.
1235
1236Sharing Files Between Recipes
1237-----------------------------
1238
1239Recipes often need to use files provided by other recipes on the
1240:term:`Build Host`. For example,
1241an application linking to a common library needs access to the library
1242itself and its associated headers. The way this access is accomplished
1243is through the :term:`Sysroot`. There is a sysroot for the target machine, and a
1244sysroot for the build host.
1245
1246Recipes should never write files directly into the sysroot. Instead,
1247files should be installed into standard locations during the
1248:ref:`ref-tasks-install` task within the ``${``\ :term:`D`\ ``}`` directory. A
1249subset of these files automatically goes into the sysroot. The reason
1250for this limitation is that almost all files that go into the sysroot
1251are cataloged in manifests in order to ensure they can be removed later
1252when a recipe is modified or removed. Thus, the sysroot is able to
1253remain free from stale files.
1254
1255Packaging
1256---------
1257
1258If you examine how build output gets into the final image on
1259the target device, it is important to understand packaging because the
1260contents of the image are expressed in terms of packages and not
1261recipes.
1262
1263During the :ref:`ref-tasks-package` task, files installed during the
1264:ref:`ref-tasks-install` task are split into one main package, which is almost
1265always named the same as the recipe, and into several other packages. This
1266separation exists because not all of those installed files are useful in every
1267image. For example, you probably do not need any of the documentation installed
1268in a production image. Consequently, for each recipe the documentation
1269files are separated into a ``-doc`` package. Recipes that package
1270software containing optional modules or plugins might undergo additional
1271package splitting as well.
1272
1273After building a recipe, you can see where files have gone by looking in
1274the ``oe-workdir/packages-split`` directory, which contains a
1275subdirectory for each package. Apart from some advanced cases, the
1276:term:`PACKAGES` and :term:`FILES` variables controls
1277splitting. The :term:`PACKAGES` variable lists all of the packages to be
1278produced, while the :term:`FILES` variable specifies which files to include
1279in each package by using an override to specify the package. For
1280example, ``FILES:${PN}`` specifies the files to go into the main package
1281(i.e. the main package has the same name as the recipe and
1282``${``\ :term:`PN`\ ``}`` evaluates to the
1283recipe name). The order of the :term:`PACKAGES` value is significant. For
1284each installed file, the first package whose :term:`FILES` value matches the
1285file is the package into which the file goes. Both the :term:`PACKAGES` and
1286:term:`FILES` variables have default values. Consequently, you might find
1287you do not even need to set these variables in your recipe unless the
1288software the recipe is building installs files into non-standard
1289locations.
1290
1291Restoring the Target Device to its Original State
1292=================================================
1293
1294If you use the ``devtool deploy-target`` command to write a recipe's
1295build output to the target, and you are working on an existing component
1296of the system, then you might find yourself in a situation where you
1297need to restore the original files that existed prior to running the
1298``devtool deploy-target`` command. Because the ``devtool deploy-target``
1299command backs up any files it overwrites, you can use the
1300``devtool undeploy-target`` command to restore those files and remove
1301any other files the recipe deployed. Consider the following example::
1302
1303 $ devtool undeploy-target lighttpd root@192.168.7.2
1304
1305If you have deployed
1306multiple applications, you can remove them all using the "-a" option
1307thus restoring the target device to its original state::
1308
1309 $ devtool undeploy-target -a root@192.168.7.2
1310
1311Information about files deployed to
1312the target as well as any backed up files are stored on the target
1313itself. This storage, of course, requires some additional space on the
1314target machine.
1315
1316.. note::
1317
1318 The ``devtool deploy-target`` and ``devtool undeploy-target`` commands do
1319 not currently interact with any package management system on the target
1320 device (e.g. RPM or OPKG). Consequently, you should not intermingle
1321 ``devtool deploy-target`` and package manager operations on the target
1322 device. Doing so could result in a conflicting set of files.
diff --git a/documentation/dev-manual/disk-space.rst b/documentation/dev-manual/disk-space.rst
index 6d1638a302..ba3afa5a2c 100644
--- a/documentation/dev-manual/disk-space.rst
+++ b/documentation/dev-manual/disk-space.rst
@@ -49,10 +49,10 @@ requires a full build environment to be available and doesn't work well
49covering multiple releases. It won't work either on limited environments 49covering multiple releases. It won't work either on limited environments
50such as BSD based NAS:: 50such as BSD based NAS::
51 51
52 sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache 52 sstate-cache-management.py --remove-duplicated --cache-dir=sstate-cache
53 53
54This command will ask you to confirm the deletions it identifies. 54This command will ask you to confirm the deletions it identifies.
55Run ``sstate-cache-management.sh`` for more details about this script. 55Run ``sstate-cache-management.py`` for more details about this script.
56 56
57.. note:: 57.. note::
58 58
diff --git a/documentation/dev-manual/efficiently-fetching-sources.rst b/documentation/dev-manual/efficiently-fetching-sources.rst
index a15f0a92ce..a3366226c0 100644
--- a/documentation/dev-manual/efficiently-fetching-sources.rst
+++ b/documentation/dev-manual/efficiently-fetching-sources.rst
@@ -6,7 +6,7 @@ Efficiently Fetching Source Files During a Build
6The OpenEmbedded build system works with source files located through 6The OpenEmbedded build system works with source files located through
7the :term:`SRC_URI` variable. When 7the :term:`SRC_URI` variable. When
8you build something using BitBake, a big part of the operation is 8you build something using BitBake, a big part of the operation is
9locating and downloading all the source tarballs. For images, 9locating and downloading all of the source code. For images,
10downloading all the source for various packages can take a significant 10downloading all the source for various packages can take a significant
11amount of time. 11amount of time.
12 12
@@ -18,7 +18,7 @@ Setting up Effective Mirrors
18============================ 18============================
19 19
20A good deal that goes into a Yocto Project build is simply downloading 20A good deal that goes into a Yocto Project build is simply downloading
21all of the source tarballs. Maybe you have been working with another 21source code. Maybe you have been working with another
22build system for which you have built up a 22build system for which you have built up a
23sizable directory of source tarballs. Or, perhaps someone else has such 23sizable directory of source tarballs. Or, perhaps someone else has such
24a directory for which you have read access. If so, you can save time by 24a directory for which you have read access. If so, you can save time by
diff --git a/documentation/dev-manual/external-scm.rst b/documentation/dev-manual/external-scm.rst
index 97a7e63e36..896b1b5ac7 100644
--- a/documentation/dev-manual/external-scm.rst
+++ b/documentation/dev-manual/external-scm.rst
@@ -12,10 +12,13 @@ revision number for changes. Currently, you can do this with Apache
12Subversion (SVN), Git, and Bazaar (BZR) repositories. 12Subversion (SVN), Git, and Bazaar (BZR) repositories.
13 13
14To enable this behavior, the :term:`PV` of 14To enable this behavior, the :term:`PV` of
15the recipe needs to reference 15the recipe needs to include a ``+`` sign in its assignment.
16:term:`SRCPV`. Here is an example:: 16Here is an example::
17 17
18 PV = "1.2.3+git${SRCPV}" 18 PV = "1.2.3+git"
19
20:term:`Bitbake` later includes the source control information in :term:`PKGV`
21during the packaging phase.
19 22
20Then, you can add the following to your 23Then, you can add the following to your
21``local.conf``:: 24``local.conf``::
diff --git a/documentation/dev-manual/figures/devtool-add-flow.png b/documentation/dev-manual/figures/devtool-add-flow.png
new file mode 100644
index 0000000000..e7d6173d2d
--- /dev/null
+++ b/documentation/dev-manual/figures/devtool-add-flow.png
Binary files differ
diff --git a/documentation/dev-manual/figures/devtool-modify-flow.png b/documentation/dev-manual/figures/devtool-modify-flow.png
new file mode 100644
index 0000000000..18ba8b7e65
--- /dev/null
+++ b/documentation/dev-manual/figures/devtool-modify-flow.png
Binary files differ
diff --git a/documentation/dev-manual/figures/devtool-upgrade-flow.png b/documentation/dev-manual/figures/devtool-upgrade-flow.png
new file mode 100644
index 0000000000..7d4f395e24
--- /dev/null
+++ b/documentation/dev-manual/figures/devtool-upgrade-flow.png
Binary files differ
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
index 9ccf60f701..63736e0abf 100644
--- a/documentation/dev-manual/index.rst
+++ b/documentation/dev-manual/index.rst
@@ -20,9 +20,11 @@ Yocto Project Development Tasks Manual
20 development-shell 20 development-shell
21 python-development-shell 21 python-development-shell
22 building 22 building
23 multiconfig
23 speeding-up-build 24 speeding-up-build
24 libraries 25 libraries
25 prebuilt-libraries 26 prebuilt-libraries
27 devtool
26 x32-psabi 28 x32-psabi
27 gobject-introspection 29 gobject-introspection
28 external-toolchain 30 external-toolchain
@@ -39,7 +41,6 @@ Yocto Project Development Tasks Manual
39 external-scm 41 external-scm
40 read-only-rootfs 42 read-only-rootfs
41 build-quality 43 build-quality
42 runtime-testing
43 debugging 44 debugging
44 licenses 45 licenses
45 security-subjects 46 security-subjects
@@ -48,5 +49,6 @@ Yocto Project Development Tasks Manual
48 error-reporting-tool 49 error-reporting-tool
49 wayland 50 wayland
50 qemu 51 qemu
52 bblock
51 53
52.. include:: /boilerplate.rst 54.. include:: /boilerplate.rst
diff --git a/documentation/dev-manual/init-manager.rst b/documentation/dev-manual/init-manager.rst
index 20d61ea830..d0fdfdf773 100644
--- a/documentation/dev-manual/init-manager.rst
+++ b/documentation/dev-manual/init-manager.rst
@@ -44,7 +44,7 @@ therefore increasing the total system boot time. systemd also substantially
44increases system size because of its multiple components and the extra 44increases system size because of its multiple components and the extra
45dependencies it pulls. 45dependencies it pulls.
46 46
47On the contrary, BusyBox init is the simplest and the lightest solution and 47By contrast, BusyBox init is the simplest and the lightest solution and
48also comes with BusyBox mdev as device manager, a lighter replacement to 48also comes with BusyBox mdev as device manager, a lighter replacement to
49:wikipedia:`udev <Udev>`, which SysVinit and systemd both use. 49:wikipedia:`udev <Udev>`, which SysVinit and systemd both use.
50 50
@@ -121,7 +121,7 @@ increasing levels of complexity and functionality:
121 :widths: 40 20 20 20 121 :widths: 40 20 20 20
122 :header-rows: 1 122 :header-rows: 1
123 123
124 * - 124 * -
125 - BusyBox init 125 - BusyBox init
126 - SysVinit 126 - SysVinit
127 - systemd 127 - systemd
diff --git a/documentation/dev-manual/layers.rst b/documentation/dev-manual/layers.rst
index f7929e630e..67482bf544 100644
--- a/documentation/dev-manual/layers.rst
+++ b/documentation/dev-manual/layers.rst
@@ -80,7 +80,7 @@ Follow these general steps to create your layer without using tools:
80 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" 80 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
81 BBFILE_PRIORITY_yoctobsp = "5" 81 BBFILE_PRIORITY_yoctobsp = "5"
82 LAYERVERSION_yoctobsp = "4" 82 LAYERVERSION_yoctobsp = "4"
83 LAYERSERIES_COMPAT_yoctobsp = "dunfell" 83 LAYERSERIES_COMPAT_yoctobsp = "walnascar"
84 84
85 Here is an explanation of the layer configuration file: 85 Here is an explanation of the layer configuration file:
86 86
@@ -306,7 +306,7 @@ The Yocto Project Compatibility Program consists of a layer application
306process that requests permission to use the Yocto Project Compatibility 306process that requests permission to use the Yocto Project Compatibility
307Logo for your layer and application. The process consists of two parts: 307Logo for your layer and application. The process consists of two parts:
308 308
309#. Successfully passing a script (``yocto-check-layer``) that when run 309#. Successfully passing a script (``yocto-check-layer``) that, when run
310 against your layer, tests it against constraints based on experiences 310 against your layer, tests it against constraints based on experiences
311 of how layers have worked in the real world and where pitfalls have 311 of how layers have worked in the real world and where pitfalls have
312 been found. Getting a "PASS" result from the script is required for 312 been found. Getting a "PASS" result from the script is required for
@@ -470,11 +470,23 @@ corresponding recipe file. For example, the append file
470means the original recipe and append filenames are version 470means the original recipe and append filenames are version
471number-specific. If the corresponding recipe is renamed to update to a 471number-specific. If the corresponding recipe is renamed to update to a
472newer version, you must also rename and possibly update the 472newer version, you must also rename and possibly update the
473corresponding ``.bbappend`` as well. During the build process, BitBake 473corresponding ``.bbappend`` as well.
474displays an error on starting if it detects a ``.bbappend`` file that 474
475does not have a corresponding recipe with a matching name. See the 475During the build process, BitBake displays an error on startup if it detects a
476:term:`BB_DANGLINGAPPENDS_WARNONLY` 476``.bbappend`` file that does not have a corresponding recipe with a matching
477variable for information on how to handle this error. 477name. To handle these errors, the best practice is to rename the ``.bbappend``
478to match the original recipe version. This also gives you the opportunity to see
479if the ``.bbappend`` is still relevant for the new version of the recipe.
480
481Another method is to use the character ``%`` in the ``.bbappend`` filename. For
482example, to append information to every ``6.*`` minor versions of the recipe
483``someapp``, the ``someapp_6.%.bbappend`` file can be created. This way, an
484error will only be triggered if the ``someapp`` recipe has a major version
485update.
486
487Finally, another method to deal with these errors is to use the variable
488:term:`BBMASK`, especially in cases where modifying the ``.bbappend`` is not
489possible.
478 490
479Overlaying a File Using Your Layer 491Overlaying a File Using Your Layer
480---------------------------------- 492----------------------------------
@@ -492,21 +504,20 @@ the "meta" layer at ``meta/recipes-bsp/formfactor``::
492 SECTION = "base" 504 SECTION = "base"
493 LICENSE = "MIT" 505 LICENSE = "MIT"
494 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" 506 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
495 PR = "r45"
496 507
497 SRC_URI = "file://config file://machconfig" 508 SRC_URI = "file://config file://machconfig"
498 S = "${WORKDIR}" 509 S = "${UNPACKDIR}"
499 510
500 PACKAGE_ARCH = "${MACHINE_ARCH}" 511 PACKAGE_ARCH = "${MACHINE_ARCH}"
501 INHIBIT_DEFAULT_DEPS = "1" 512 INHIBIT_DEFAULT_DEPS = "1"
502 513
503 do_install() { 514 do_install() {
504 # Install file only if it has contents 515 # Install file only if it has contents
505 install -d ${D}${sysconfdir}/formfactor/ 516 install -d ${D}${sysconfdir}/formfactor/
506 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ 517 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
507 if [ -s "${S}/machconfig" ]; then 518 if [ -s "${S}/machconfig" ]; then
508 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ 519 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
509 fi 520 fi
510 } 521 }
511 522
512In the main recipe, note the :term:`SRC_URI` 523In the main recipe, note the :term:`SRC_URI`
@@ -570,11 +581,10 @@ Directory`. Here is the main ``xserver-xf86-config`` recipe, which is named
570 SECTION = "x11/base" 581 SECTION = "x11/base"
571 LICENSE = "MIT" 582 LICENSE = "MIT"
572 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" 583 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
573 PR = "r33"
574 584
575 SRC_URI = "file://xorg.conf" 585 SRC_URI = "file://xorg.conf"
576 586
577 S = "${WORKDIR}" 587 S = "${UNPACKDIR}"
578 588
579 CONFFILES:${PN} = "${sysconfdir}/X11/xorg.conf" 589 CONFFILES:${PN} = "${sysconfdir}/X11/xorg.conf"
580 590
@@ -582,10 +592,10 @@ Directory`. Here is the main ``xserver-xf86-config`` recipe, which is named
582 ALLOW_EMPTY:${PN} = "1" 592 ALLOW_EMPTY:${PN} = "1"
583 593
584 do_install () { 594 do_install () {
585 if test -s ${WORKDIR}/xorg.conf; then 595 if test -s ${UNPACKDIR}/xorg.conf; then
586 install -d ${D}/${sysconfdir}/X11 596 install -d ${D}/${sysconfdir}/X11
587 install -m 0644 ${WORKDIR}/xorg.conf ${D}/${sysconfdir}/X11/ 597 install -m 0644 ${UNPACKDIR}/xorg.conf ${D}/${sysconfdir}/X11/
588 fi 598 fi
589 } 599 }
590 600
591Here is the append file, which is named ``xserver-xf86-config_%.bbappend`` 601Here is the append file, which is named ``xserver-xf86-config_%.bbappend``
@@ -602,8 +612,8 @@ file is in the layer at ``recipes-graphics/xorg-xserver``::
602 PITFT="${@bb.utils.contains("MACHINE_FEATURES", "pitft", "1", "0", d)}" 612 PITFT="${@bb.utils.contains("MACHINE_FEATURES", "pitft", "1", "0", d)}"
603 if [ "${PITFT}" = "1" ]; then 613 if [ "${PITFT}" = "1" ]; then
604 install -d ${D}/${sysconfdir}/X11/xorg.conf.d/ 614 install -d ${D}/${sysconfdir}/X11/xorg.conf.d/
605 install -m 0644 ${WORKDIR}/xorg.conf.d/98-pitft.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ 615 install -m 0644 ${UNPACKDIR}/xorg.conf.d/98-pitft.conf ${D}/${sysconfdir}/X11/xorg.conf.d/
606 install -m 0644 ${WORKDIR}/xorg.conf.d/99-calibration.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ 616 install -m 0644 ${UNPACKDIR}/xorg.conf.d/99-calibration.conf ${D}/${sysconfdir}/X11/xorg.conf.d/
607 fi 617 fi
608 } 618 }
609 619
@@ -644,6 +654,96 @@ variable and append the layer's root name::
644 order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake 654 order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake
645 might address this. 655 might address this.
646 656
657Providing Global-level Configurations With Your Layer
658-----------------------------------------------------
659
660When creating a layer, you may need to define configurations that should take
661effect globally in your build environment when the layer is part of the build.
662The ``layer.conf`` file is a :term:`configuration file` that affects the build
663system globally, so it is a candidate for this use-case.
664
665.. warning::
666
667 Providing unconditional global level configuration from the ``layer.conf``
668 file is *not* a good practice, and should be avoided. For this reason, the
669 section :ref:`ref-conditional-layer-confs` below shows how the ``layer.conf``
670 file can be used to provide configurations only if a certain condition is
671 met.
672
673For example, if your layer provides a Linux kernel recipe named
674``linux-custom``, you may want to make :term:`PREFERRED_PROVIDER_virtual/kernel
675<PREFERRED_PROVIDER>` point to ``linux-custom``::
676
677 PREFERRED_PROVIDER_virtual/kernel = "linux-custom"
678
679This can be defined in the ``layer.conf`` file. If your layer is at the last
680position in the :term:`BBLAYERS` list, it will take precedence over previous
681``PREFERRED_PROVIDER_virtual/kernel`` assignments (unless one is set from a
682:term:`configuration file` that is parsed later, such as machine or distro
683configuration files).
684
685.. _ref-conditional-layer-confs:
686
687Conditionally Provide Global-level Configurations With Your Layer
688^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
689
690In some cases, your layer may provide global configurations only if some
691features it provides are enabled. Since the ``layer.conf`` file is parsed at an
692earlier stage in the parsing process, the :term:`DISTRO_FEATURES` and
693:term:`MACHINE_FEATURES` variables are not yet available to ``layer.conf``, and
694declaring conditional assignments based on these variables is not possible. The
695following technique shows a way to bypass this limitation by using the
696:term:`USER_CLASSES` variable and a conditional ``require`` command.
697
698In the following steps, let's assume our layer is named ``meta-mylayer`` and
699that this layer defines a custom :ref:`distro feature <ref-features-distro>`
700named ``mylayer-kernel``. We will set the :term:`PREFERRED_PROVIDER` variable
701for the kernel only if our feature ``mylayer-kernel`` is part of the
702:term:`DISTRO_FEATURES`:
703
704#. Create an include file in the directory
705 ``meta-mylayer/conf/distro/include/``, for example a file named
706 ``mylayer-kernel-provider.inc`` that sets the kernel provider to
707 ``linux-custom``::
708
709 PREFERRED_PROVIDER_virtual/kernel = "linux-custom"
710
711#. Provide a path to this include file in your ``layer.conf``::
712
713 META_MYLAYER_KERNEL_PROVIDER_PATH = "${LAYERDIR}/conf/distro/include/mylayer-kernel-provider.inc"
714
715#. Create a new class in ``meta-mylayer/classes-global/``, for example a class
716 ``meta-mylayer-cfg.bbclass``. Make it conditionally require the file
717 ``mylayer-kernel-provider.inc`` defined above, using the variable
718 ``META_MYLAYER_KERNEL_PROVIDER_PATH`` defined in ``layer.conf``::
719
720 require ${@bb.utils.contains('DISTRO_FEATURES', 'mylayer-kernel', '${META_MYLAYER_KERNEL_PROVIDER_PATH}', '', d)}
721
722 For details on the ``bb.utils.contains`` function, see its definition in
723 :bitbake_git:`lib/bb/utils.py </tree/lib/bb/utils.py>`.
724
725 .. note::
726
727 The ``require`` command is designed to not fail if the function
728 ``bb.utils.contains`` returns an empty string.
729
730#. Back to your ``layer.conf`` file, add the class ``meta-mylayer-cfg`` class to
731 the :term:`USER_CLASSES` variable::
732
733 USER_CLASSES:append = " meta-mylayer-cfg"
734
735 This will add the class ``meta-mylayer-cfg`` to the list of classes to
736 globally inherit. Since the ``require`` command is conditional in
737 ``meta-mylayer-cfg.bbclass``, even though inherited the class will have no
738 effect unless the feature ``mylayer-kernel`` is enabled through
739 :term:`DISTRO_FEATURES`.
740
741This technique can also be used for :ref:`Machine features
742<ref-features-machine>` by following the same steps. Though not mandatory, it is
743recommended to put include files for :term:`DISTRO_FEATURES` in your layer's
744``conf/distro/include`` and the ones for :term:`MACHINE_FEATURES` in your
745layer's ``conf/machine/include``.
746
647Managing Layers 747Managing Layers
648=============== 748===============
649 749
@@ -732,7 +832,7 @@ The following list describes the available commands:
732 832
733- ``save-build-conf``: Saves the currently active build configuration 833- ``save-build-conf``: Saves the currently active build configuration
734 (``conf/local.conf``, ``conf/bblayers.conf``) as a template into a layer. 834 (``conf/local.conf``, ``conf/bblayers.conf``) as a template into a layer.
735 This template can later be used for setting up builds via :term:``TEMPLATECONF``. 835 This template can later be used for setting up builds via :term:`TEMPLATECONF`.
736 For information about saving and using configuration templates, see 836 For information about saving and using configuration templates, see
737 ":ref:`dev-manual/custom-template-configuration-directory:creating a custom template configuration directory`". 837 ":ref:`dev-manual/custom-template-configuration-directory:creating a custom template configuration directory`".
738 838
diff --git a/documentation/dev-manual/libraries.rst b/documentation/dev-manual/libraries.rst
index 521dbb9a7c..a8c38128ea 100644
--- a/documentation/dev-manual/libraries.rst
+++ b/documentation/dev-manual/libraries.rst
@@ -37,40 +37,10 @@ library files.
37 Some previously released versions of the Yocto Project defined the 37 Some previously released versions of the Yocto Project defined the
38 static library files through ``${PN}-dev``. 38 static library files through ``${PN}-dev``.
39 39
40Here is the part of the BitBake configuration file, where you can see 40Here is a small part of the BitBake configuration file, where you can see
41how the static library files are defined:: 41how the static library files are defined::
42 42
43 PACKAGE_BEFORE_PN ?= ""
44 PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" 43 PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
45 PACKAGES_DYNAMIC = "^${PN}-locale-.*"
46 FILES = ""
47
48 FILES:${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
49 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
50 ${base_bindir}/* ${base_sbindir}/* \
51 ${base_libdir}/*${SOLIBS} \
52 ${base_prefix}/lib/udev ${prefix}/lib/udev \
53 ${base_libdir}/udev ${libdir}/udev \
54 ${datadir}/${BPN} ${libdir}/${BPN}/* \
55 ${datadir}/pixmaps ${datadir}/applications \
56 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
57 ${libdir}/bonobo/servers"
58
59 FILES:${PN}-bin = "${bindir}/* ${sbindir}/*"
60
61 FILES:${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
62 ${datadir}/gnome/help"
63 SECTION:${PN}-doc = "doc"
64
65 FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
66 FILES:${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
67 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
68 ${datadir}/aclocal ${base_libdir}/*.o \
69 ${libdir}/${BPN}/*.la ${base_libdir}/*.la \
70 ${libdir}/cmake ${datadir}/cmake"
71 SECTION:${PN}-dev = "devel"
72 ALLOW_EMPTY:${PN}-dev = "1"
73 RDEPENDS:${PN}-dev = "${PN} (= ${EXTENDPKGV})"
74 44
75 FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" 45 FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
76 SECTION:${PN}-staticdev = "devel" 46 SECTION:${PN}-staticdev = "devel"
diff --git a/documentation/dev-manual/licenses.rst b/documentation/dev-manual/licenses.rst
index bffff3675f..7d6636eeff 100644
--- a/documentation/dev-manual/licenses.rst
+++ b/documentation/dev-manual/licenses.rst
@@ -55,11 +55,11 @@ Consider this next example::
55 55
56 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ 56 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
57 md5=bb14ed3c4cda583abc85401304b5cd4e" 57 md5=bb14ed3c4cda583abc85401304b5cd4e"
58 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" 58 LIC_FILES_CHKSUM = "file://${UNPACKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
59 59
60The first line locates a file in ``${S}/src/ls.c`` and isolates lines 60The first line locates a file in ``${S}/src/ls.c`` and isolates lines
61five through 16 as license text. The second line refers to a file in 61five through 16 as license text. The second line refers to a file in
62:term:`WORKDIR`. 62:term:`UNPACKDIR`.
63 63
64Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes, 64Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes,
65unless the :term:`LICENSE` variable is set to "CLOSED". 65unless the :term:`LICENSE` variable is set to "CLOSED".
diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst
new file mode 100644
index 0000000000..71fe542efb
--- /dev/null
+++ b/documentation/dev-manual/multiconfig.rst
@@ -0,0 +1,312 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Building Images for Multiple Targets With Multiconfig
4*****************************************************
5
6You can use a single ``bitbake`` command to build multiple images or
7packages for different targets where each image or package requires a
8different configuration (multiple configuration builds). The builds, in
9this scenario, are sometimes referred to as "multiconfigs", and this
10section uses that term throughout.
11
12This section describes how to set up for multiple configuration builds
13and how to account for cross-build dependencies between the
14multiconfigs.
15
16Setting Up and Running a Multiple Configuration Build
17=====================================================
18
19To accomplish a multiple configuration build, you must define each
20target's configuration separately using a parallel :term:`configuration file` in
21the :term:`Build Directory` or configuration directory within a layer, and you
22must follow a required file hierarchy. Additionally, you must enable the
23multiple configuration builds in your ``local.conf`` file.
24
25Follow these steps to set up and execute multiple configuration builds:
26
27- *Create Separate Configuration Files*: You need to create a single
28 :term:`Configuration File` for each build target (each multiconfig).
29 The configuration definitions are implementation dependent but often
30 each configuration file will define the :term:`MACHINE` and the
31 temporary directory (:term:`TMPDIR`) BitBake uses for the build.
32
33 .. note::
34
35 Whether the same temporary directory (:term:`TMPDIR`) can be shared will
36 depend on what is similar and what is different between the
37 configurations. Multiple :term:`MACHINE` targets can share the same
38 :term:`TMPDIR` as long as the rest of the configuration is the same,
39 multiple :term:`DISTRO` settings would need separate :term:`TMPDIR`
40 directories.
41
42 For example, consider a scenario with two different multiconfigs for the same
43 :term:`MACHINE`: "qemux86" built for two distributions such as "poky" and
44 "poky-lsb". In this case, you would need to use two different :term:`TMPDIR`.
45
46 In the general case, using separate :term:`TMPDIR` for the different
47 multiconfigs is strongly recommended.
48
49 The location for these multiconfig configuration files is specific.
50 They must reside in the current :term:`Build Directory` in a sub-directory of
51 ``conf`` named ``multiconfig`` or within a :term:`Layer`'s ``conf`` directory
52 under a directory named ``multiconfig``. Here is an example that defines
53 two configuration files for the "x86" and "arm" multiconfigs:
54
55 .. image:: figures/multiconfig_files.png
56 :align: center
57 :width: 50%
58
59 The usual :term:`BBPATH` search path is used to locate multiconfig files in
60 a similar way to other configuration files.
61
62 Here is an example showing the minimal statements needed in a
63 :term:`configuration file` named ``qemux86.conf`` for a ``qemux86`` target
64 whose temporary build directory is ``tmp-qemux86``::
65
66 MACHINE = "qemux86"
67 TMPDIR .= "-${BB_CURRENT_MC}"
68
69 BitBake will expand the :term:`BB_CURRENT_MC` variable to the value of the
70 current multiconfig in use. We append this value to :term:`TMPDIR` so that
71 any change on the definition of :term:`TMPDIR` will automatically affect the
72 value of :term:`TMPDIR` for each multiconfig.
73
74- *Add the BitBake Multi-configuration Variable to the Local
75 Configuration File*: Use the
76 :term:`BBMULTICONFIG`
77 variable in your ``conf/local.conf`` configuration file to specify
78 each multiconfig. Continuing with the example from the previous
79 figure, the :term:`BBMULTICONFIG` variable needs to enable two
80 multiconfigs: "x86" and "arm" by specifying each configuration file::
81
82 BBMULTICONFIG = "x86 arm"
83
84 .. note::
85
86 A "default" configuration already exists by definition. This
87 configuration is named: "" (i.e. empty string) and is defined by
88 the variables coming from your ``local.conf``
89 file. Consequently, the previous example actually adds two
90 additional configurations to your build: "arm" and "x86" along
91 with "".
92
93- *Launch BitBake*: Use the following BitBake command form to launch
94 the multiple configuration build::
95
96 $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ]
97
98 For the example in this section, the following command applies::
99
100 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
101
102 The previous BitBake command builds several components:
103
104 - A ``core-image-minimal`` image that is configured through the ``x86.conf``
105 configuration file
106
107 - A ``core-image-sato`` image that is configured through the ``arm.conf``
108 configuration file
109
110 - A ``core-image-base`` that is configured through your ``local.conf``
111 configuration file
112
113.. note::
114
115 Support for multiple configuration builds in the Yocto Project &DISTRO;
116 (&DISTRO_NAME;) Release does not include Shared State (sstate)
117 optimizations. Consequently, if a build uses the same object twice
118 in, for example, two different :term:`TMPDIR`
119 directories, the build either loads from an existing sstate cache for
120 that build at the start or builds the object fresh.
121
122Enabling Multiple Configuration Build Dependencies
123==================================================
124
125Sometimes dependencies can exist between targets (multiconfigs) in a
126multiple configuration build. For example, suppose that in order to
127build a ``core-image-sato`` image for an "x86" multiconfig, the root
128filesystem of an "arm" multiconfig must exist. This dependency is
129essentially that the
130:ref:`ref-tasks-image` task in the
131``core-image-sato`` recipe depends on the completion of the
132:ref:`ref-tasks-rootfs` task of the
133``core-image-minimal`` recipe.
134
135To enable dependencies in a multiple configuration build, you must
136declare the dependencies in the recipe using the following statement
137form::
138
139 task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
140
141To better show how to use this statement, consider the example scenario
142from the first paragraph of this section. The following statement needs
143to be added to the recipe that builds the ``core-image-sato`` image::
144
145 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
146
147In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The
148task on which the :ref:`ref-tasks-image` task in the recipe depends is the
149:ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated
150with the "arm" multiconfig.
151
152Once you set up this dependency, you can build the "x86" multiconfig
153using a BitBake command as follows::
154
155 $ bitbake mc:x86:core-image-sato
156
157This command executes all the tasks needed to create the
158``core-image-sato`` image for the "x86" multiconfig. Because of the
159dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the
160"arm" multiconfig build.
161
162Having a recipe depend on the root filesystem of another build might not
163seem that useful. Consider this change to the statement in the
164``core-image-sato`` recipe::
165
166 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
167
168In this case, BitBake must
169create the ``core-image-minimal`` image for the "arm" build since the
170"x86" build depends on it.
171
172Because "x86" and "arm" are enabled for multiple configuration builds
173and have separate configuration files, BitBake places the artifacts for
174each build in the respective temporary build directories (i.e.
175:term:`TMPDIR`).
176
177Suggested best practices
178========================
179
180- :term:`TMPDIR` (other than the default set in bitbake.conf) is only set in
181 ``local.conf`` by the user. This means that we should **not** manipulate
182 :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration
183 file`.
184
185- A multiconfig should specify a :term:`TMPDIR`, and should specify it by
186 appending the multiconfig name with :term:`BB_CURRENT_MC`.
187
188- Recipes that are used to transfer the output from a multiconfig build to
189 another should use ``do_task[mcdepends]`` to trigger the build of the
190 component, and then transfer the item to the current configuration in
191 :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy`, assuming the value of
192 the deployed item based on :term:`TMPDIR`.
193
194 The :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy` tasks should look
195 like this::
196
197 do_install() {
198 install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${D}/some/path
199 }
200
201 do_deploy() {
202 install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${DEPLOYDIR}/somefile
203 }
204
205 In the example above:
206
207 - ``<multiconfig>`` is the multiconfig name as set by the multiconfig
208 :term:`configuration file` (see the :ref:`dev-manual/multiconfig:Setting
209 Up and Running a Multiple Configuration Build` section above).
210
211 - ``<machine>`` must be the :term:`MACHINE` for which ``somefile`` was built
212 and deployed. This value may differ from the current :term:`MACHINE` if
213 the multiconfig :term:`configuration file` overrides it.
214
215- Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1``
216 if they don't rely on default dependencies such as the standard C library.
217
218Common use case: building baremetal firmware alongside a Linux build
219====================================================================
220
221A common use case for multiconfig is to use the default configuration as the
222regular Linux build, while one or more multiconfigs can be used to build special
223components, such as baremetal firmware. It would also apply to a scenario where
224a microcontroller, for example, is companion to a main processor where Linux is
225running. This section details how one can achieve these kinds of scenarios with
226a multiconfig build.
227
228Adding a multiconfig configuration file and recipe for a baremetal firmware
229---------------------------------------------------------------------------
230
231As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple
232Configuration Build`, each multiconfig will require a separate
233:term:`Configuration File`. In addition, we will define a separate
234:term:`TMPDIR` for our baremetal firmware build configuration.
235
236For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf``
237as follows::
238
239 TMPDIR .= "-${BB_CURRENT_MC}"
240 TCLIBC = "newlib"
241
242The ``baremetal-firmware.conf`` file configures a separate :term:`TMPDIR` for
243holding binaries compiled with the `newlib <https://sourceware.org/newlib/>`__
244toolchain (see :term:`TCLIBC`).
245
246.. note::
247
248 Here, the default :term:`MACHINE` is not overridden by the multiconfig
249 configuration file. As a consequence, the architecture of the built baremetal
250 binaries will be the same. In other cases, where the firmware runs on a
251 completely different architecture, the :term:`MACHINE` must be overridden.
252
253We then create a recipe ``my-firmware.bb`` that defines how the baremetal
254firmware is built. The recipe should contain enough information for the
255:term:`OpenEmbedded build system` to properly compile the firmware with our
256toolchain. The building tasks may vary depending on the nature of the firmware.
257However, the recipe should define a :ref:`ref-classes-deploy` task that deploys
258the output into the :term:`DEPLOYDIR` directory. We will consider in the
259following that the file is named ``my-firmware.elf``.
260
261Building the firmware
262---------------------
263
264The firmware can be built with BitBake with the following command::
265
266 $ bitbake mc:baremetal-firmware:my-firmware
267
268However, we would prefer for ``my-firmware`` to be automatically built when
269triggering a normal Linux build.
270
271Using a ``mcdepend``, a recipe belonging to the Linux build can trigger the
272build of ``my-firmware``. For example, let's consider that our Linux build needs
273to assemble a "special" firmware that uses the output of our ``my-firmware``
274recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this
275dependency in ``my-parent-firmware.bb`` with::
276
277 do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy"
278
279The above will ensure that when the :ref:`ref-tasks-compile` task of
280``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of
281``my-firmware`` will already have run successfully.
282
283Using the output of ``my-firmware``
284-----------------------------------
285
286After ``my-firmware`` recipe has deployed ``my-firmware.elf``, we need to use
287the output in some way. We can make a series of assumptions, based on the
288default Yocto Project variables in order to get the binary for packaging.
289
290First, we can set the following in ``my-parent-firmware.bb``::
291
292 FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf"
293 FIRMWARE_FILE[vardepsexclude] += "TMPDIR"
294
295The first assignment stores the value of the path to the firmware built and
296deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the
297:term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies ---
298meaning that changing the value of :term:`TMPDIR` (for example, changing the
299host on which the firmware is built) will not invalidate the :ref:`shared state
300cache <overview-manual/concepts:shared state cache>`.
301
302Additionally, ``<machine>`` should be replaced by the :term:`MACHINE` for which
303we are building in the baremetal-firmware context.
304
305We can then add a :ref:`ref-tasks-install` task to ``my-parent-firmware``::
306
307 do_install() {
308 install -Dm 0644 ${FIRMWARE_FILE} ${D}/lib/firmware/my-firmware.elf
309 }
310
311Doing the above will allow the firmware binary to be transferred and packaged
312into the Linux context and rootfs.
diff --git a/documentation/dev-manual/new-recipe.rst b/documentation/dev-manual/new-recipe.rst
index 61fc2eb122..832aa300e1 100644
--- a/documentation/dev-manual/new-recipe.rst
+++ b/documentation/dev-manual/new-recipe.rst
@@ -56,7 +56,7 @@ necessary when adding a recipe to build a new piece of software to be
56included in a build. 56included in a build.
57 57
58You can find a complete description of the ``devtool add`` command in 58You can find a complete description of the ``devtool add`` command in
59the ":ref:`sdk-manual/extensible:a closer look at \`\`devtool add\`\``" section 59the ":ref:`dev-manual/devtool:a closer look at ``devtool add```" section
60in the Yocto Project Application Development and the Extensible Software 60in the Yocto Project Application Development and the Extensible Software
61Development Kit (eSDK) manual. 61Development Kit (eSDK) manual.
62 62
@@ -188,13 +188,14 @@ the recipe.
188 Use lower-cased characters and do not include the reserved suffixes 188 Use lower-cased characters and do not include the reserved suffixes
189 ``-native``, ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use 189 ``-native``, ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use
190 them as part of your recipe name unless the string applies). Here are some 190 them as part of your recipe name unless the string applies). Here are some
191 examples: 191 examples (which includes the use of the string "git" as a special case of a
192 version identifier):
192 193
193 .. code-block:: none 194 .. code-block:: none
194 195
195 cups_1.7.0.bb 196 cups_2.4.12.bb
196 gawk_4.0.2.bb 197 gawk_5.3.2.bb
197 irssi_0.8.16-rc1.bb 198 psplash_git.bb
198 199
199Running a Build on the Recipe 200Running a Build on the Recipe
200============================= 201=============================
@@ -276,11 +277,11 @@ upgrading the recipe to a future version is as simple as renaming the
276recipe to match the new version. 277recipe to match the new version.
277 278
278Here is a simple example from the 279Here is a simple example from the
279``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source 280:oe_git:`strace recipe </openembedded-core/tree/meta/recipes-devtools/strace>`
280comes from a single tarball. Notice the use of the 281where the source comes from a single tarball. Notice the use of the
281:term:`PV` variable:: 282:term:`PV` variable::
282 283
283 SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \ 284 SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/strace-${PV}.tar.xz \
284 285
285Files mentioned in :term:`SRC_URI` whose names end in a typical archive 286Files mentioned in :term:`SRC_URI` whose names end in a typical archive
286extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so 287extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so
@@ -291,13 +292,13 @@ another example that specifies these types of files, see the
291 292
292Another way of specifying source is from an SCM. For Git repositories, 293Another way of specifying source is from an SCM. For Git repositories,
293you must specify :term:`SRCREV` and you should specify :term:`PV` to include 294you must specify :term:`SRCREV` and you should specify :term:`PV` to include
294the revision with :term:`SRCPV`. Here is an example from the recipe 295a ``+`` sign in its definition. Here is an example from the recipe
295``meta/recipes-core/musl/gcompat_git.bb``:: 296:oe_git:`l3afpad_git.bb </openembedded-core/tree/meta/recipes-sato/l3afpad/l3afpad_git.bb>`::
296 297
297 SRC_URI = "git://git.adelielinux.org/adelie/gcompat.git;protocol=https;branch=current" 298 SRC_URI = "git://github.com/stevenhoneyman/l3afpad.git;branch=master;protocol=https"
298 299
299 PV = "1.0.0+1.1+git${SRCPV}" 300 PV = "0.8.18.1.11+git"
300 SRCREV = "af5a49e489fdc04b9cf02547650d7aeaccd43793" 301 SRCREV ="3cdccdc9505643e50f8208171d9eee5de11a42ff"
301 302
302If your :term:`SRC_URI` statement includes URLs pointing to individual files 303If your :term:`SRC_URI` statement includes URLs pointing to individual files
303fetched from a remote server other than a version control system, 304fetched from a remote server other than a version control system,
@@ -347,8 +348,8 @@ paste them into your recipe and then run the build again to continue.
347 continuing with the build. 348 continuing with the build.
348 349
349This final example is a bit more complicated and is from the 350This final example is a bit more complicated and is from the
350``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The 351:oe_git:`rxvt-unicode </openembedded-core/tree/meta/recipes-sato/rxvt-unicode>`
351example's :term:`SRC_URI` statement identifies multiple files as the source 352recipe. The example's :term:`SRC_URI` statement identifies multiple files as the source
352files for the recipe: a tarball, a patch file, a desktop file, and an icon:: 353files for the recipe: a tarball, a patch file, a desktop file, and an icon::
353 354
354 SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ 355 SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \
@@ -706,7 +707,7 @@ hierarchy to locations that would mirror their locations on the target
706device. The installation process copies files from the 707device. The installation process copies files from the
707``${``\ :term:`S`\ ``}``, 708``${``\ :term:`S`\ ``}``,
708``${``\ :term:`B`\ ``}``, and 709``${``\ :term:`B`\ ``}``, and
709``${``\ :term:`WORKDIR`\ ``}`` 710``${``\ :term:`UNPACKDIR`\ ``}``
710directories to the ``${``\ :term:`D`\ ``}`` 711directories to the ``${``\ :term:`D`\ ``}``
711directory to create the structure as it should appear on the target 712directory to create the structure as it should appear on the target
712system. 713system.
@@ -1145,7 +1146,7 @@ Building an application from a single file that is stored locally (e.g. under
1145``files``) requires a recipe that has the file listed in the :term:`SRC_URI` 1146``files``) requires a recipe that has the file listed in the :term:`SRC_URI`
1146variable. Additionally, you need to manually write the :ref:`ref-tasks-compile` 1147variable. Additionally, you need to manually write the :ref:`ref-tasks-compile`
1147and :ref:`ref-tasks-install` tasks. The :term:`S` variable defines the 1148and :ref:`ref-tasks-install` tasks. The :term:`S` variable defines the
1148directory containing the source code, which is set to :term:`WORKDIR` in this 1149directory containing the source code, which is set to :term:`UNPACKDIR` in this
1149case --- the directory BitBake uses for the build:: 1150case --- the directory BitBake uses for the build::
1150 1151
1151 SUMMARY = "Simple helloworld application" 1152 SUMMARY = "Simple helloworld application"
@@ -1155,7 +1156,7 @@ case --- the directory BitBake uses for the build::
1155 1156
1156 SRC_URI = "file://helloworld.c" 1157 SRC_URI = "file://helloworld.c"
1157 1158
1158 S = "${WORKDIR}" 1159 S = "${UNPACKDIR}"
1159 1160
1160 do_compile() { 1161 do_compile() {
1161 ${CC} ${LDFLAGS} helloworld.c -o helloworld 1162 ${CC} ${LDFLAGS} helloworld.c -o helloworld
@@ -1211,8 +1212,6 @@ In the following example, ``lz4`` is a makefile-based package::
1211 " 1212 "
1212 UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)" 1213 UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)"
1213 1214
1214 S = "${WORKDIR}/git"
1215
1216 CVE_STATUS[CVE-2014-4715] = "fixed-version: Fixed in r118, which is larger than the current version" 1215 CVE_STATUS[CVE-2014-4715] = "fixed-version: Fixed in r118, which is larger than the current version"
1217 1216
1218 EXTRA_OEMAKE = "PREFIX=${prefix} CC='${CC}' CFLAGS='${CFLAGS}' DESTDIR=${D} LIBDIR=${libdir} INCLUDEDIR=${includedir} BUILD_STATIC=no" 1217 EXTRA_OEMAKE = "PREFIX=${prefix} CC='${CC}' CFLAGS='${CFLAGS}' DESTDIR=${D} LIBDIR=${libdir} INCLUDEDIR=${includedir} BUILD_STATIC=no"
@@ -1271,8 +1270,6 @@ is a simple example of an application without dependencies::
1271 SRC_URI = "git://gitlab.com/ipcalc/ipcalc.git;protocol=https;branch=master" 1270 SRC_URI = "git://gitlab.com/ipcalc/ipcalc.git;protocol=https;branch=master"
1272 SRCREV = "4c4261a47f355946ee74013d4f5d0494487cc2d6" 1271 SRCREV = "4c4261a47f355946ee74013d4f5d0494487cc2d6"
1273 1272
1274 S = "${WORKDIR}/git"
1275
1276 inherit meson 1273 inherit meson
1277 1274
1278Applications with dependencies are likely to inherit the 1275Applications with dependencies are likely to inherit the
@@ -1397,11 +1394,31 @@ doing the following:
1397 where you have installed them and whether those files are in 1394 where you have installed them and whether those files are in
1398 different locations than the defaults. 1395 different locations than the defaults.
1399 1396
1397As a basic example of a :ref:`ref-classes-bin-package`-style recipe, consider
1398this snippet from the
1399:oe_git:`wireless-regdb </openembedded-core/tree/meta/recipes-kernel/wireless-regdb>`
1400recipe file, which fetches a single tarball of binary content and manually
1401installs with no need for any configuration or compilation::
1402
1403 SRC_URI = "https://www.kernel.org/pub/software/network/${BPN}/${BP}.tar.xz"
1404 SRC_URI[sha256sum] = "57f8e7721cf5a880c13ae0c202edbb21092a060d45f9e9c59bcd2a8272bfa456"
1405
1406 inherit bin_package allarch
1407
1408 do_install() {
1409 install -d -m0755 ${D}${nonarch_libdir}/crda
1410 install -d -m0755 ${D}${sysconfdir}/wireless-regdb/pubkeys
1411 install -m 0644 regulatory.bin ${D}${nonarch_libdir}/crda/regulatory.bin
1412 install -m 0644 wens.key.pub.pem ${D}${sysconfdir}/wireless-regdb/pubkeys/wens.key.pub.pem
1413 install -m 0644 -D regulatory.db ${D}${nonarch_base_libdir}/firmware/regulatory.db
1414 install -m 0644 regulatory.db.p7s ${D}${nonarch_base_libdir}/firmware/regulatory.db.p7s
1415 }
1416
1400Following Recipe Style Guidelines 1417Following Recipe Style Guidelines
1401================================= 1418=================================
1402 1419
1403When writing recipes, it is good to conform to existing style guidelines. 1420When writing recipes, it is good to conform to existing style guidelines.
1404See the ":doc:`../contributor-guide/recipe-style-guide`" in the Yocto Project 1421See the ":doc:`/contributor-guide/recipe-style-guide`" in the Yocto Project
1405and OpenEmbedded Contributor Guide for reference. 1422and OpenEmbedded Contributor Guide for reference.
1406 1423
1407It is common for existing recipes to deviate a bit from this style. 1424It is common for existing recipes to deviate a bit from this style.
@@ -1428,7 +1445,7 @@ chapter of the BitBake User Manual.
1428 The following example shows some of the ways you can use variables in 1445 The following example shows some of the ways you can use variables in
1429 recipes:: 1446 recipes::
1430 1447
1431 S = "${WORKDIR}/postfix-${PV}" 1448 S = "${UNPACKDIR}/postfix-${PV}"
1432 CFLAGS += "-DNO_ASM" 1449 CFLAGS += "-DNO_ASM"
1433 CFLAGS:append = " --enable-important-feature" 1450 CFLAGS:append = " --enable-important-feature"
1434 1451
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst
index e5028fffdc..8bd48c8e8f 100644
--- a/documentation/dev-manual/packages.rst
+++ b/documentation/dev-manual/packages.rst
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages:
16- :ref:`dev-manual/packages:generating and using signed packages` 16- :ref:`dev-manual/packages:generating and using signed packages`
17 17
18- :ref:`Setting up and running package test 18- :ref:`Setting up and running package test
19 (ptest) <dev-manual/packages:testing packages with ptest>` 19 (ptest) <test-manual/ptest:testing packages with ptest>`
20 20
21- :ref:`dev-manual/packages:creating node package manager (npm) packages` 21- :ref:`dev-manual/packages:creating node package manager (npm) packages`
22 22
@@ -84,10 +84,6 @@ the following:
84 84
85- :term:`PR`: The recipe revision. 85- :term:`PR`: The recipe revision.
86 86
87- :term:`SRCPV`: The OpenEmbedded
88 build system uses this string to help define the value of :term:`PV` when
89 the source code revision needs to be included in it.
90
91- :yocto_wiki:`PR Service </PR_Service>`: A 87- :yocto_wiki:`PR Service </PR_Service>`: A
92 network-based service that helps automate keeping package feeds 88 network-based service that helps automate keeping package feeds
93 compatible with existing package manager applications such as RPM, 89 compatible with existing package manager applications such as RPM,
@@ -256,15 +252,14 @@ the software::
256 252
257 SRCREV = "${AUTOREV}" 253 SRCREV = "${AUTOREV}"
258 254
259Furthermore, you need to reference :term:`SRCPV` in :term:`PV` in order to 255Furthermore, you need to include a ``+`` sign in :term:`PV` in order to
260automatically update the version whenever the revision of the source 256automatically update the version whenever the revision of the source
261code changes. Here is an example:: 257code changes. Here is an example::
262 258
263 PV = "1.0+git${SRCPV}" 259 PV = "1.0+git"
264
265The OpenEmbedded build system substitutes :term:`SRCPV` with the following:
266 260
267.. code-block:: none 261The OpenEmbedded build system will automatically add the source control
262information to the end of the variable :term:`PKGV`, in this format::
268 263
269 AUTOINC+source_code_revision 264 AUTOINC+source_code_revision
270 265
@@ -824,9 +819,19 @@ to use signed package feeds (repositories) when doing a build.
824Signing RPM Packages 819Signing RPM Packages
825-------------------- 820--------------------
826 821
827To enable signing RPM packages, you must set up the following 822To enable signing RPM packages, you must modify the ``rpm``
828configurations in either your ``local.config`` or ``distro.config`` 823recipe configuration to include support for OpenPGP signing.
829file:: 824That may be done either in a ``.bbappend`` for the ``rpm`` recipe::
825
826 PACKAGECONFIG:append = " sequoia"
827
828or in a :term:`Configuration File`::
829
830 PACKAGECONFIG:append:pn-rpm-native = " sequoia"
831 PACKAGECONFIG:append:pn-rpm = " sequoia"
832
833You must also set up the following settings in a
834:term:`Configuration File`::
830 835
831 # Inherit sign_rpm.bbclass to enable signing functionality 836 # Inherit sign_rpm.bbclass to enable signing functionality
832 INHERIT += " sign_rpm" 837 INHERIT += " sign_rpm"
@@ -887,114 +892,8 @@ related to signed package feeds are available:
887Testing Packages With ptest 892Testing Packages With ptest
888=========================== 893===========================
889 894
890A Package Test (ptest) runs tests against packages built by the 895See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the
891OpenEmbedded build system on the target machine. A ptest contains at 896Yocto Project Test Environment Manual.
892least two items: the actual test, and a shell script (``run-ptest``)
893that starts the test. The shell script that starts the test must not
894contain the actual test --- the script only starts the test. On the other
895hand, the test can be anything from a simple shell script that runs a
896binary and checks the output to an elaborate system of test binaries and
897data files.
898
899The test generates output in the format used by Automake::
900
901 result: testname
902
903where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
904the testname can be any identifying string.
905
906For a list of Yocto Project recipes that are already enabled with ptest,
907see the :yocto_wiki:`Ptest </Ptest>` wiki page.
908
909.. note::
910
911 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
912 class.
913
914Adding ptest to Your Build
915--------------------------
916
917To add package testing to your build, add the :term:`DISTRO_FEATURES` and
918:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
919is found in the :term:`Build Directory`::
920
921 DISTRO_FEATURES:append = " ptest"
922 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
923
924Once your build is complete, the ptest files are installed into the
925``/usr/lib/package/ptest`` directory within the image, where ``package``
926is the name of the package.
927
928Running ptest
929-------------
930
931The ``ptest-runner`` package installs a shell script that loops through
932all installed ptest test suites and runs them in sequence. Consequently,
933you might want to add this package to your image.
934
935Getting Your Package Ready
936--------------------------
937
938In order to enable a recipe to run installed ptests on target hardware,
939you need to prepare the recipes that build the packages you want to
940test. Here is what you have to do for each recipe:
941
942- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
943 Include the following line in each recipe::
944
945 inherit ptest
946
947- *Create run-ptest:* This script starts your test. Locate the
948 script where you will refer to it using
949 :term:`SRC_URI`. Here is an
950 example that starts a test for ``dbus``::
951
952 #!/bin/sh
953 cd test
954 make -k runtest-TESTS
955
956- *Ensure dependencies are met:* If the test adds build or runtime
957 dependencies that normally do not exist for the package (such as
958 requiring "make" to run the test suite), use the
959 :term:`DEPENDS` and
960 :term:`RDEPENDS` variables in
961 your recipe in order for the package to meet the dependencies. Here
962 is an example where the package has a runtime dependency on "make"::
963
964 RDEPENDS:${PN}-ptest += "make"
965
966- *Add a function to build the test suite:* Not many packages support
967 cross-compilation of their test suites. Consequently, you usually
968 need to add a cross-compilation function to the package.
969
970 Many packages based on Automake compile and run the test suite by
971 using a single command such as ``make check``. However, the host
972 ``make check`` builds and runs on the same computer, while
973 cross-compiling requires that the package is built on the host but
974 executed for the target architecture (though often, as in the case
975 for ptest, the execution occurs on the host). The built version of
976 Automake that ships with the Yocto Project includes a patch that
977 separates building and execution. Consequently, packages that use the
978 unaltered, patched version of ``make check`` automatically
979 cross-compiles.
980
981 Regardless, you still must add a ``do_compile_ptest`` function to
982 build the test suite. Add a function similar to the following to your
983 recipe::
984
985 do_compile_ptest() {
986 oe_runmake buildtest-TESTS
987 }
988
989- *Ensure special configurations are set:* If the package requires
990 special configurations prior to compiling the test code, you must
991 insert a ``do_configure_ptest`` function into the recipe.
992
993- *Install the test suite:* The :ref:`ref-classes-ptest` class
994 automatically copies the file ``run-ptest`` to the target and then runs make
995 ``install-ptest`` to run the tests. If this is not enough, you need
996 to create a ``do_install_ptest`` function and make sure it gets
997 called after the "make install-ptest" completes.
998 897
999Creating Node Package Manager (NPM) Packages 898Creating Node Package Manager (NPM) Packages
1000============================================ 899============================================
@@ -1125,7 +1024,7 @@ The ``devtool edit-recipe`` command lets you take a look at the recipe::
1125 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ 1024 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
1126 " 1025 "
1127 1026
1128 S = "${WORKDIR}/npm" 1027 S = "${UNPACKDIR}/npm"
1129 1028
1130 inherit npm 1029 inherit npm
1131 1030
diff --git a/documentation/dev-manual/prebuilt-libraries.rst b/documentation/dev-manual/prebuilt-libraries.rst
index a05f39ca1e..59621ca16d 100644
--- a/documentation/dev-manual/prebuilt-libraries.rst
+++ b/documentation/dev-manual/prebuilt-libraries.rst
@@ -97,7 +97,7 @@ The complete recipe would look like this::
97 # we use a local link. 97 # we use a local link.
98 SRC_URI = "file://libft4222-linux-${PV}.tgz" 98 SRC_URI = "file://libft4222-linux-${PV}.tgz"
99 99
100 S = "${WORKDIR}" 100 S = "${UNPACKDIR}"
101 101
102 ARCH_DIR:x86-64 = "build-x86_64" 102 ARCH_DIR:x86-64 = "build-x86_64"
103 ARCH_DIR:i586 = "build-i386" 103 ARCH_DIR:i586 = "build-i386"
@@ -170,7 +170,7 @@ as follows::
170The modifications cause the ``.so`` file to be the real library 170The modifications cause the ``.so`` file to be the real library
171and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into 171and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into
172``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed, 172``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed,
173``${PN}-dev`` collects files before `${PN}`. ``${PN}-dev`` must not collect any of 173``${PN}-dev`` collects files before ``${PN}``. ``${PN}-dev`` must not collect any of
174the files you want in ``${PN}``. 174the files you want in ``${PN}``.
175 175
176Finally, loadable modules, essentially unversioned libraries that are linked 176Finally, loadable modules, essentially unversioned libraries that are linked
@@ -204,6 +204,6 @@ versioned library example. The "magic" is setting the :term:`SOLIBS` and
204 204
205 do_install () { 205 do_install () {
206 install -d ${D}${libdir} 206 install -d ${D}${libdir}
207 install -m 0755 ${WORKDIR}/libfoo.so ${D}${libdir} 207 install -m 0755 ${UNPACKDIR}/libfoo.so ${D}${libdir}
208 } 208 }
209 209
diff --git a/documentation/dev-manual/qemu.rst b/documentation/dev-manual/qemu.rst
index 19f3e40d63..92c93a82ab 100644
--- a/documentation/dev-manual/qemu.rst
+++ b/documentation/dev-manual/qemu.rst
@@ -75,7 +75,7 @@ available. Follow these general steps to run QEMU:
75 your :term:`Build Directory`. 75 your :term:`Build Directory`.
76 76
77 - If you have not built an image, you can go to the 77 - If you have not built an image, you can go to the
78 :yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu/>` area and download a 78 :yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu/>` area and download a
79 pre-built image that matches your architecture and can be run on 79 pre-built image that matches your architecture and can be run on
80 QEMU. 80 QEMU.
81 81
@@ -280,12 +280,11 @@ present, the toolchain is also automatically used.
280 networking. 280 networking.
281 281
282 - SSH servers are available in some QEMU images. The ``core-image-sato`` 282 - SSH servers are available in some QEMU images. The ``core-image-sato``
283 QEMU image has a Dropbear secure shell (SSH) server that runs with 283 QEMU image has a Dropbear secure shell (SSH) server that runs with the
284 the root password disabled. The ``core-image-full-cmdline`` and 284 root password disabled. The ``core-image-full-cmdline`` QEMU image has
285 ``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear. 285 OpenSSH instead of Dropbear. Including these SSH servers allow you to use
286 Including these SSH servers allow you to use standard ``ssh`` and 286 standard ``ssh`` and ``scp`` commands. The ``core-image-minimal`` QEMU
287 ``scp`` commands. The ``core-image-minimal`` QEMU image, however, 287 image, however, contains no SSH server.
288 contains no SSH server.
289 288
290 - You can use a provided, user-space NFS server to boot the QEMU 289 - You can use a provided, user-space NFS server to boot the QEMU
291 session using a local copy of the root filesystem on the host. In 290 session using a local copy of the root filesystem on the host. In
@@ -368,9 +367,6 @@ command line:
368 367
369- `VM`: The virtual machine image, which must be a ``.wic.vmdk`` 368- `VM`: The virtual machine image, which must be a ``.wic.vmdk``
370 file. Use this option when you want to boot a ``.wic.vmdk`` image. 369 file. Use this option when you want to boot a ``.wic.vmdk`` image.
371 The image filename you provide must contain one of the following
372 strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64",
373 "qemumips", "qemuppc", or "qemush4".
374 370
375- `ROOTFS`: A root filesystem that has one of the following filetype 371- `ROOTFS`: A root filesystem that has one of the following filetype
376 extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If 372 extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If
diff --git a/documentation/dev-manual/runtime-testing.rst b/documentation/dev-manual/runtime-testing.rst
deleted file mode 100644
index 7a2b42f25a..0000000000
--- a/documentation/dev-manual/runtime-testing.rst
+++ /dev/null
@@ -1,594 +0,0 @@
1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Performing Automated Runtime Testing
4************************************
5
6The OpenEmbedded build system makes available a series of automated
7tests for images to verify runtime functionality. You can run these
8tests on either QEMU or actual target hardware. Tests are written in
9Python making use of the ``unittest`` module, and the majority of them
10run commands on the target system over SSH. This section describes how
11you set up the environment to use these tests, run available tests, and
12write and add your own tests.
13
14For information on the test and QA infrastructure available within the
15Yocto Project, see the ":ref:`ref-manual/release-process:testing and quality assurance`"
16section in the Yocto Project Reference Manual.
17
18Enabling Tests
19==============
20
21Depending on whether you are planning to run tests using QEMU or on the
22hardware, you have to take different steps to enable the tests. See the
23following subsections for information on how to enable both types of
24tests.
25
26Enabling Runtime Tests on QEMU
27------------------------------
28
29In order to run tests, you need to do the following:
30
31- *Set up to avoid interaction with sudo for networking:* To
32 accomplish this, you must do one of the following:
33
34 - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all
35 commands or just for ``runqemu-ifup``. You must provide the full
36 path as that can change if you are using multiple clones of the
37 source repository.
38
39 .. note::
40
41 On some distributions, you also need to comment out "Defaults
42 requiretty" in ``/etc/sudoers``.
43
44 - Manually configure a tap interface for your system.
45
46 - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which
47 should generate a list of tap devices. This is the option
48 typically chosen for Autobuilder-type environments.
49
50 .. note::
51
52 - Be sure to use an absolute path when calling this script
53 with sudo.
54
55 - Ensure that your host has the package ``iptables`` installed.
56
57 - The package recipe ``qemu-helper-native`` is required to run
58 this script. Build the package using the following command::
59
60 $ bitbake qemu-helper-native
61
62- *Set the DISPLAY variable:* You need to set this variable so that
63 you have an X server available (e.g. start ``vncserver`` for a
64 headless machine).
65
66- *Be sure your host's firewall accepts incoming connections from
67 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an
68 HTTP server on a random high number port, which is used to serve
69 files to the target. The DNF module serves
70 ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands.
71 That means your host's firewall must accept incoming connections from
72 192.168.7.0/24, which is the default IP range used for tap devices by
73 ``runqemu``.
74
75- *Be sure your host has the correct packages installed:* Depending
76 your host's distribution, you need to have the following packages
77 installed:
78
79 - Ubuntu and Debian: ``sysstat`` and ``iproute2``
80
81 - openSUSE: ``sysstat`` and ``iproute2``
82
83 - Fedora: ``sysstat`` and ``iproute``
84
85 - CentOS: ``sysstat`` and ``iproute``
86
87Once you start running the tests, the following happens:
88
89#. A copy of the root filesystem is written to ``${WORKDIR}/testimage``.
90
91#. The image is booted under QEMU using the standard ``runqemu`` script.
92
93#. A default timeout of 500 seconds occurs to allow for the boot process
94 to reach the login prompt. You can change the timeout period by
95 setting
96 :term:`TEST_QEMUBOOT_TIMEOUT`
97 in the ``local.conf`` file.
98
99#. Once the boot process is reached and the login prompt appears, the
100 tests run. The full boot log is written to
101 ``${WORKDIR}/testimage/qemu_boot_log``.
102
103#. Each test module loads in the order found in :term:`TEST_SUITES`. You can
104 find the full output of the commands run over SSH in
105 ``${WORKDIR}/testimgage/ssh_target_log``.
106
107#. If no failures occur, the task running the tests ends successfully.
108 You can find the output from the ``unittest`` in the task log at
109 ``${WORKDIR}/temp/log.do_testimage``.
110
111Enabling Runtime Tests on Hardware
112----------------------------------
113
114The OpenEmbedded build system can run tests on real hardware, and for
115certain devices it can also deploy the image to be tested onto the
116device beforehand.
117
118For automated deployment, a "controller image" is installed onto the
119hardware once as part of setup. Then, each time tests are to be run, the
120following occurs:
121
122#. The controller image is booted into and used to write the image to be
123 tested to a second partition.
124
125#. The device is then rebooted using an external script that you need to
126 provide.
127
128#. The device boots into the image to be tested.
129
130When running tests (independent of whether the image has been deployed
131automatically or not), the device is expected to be connected to a
132network on a pre-determined IP address. You can either use static IP
133addresses written into the image, or set the image to use DHCP and have
134your DHCP server on the test network assign a known IP address based on
135the MAC address of the device.
136
137In order to run tests on hardware, you need to set :term:`TEST_TARGET` to an
138appropriate value. For QEMU, you do not have to change anything, the
139default value is "qemu". For running tests on hardware, the following
140options are available:
141
142- *"simpleremote":* Choose "simpleremote" if you are going to run tests
143 on a target system that is already running the image to be tested and
144 is available on the network. You can use "simpleremote" in
145 conjunction with either real hardware or an image running within a
146 separately started QEMU or any other virtual machine manager.
147
148- *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is
149 an EFI-based machine with ``systemd-boot`` as bootloader and
150 ``core-image-testmaster`` (or something similar) is installed. Also,
151 your hardware under test must be in a DHCP-enabled network that gives
152 it the same IP address for each reboot.
153
154 If you choose "SystemdbootTarget", there are additional requirements
155 and considerations. See the
156 ":ref:`dev-manual/runtime-testing:selecting systemdboottarget`" section, which
157 follows, for more information.
158
159- *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying
160 images and running tests on the BeagleBone "Black" or original
161 "White" hardware. For information on how to use these tests, see the
162 comments at the top of the BeagleBoneTarget
163 ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file.
164
165- *"GrubTarget":* Choose "GrubTarget" if you are deploying images and running
166 tests on any generic PC that boots using GRUB. For information on how
167 to use these tests, see the comments at the top of the GrubTarget
168 ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file.
169
170- *"your-target":* Create your own custom target if you want to run
171 tests when you are deploying images and running tests on a custom
172 machine within your BSP layer. To do this, you need to add a Python
173 unit that defines the target class under ``lib/oeqa/controllers/``
174 within your layer. You must also provide an empty ``__init__.py``.
175 For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``.
176
177Selecting SystemdbootTarget
178---------------------------
179
180If you did not set :term:`TEST_TARGET` to "SystemdbootTarget", then you do
181not need any information in this section. You can skip down to the
182":ref:`dev-manual/runtime-testing:running tests`" section.
183
184If you did set :term:`TEST_TARGET` to "SystemdbootTarget", you also need to
185perform a one-time setup of your controller image by doing the following:
186
187#. *Set EFI_PROVIDER:* Be sure that :term:`EFI_PROVIDER` is as follows::
188
189 EFI_PROVIDER = "systemd-boot"
190
191#. *Build the controller image:* Build the ``core-image-testmaster`` image.
192 The ``core-image-testmaster`` recipe is provided as an example for a
193 "controller" image and you can customize the image recipe as you would
194 any other recipe.
195
196 Image recipe requirements are:
197
198 - Inherits ``core-image`` so that kernel modules are installed.
199
200 - Installs normal linux utilities not BusyBox ones (e.g. ``bash``,
201 ``coreutils``, ``tar``, ``gzip``, and ``kmod``).
202
203 - Uses a custom :term:`Initramfs` image with a custom
204 installer. A normal image that you can install usually creates a
205 single root filesystem partition. This image uses another installer that
206 creates a specific partition layout. Not all Board Support
207 Packages (BSPs) can use an installer. For such cases, you need to
208 manually create the following partition layout on the target:
209
210 - First partition mounted under ``/boot``, labeled "boot".
211
212 - The main root filesystem partition where this image gets installed,
213 which is mounted under ``/``.
214
215 - Another partition labeled "testrootfs" where test images get
216 deployed.
217
218#. *Install image:* Install the image that you just built on the target
219 system.
220
221The final thing you need to do when setting :term:`TEST_TARGET` to
222"SystemdbootTarget" is to set up the test image:
223
224#. *Set up your local.conf file:* Make sure you have the following
225 statements in your ``local.conf`` file::
226
227 IMAGE_FSTYPES += "tar.gz"
228 IMAGE_CLASSES += "testimage"
229 TEST_TARGET = "SystemdbootTarget"
230 TEST_TARGET_IP = "192.168.2.3"
231
232#. *Build your test image:* Use BitBake to build the image::
233
234 $ bitbake core-image-sato
235
236Power Control
237-------------
238
239For most hardware targets other than "simpleremote", you can control
240power:
241
242- You can use :term:`TEST_POWERCONTROL_CMD` together with
243 :term:`TEST_POWERCONTROL_EXTRA_ARGS` as a command that runs on the host
244 and does power cycling. The test code passes one argument to that
245 command: off, on or cycle (off then on). Here is an example that
246 could appear in your ``local.conf`` file::
247
248 TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1"
249
250 In this example, the expect
251 script does the following:
252
253 .. code-block:: shell
254
255 ssh test@10.11.12.1 "pyctl nuc1 arg"
256
257 It then runs a Python script that controls power for a label called
258 ``nuc1``.
259
260 .. note::
261
262 You need to customize :term:`TEST_POWERCONTROL_CMD` and
263 :term:`TEST_POWERCONTROL_EXTRA_ARGS` for your own setup. The one requirement
264 is that it accepts "on", "off", and "cycle" as the last argument.
265
266- When no command is defined, it connects to the device over SSH and
267 uses the classic reboot command to reboot the device. Classic reboot
268 is fine as long as the machine actually reboots (i.e. the SSH test
269 has not failed). It is useful for scenarios where you have a simple
270 setup, typically with a single board, and where some manual
271 interaction is okay from time to time.
272
273If you have no hardware to automatically perform power control but still
274wish to experiment with automated hardware testing, you can use the
275``dialog-power-control`` script that shows a dialog prompting you to perform
276the required power action. This script requires either KDialog or Zenity
277to be installed. To use this script, set the
278:term:`TEST_POWERCONTROL_CMD`
279variable as follows::
280
281 TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control"
282
283Serial Console Connection
284-------------------------
285
286For test target classes requiring a serial console to interact with the
287bootloader (e.g. BeagleBoneTarget and GrubTarget),
288you need to specify a command to use to connect to the serial console of
289the target machine by using the
290:term:`TEST_SERIALCONTROL_CMD`
291variable and optionally the
292:term:`TEST_SERIALCONTROL_EXTRA_ARGS`
293variable.
294
295These cases could be a serial terminal program if the machine is
296connected to a local serial port, or a ``telnet`` or ``ssh`` command
297connecting to a remote console server. Regardless of the case, the
298command simply needs to connect to the serial console and forward that
299connection to standard input and output as any normal terminal program
300does. For example, to use the picocom terminal program on serial device
301``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows::
302
303 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
304
305For local
306devices where the serial port device disappears when the device reboots,
307an additional "serdevtry" wrapper script is provided. To use this
308wrapper, simply prefix the terminal command with
309``${COREBASE}/scripts/contrib/serdevtry``::
310
311 TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0"
312
313Running Tests
314=============
315
316You can start the tests automatically or manually:
317
318- *Automatically running tests:* To run the tests automatically after the
319 OpenEmbedded build system successfully creates an image, first set the
320 :term:`TESTIMAGE_AUTO` variable to "1" in your ``local.conf`` file in the
321 :term:`Build Directory`::
322
323 TESTIMAGE_AUTO = "1"
324
325 Next, build your image. If the image successfully builds, the
326 tests run::
327
328 bitbake core-image-sato
329
330- *Manually running tests:* To manually run the tests, first globally
331 inherit the :ref:`ref-classes-testimage` class by editing your
332 ``local.conf`` file::
333
334 IMAGE_CLASSES += "testimage"
335
336 Next, use BitBake to run the tests::
337
338 bitbake -c testimage image
339
340All test files reside in ``meta/lib/oeqa/runtime/cases`` in the
341:term:`Source Directory`. A test name maps
342directly to a Python module. Each test module may contain a number of
343individual tests. Tests are usually grouped together by the area tested
344(e.g tests for systemd reside in ``meta/lib/oeqa/runtime/cases/systemd.py``).
345
346You can add tests to any layer provided you place them in the proper
347area and you extend :term:`BBPATH` in
348the ``local.conf`` file as normal. Be sure that tests reside in
349``layer/lib/oeqa/runtime/cases``.
350
351.. note::
352
353 Be sure that module names do not collide with module names used in
354 the default set of test modules in ``meta/lib/oeqa/runtime/cases``.
355
356You can change the set of tests run by appending or overriding
357:term:`TEST_SUITES` variable in
358``local.conf``. Each name in :term:`TEST_SUITES` represents a required test
359for the image. Test modules named within :term:`TEST_SUITES` cannot be
360skipped even if a test is not suitable for an image (e.g. running the
361RPM tests on an image without ``rpm``). Appending "auto" to
362:term:`TEST_SUITES` causes the build system to try to run all tests that are
363suitable for the image (i.e. each test module may elect to skip itself).
364
365The order you list tests in :term:`TEST_SUITES` is important and influences
366test dependencies. Consequently, tests that depend on other tests should
367be added after the test on which they depend. For example, since the
368``ssh`` test depends on the ``ping`` test, "ssh" needs to come after
369"ping" in the list. The test class provides no re-ordering or dependency
370handling.
371
372.. note::
373
374 Each module can have multiple classes with multiple test methods.
375 And, Python ``unittest`` rules apply.
376
377Here are some things to keep in mind when running tests:
378
379- The default tests for the image are defined as::
380
381 DEFAULT_TEST_SUITES:pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg"
382
383- Add your own test to the list of the by using the following::
384
385 TEST_SUITES:append = " mytest"
386
387- Run a specific list of tests as follows::
388
389 TEST_SUITES = "test1 test2 test3"
390
391 Remember, order is important. Be sure to place a test that is
392 dependent on another test later in the order.
393
394Exporting Tests
395===============
396
397You can export tests so that they can run independently of the build
398system. Exporting tests is required if you want to be able to hand the
399test execution off to a scheduler. You can only export tests that are
400defined in :term:`TEST_SUITES`.
401
402If your image is already built, make sure the following are set in your
403``local.conf`` file::
404
405 INHERIT += "testexport"
406 TEST_TARGET_IP = "IP-address-for-the-test-target"
407 TEST_SERVER_IP = "IP-address-for-the-test-server"
408
409You can then export the tests with the
410following BitBake command form::
411
412 $ bitbake image -c testexport
413
414Exporting the tests places them in the :term:`Build Directory` in
415``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR`
416variable.
417
418You can now run the tests outside of the build environment::
419
420 $ cd tmp/testexport/image
421 $ ./runexported.py testdata.json
422
423Here is a complete example that shows IP addresses and uses the
424``core-image-sato`` image::
425
426 INHERIT += "testexport"
427 TEST_TARGET_IP = "192.168.7.2"
428 TEST_SERVER_IP = "192.168.7.1"
429
430Use BitBake to export the tests::
431
432 $ bitbake core-image-sato -c testexport
433
434Run the tests outside of
435the build environment using the following::
436
437 $ cd tmp/testexport/core-image-sato
438 $ ./runexported.py testdata.json
439
440Writing New Tests
441=================
442
443As mentioned previously, all new test files need to be in the proper
444place for the build system to find them. New tests for additional
445functionality outside of the core should be added to the layer that adds
446the functionality, in ``layer/lib/oeqa/runtime/cases`` (as long as
447:term:`BBPATH` is extended in the
448layer's ``layer.conf`` file as normal). Just remember the following:
449
450- Filenames need to map directly to test (module) names.
451
452- Do not use module names that collide with existing core tests.
453
454- Minimally, an empty ``__init__.py`` file must be present in the runtime
455 directory.
456
457To create a new test, start by copying an existing module (e.g.
458``oe_syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use
459code from ``meta/lib/oeqa/utils``, which are helper classes.
460
461.. note::
462
463 Structure shell commands such that you rely on them and they return a
464 single code for success. Be aware that sometimes you will need to
465 parse the output. See the ``df.py`` and ``date.py`` modules for examples.
466
467You will notice that all test classes inherit ``oeRuntimeTest``, which
468is found in ``meta/lib/oetest.py``. This base class offers some helper
469attributes, which are described in the following sections:
470
471Class Methods
472-------------
473
474Class methods are as follows:
475
476- *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed
477 package list of the image, which is based on the manifest file that
478 is generated during the :ref:`ref-tasks-rootfs` task.
479
480- *hasFeature(feature):* Returns "True" if the feature is in
481 :term:`IMAGE_FEATURES` or
482 :term:`DISTRO_FEATURES`.
483
484Class Attributes
485----------------
486
487Class attributes are as follows:
488
489- *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image.
490 Otherwise, ``pscmd`` equals "ps" (busybox).
491
492- *tc:* The called test context, which gives access to the
493 following attributes:
494
495 - *d:* The BitBake datastore, which allows you to use stuff such
496 as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``.
497
498 - *testslist and testsrequired:* Used internally. The tests
499 do not need these.
500
501 - *filesdir:* The absolute path to
502 ``meta/lib/oeqa/runtime/files``, which contains helper files for
503 tests meant for copying on the target such as small files written
504 in C for compilation.
505
506 - *target:* The target controller object used to deploy and
507 start an image on a particular target (e.g. Qemu, SimpleRemote,
508 and SystemdbootTarget). Tests usually use the following:
509
510 - *ip:* The target's IP address.
511
512 - *server_ip:* The host's IP address, which is usually used
513 by the DNF test suite.
514
515 - *run(cmd, timeout=None):* The single, most used method.
516 This command is a wrapper for: ``ssh root@host "cmd"``. The
517 command returns a tuple: (status, output), which are what their
518 names imply - the return code of "cmd" and whatever output it
519 produces. The optional timeout argument represents the number
520 of seconds the test should wait for "cmd" to return. If the
521 argument is "None", the test uses the default instance's
522 timeout period, which is 300 seconds. If the argument is "0",
523 the test runs until the command returns.
524
525 - *copy_to(localpath, remotepath):*
526 ``scp localpath root@ip:remotepath``.
527
528 - *copy_from(remotepath, localpath):*
529 ``scp root@host:remotepath localpath``.
530
531Instance Attributes
532-------------------
533
534There is a single instance attribute, which is ``target``. The ``target``
535instance attribute is identical to the class attribute of the same name,
536which is described in the previous section. This attribute exists as
537both an instance and class attribute so tests can use
538``self.target.run(cmd)`` in instance methods instead of
539``oeRuntimeTest.tc.target.run(cmd)``.
540
541Installing Packages in the DUT Without the Package Manager
542==========================================================
543
544When a test requires a package built by BitBake, it is possible to
545install that package. Installing the package does not require a package
546manager be installed in the device under test (DUT). It does, however,
547require an SSH connection and the target must be using the
548``sshcontrol`` class.
549
550.. note::
551
552 This method uses ``scp`` to copy files from the host to the target, which
553 causes permissions and special attributes to be lost.
554
555A JSON file is used to define the packages needed by a test. This file
556must be in the same path as the file used to define the tests.
557Furthermore, the filename must map directly to the test module name with
558a ``.json`` extension.
559
560The JSON file must include an object with the test name as keys of an
561object or an array. This object (or array of objects) uses the following
562data:
563
564- "pkg" --- a mandatory string that is the name of the package to be
565 installed.
566
567- "rm" --- an optional boolean, which defaults to "false", that specifies
568 to remove the package after the test.
569
570- "extract" --- an optional boolean, which defaults to "false", that
571 specifies if the package must be extracted from the package format.
572 When set to "true", the package is not automatically installed into
573 the DUT.
574
575Here is an example JSON file that handles test "foo" installing
576package "bar" and test "foobar" installing packages "foo" and "bar".
577Once the test is complete, the packages are removed from the DUT::
578
579 {
580 "foo": {
581 "pkg": "bar"
582 },
583 "foobar": [
584 {
585 "pkg": "foo",
586 "rm": true
587 },
588 {
589 "pkg": "bar",
590 "rm": true
591 }
592 ]
593 }
594
diff --git a/documentation/dev-manual/sbom.rst b/documentation/dev-manual/sbom.rst
index b72bad1554..ca0fc8b9d6 100644
--- a/documentation/dev-manual/sbom.rst
+++ b/documentation/dev-manual/sbom.rst
@@ -24,12 +24,20 @@ users can read in standardized format.
24:term:`SBOM` information is also critical to performing vulnerability exposure 24:term:`SBOM` information is also critical to performing vulnerability exposure
25assessments, as all the components used in the Software Supply Chain are listed. 25assessments, as all the components used in the Software Supply Chain are listed.
26 26
27The OpenEmbedded build system doesn't generate such information by default. 27The OpenEmbedded build system doesn't generate such information by default,
28To make this happen, you must inherit the 28though the :term:`Poky` reference distribution has it enabled out of the box.
29:ref:`ref-classes-create-spdx` class from a configuration file:: 29
30To enable it, inherit the :ref:`ref-classes-create-spdx` class from a
31configuration file::
30 32
31 INHERIT += "create-spdx" 33 INHERIT += "create-spdx"
32 34
35In the :term:`Poky` reference distribution, :term:`SPDX` generation does
36consume some build time resources and thus if needed it can be disabled from a
37:term:`configuration file`::
38
39 INHERIT:remove = "create-spdx"
40
33Upon building an image, you will then get: 41Upon building an image, you will then get:
34 42
35- :term:`SPDX` output in JSON format as an ``IMAGE-MACHINE.spdx.json`` file in 43- :term:`SPDX` output in JSON format as an ``IMAGE-MACHINE.spdx.json`` file in
@@ -52,6 +60,9 @@ more information in the output :term:`SPDX` data:
52- Add a description of the source files used to generate host tools and target 60- Add a description of the source files used to generate host tools and target
53 packages (:term:`SPDX_INCLUDE_SOURCES`) 61 packages (:term:`SPDX_INCLUDE_SOURCES`)
54 62
63- Add a description of the **compiled** source files used to generate host tools
64 and target packages (:term:`SPDX_INCLUDE_COMPILED_SOURCES`)
65
55- Add archives of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`). 66- Add archives of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`).
56 67
57Though the toplevel :term:`SPDX` output is available in 68Though the toplevel :term:`SPDX` output is available in
diff --git a/documentation/dev-manual/securing-images.rst b/documentation/dev-manual/securing-images.rst
index e5791d3d6d..f4b528e559 100644
--- a/documentation/dev-manual/securing-images.rst
+++ b/documentation/dev-manual/securing-images.rst
@@ -107,18 +107,18 @@ Considerations Specific to the OpenEmbedded Build System
107You can take some steps that are specific to the OpenEmbedded build 107You can take some steps that are specific to the OpenEmbedded build
108system to make your images more secure: 108system to make your images more secure:
109 109
110- Ensure "debug-tweaks" is not one of your selected 110- Ensure that "allow-empty-password", "allow-root-login", or
111 :term:`IMAGE_FEATURES`. 111 "empty-root-password" are not one of your selected :term:`IMAGE_FEATURES`.
112 When creating a new project, the default is to provide you with an 112 When creating a new project, the default is to provide you with an
113 initial ``local.conf`` file that enables this feature using the 113 initial ``local.conf`` file that enables these features using the
114 :term:`EXTRA_IMAGE_FEATURES` 114 :term:`EXTRA_IMAGE_FEATURES`
115 variable with the line:: 115 variable with the line::
116 116
117 EXTRA_IMAGE_FEATURES = "debug-tweaks" 117 EXTRA_IMAGE_FEATURES = "allow-empty-password empty-root-password allow-root-login"
118 118
119 To disable that feature, simply comment out that line in your 119 To disable these features, simply comment out that line in your
120 ``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain 120 ``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain
121 "debug-tweaks" before producing your final image. Among other things, 121 any of these features before producing your final image. Among other things,
122 leaving this in place sets the root password as blank, which makes 122 leaving this in place sets the root password as blank, which makes
123 logging in for debugging or inspection easy during development but 123 logging in for debugging or inspection easy during development but
124 also means anyone can easily log in during production. 124 also means anyone can easily log in during production.
diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst
index 8539bc0889..44bd2de137 100644
--- a/documentation/dev-manual/start.rst
+++ b/documentation/dev-manual/start.rst
@@ -109,7 +109,7 @@ particular working environment and set of practices.
109 109
110 - Keep your cross-development toolchains updated. You can do this 110 - Keep your cross-development toolchains updated. You can do this
111 through provisioning either as new toolchain downloads or as 111 through provisioning either as new toolchain downloads or as
112 updates through a package update mechanism using ``opkg`` to 112 updates through a package update mechanism to
113 provide updates to an existing toolchain. The exact mechanics of 113 provide updates to an existing toolchain. The exact mechanics of
114 how and when to do this depend on local policy. 114 how and when to do this depend on local policy.
115 115
@@ -159,7 +159,7 @@ particular working environment and set of practices.
159 are made. 159 are made.
160 160
161 - Allows triggering of automated image booting and testing under 161 - Allows triggering of automated image booting and testing under
162 the QuickEMUlator (QEMU). 162 the Quick EMUlator (QEMU).
163 163
164 - Supports incremental build testing and from-scratch builds. 164 - Supports incremental build testing and from-scratch builds.
165 165
@@ -234,7 +234,7 @@ particular working environment and set of practices.
234 - The Yocto Project community encourages you to send patches to the 234 - The Yocto Project community encourages you to send patches to the
235 project to fix bugs or add features. If you do submit patches, 235 project to fix bugs or add features. If you do submit patches,
236 follow the project commit guidelines for writing good commit 236 follow the project commit guidelines for writing good commit
237 messages. See the ":doc:`../contributor-guide/submit-changes`" 237 messages. See the ":doc:`/contributor-guide/submit-changes`"
238 section in the Yocto Project and OpenEmbedded Contributor Guide. 238 section in the Yocto Project and OpenEmbedded Contributor Guide.
239 239
240 - Send changes to the core sooner than later as others are likely 240 - Send changes to the core sooner than later as others are likely
@@ -310,7 +310,7 @@ Project Build Host:
310 310
311 - GNU make &MIN_MAKE_VERSION; or greater 311 - GNU make &MIN_MAKE_VERSION; or greater
312 312
313 If your build host does not meet any of these listed version 313 If your build host does not satisfy all of these listed version
314 requirements, you can take steps to prepare the system so that you 314 requirements, you can take steps to prepare the system so that you
315 can still use the Yocto Project. See the 315 can still use the Yocto Project. See the
316 ":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`" 316 ":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
@@ -568,7 +568,7 @@ extension accordingly.
568Locating Yocto Project Source Files 568Locating Yocto Project Source Files
569=================================== 569===================================
570 570
571This section shows you how to locate, fetch and configure the source 571This section shows you how to locate, fetch, unpack, patch and configure the source
572files you'll need to work with the Yocto Project. 572files you'll need to work with the Yocto Project.
573 573
574.. note:: 574.. note::
@@ -615,11 +615,11 @@ Accessing Source Archives
615The Yocto Project also provides source archives of its releases, which 615The Yocto Project also provides source archives of its releases, which
616are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory 616are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory
617containing the release you wish to use, for example 617containing the release you wish to use, for example
618:yocto_dl:`yocto-&DISTRO; </releases/yocto/yocto-&DISTRO;/>`. 618:yocto_dl:`&DISTRO_REL_LATEST_TAG; </releases/yocto/&DISTRO_REL_LATEST_TAG;/>`.
619 619
620You will find there source archives of individual components (if you wish 620You will find there source archives of individual components (if you wish
621to use them individually), and of the corresponding Poky release bundling 621to use them individually), and of the corresponding Poky release bundling
622a selection of these components. 622a selection of these components.
623 623
624.. note:: 624.. note::
625 625
@@ -720,11 +720,11 @@ Follow these steps to create a local version of the upstream
720 $ git branch 720 $ git branch
721 * master 721 * master
722 722
723 Your local repository of poky is identical to the 723 Your local repository of poky is initially identical to the
724 upstream poky repository at the time from which it was cloned. As you 724 upstream poky repository from which it was cloned. As you
725 work with the local branch, you can periodically use the 725 work with the local branch, you can periodically use the
726 ``git pull --rebase`` command to be sure you are up-to-date 726 ``git pull`` command to be sure you stay up-to-date
727 with the upstream branch. 727 with the upstream poky branch.
728 728
729Checking Out by Branch in Poky 729Checking Out by Branch in Poky
730------------------------------ 730------------------------------
@@ -853,3 +853,14 @@ similar to checking out by branch name except you use tag names.
853 ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;" 853 ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;"
854 development branch at the point where Yocto Project &DISTRO; was 854 development branch at the point where Yocto Project &DISTRO; was
855 released. 855 released.
856
857Initializing the Build Environment
858==================================
859
860Before you can use Yocto you need to setup the build environment.
861From within the ``poky`` directory, source the :ref:`ref-manual/structure:``oe-init-build-env``` environment
862setup script to define Yocto Project's build environment on your build host::
863
864 $ source oe-init-build-env
865
866Note, that this step will have to be repeated every time you open a new shell.
diff --git a/documentation/dev-manual/temporary-source-code.rst b/documentation/dev-manual/temporary-source-code.rst
index 08bf68d982..9a7cd0f771 100644
--- a/documentation/dev-manual/temporary-source-code.rst
+++ b/documentation/dev-manual/temporary-source-code.rst
@@ -18,11 +18,10 @@ build packages is available in the :term:`Build Directory` as defined by the
18defined in the ``meta/conf/bitbake.conf`` configuration file in the 18defined in the ``meta/conf/bitbake.conf`` configuration file in the
19:term:`Source Directory`:: 19:term:`Source Directory`::
20 20
21 S = "${WORKDIR}/${BP}" 21 S = "${UNPACKDIR}/${BP}"
22 22
23You should be aware that many recipes override the 23You should be aware that many recipes override the
24:term:`S` variable. For example, recipes that fetch their source from Git 24:term:`S` variable when the default isn't accurate.
25usually set :term:`S` to ``${WORKDIR}/git``.
26 25
27.. note:: 26.. note::
28 27
@@ -31,8 +30,16 @@ usually set :term:`S` to ``${WORKDIR}/git``.
31 30
32 BP = "${BPN}-${PV}" 31 BP = "${BPN}-${PV}"
33 32
33 This matches the location that the git fetcher unpacks to, and usually
34 matches unpacked content of release tarballs (e.g. they contain a single
35 directory which matches value of ${BP} exactly).
34 36
35The path to the work directory for the recipe 37The path to the unpack directory for the recipe
38(:term:`UNPACKDIR`) is defined as follows::
39
40 ${WORKDIR}/sources
41
42In turn, the path to the work directory for the recipe
36(:term:`WORKDIR`) is defined as 43(:term:`WORKDIR`) is defined as
37follows:: 44follows::
38 45
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst
index 4fac78bdfb..2e65678679 100644
--- a/documentation/dev-manual/upgrading-recipes.rst
+++ b/documentation/dev-manual/upgrading-recipes.rst
@@ -203,7 +203,7 @@ As mentioned earlier, an alternative method for upgrading recipes to
203newer versions is to use 203newer versions is to use
204:doc:`devtool upgrade </ref-manual/devtool-reference>`. 204:doc:`devtool upgrade </ref-manual/devtool-reference>`.
205You can read about ``devtool upgrade`` in general in the 205You can read about ``devtool upgrade`` in general in the
206":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" 206":ref:`dev-manual/devtool:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`"
207section in the Yocto Project Application Development and the Extensible 207section in the Yocto Project Application Development and the Extensible
208Software Development Kit (eSDK) Manual. 208Software Development Kit (eSDK) Manual.
209 209
@@ -333,7 +333,7 @@ Manually Upgrading a Recipe
333 333
334If for some reason you choose not to upgrade recipes using 334If for some reason you choose not to upgrade recipes using
335:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or 335:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or
336by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, 336by :ref:`dev-manual/upgrading-recipes:Using ``devtool upgrade```,
337you can manually edit the recipe files to upgrade the versions. 337you can manually edit the recipe files to upgrade the versions.
338 338
339.. note:: 339.. note::
diff --git a/documentation/dev-manual/vulnerabilities.rst b/documentation/dev-manual/vulnerabilities.rst
index 1bc2a85929..5331a63991 100644
--- a/documentation/dev-manual/vulnerabilities.rst
+++ b/documentation/dev-manual/vulnerabilities.rst
@@ -22,7 +22,7 @@ issues may be impacting Poky and OE-Core. It is up to the maintainers, users,
22contributors and anyone interested in the issues to investigate and possibly fix them by 22contributors and anyone interested in the issues to investigate and possibly fix them by
23updating software components to newer versions or by applying patches to address them. 23updating software components to newer versions or by applying patches to address them.
24It is recommended to work with Poky and OE-Core upstream maintainers and submit 24It is recommended to work with Poky and OE-Core upstream maintainers and submit
25patches to fix them, see ":doc:`../contributor-guide/submit-changes`" for details. 25patches to fix them, see ":doc:`/contributor-guide/submit-changes`" for details.
26 26
27Vulnerability check at build time 27Vulnerability check at build time
28================================= 28=================================
@@ -57,42 +57,86 @@ applied and that the issue needs to be investigated. ``Ignored`` means that afte
57analysis, it has been deemed to ignore the issue as it for example affects 57analysis, it has been deemed to ignore the issue as it for example affects
58the software component on a different operating system platform. 58the software component on a different operating system platform.
59 59
60By default, no NVD API key is used to retrieve data from the CVE database, which
61results in larger delays between NVD API requests. See the :term:`NVDCVE_API_KEY`
62documentation on how to request and set a NVD API key.
63
60After a build with CVE check enabled, reports for each compiled source recipe will be 64After a build with CVE check enabled, reports for each compiled source recipe will be
61found in ``build/tmp/deploy/cve``. 65found in ``build/tmp/deploy/cve``.
62 66
63For example the CVE check report for the ``flex-native`` recipe looks like:: 67For example the CVE check report for the ``flex-native`` recipe looks like::
64 68
65 $ cat poky/build/tmp/deploy/cve/flex-native 69 $ cat ./tmp/deploy/cve/flex-native_cve.json
66 LAYER: meta 70 {
67 PACKAGE NAME: flex-native 71 "version": "1",
68 PACKAGE VERSION: 2.6.4 72 "package": [
69 CVE: CVE-2016-6354 73 {
70 CVE STATUS: Patched 74 "name": "flex-native",
71 CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read. 75 "layer": "meta",
72 CVSS v2 BASE SCORE: 7.5 76 "version": "2.6.4",
73 CVSS v3 BASE SCORE: 9.8 77 "products": [
74 VECTOR: NETWORK 78 {
75 MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354 79 "product": "flex",
76 80 "cvesInRecord": "No"
77 LAYER: meta 81 },
78 PACKAGE NAME: flex-native 82 {
79 PACKAGE VERSION: 2.6.4 83 "product": "flex",
80 CVE: CVE-2019-6293 84 "cvesInRecord": "Yes"
81 CVE STATUS: Ignored 85 }
82 CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service. 86 ],
83 CVSS v2 BASE SCORE: 4.3 87 "issue": [
84 CVSS v3 BASE SCORE: 5.5 88 {
85 VECTOR: NETWORK 89 "id": "CVE-2006-0459",
86 MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293 90 "status": "Patched",
91 "link": "https://nvd.nist.gov/vuln/detail/CVE-2006-0459",
92 "summary": "flex.skl in Will Estes and John Millaway Fast Lexical Analyzer Generator (flex) before 2.5.33 does not allocate enough memory for grammars containing (1) REJECT statements or (2) trailing context rules, which causes flex to generate code that contains a buffer overflow that might allow context-dependent attackers to execute arbitrary code.",
93 "scorev2": "7.5",
94 "scorev3": "0.0",
95 "scorev4": "0.0",
96 "modified": "2024-11-21T00:06Z",
97 "vector": "NETWORK",
98 "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P",
99 "detail": "version-not-in-range"
100 },
101 {
102 "id": "CVE-2016-6354",
103 "status": "Patched",
104 "link": "https://nvd.nist.gov/vuln/detail/CVE-2016-6354",
105 "summary": "Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.",
106 "scorev2": "7.5",
107 "scorev3": "9.8",
108 "scorev4": "0.0",
109 "modified": "2024-11-21T02:55Z",
110 "vector": "NETWORK",
111 "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P",
112 "detail": "version-not-in-range"
113 },
114 {
115 "id": "CVE-2019-6293",
116 "status": "Ignored",
117 "link": "https://nvd.nist.gov/vuln/detail/CVE-2019-6293",
118 "summary": "An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.",
119 "scorev2": "4.3",
120 "scorev3": "5.5",
121 "scorev4": "0.0",
122 "modified": "2024-11-21T04:46Z",
123 "vector": "NETWORK",
124 "vectorString": "AV:N/AC:M/Au:N/C:N/I:N/A:P",
125 "detail": "upstream-wontfix",
126 "description": "there is stack exhaustion but no bug and it is building the parser, not running it, effectively similar to a compiler ICE. Upstream no plans to address this."
127 }
128 ]
129 }
130 ]
131 }
87 132
88For images, a summary of all recipes included in the image and their CVEs is also 133For images, a summary of all recipes included in the image and their CVEs is also
89generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found 134generated in the JSON format. These ``.json`` reports can be found
90in the ``tmp/deploy/images`` directory for each compiled image. 135in the ``tmp/deploy/images`` directory for each compiled image.
91 136
92At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: 137At build time CVE check will also throw warnings about ``Unpatched`` CVEs::
93 138
94 WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log 139 WARNING: qemu-native-9.2.0-r0 do_cve_check: Found unpatched CVE (CVE-2023-1386)
95 WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log
96 140
97It is also possible to check the CVE status of individual packages as follows:: 141It is also possible to check the CVE status of individual packages as follows::
98 142
@@ -111,10 +155,10 @@ upstream `NIST CVE database <https://nvd.nist.gov/>`__.
111 155
112The variable supports using vendor and product names like this:: 156The variable supports using vendor and product names like this::
113 157
114 CVE_PRODUCT = "flex_project:flex" 158 CVE_PRODUCT = "flex_project:flex westes:flex"
115 159
116In this example the vendor name used in the CVE database is ``flex_project`` and the 160In this example we have two possible vendors names, ``flex_project`` and ``westes``,
117product is ``flex``. With this setting the ``flex`` recipe only maps to this specific 161with the product name ``flex``. With this setting the ``flex`` recipe only maps to this specific
118product and not products from other vendors with same name ``flex``. 162product and not products from other vendors with same name ``flex``.
119 163
120Similarly, when the recipe version :term:`PV` is not compatible with software versions used by 164Similarly, when the recipe version :term:`PV` is not compatible with software versions used by
diff --git a/documentation/dev-manual/wic.rst b/documentation/dev-manual/wic.rst
index 05e9cb381b..fced0e170c 100644
--- a/documentation/dev-manual/wic.rst
+++ b/documentation/dev-manual/wic.rst
@@ -139,20 +139,20 @@ individual images through the ``list`` command. You can use the ``list``
139command to return the available Wic images as follows:: 139command to return the available Wic images as follows::
140 140
141 $ wic list images 141 $ wic list images
142 genericx86 Create an EFI disk image for genericx86* 142 genericx86 Create an EFI disk image for genericx86*
143 beaglebone-yocto Create SD card image for Beaglebone 143 beaglebone-yocto Create SD card image for Beaglebone
144 qemuriscv Create qcow2 image for RISC-V QEMU machines 144 qemuriscv Create qcow2 image for RISC-V QEMU machines
145 mkefidisk Create an EFI disk image 145 mkefidisk Create an EFI disk image
146 qemuloongarch Create qcow2 image for LoongArch QEMU machines 146 qemuloongarch Create qcow2 image for LoongArch QEMU machines
147 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin 147 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
148 directdisk Create a 'pcbios' direct disk image 148 directdisk Create a 'pcbios' direct disk image
149 efi-bootdisk 149 efi-bootdisk
150 mkhybridiso Create a hybrid ISO image 150 mkhybridiso Create a hybrid ISO image
151 directdisk-gpt Create a 'pcbios' direct disk image 151 directdisk-gpt Create a 'pcbios' direct disk image
152 systemd-bootdisk Create an EFI disk image with systemd-boot 152 systemd-bootdisk Create an EFI disk image with systemd-boot
153 sdimage-bootpart Create SD card image with a boot partition 153 sdimage-bootpart Create SD card image with a boot partition
154 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image 154 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
155 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config 155 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
156 156
157Once you know the list of available 157Once you know the list of available
158Wic images, you can use ``help`` with the command to get help on a 158Wic images, you can use ``help`` with the command to get help on a
@@ -282,20 +282,20 @@ following two locations::
282Use the following command to list the available kickstart files:: 282Use the following command to list the available kickstart files::
283 283
284 $ wic list images 284 $ wic list images
285 genericx86 Create an EFI disk image for genericx86* 285 genericx86 Create an EFI disk image for genericx86*
286 beaglebone-yocto Create SD card image for Beaglebone 286 beaglebone-yocto Create SD card image for Beaglebone
287 qemuriscv Create qcow2 image for RISC-V QEMU machines 287 qemuriscv Create qcow2 image for RISC-V QEMU machines
288 mkefidisk Create an EFI disk image 288 mkefidisk Create an EFI disk image
289 qemuloongarch Create qcow2 image for LoongArch QEMU machines 289 qemuloongarch Create qcow2 image for LoongArch QEMU machines
290 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin 290 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
291 directdisk Create a 'pcbios' direct disk image 291 directdisk Create a 'pcbios' direct disk image
292 efi-bootdisk 292 efi-bootdisk
293 mkhybridiso Create a hybrid ISO image 293 mkhybridiso Create a hybrid ISO image
294 directdisk-gpt Create a 'pcbios' direct disk image 294 directdisk-gpt Create a 'pcbios' direct disk image
295 systemd-bootdisk Create an EFI disk image with systemd-boot 295 systemd-bootdisk Create an EFI disk image with systemd-boot
296 sdimage-bootpart Create SD card image with a boot partition 296 sdimage-bootpart Create SD card image with a boot partition
297 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image 297 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
298 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config 298 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
299 299
300When you use an existing file, you 300When you use an existing file, you
301do not have to use the ``.wks`` extension. Here is an example in Raw 301do not have to use the ``.wks`` extension. Here is an example in Raw
@@ -513,7 +513,7 @@ or ::
513 513
514 For more information on how to use the ``bmaptool`` 514 For more information on how to use the ``bmaptool``
515 to flash a device with an image, see the 515 to flash a device with an image, see the
516 ":ref:`dev-manual/bmaptool:flashing images using \`\`bmaptool\`\``" 516 ":ref:`dev-manual/bmaptool:flashing images using \`bmaptool\``"
517 section. 517 section.
518 518
519Using a Modified Kickstart File 519Using a Modified Kickstart File
@@ -721,7 +721,7 @@ the existing kernel, and then inserts a new kernel:
721 721
722 Once the new kernel is added back into the image, you can use the 722 Once the new kernel is added back into the image, you can use the
723 ``dd`` command or :ref:`bmaptool 723 ``dd`` command or :ref:`bmaptool
724 <dev-manual/bmaptool:flashing images using \`\`bmaptool\`\`>` 724 <dev-manual/bmaptool:flashing images using \`bmaptool\`>` commands
725 to flash your wic image onto an SD card or USB stick and test your 725 to flash your wic image onto an SD card or USB stick and test your
726 target. 726 target.
727 727