diff options
Diffstat (limited to 'documentation/dev-manual')
32 files changed, 2285 insertions, 1099 deletions
diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst new file mode 100644 index 0000000000..605bb75655 --- /dev/null +++ b/documentation/dev-manual/bblock.rst | |||
@@ -0,0 +1,129 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Locking and Unlocking Recipes Using ``bblock`` | ||
4 | ********************************************** | ||
5 | |||
6 | By design, the OpenEmbedded build system builds everything from scratch | ||
7 | unless BitBake determines that specific tasks do not require rebuilding. | ||
8 | At startup, it computes a signature for all tasks, based on the task's input. | ||
9 | Then, it compares these signatures with the ones from the sstate cache (if they | ||
10 | exist). Any changes cause the task to rerun. | ||
11 | |||
12 | During development, changes might trigger BitBake to rebuild certain | ||
13 | recipes, even when we know they do not require rebuilding at that stage. | ||
14 | For example, modifying a recipe can lead to rebuilding its native | ||
15 | counterpart, which might prove unnecessary. Editing the ``python3`` recipe, | ||
16 | for instance, can prompt BitBake to rebuild ``python3-native`` along with any | ||
17 | recipes that depend on it. | ||
18 | |||
19 | To prevent this, use ``bblock`` to lock specific tasks or recipes to | ||
20 | specific signatures, forcing BitBake to use the sstate cache for them. | ||
21 | |||
22 | .. warning:: | ||
23 | |||
24 | Use ``bblock`` only during the development phase. | ||
25 | |||
26 | Forcing BitBake to use the sstate cache, regardless of input changes, means | ||
27 | the recipe metadata no longer directly reflect the output. Use this feature | ||
28 | with caution. If you do not understand why signatures change, see the section | ||
29 | on :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`. | ||
30 | |||
31 | |||
32 | Locking tasks and recipes | ||
33 | ------------------------- | ||
34 | |||
35 | To lock a recipe, use:: | ||
36 | |||
37 | $ bblock recipe | ||
38 | |||
39 | You can also use a space-separated list of recipes to lock multiple recipes:: | ||
40 | |||
41 | $ bblock recipe1 recipe2 | ||
42 | |||
43 | Locking a recipe means locking all tasks of the recipe. If you need to | ||
44 | lock only particular tasks, use the `-t` option with a comma-separated | ||
45 | list of tasks:: | ||
46 | |||
47 | $ bblock -t task1,task2 recipe | ||
48 | |||
49 | |||
50 | Unlocking tasks and recipes | ||
51 | --------------------------- | ||
52 | |||
53 | To unlock a recipe, use the ``-r`` option:: | ||
54 | |||
55 | $ bblock -r recipe | ||
56 | |||
57 | You can also use a space-separated list of recipes to unlock multiple recipes:: | ||
58 | |||
59 | $ bblock -r recipe1 recipe2 | ||
60 | |||
61 | Unlocking a recipe means unlocking all tasks of the recipe. If you need to | ||
62 | unlock only particular tasks use the ``-t`` option with a comma-separated | ||
63 | list of tasks:: | ||
64 | |||
65 | $ bblock -r -t task1,task2 recipe | ||
66 | |||
67 | To unlock all recipes, do not specify any recipe:: | ||
68 | |||
69 | $ bblock -r | ||
70 | |||
71 | |||
72 | Configuration file | ||
73 | ------------------ | ||
74 | |||
75 | ``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file, | ||
76 | included by default in :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`. | ||
77 | |||
78 | To dump the file, use the ``-d`` option:: | ||
79 | |||
80 | $ bblock -d | ||
81 | |||
82 | |||
83 | Locking mechanism | ||
84 | ----------------- | ||
85 | |||
86 | ``bblock`` computes the signature(s) of the task(s) and sets the 3 following | ||
87 | variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES` | ||
88 | and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`. | ||
89 | |||
90 | In particular, ``bblock`` sets:: | ||
91 | |||
92 | SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" | ||
93 | SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" | ||
94 | |||
95 | SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>" | ||
96 | |||
97 | This produces architecture specific locks and reminds user that some tasks | ||
98 | have locked signatures. | ||
99 | |||
100 | Example | ||
101 | ------- | ||
102 | |||
103 | When working on the ``python3`` recipe, we can lock ``python3-native`` with | ||
104 | the following:: | ||
105 | |||
106 | $ bblock python3-native | ||
107 | $ bblock -d | ||
108 | # Generated by bblock | ||
109 | SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info" | ||
110 | SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}" | ||
111 | |||
112 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7" | ||
113 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605" | ||
114 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a" | ||
115 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25" | ||
116 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f" | ||
117 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42" | ||
118 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482" | ||
119 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9" | ||
120 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d" | ||
121 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882" | ||
122 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4" | ||
123 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5" | ||
124 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020" | ||
125 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4" | ||
126 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd" | ||
127 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28" | ||
128 | SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7" | ||
129 | |||
diff --git a/documentation/dev-manual/bmaptool.rst b/documentation/dev-manual/bmaptool.rst index f6f0e6afaf..87162a49c9 100644 --- a/documentation/dev-manual/bmaptool.rst +++ b/documentation/dev-manual/bmaptool.rst | |||
@@ -1,13 +1,13 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | 1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK |
2 | 2 | ||
3 | Flashing Images Using ``bmaptool`` | 3 | Flashing Images Using `bmaptool` |
4 | ********************************** | 4 | ******************************** |
5 | 5 | ||
6 | A fast and easy way to flash an image to a bootable device is to use | 6 | A fast and easy way to flash an image to a bootable device is to use |
7 | bmaptool, which is integrated into the OpenEmbedded build system. | 7 | `bmaptool`, which is integrated into the OpenEmbedded build system. |
8 | bmaptool is a generic tool that creates a file's block map (bmap) and | 8 | `bmaptool` is a generic tool that creates a file's block map (bmap) and |
9 | then uses that map to copy the file. As compared to traditional tools | 9 | then uses that map to copy the file. As compared to traditional tools |
10 | such as dd or cp, bmaptool can copy (or flash) large files like raw | 10 | such as `dd` or `cp`, `bmaptool` can copy (or flash) large files like raw |
11 | system image files much faster. | 11 | system image files much faster. |
12 | 12 | ||
13 | .. note:: | 13 | .. note:: |
@@ -20,13 +20,13 @@ system image files much faster. | |||
20 | $ sudo apt install bmap-tools | 20 | $ sudo apt install bmap-tools |
21 | 21 | ||
22 | - If you are unable to install the ``bmap-tools`` package, you will | 22 | - If you are unable to install the ``bmap-tools`` package, you will |
23 | need to build bmaptool before using it. Use the following command:: | 23 | need to build `bmaptool` before using it. Use the following command:: |
24 | 24 | ||
25 | $ bitbake bmaptool-native | 25 | $ bitbake bmaptool-native -caddto_recipe_sysroot |
26 | 26 | ||
27 | Following, is an example that shows how to flash a Wic image. Realize | 27 | Following, is an example that shows how to flash a Wic image. Realize |
28 | that while this example uses a Wic image, you can use bmaptool to flash | 28 | that while this example uses a Wic image, you can use `bmaptool` to flash |
29 | any type of image. Use these steps to flash an image using bmaptool: | 29 | any type of image. Use these steps to flash an image using `bmaptool`: |
30 | 30 | ||
31 | #. *Update your local.conf File:* You need to have the following set | 31 | #. *Update your local.conf File:* You need to have the following set |
32 | in your ``local.conf`` file before building your image:: | 32 | in your ``local.conf`` file before building your image:: |
@@ -39,18 +39,17 @@ any type of image. Use these steps to flash an image using bmaptool: | |||
39 | 39 | ||
40 | $ bitbake image | 40 | $ bitbake image |
41 | 41 | ||
42 | #. *Flash the Device:* Flash the device with the image by using bmaptool | 42 | #. *Flash the Device:* Flash the device with the image by using `bmaptool` |
43 | depending on your particular setup. The following commands assume the | 43 | depending on your particular setup. The following commands assume the |
44 | image resides in the :term:`Build Directory`'s ``deploy/images/`` area: | 44 | image resides in the :term:`Build Directory`'s ``deploy/images/`` area: |
45 | 45 | ||
46 | - If you have write access to the media, use this command form:: | 46 | - If you installed the package for `bmaptool`, you can directly run:: |
47 | 47 | ||
48 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | 48 | $ sudo bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX |
49 | 49 | ||
50 | - If you do not have write access to the media, set your permissions | 50 | - Otherwise, if you built `bmaptool` with BitBake, run:: |
51 | first and then use the same command form:: | ||
52 | 51 | ||
53 | $ sudo chmod 666 /dev/sdX | 52 | $ sudo chmod a+w /dev/sdX # get write access to the media, needed only once after booting |
54 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | 53 | $ oe-run-native bmaptool-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX |
55 | 54 | ||
56 | For help on the ``bmaptool`` command, use the following command:: | 55 | For help on the ``bmaptool`` command, use the following command:: |
diff --git a/documentation/dev-manual/build-quality.rst b/documentation/dev-manual/build-quality.rst index 713ea3a48e..fbe5fb6f0b 100644 --- a/documentation/dev-manual/build-quality.rst +++ b/documentation/dev-manual/build-quality.rst | |||
@@ -236,7 +236,7 @@ Here is an example of ``image-info.txt``: | |||
236 | DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be | 236 | DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be |
237 | USER_CLASSES = buildstats | 237 | USER_CLASSES = buildstats |
238 | IMAGE_CLASSES = qemuboot qemuboot license_image | 238 | IMAGE_CLASSES = qemuboot qemuboot license_image |
239 | IMAGE_FEATURES = debug-tweaks | 239 | IMAGE_FEATURES = allow-empty-password empty-root-password allow-root-login post-install-logging |
240 | IMAGE_LINGUAS = | 240 | IMAGE_LINGUAS = |
241 | IMAGE_INSTALL = packagegroup-core-boot speex speexdsp | 241 | IMAGE_INSTALL = packagegroup-core-boot speex speexdsp |
242 | BAD_RECOMMENDATIONS = | 242 | BAD_RECOMMENDATIONS = |
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst index fe502690dd..32c7aa5da0 100644 --- a/documentation/dev-manual/building.rst +++ b/documentation/dev-manual/building.rst | |||
@@ -48,7 +48,7 @@ The following figure and list overviews the build process: | |||
48 | :width: 100% | 48 | :width: 100% |
49 | 49 | ||
50 | #. *Set up Your Host Development System to Support Development Using the | 50 | #. *Set up Your Host Development System to Support Development Using the |
51 | Yocto Project*: See the ":doc:`start`" section for options on how to get a | 51 | Yocto Project*: See the ":doc:`/dev-manual/start`" section for options on how to get a |
52 | build host ready to use the Yocto Project. | 52 | build host ready to use the Yocto Project. |
53 | 53 | ||
54 | #. *Initialize the Build Environment:* Initialize the build environment | 54 | #. *Initialize the Build Environment:* Initialize the build environment |
@@ -113,160 +113,8 @@ The following figure and list overviews the build process: | |||
113 | Building Images for Multiple Targets Using Multiple Configurations | 113 | Building Images for Multiple Targets Using Multiple Configurations |
114 | ================================================================== | 114 | ================================================================== |
115 | 115 | ||
116 | You can use a single ``bitbake`` command to build multiple images or | 116 | See the :doc:`/dev-manual/multiconfig` section of the Yocto Project Development Tasks |
117 | packages for different targets where each image or package requires a | 117 | Manual. |
118 | different configuration (multiple configuration builds). The builds, in | ||
119 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
120 | section uses that term throughout. | ||
121 | |||
122 | This section describes how to set up for multiple configuration builds | ||
123 | and how to account for cross-build dependencies between the | ||
124 | multiconfigs. | ||
125 | |||
126 | Setting Up and Running a Multiple Configuration Build | ||
127 | ----------------------------------------------------- | ||
128 | |||
129 | To accomplish a multiple configuration build, you must define each | ||
130 | target's configuration separately using a parallel configuration file in | ||
131 | the :term:`Build Directory` or configuration directory within a layer, and you | ||
132 | must follow a required file hierarchy. Additionally, you must enable the | ||
133 | multiple configuration builds in your ``local.conf`` file. | ||
134 | |||
135 | Follow these steps to set up and execute multiple configuration builds: | ||
136 | |||
137 | - *Create Separate Configuration Files*: You need to create a single | ||
138 | configuration file for each build target (each multiconfig). | ||
139 | The configuration definitions are implementation dependent but often | ||
140 | each configuration file will define the machine and the | ||
141 | temporary directory BitBake uses for the build. Whether the same | ||
142 | temporary directory (:term:`TMPDIR`) can be shared will depend on what is | ||
143 | similar and what is different between the configurations. Multiple MACHINE | ||
144 | targets can share the same (:term:`TMPDIR`) as long as the rest of the | ||
145 | configuration is the same, multiple :term:`DISTRO` settings would need separate | ||
146 | (:term:`TMPDIR`) directories. | ||
147 | |||
148 | For example, consider a scenario with two different multiconfigs for the same | ||
149 | :term:`MACHINE`: "qemux86" built | ||
150 | for two distributions such as "poky" and "poky-lsb". In this case, | ||
151 | you would need to use the different :term:`TMPDIR`. | ||
152 | |||
153 | Here is an example showing the minimal statements needed in a | ||
154 | configuration file for a "qemux86" target whose temporary build | ||
155 | directory is ``tmpmultix86``:: | ||
156 | |||
157 | MACHINE = "qemux86" | ||
158 | TMPDIR = "${TOPDIR}/tmpmultix86" | ||
159 | |||
160 | The location for these multiconfig configuration files is specific. | ||
161 | They must reside in the current :term:`Build Directory` in a sub-directory of | ||
162 | ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory | ||
163 | under a directory named ``multiconfig``. Here is an example that defines | ||
164 | two configuration files for the "x86" and "arm" multiconfigs: | ||
165 | |||
166 | .. image:: figures/multiconfig_files.png | ||
167 | :align: center | ||
168 | :width: 50% | ||
169 | |||
170 | The usual :term:`BBPATH` search path is used to locate multiconfig files in | ||
171 | a similar way to other conf files. | ||
172 | |||
173 | - *Add the BitBake Multi-configuration Variable to the Local | ||
174 | Configuration File*: Use the | ||
175 | :term:`BBMULTICONFIG` | ||
176 | variable in your ``conf/local.conf`` configuration file to specify | ||
177 | each multiconfig. Continuing with the example from the previous | ||
178 | figure, the :term:`BBMULTICONFIG` variable needs to enable two | ||
179 | multiconfigs: "x86" and "arm" by specifying each configuration file:: | ||
180 | |||
181 | BBMULTICONFIG = "x86 arm" | ||
182 | |||
183 | .. note:: | ||
184 | |||
185 | A "default" configuration already exists by definition. This | ||
186 | configuration is named: "" (i.e. empty string) and is defined by | ||
187 | the variables coming from your ``local.conf`` | ||
188 | file. Consequently, the previous example actually adds two | ||
189 | additional configurations to your build: "arm" and "x86" along | ||
190 | with "". | ||
191 | |||
192 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
193 | the multiple configuration build:: | ||
194 | |||
195 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
196 | |||
197 | For the example in this section, the following command applies:: | ||
198 | |||
199 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
200 | |||
201 | The previous BitBake command builds a ``core-image-minimal`` image | ||
202 | that is configured through the ``x86.conf`` configuration file, a | ||
203 | ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
204 | configuration file and a ``core-image-base`` that is configured | ||
205 | through your ``local.conf`` configuration file. | ||
206 | |||
207 | .. note:: | ||
208 | |||
209 | Support for multiple configuration builds in the Yocto Project &DISTRO; | ||
210 | (&DISTRO_NAME;) Release does not include Shared State (sstate) | ||
211 | optimizations. Consequently, if a build uses the same object twice | ||
212 | in, for example, two different :term:`TMPDIR` | ||
213 | directories, the build either loads from an existing sstate cache for | ||
214 | that build at the start or builds the object fresh. | ||
215 | |||
216 | Enabling Multiple Configuration Build Dependencies | ||
217 | -------------------------------------------------- | ||
218 | |||
219 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
220 | multiple configuration build. For example, suppose that in order to | ||
221 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
222 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
223 | essentially that the | ||
224 | :ref:`ref-tasks-image` task in the | ||
225 | ``core-image-sato`` recipe depends on the completion of the | ||
226 | :ref:`ref-tasks-rootfs` task of the | ||
227 | ``core-image-minimal`` recipe. | ||
228 | |||
229 | To enable dependencies in a multiple configuration build, you must | ||
230 | declare the dependencies in the recipe using the following statement | ||
231 | form:: | ||
232 | |||
233 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
234 | |||
235 | To better show how to use this statement, consider the example scenario | ||
236 | from the first paragraph of this section. The following statement needs | ||
237 | to be added to the recipe that builds the ``core-image-sato`` image:: | ||
238 | |||
239 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
240 | |||
241 | In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The | ||
242 | task on which the :ref:`ref-tasks-image` task in the recipe depends is the | ||
243 | :ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated | ||
244 | with the "arm" multiconfig. | ||
245 | |||
246 | Once you set up this dependency, you can build the "x86" multiconfig | ||
247 | using a BitBake command as follows:: | ||
248 | |||
249 | $ bitbake mc:x86:core-image-sato | ||
250 | |||
251 | This command executes all the tasks needed to create the | ||
252 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
253 | dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the | ||
254 | "arm" multiconfig build. | ||
255 | |||
256 | Having a recipe depend on the root filesystem of another build might not | ||
257 | seem that useful. Consider this change to the statement in the | ||
258 | ``core-image-sato`` recipe:: | ||
259 | |||
260 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
261 | |||
262 | In this case, BitBake must | ||
263 | create the ``core-image-minimal`` image for the "arm" build since the | ||
264 | "x86" build depends on it. | ||
265 | |||
266 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
267 | and have separate configuration files, BitBake places the artifacts for | ||
268 | each build in the respective temporary build directories (i.e. | ||
269 | :term:`TMPDIR`). | ||
270 | 118 | ||
271 | Building an Initial RAM Filesystem (Initramfs) Image | 119 | Building an Initial RAM Filesystem (Initramfs) Image |
272 | ==================================================== | 120 | ==================================================== |
@@ -280,7 +128,9 @@ Follow these steps to create an :term:`Initramfs` image: | |||
280 | #. *Create the Initramfs Image Recipe:* You can reference the | 128 | #. *Create the Initramfs Image Recipe:* You can reference the |
281 | ``core-image-minimal-initramfs.bb`` recipe found in the | 129 | ``core-image-minimal-initramfs.bb`` recipe found in the |
282 | ``meta/recipes-core`` directory of the :term:`Source Directory` | 130 | ``meta/recipes-core`` directory of the :term:`Source Directory` |
283 | as an example from which to work. | 131 | as an example from which to work. The ``core-image-minimal-initramfs`` recipe |
132 | is based on the :ref:`initramfs-framework <dev-manual/building:Customizing an | ||
133 | Initramfs using \`\`initramfs-framework\`\`>` recipe described below. | ||
284 | 134 | ||
285 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel | 135 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel |
286 | Image:* If you want the :term:`Initramfs` image that is built to be bundled | 136 | Image:* If you want the :term:`Initramfs` image that is built to be bundled |
@@ -308,6 +158,86 @@ Follow these steps to create an :term:`Initramfs` image: | |||
308 | and bundled with the kernel image if you used the | 158 | and bundled with the kernel image if you used the |
309 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. | 159 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. |
310 | 160 | ||
161 | Customizing an Initramfs using ``initramfs-framework`` | ||
162 | ------------------------------------------------------ | ||
163 | |||
164 | The ``core-image-minimal-initramfs.bb`` recipe found in | ||
165 | :oe_git:`meta/recipes-core/images | ||
166 | </openembedded-core/tree/meta/recipes-core/images>` uses the | ||
167 | :oe_git:`initramfs-framework_1.0.bb | ||
168 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb>` | ||
169 | recipe as its base component. The goal of the ``initramfs-framework`` recipe is | ||
170 | to provide the building blocks to build a customized :term:`Initramfs`. | ||
171 | |||
172 | The ``initramfs-framework`` recipe relies on shell initialization scripts | ||
173 | defined in :oe_git:`meta/recipes-core/initrdscripts/initramfs-framework | ||
174 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework>`. Since some of | ||
175 | these scripts do not apply for all use cases, the ``initramfs-framework`` recipe | ||
176 | defines different packages: | ||
177 | |||
178 | - ``initramfs-framework-base``: this package installs the basic components of | ||
179 | an :term:`Initramfs`, such as the ``init`` script or the ``/dev/console`` | ||
180 | character special file. As this package is a runtime dependency of all | ||
181 | modules listed below, it is automatically pulled in when one of the modules | ||
182 | is installed in the image. | ||
183 | - ``initramfs-module-exec``: support for execution of applications. | ||
184 | - ``initramfs-module-mdev``: support for `mdev | ||
185 | <https://wiki.gentoo.org/wiki/Mdev>`__. | ||
186 | - ``initramfs-module-udev``: support for :wikipedia:`Udev <Udev>`. | ||
187 | - ``initramfs-module-e2fs``: support for :wikipedia:`ext4/ext3/ext2 | ||
188 | <Extended_file_system>` filesystems. | ||
189 | - ``initramfs-module-nfsrootfs``: support for locating and mounting the root | ||
190 | partition via :wikipedia:`NFS <Network_File_System>`. | ||
191 | - ``initramfs-module-rootfs``: support for locating and mounting the root | ||
192 | partition. | ||
193 | - ``initramfs-module-debug``: dynamic debug support. | ||
194 | - ``initramfs-module-lvm``: :wikipedia:`LVM <Logical_volume_management>` rootfs support. | ||
195 | - ``initramfs-module-overlayroot``: support for mounting a read-write overlay | ||
196 | on top of a read-only root filesystem. | ||
197 | |||
198 | In addition to the packages defined by the ``initramfs-framework`` recipe | ||
199 | itself, the following packages are defined by the recipes present in | ||
200 | :oe_git:`meta/recipes-core/initrdscripts </openembedded-core/tree/meta/recipes-core/initrdscripts>`: | ||
201 | |||
202 | - ``initramfs-module-install``: module to create and install a partition layout | ||
203 | on a selected block device. | ||
204 | - ``initramfs-module-install-efi``: module to create and install an EFI | ||
205 | partition layout on a selected block device. | ||
206 | - ``initramfs-module-setup-live``: module to start a shell in the | ||
207 | :term:`Initramfs` if ``root=/dev/ram0`` in passed in the `Kernel command-line | ||
208 | <https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html>`__ | ||
209 | or the ``root=`` parameter was not passed. | ||
210 | |||
211 | To customize the :term:`Initramfs`, you can add or remove packages listed | ||
212 | earlier from the :term:`PACKAGE_INSTALL` variable with a :ref:`bbappend | ||
213 | <dev-manual/layers:Appending Other Layers Metadata With Your Layer>` on the | ||
214 | ``core-image-minimal-initramfs`` recipe, or create a custom recipe for the | ||
215 | :term:`Initramfs` taking ``core-image-minimal-initramfs`` as example. | ||
216 | |||
217 | Custom scripts can be added to the :term:`Initramfs` by writing your own | ||
218 | recipes. The recipes are conventionally named ``initramfs-module-<module name>`` | ||
219 | where ``<module name>`` is the name of the module. The recipe should set its | ||
220 | :term:`RDEPENDS` package-specific variables to include | ||
221 | ``initramfs-framework-base`` and the other packages on which the module depends | ||
222 | at runtime. | ||
223 | |||
224 | The recipe must install shell initialization scripts in :term:`${D} <D>`\ | ||
225 | ``/init.d`` and must follow the ``<number>-<script name>`` naming scheme where: | ||
226 | |||
227 | - ``<number>`` is a *two-digit* number that affects the execution order of the | ||
228 | script compared to others. For example, the script ``80-setup-live`` would be | ||
229 | executed after ``01-udev`` because 80 is greater than 01. | ||
230 | |||
231 | This number being two-digits is important here as the scripts are executed | ||
232 | alphabetically. For example, the script ``10-script`` would be executed | ||
233 | before the script ``8-script``, because ``1`` is inferior to ``8``. | ||
234 | Therefore, the script should be named ``08-script``. | ||
235 | |||
236 | - ``<script name>`` is the script name which you can choose freely. | ||
237 | |||
238 | If two script use the same ``<number>``, they are sorted alphabetically based | ||
239 | on ``<script name>``. | ||
240 | |||
311 | Bundling an Initramfs Image From a Separate Multiconfig | 241 | Bundling an Initramfs Image From a Separate Multiconfig |
312 | ------------------------------------------------------- | 242 | ------------------------------------------------------- |
313 | 243 | ||
@@ -661,7 +591,7 @@ If build speed and package feed maintenance are considerations, you | |||
661 | should consider the points in this section that can help you optimize | 591 | should consider the points in this section that can help you optimize |
662 | your tunings to best consider build times and package feed maintenance. | 592 | your tunings to best consider build times and package feed maintenance. |
663 | 593 | ||
664 | - *Share the :term:`Build Directory`:* If at all possible, share the | 594 | - *Share the* :term:`Build Directory` *:* If at all possible, share the |
665 | :term:`TMPDIR` across builds. The Yocto Project supports switching between | 595 | :term:`TMPDIR` across builds. The Yocto Project supports switching between |
666 | different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice | 596 | different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice |
667 | is well supported and regularly used by developers when building for | 597 | is well supported and regularly used by developers when building for |
@@ -883,7 +813,7 @@ directory: | |||
883 | 813 | ||
884 | #. *Using Local Files Only:* Inside your ``local.conf`` file, add the | 814 | #. *Using Local Files Only:* Inside your ``local.conf`` file, add the |
885 | :term:`SOURCE_MIRROR_URL` variable, inherit the | 815 | :term:`SOURCE_MIRROR_URL` variable, inherit the |
886 | :ref:`ref-classes-own-mirrors` class, and use the | 816 | :ref:`ref-classes-own-mirrors` class, and add the |
887 | :term:`BB_NO_NETWORK` variable to your ``local.conf``:: | 817 | :term:`BB_NO_NETWORK` variable to your ``local.conf``:: |
888 | 818 | ||
889 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | 819 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" |
diff --git a/documentation/dev-manual/custom-distribution.rst b/documentation/dev-manual/custom-distribution.rst index 47faed0d04..0bc386d606 100644 --- a/documentation/dev-manual/custom-distribution.rst +++ b/documentation/dev-manual/custom-distribution.rst | |||
@@ -4,10 +4,16 @@ Creating Your Own Distribution | |||
4 | ****************************** | 4 | ****************************** |
5 | 5 | ||
6 | When you build an image using the Yocto Project and do not alter any | 6 | When you build an image using the Yocto Project and do not alter any |
7 | distribution :term:`Metadata`, you are | 7 | distribution :term:`Metadata`, you are using the Poky distribution. |
8 | creating a Poky distribution. If you wish to gain more control over | 8 | Poky is explicitly a *reference* distribution for testing and |
9 | package alternative selections, compile-time options, and other | 9 | development purposes. It enables most hardware and software features |
10 | low-level configurations, you can create your own distribution. | 10 | so that they can be tested, but this also means that from a security |
11 | point of view the attack surface is very large. Additionally, at some | ||
12 | point it is likely that you will want to gain more control over package | ||
13 | alternative selections, compile-time options, and other low-level | ||
14 | configurations. For both of these reasons, if you are using the Yocto | ||
15 | Project for production use then you are strongly encouraged to create | ||
16 | your own distribution. | ||
11 | 17 | ||
12 | To create your own distribution, the basic steps consist of creating | 18 | To create your own distribution, the basic steps consist of creating |
13 | your own distribution layer, creating your own distribution | 19 | your own distribution layer, creating your own distribution |
@@ -107,3 +113,23 @@ layer. The following steps provide some more detail: | |||
107 | For information on append files, see the | 113 | For information on append files, see the |
108 | ":ref:`dev-manual/layers:appending other layers metadata with your layer`" | 114 | ":ref:`dev-manual/layers:appending other layers metadata with your layer`" |
109 | section. | 115 | section. |
116 | |||
117 | Copying and modifying the Poky distribution | ||
118 | =========================================== | ||
119 | |||
120 | Instead of creating a custom distribution from scratch as per above, you may | ||
121 | wish to start your custom distribution configuration by copying the Poky | ||
122 | distribution provided within the ``meta-poky`` layer and then modifying it. | ||
123 | This is fine, however if you do this you should keep the following in mind: | ||
124 | |||
125 | - Every reference to Poky needs to be updated in your copy so that it | ||
126 | will still apply. This includes override usage within files (e.g. ``:poky``) | ||
127 | and in directory names. This is a good opportunity to evaluate each one of | ||
128 | these customizations to see if they are needed for your use case. | ||
129 | |||
130 | - Unless you also intend to use them, the ``poky-tiny``, ``poky-altcfg`` and | ||
131 | ``poky-bleeding`` variants and any references to them can be removed. | ||
132 | |||
133 | - More generally, the Poky distribution configuration enables a lot more | ||
134 | than you likely need for your production use case. You should evaluate *every* | ||
135 | configuration choice made in your copy to determine if it is needed. | ||
diff --git a/documentation/dev-manual/customizing-images.rst b/documentation/dev-manual/customizing-images.rst index 5b18958ade..6eed9ef56a 100644 --- a/documentation/dev-manual/customizing-images.rst +++ b/documentation/dev-manual/customizing-images.rst | |||
@@ -62,8 +62,7 @@ To understand how these features work, the best reference is | |||
62 | :ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`. | 62 | :ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`. |
63 | This class lists out the available | 63 | This class lists out the available |
64 | :term:`IMAGE_FEATURES` of which most map to package groups while some, such | 64 | :term:`IMAGE_FEATURES` of which most map to package groups while some, such |
65 | as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general | 65 | as ``read-only-rootfs``, resolve as general configuration settings. |
66 | configuration settings. | ||
67 | 66 | ||
68 | In summary, the file looks at the contents of the :term:`IMAGE_FEATURES` | 67 | In summary, the file looks at the contents of the :term:`IMAGE_FEATURES` |
69 | variable and then maps or configures the feature accordingly. Based on | 68 | variable and then maps or configures the feature accordingly. Based on |
@@ -80,15 +79,14 @@ recipe that are enabled with :term:`IMAGE_FEATURES`. The value of | |||
80 | :term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within | 79 | :term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within |
81 | ``meta/conf/bitbake.conf``. | 80 | ``meta/conf/bitbake.conf``. |
82 | 81 | ||
83 | To illustrate how you can use these variables to modify your image, | 82 | To illustrate how you can use these variables to modify your image, consider an |
84 | consider an example that selects the SSH server. The Yocto Project ships | 83 | example that selects the SSH server. The Yocto Project ships with two SSH |
85 | with two SSH servers you can use with your images: Dropbear and OpenSSH. | 84 | servers you can use with your images: Dropbear and OpenSSH. Dropbear is a |
86 | Dropbear is a minimal SSH server appropriate for resource-constrained | 85 | minimal SSH server appropriate for resource-constrained environments, while |
87 | environments, while OpenSSH is a well-known standard SSH server | 86 | OpenSSH is a well-known standard SSH server implementation. By default, the |
88 | implementation. By default, the ``core-image-sato`` image is configured | 87 | ``core-image-sato`` image is configured to use Dropbear. The |
89 | to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb`` | 88 | ``core-image-full-cmdline`` image includes OpenSSH. The ``core-image-minimal`` |
90 | images both include OpenSSH. The ``core-image-minimal`` image does not | 89 | image does not contain an SSH server. |
91 | contain an SSH server. | ||
92 | 90 | ||
93 | You can customize your image and change these defaults. Edit the | 91 | You can customize your image and change these defaults. Edit the |
94 | :term:`IMAGE_FEATURES` variable in your recipe or use the | 92 | :term:`IMAGE_FEATURES` variable in your recipe or use the |
diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst index e20637e1c6..6c45ccf652 100644 --- a/documentation/dev-manual/debugging.rst +++ b/documentation/dev-manual/debugging.rst | |||
@@ -36,7 +36,7 @@ section: | |||
36 | use the BitBake ``-e`` option to examine variable values after a | 36 | use the BitBake ``-e`` option to examine variable values after a |
37 | recipe has been parsed. | 37 | recipe has been parsed. |
38 | 38 | ||
39 | - ":ref:`dev-manual/debugging:viewing package information with \`\`oe-pkgdata-util\`\``" | 39 | - ":ref:`dev-manual/debugging:viewing package information with ``oe-pkgdata-util```" |
40 | describes how to use the ``oe-pkgdata-util`` utility to query | 40 | describes how to use the ``oe-pkgdata-util`` utility to query |
41 | :term:`PKGDATA_DIR` and | 41 | :term:`PKGDATA_DIR` and |
42 | display package-related information for built packages. | 42 | display package-related information for built packages. |
@@ -270,13 +270,17 @@ format and can be converted to images (e.g. using the ``dot`` tool from | |||
270 | displays paths between graph nodes. | 270 | displays paths between graph nodes. |
271 | 271 | ||
272 | You can use a different method to view dependency information by using | 272 | You can use a different method to view dependency information by using |
273 | the following command:: | 273 | either:: |
274 | 274 | ||
275 | $ bitbake -g -u taskexp recipename | 275 | $ bitbake -g -u taskexp recipename |
276 | 276 | ||
277 | This command | 277 | or:: |
278 | displays a GUI window from which you can view build-time and runtime | 278 | |
279 | dependencies for the recipes involved in building recipename. | 279 | $ bitbake -g -u taskexp_ncurses recipename |
280 | |||
281 | The ``-u taskdep`` option GUI window from which you can view build-time and | ||
282 | runtime dependencies for the recipes involved in building recipename. The | ||
283 | ``-u taskexp_ncurses`` option uses ncurses instead of GTK to render the UI. | ||
280 | 284 | ||
281 | Viewing Task Variable Dependencies | 285 | Viewing Task Variable Dependencies |
282 | ================================== | 286 | ================================== |
@@ -886,7 +890,7 @@ The build should work without issue. | |||
886 | As with all solved problems, if they originated upstream, you need to | 890 | As with all solved problems, if they originated upstream, you need to |
887 | submit the fix for the recipe in OE-Core and upstream so that the | 891 | submit the fix for the recipe in OE-Core and upstream so that the |
888 | problem is taken care of at its source. See the | 892 | problem is taken care of at its source. See the |
889 | ":doc:`../contributor-guide/submit-changes`" section for more information. | 893 | ":doc:`/contributor-guide/submit-changes`" section for more information. |
890 | 894 | ||
891 | Debugging With the GNU Project Debugger (GDB) Remotely | 895 | Debugging With the GNU Project Debugger (GDB) Remotely |
892 | ====================================================== | 896 | ====================================================== |
@@ -1257,7 +1261,7 @@ Here are some other tips that you might find useful: | |||
1257 | :yocto_bugs:`Bugzilla <>`. For information on | 1261 | :yocto_bugs:`Bugzilla <>`. For information on |
1258 | how to submit a bug against the Yocto Project, see the Yocto Project | 1262 | how to submit a bug against the Yocto Project, see the Yocto Project |
1259 | Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` | 1263 | Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` |
1260 | and the ":doc:`../contributor-guide/report-defect`" section. | 1264 | and the ":doc:`/contributor-guide/report-defect`" section. |
1261 | 1265 | ||
1262 | .. note:: | 1266 | .. note:: |
1263 | 1267 | ||
diff --git a/documentation/dev-manual/devtool.rst b/documentation/dev-manual/devtool.rst new file mode 100644 index 0000000000..c82dc9c333 --- /dev/null +++ b/documentation/dev-manual/devtool.rst | |||
@@ -0,0 +1,1322 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Using the ``devtool`` command-line tool | ||
4 | *************************************** | ||
5 | |||
6 | The ``devtool`` command-line tool provides a number of features that | ||
7 | help you build, test, and package software. This command is available | ||
8 | alongside the ``bitbake`` command. Additionally, the ``devtool`` command | ||
9 | is a key part of the :term:`Extensible Software Development Kit (eSDK)`. | ||
10 | |||
11 | Use ``devtool add`` to Add an Application | ||
12 | ========================================= | ||
13 | |||
14 | The ``devtool add`` command generates a new recipe based on existing | ||
15 | source code. This command takes advantage of the | ||
16 | :ref:`devtool-the-workspace-layer-structure` | ||
17 | layer that many ``devtool`` commands use. The command is flexible enough | ||
18 | to allow you to extract source code into both the workspace or a | ||
19 | separate local Git repository and to use existing code that does not | ||
20 | need to be extracted. | ||
21 | |||
22 | Depending on your particular scenario, the arguments and options you use | ||
23 | with ``devtool add`` form different combinations. The following diagram | ||
24 | shows common development flows you would use with the ``devtool add`` | ||
25 | command: | ||
26 | |||
27 | .. image:: figures/devtool-add-flow.png | ||
28 | :width: 100% | ||
29 | |||
30 | #. *Generating the New Recipe*: The top part of the flow shows three | ||
31 | scenarios by which you could use ``devtool add`` to generate a recipe | ||
32 | based on existing source code. | ||
33 | |||
34 | In a shared development environment, it is typical for other | ||
35 | developers to be responsible for various areas of source code. As a | ||
36 | developer, you are probably interested in using that source code as | ||
37 | part of your development within the Yocto Project. All you need is | ||
38 | access to the code, a recipe, and a controlled area in which to do | ||
39 | your work. | ||
40 | |||
41 | Within the diagram, three possible scenarios feed into the | ||
42 | ``devtool add`` workflow: | ||
43 | |||
44 | - *Left*: The left scenario in the figure represents a common | ||
45 | situation where the source code does not exist locally and needs | ||
46 | to be extracted. In this situation, the source code is extracted | ||
47 | to the default workspace --- you do not want the files in some | ||
48 | specific location outside of the workspace. Thus, everything you | ||
49 | need will be located in the workspace:: | ||
50 | |||
51 | $ devtool add recipe fetchuri | ||
52 | |||
53 | With this command, ``devtool`` extracts the upstream | ||
54 | source files into a local Git repository within the ``sources`` | ||
55 | folder. The command then creates a recipe named recipe and a | ||
56 | corresponding append file in the workspace. If you do not provide | ||
57 | recipe, the command makes an attempt to determine the recipe name. | ||
58 | |||
59 | - *Middle*: The middle scenario in the figure also represents a | ||
60 | situation where the source code does not exist locally. In this | ||
61 | case, the code is again upstream and needs to be extracted to some | ||
62 | local area --- this time outside of the default workspace. | ||
63 | |||
64 | .. note:: | ||
65 | |||
66 | If required, ``devtool`` always creates a Git repository locally | ||
67 | during the extraction. | ||
68 | |||
69 | Furthermore, the first positional argument ``srctree`` in this case | ||
70 | identifies where the ``devtool add`` command will locate the | ||
71 | extracted code outside of the workspace. You need to specify an | ||
72 | empty directory:: | ||
73 | |||
74 | $ devtool add recipe srctree fetchuri | ||
75 | |||
76 | In summary, the source code is pulled from fetchuri and extracted into the | ||
77 | location defined by ``srctree`` as a local Git repository. | ||
78 | |||
79 | Within workspace, ``devtool`` creates a recipe named recipe along | ||
80 | with an associated append file. | ||
81 | |||
82 | - *Right*: The right scenario in the figure represents a situation | ||
83 | where the ``srctree`` has been previously prepared outside of the | ||
84 | ``devtool`` workspace. | ||
85 | |||
86 | The following command provides a new recipe name and identifies | ||
87 | the existing source tree location:: | ||
88 | |||
89 | $ devtool add recipe srctree | ||
90 | |||
91 | The command examines the source code and creates a recipe named | ||
92 | recipe for the code and places the recipe into the workspace. | ||
93 | |||
94 | Because the extracted source code already exists, ``devtool`` does | ||
95 | not try to relocate the source code into the workspace --- only the | ||
96 | new recipe is placed in the workspace. | ||
97 | |||
98 | Aside from a recipe folder, the command also creates an associated | ||
99 | append folder and places an initial ``*.bbappend`` file within. | ||
100 | |||
101 | #. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the | ||
102 | editor as defined by the ``$EDITOR`` environment variable and modify | ||
103 | the file:: | ||
104 | |||
105 | $ devtool edit-recipe recipe | ||
106 | |||
107 | From within the editor, you can make modifications to the recipe that | ||
108 | take effect when you build it later. | ||
109 | |||
110 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
111 | depends on what you are going to do with the new code. | ||
112 | |||
113 | If you need to eventually move the build output to the target | ||
114 | hardware, use the following ``devtool`` command:: | ||
115 | |||
116 | $ devtool build recipe | ||
117 | |||
118 | On the other hand, if you want an image to contain the recipe's | ||
119 | packages from the workspace for immediate deployment onto a device | ||
120 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
121 | command:: | ||
122 | |||
123 | $ devtool build-image image | ||
124 | |||
125 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
126 | to build out your recipe, you probably want to see if the resulting | ||
127 | build output works as expected on the target hardware. | ||
128 | |||
129 | .. note:: | ||
130 | |||
131 | This step assumes you have a previously built image that is | ||
132 | already either running in QEMU or is running on actual hardware. | ||
133 | Also, it is assumed that for deployment of the image to the | ||
134 | target, SSH is installed in the image and, if the image is running | ||
135 | on real hardware, you have network access to and from your | ||
136 | development machine. | ||
137 | |||
138 | You can deploy your build output to that target hardware by using the | ||
139 | ``devtool deploy-target`` command:: | ||
140 | |||
141 | $ devtool deploy-target recipe target | ||
142 | |||
143 | The target is a live target machine running as an SSH server. | ||
144 | |||
145 | You can, of course, also deploy the image you build to actual | ||
146 | hardware by using the ``devtool build-image`` command. However, | ||
147 | ``devtool`` does not provide a specific command that allows you to | ||
148 | deploy the image to actual hardware. | ||
149 | |||
150 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
151 | creates any patches corresponding to commits in the local Git | ||
152 | repository, moves the new recipe to a more permanent layer, and then | ||
153 | resets the recipe so that the recipe is built normally rather than | ||
154 | from the workspace:: | ||
155 | |||
156 | $ devtool finish recipe layer | ||
157 | |||
158 | .. note:: | ||
159 | |||
160 | Any changes you want to turn into patches must be committed to the | ||
161 | Git repository in the source tree. | ||
162 | |||
163 | As mentioned, the ``devtool finish`` command moves the final recipe | ||
164 | to its permanent layer. | ||
165 | |||
166 | As a final process of the ``devtool finish`` command, the state of | ||
167 | the standard layers and the upstream source is restored so that you | ||
168 | can build the recipe from those areas rather than the workspace. | ||
169 | |||
170 | .. note:: | ||
171 | |||
172 | You can use the ``devtool reset`` command to put things back should you | ||
173 | decide you do not want to proceed with your work. If you do use this | ||
174 | command, realize that the source tree is preserved. | ||
175 | |||
176 | Use ``devtool modify`` to Modify the Source of an Existing Component | ||
177 | ==================================================================== | ||
178 | |||
179 | The ``devtool modify`` command prepares the way to work on existing code | ||
180 | that already has a local recipe in place that is used to build the | ||
181 | software. The command is flexible enough to allow you to extract code | ||
182 | from an upstream source, specify the existing recipe, and keep track of | ||
183 | and gather any patch files from other developers that are associated | ||
184 | with the code. | ||
185 | |||
186 | Depending on your particular scenario, the arguments and options you use | ||
187 | with ``devtool modify`` form different combinations. The following | ||
188 | diagram shows common development flows for the ``devtool modify`` | ||
189 | command: | ||
190 | |||
191 | .. image:: figures/devtool-modify-flow.png | ||
192 | :width: 100% | ||
193 | |||
194 | #. *Preparing to Modify the Code*: The top part of the flow shows three | ||
195 | scenarios by which you could use ``devtool modify`` to prepare to | ||
196 | work on source files. Each scenario assumes the following: | ||
197 | |||
198 | - The recipe exists locally in a layer external to the ``devtool`` | ||
199 | workspace. | ||
200 | |||
201 | - The source files exist either upstream in an un-extracted state or | ||
202 | locally in a previously extracted state. | ||
203 | |||
204 | The typical situation is where another developer has created a layer | ||
205 | for use with the Yocto Project and their recipe already resides in | ||
206 | that layer. Furthermore, their source code is readily available | ||
207 | either upstream or locally. | ||
208 | |||
209 | - *Left*: The left scenario in the figure represents a common | ||
210 | situation where the source code does not exist locally and it | ||
211 | needs to be extracted from an upstream source. In this situation, | ||
212 | the source is extracted into the default ``devtool`` workspace | ||
213 | location. The recipe, in this scenario, is in its own layer | ||
214 | outside the workspace (i.e. ``meta-``\ layername). | ||
215 | |||
216 | The following command identifies the recipe and, by default, | ||
217 | extracts the source files:: | ||
218 | |||
219 | $ devtool modify recipe | ||
220 | |||
221 | Once ``devtool`` locates the recipe, ``devtool`` uses the recipe's | ||
222 | :term:`SRC_URI` statements to locate the source code and any local | ||
223 | patch files from other developers. | ||
224 | |||
225 | With this scenario, there is no ``srctree`` argument. Consequently, the | ||
226 | default behavior of the ``devtool modify`` command is to extract | ||
227 | the source files pointed to by the :term:`SRC_URI` statements into a | ||
228 | local Git structure. Furthermore, the location for the extracted | ||
229 | source is the default area within the ``devtool`` workspace. The | ||
230 | result is that the command sets up both the source code and an | ||
231 | append file within the workspace while the recipe remains in its | ||
232 | original location. | ||
233 | |||
234 | Additionally, if you have any non-patch local files (i.e. files | ||
235 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
236 | excluding ``*.patch/`` or ``*.diff``), these files are copied to | ||
237 | an ``oe-local-files`` folder under the newly created source tree. | ||
238 | Copying the files here gives you a convenient area from which you | ||
239 | can modify the files. Any changes or additions you make to those | ||
240 | files are incorporated into the build the next time you build the | ||
241 | software just as are other changes you might have made to the | ||
242 | source. | ||
243 | |||
244 | - *Middle*: The middle scenario in the figure represents a situation | ||
245 | where the source code also does not exist locally. In this case, | ||
246 | the code is again upstream and needs to be extracted to some local | ||
247 | area as a Git repository. The recipe, in this scenario, is again | ||
248 | local and in its own layer outside the workspace. | ||
249 | |||
250 | The following command tells ``devtool`` the recipe with which to | ||
251 | work and, in this case, identifies a local area for the extracted | ||
252 | source files that exists outside of the default ``devtool`` | ||
253 | workspace:: | ||
254 | |||
255 | $ devtool modify recipe srctree | ||
256 | |||
257 | .. note:: | ||
258 | |||
259 | You cannot provide a URL for ``srctree`` using the ``devtool`` command. | ||
260 | |||
261 | As with all extractions, the command uses the recipe's :term:`SRC_URI` | ||
262 | statements to locate the source files and any associated patch | ||
263 | files. Non-patch files are copied to an ``oe-local-files`` folder | ||
264 | under the newly created source tree. | ||
265 | |||
266 | Once the files are located, the command by default extracts them | ||
267 | into ``srctree``. | ||
268 | |||
269 | Within workspace, ``devtool`` creates an append file for the | ||
270 | recipe. The recipe remains in its original location but the source | ||
271 | files are extracted to the location you provide with ``srctree``. | ||
272 | |||
273 | - *Right*: The right scenario in the figure represents a situation | ||
274 | where the source tree (``srctree``) already exists locally as a | ||
275 | previously extracted Git structure outside of the ``devtool`` | ||
276 | workspace. In this example, the recipe also exists elsewhere | ||
277 | locally in its own layer. | ||
278 | |||
279 | The following command tells ``devtool`` the recipe with which to | ||
280 | work, uses the "-n" option to indicate source does not need to be | ||
281 | extracted, and uses ``srctree`` to point to the previously extracted | ||
282 | source files:: | ||
283 | |||
284 | $ devtool modify -n recipe srctree | ||
285 | |||
286 | If an ``oe-local-files`` subdirectory happens to exist and it | ||
287 | contains non-patch files, the files are used. However, if the | ||
288 | subdirectory does not exist and you run the ``devtool finish`` | ||
289 | command, any non-patch files that might exist next to the recipe | ||
290 | are removed because it appears to ``devtool`` that you have | ||
291 | deleted those files. | ||
292 | |||
293 | Once the ``devtool modify`` command finishes, it creates only an | ||
294 | append file for the recipe in the ``devtool`` workspace. The | ||
295 | recipe and the source code remain in their original locations. | ||
296 | |||
297 | #. *Edit the Source*: Once you have used the ``devtool modify`` command, | ||
298 | you are free to make changes to the source files. You can use any | ||
299 | editor you like to make and save your source code modifications. | ||
300 | |||
301 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
302 | depends on what you are going to do with the new code. | ||
303 | |||
304 | If you need to eventually move the build output to the target | ||
305 | hardware, use the following ``devtool`` command:: | ||
306 | |||
307 | $ devtool build recipe | ||
308 | |||
309 | On the other hand, if you want an image to contain the recipe's | ||
310 | packages from the workspace for immediate deployment onto a device | ||
311 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
312 | command:: | ||
313 | |||
314 | $ devtool build-image image | ||
315 | |||
316 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
317 | to build out your recipe, you probably want to see if the resulting | ||
318 | build output works as expected on target hardware. | ||
319 | |||
320 | .. note:: | ||
321 | |||
322 | This step assumes you have a previously built image that is | ||
323 | already either running in QEMU or running on actual hardware. | ||
324 | Also, it is assumed that for deployment of the image to the | ||
325 | target, SSH is installed in the image and if the image is running | ||
326 | on real hardware that you have network access to and from your | ||
327 | development machine. | ||
328 | |||
329 | You can deploy your build output to that target hardware by using the | ||
330 | ``devtool deploy-target`` command:: | ||
331 | |||
332 | $ devtool deploy-target recipe target | ||
333 | |||
334 | The target is a live target machine running as an SSH server. | ||
335 | |||
336 | You can, of course, use other methods to deploy the image you built | ||
337 | using the ``devtool build-image`` command to actual hardware. | ||
338 | ``devtool`` does not provide a specific command to deploy the image | ||
339 | to actual hardware. | ||
340 | |||
341 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
342 | creates any patches corresponding to commits in the local Git | ||
343 | repository, updates the recipe to point to them (or creates a | ||
344 | ``.bbappend`` file to do so, depending on the specified destination | ||
345 | layer), and then resets the recipe so that the recipe is built | ||
346 | normally rather than from the workspace:: | ||
347 | |||
348 | $ devtool finish recipe layer | ||
349 | |||
350 | .. note:: | ||
351 | |||
352 | Any changes you want to turn into patches must be staged and | ||
353 | committed within the local Git repository before you use the | ||
354 | ``devtool finish`` command. | ||
355 | |||
356 | Because there is no need to move the recipe, ``devtool finish`` | ||
357 | either updates the original recipe in the original layer or the | ||
358 | command creates a ``.bbappend`` file in a different layer as provided | ||
359 | by layer. Any work you did in the ``oe-local-files`` directory is | ||
360 | preserved in the original files next to the recipe during the | ||
361 | ``devtool finish`` command. | ||
362 | |||
363 | As a final process of the ``devtool finish`` command, the state of | ||
364 | the standard layers and the upstream source is restored so that you | ||
365 | can build the recipe from those areas rather than from the workspace. | ||
366 | |||
367 | .. note:: | ||
368 | |||
369 | You can use the ``devtool reset`` command to put things back should you | ||
370 | decide you do not want to proceed with your work. If you do use this | ||
371 | command, realize that the source tree is preserved. | ||
372 | |||
373 | ``devtool ide-sdk`` configures IDEs and bootstraps SDKs | ||
374 | ======================================================= | ||
375 | |||
376 | The ``devtool ide-sdk`` command can provide an IDE configuration for IDEs when | ||
377 | working on the source code of one or more recipes. | ||
378 | Depending on the programming language, and the build system used by the recipe, | ||
379 | the tools required for cross-development and remote debugging are different. | ||
380 | For example: | ||
381 | |||
382 | - A C/C++ project usually uses CMake or Meson. | ||
383 | |||
384 | - A Python project uses setuptools or one of its successors. | ||
385 | |||
386 | - A Rust project uses Cargo. | ||
387 | |||
388 | Also, the IDE plugins needed for the integration of a build system with the | ||
389 | IDE and the corresponding settings are usually specific to these build-systems. | ||
390 | To hide all these details from the user, ``devtool ide-sdk`` does two things: | ||
391 | |||
392 | - It generates any kind of SDK needed for cross-development and remote | ||
393 | debugging of the specified recipes. | ||
394 | |||
395 | - It generates the configuration for the IDE (and the IDE plugins) for using | ||
396 | the cross-toolchain and remote debugging tools provided by the SDK directly | ||
397 | from the IDE. | ||
398 | |||
399 | For supported build systems the configurations generated by ``devtool ide-sdk`` | ||
400 | combine the advantages of the ``devtool modify`` based workflow | ||
401 | (see :ref:`using_devtool`) with the advantages of the simple Environment Setup | ||
402 | script based workflow (see :ref:`running_the_ext_sdk_env`) provided by Yocto's | ||
403 | SDK or eSDK: | ||
404 | |||
405 | - The source code of the recipe is in the workspace created by | ||
406 | ``devtool modify`` or ``devtool add``. | ||
407 | Using ``devtool build``, ``devtool build-image``, | ||
408 | ``devtool deploy-target`` or ``bitbake`` is possible. | ||
409 | Also ``devtool ide-sdk`` can be used to update the SDK and the IDE | ||
410 | configuration at any time. | ||
411 | |||
412 | - ``devtool ide-sdk`` aims to support multiple programming languages and | ||
413 | multiple IDEs natively. "Natively" means that the IDE is configured to call | ||
414 | the build tool (e.g. ``cmake`` or ``meson``) directly. This has several | ||
415 | advantages. | ||
416 | First of all, it is usually much faster to call for example ``cmake`` than | ||
417 | ``devtool build``. | ||
418 | It also allows to benefit from the very good integration that IDEs like | ||
419 | VSCode offer for tools like CMake or GDB. | ||
420 | |||
421 | However, supporting many programming languages and multiple | ||
422 | IDEs is quite an elaborate and constantly evolving thing. Support for IDEs | ||
423 | is therefore implemented as plugins. Plugins can also be provided by | ||
424 | optional layers. | ||
425 | |||
426 | So much about the introduction to the default mode of ``devtool sdk-ide`` which | ||
427 | is called the "modified" mode because it uses the workspace created by | ||
428 | ``devtool modify`` and the per recipe :term:`Sysroots <Sysroot>` of BitBake. | ||
429 | |||
430 | For some recipes and use cases, this default behavior of ``devtool ide-sdk`` | ||
431 | with full ``devtool`` and ``bitbake`` integration might not be suitable. | ||
432 | To offer full feature parity with the SDK and the eSDK, ``devtool ide-sdk`` has | ||
433 | a second mode called "shared" mode. | ||
434 | If ``devtool ide-sdk`` is called with the ``--mode=shared`` option, it | ||
435 | bootstraps an SDK directly from the BitBake environment, which offers the same | ||
436 | Environment Setup script as described in :ref:`running_the_ext_sdk_env`. | ||
437 | In addition to the (e)SDK installer-based setup, the IDE gets configured | ||
438 | to use the shared :term:`Sysroots <Sysroot>` and the tools from the SDK. | ||
439 | ``devtool ide-sdk --mode=shared`` is basically a wrapper for the setup of the | ||
440 | extensible SDK as described in :ref:`setting_up_ext_sdk_in_build`. | ||
441 | |||
442 | The use of ``devtool ide-sdk`` is an alternative to using one of the SDK | ||
443 | installers. | ||
444 | ``devtool ide-sdk`` allows the creation of SDKs that offer all the | ||
445 | functionality of the SDK and the eSDK installers. Compared to the installers, | ||
446 | however, the SDK created with ``devtool ide-sdk`` is much more flexible. | ||
447 | For example, it is very easy to change the :term:`MACHINE` in the | ||
448 | ``local.conf`` file, update the layer meta data and then regenerate the SDK. | ||
449 | |||
450 | Let's take a look at an example of how to use ``devtool ide-sdk`` in each of | ||
451 | the two modes: | ||
452 | |||
453 | #. *Modified mode*: | ||
454 | |||
455 | In order to use the ``devtool ide-sdk``, a few settings are needed. As a | ||
456 | starting example, the following lines of code can be added to the | ||
457 | ``local.conf`` file:: | ||
458 | |||
459 | # Build the companion debug file system | ||
460 | IMAGE_GEN_DEBUGFS = "1" | ||
461 | # Optimize build time: with devtool ide-sdk the dbg tar is not needed | ||
462 | IMAGE_FSTYPES_DEBUGFS = "" | ||
463 | # Without copying the binaries into roofs-dbg, GDB does not find all source files. | ||
464 | IMAGE_CLASSES += "image-combined-dbg" | ||
465 | |||
466 | # SSH is mandatory, no password simplifies the usage | ||
467 | EXTRA_IMAGE_FEATURES += "\ | ||
468 | ssh-server-openssh \ | ||
469 | allow-empty-password \ | ||
470 | allow-root-login \ | ||
471 | empty-root-password \ | ||
472 | " | ||
473 | |||
474 | # Remote debugging needs gdbserver on the target device | ||
475 | IMAGE_INSTALL:append = " gdbserver" | ||
476 | |||
477 | # Add the recipes which should be modified to the image | ||
478 | # Otherwise some dependencies might be missing. | ||
479 | IMAGE_INSTALL:append = " my-recipe" | ||
480 | |||
481 | Assuming the BitBake environment is set up correctly and a workspace has | ||
482 | been created for the recipe using ``devtool modify my-recipe`` or probably | ||
483 | even better by using ``devtool modify my-recipe --debug-build``, the | ||
484 | following command can create the SDK and the configuration for VSCode in | ||
485 | the recipe workspace:: | ||
486 | |||
487 | $ devtool ide-sdk my-recipe core-image-minimal --target root@192.168.7.2 | ||
488 | |||
489 | The command requires an image recipe (``core-image-minimal`` for this | ||
490 | example) that is used to create the SDK. | ||
491 | This firmware image should also be installed on the target device. | ||
492 | It is possible to pass multiple package recipes:: | ||
493 | |||
494 | $ devtool ide-sdk my-recipe-1 my-recipe-2 core-image-minimal --target root@192.168.7.2 | ||
495 | |||
496 | ``devtool ide-sdk`` tries to create an IDE configuration for all package | ||
497 | recipes. | ||
498 | |||
499 | What this command does exactly depends on the recipe, more precisely on the | ||
500 | build tool used by the recipe. The basic idea is to configure the IDE so | ||
501 | that it calls the build tool exactly as ``bitbake`` does. | ||
502 | |||
503 | For example, a CMake preset is created for a recipe that inherits | ||
504 | :ref:`ref-classes-cmake`. In the case of VSCode, CMake presets are supported | ||
505 | by the CMake Tools plugin. This is an example of how the build configuration | ||
506 | used by ``bitbake`` is exported to an IDE configuration that gives exactly | ||
507 | the same build results. | ||
508 | |||
509 | Support for remote debugging with seamless integration into the IDE is | ||
510 | important for a cross-SDK. ``devtool ide-sdk`` automatically generates the | ||
511 | necessary helper scripts for deploying the compiled artifacts to the target | ||
512 | device as well as the necessary configuration for the debugger and the IDE. | ||
513 | |||
514 | .. note:: | ||
515 | |||
516 | To ensure that the debug symbols on the build machine match the binaries | ||
517 | running on the target device, it is essential that the image built by | ||
518 | ``devtool ide-sdk`` is running on the target device. | ||
519 | |||
520 | The default IDE is VSCode. Some hints about using VSCode: | ||
521 | |||
522 | - VSCode can be used to work on the BitBake recipes or the application | ||
523 | source code. | ||
524 | Usually there is one instance of VSCode running in the folder where the | ||
525 | BitBake recipes are. This instance has the | ||
526 | `Yocto Project BitBake plugin <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`_ | ||
527 | running. | ||
528 | |||
529 | .. warning:: | ||
530 | |||
531 | Some VSCode plugins (Python, BitBake and others) need a reasonable | ||
532 | configuration to work as expected. Otherwise, some plugins try to | ||
533 | index the build directory of BitBake, which keeps your system quite | ||
534 | busy until an out of memory exception stops this nonsense. | ||
535 | Other plugins, such as the BitBake plugin, do not behave as expected. | ||
536 | |||
537 | To work around such issues, the ``oe-init-build-env`` script creates | ||
538 | an initial ``.vscode/settings.json`` file if ``code`` can be found | ||
539 | and the ``.vscode`` folder does not yet exist. | ||
540 | It is best to run ``oe-init-build-env`` once before starting VSCode. | ||
541 | An alternative approach is to use a build folder outside the layers, | ||
542 | e.g. ``oe-init-build-env ../build``. | ||
543 | |||
544 | The BitBake plugin also offers to create devtool workspaces and run | ||
545 | ``devtool ide-sdk`` with a few mouse clicks. | ||
546 | Of course, issuing commands in the terminal works as well. | ||
547 | |||
548 | - To work on the source code of a recipe another instance of VSCode is | ||
549 | started in the recipe's workspace. Example:: | ||
550 | |||
551 | code build/workspace/sources/my-recipe | ||
552 | |||
553 | This instance of VSCode uses plugins that are useful for the development | ||
554 | of the application. ``devtool ide-sdk`` generates the necessary | ||
555 | ``extensions.json``, ``settings.json``, ``tasks.json`` and ``launch.json`` | ||
556 | configuration files for all the involved plugins. | ||
557 | |||
558 | When the source code folder present in the workspace folder is opened in | ||
559 | VSCode for the first time, a pop-up message recommends installing the | ||
560 | required plugins. | ||
561 | After accepting the installation of the plugins, working with the source | ||
562 | code or some debugging tasks should work as usual with VSCode. | ||
563 | |||
564 | Starting the VSCode instances in the recipe workspace folders can also be | ||
565 | done by a mouse click on the recipe workspaces in the first VSCode | ||
566 | instance. | ||
567 | |||
568 | - To work with CMake press ``Ctrl + Shift + p``, type ``cmake``. This will | ||
569 | show some possible commands like selecting a CMake preset, compiling or | ||
570 | running CTest. | ||
571 | |||
572 | For recipes inheriting :ref:`ref-classes-cmake-qemu` rather than | ||
573 | :ref:`ref-classes-cmake`, executing cross-compiled unit tests on the host | ||
574 | can be supported transparently with QEMU user-mode. | ||
575 | |||
576 | - To work with Meson press ``Ctrl + Shift + p``, type ``meson``. This will | ||
577 | show some possible commands like compiling or executing the unit tests. | ||
578 | |||
579 | A note on running cross-compiled unit tests on the host: Meson enables | ||
580 | support for QEMU user mode by default. It is expected that the execution | ||
581 | of the unit tests from the IDE will work without any additional steps, | ||
582 | given that the code is suitable for the execution on the host machine. | ||
583 | |||
584 | - For the deployment to the target device, just press ``Ctrl + Shift + p``, | ||
585 | type ``task``. Select ``install && deploy-target``. | ||
586 | |||
587 | - For remote debugging, switch to the debugging view by pressing the "play" | ||
588 | button with the ``bug icon`` on the left side. This will provide a green | ||
589 | play button with a drop-down list where a debug configuration can be | ||
590 | selected. After selecting one of the generated configurations, press the | ||
591 | "play" button. | ||
592 | |||
593 | Starting a remote debugging session automatically initiates the | ||
594 | deployment to the target device. If this is not desired, the | ||
595 | ``"dependsOn": ["install && deploy-target...]`` parameter of the tasks | ||
596 | with ``"label": "gdbserver start...`` can be removed from the | ||
597 | ``tasks.json`` file. | ||
598 | |||
599 | VSCode supports GDB with many different setups and configurations for | ||
600 | many different use cases. However, most of these setups have some | ||
601 | limitations when it comes to cross-development, support only a few target | ||
602 | architectures or require a high performance target device. Therefore | ||
603 | ``devtool ide-sdk`` supports the classic, generic setup with GDB on the | ||
604 | development host and gdbserver on the target device. | ||
605 | |||
606 | Roughly summarized, this means: | ||
607 | |||
608 | - The binaries are copied via SSH to the remote target device by a | ||
609 | script referred by ``tasks.json``. | ||
610 | |||
611 | - gdbserver is started on the remote target device via SSH by a script | ||
612 | referred by ``tasks.json``. | ||
613 | |||
614 | Changing the parameters that are passed to the debugging executable | ||
615 | requires modifying the generated script. The script is located at | ||
616 | ``oe-scripts/gdbserver_*``. Defining the parameters in the ``args`` | ||
617 | field in the ``launch.json`` file does not work. | ||
618 | |||
619 | - VSCode connects to gdbserver as documented in | ||
620 | `Remote debugging or debugging with a local debugger server | ||
621 | <https://code.visualstudio.com/docs/cpp/launch-json-reference#_remote-debugging-or-debugging-with-a-local-debugger-server>`__. | ||
622 | |||
623 | Additionally ``--ide=none`` is supported. With the ``none`` IDE parameter, | ||
624 | some generic configuration files like ``gdbinit`` files and some helper | ||
625 | scripts starting gdbserver remotely on the target device as well as the GDB | ||
626 | client on the host are generated. | ||
627 | |||
628 | Here is a usage example for the ``cmake-example`` recipe from the | ||
629 | ``meta-selftest`` layer which inherits :ref:`ref-classes-cmake-qemu`: | ||
630 | |||
631 | .. code-block:: sh | ||
632 | |||
633 | # Create the SDK | ||
634 | devtool modify cmake-example --debug-build | ||
635 | devtool ide-sdk cmake-example core-image-minimal -c --ide=none | ||
636 | |||
637 | # Install the firmware on a target device or start QEMU | ||
638 | runqemu | ||
639 | |||
640 | # From exploring the workspace of cmake-example | ||
641 | cd build/workspace/sources/cmake-example | ||
642 | |||
643 | # Find cmake-native and save the path into a variable | ||
644 | # Note: using just cmake instead of $CMAKE_NATIVE would work in many cases | ||
645 | CMAKE_NATIVE="$(jq -r '.configurePresets[0] | "\(.cmakeExecutable)"' CMakeUserPresets.json)" | ||
646 | |||
647 | # List available CMake presets | ||
648 | "$CMAKE_NATIVE" --list-presets | ||
649 | Available configure presets: | ||
650 | |||
651 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
652 | |||
653 | # Re-compile the already compiled sources | ||
654 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 | ||
655 | ninja: no work to do. | ||
656 | # Do a clean re-build | ||
657 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target clean | ||
658 | [1/1] Cleaning all built files... | ||
659 | Cleaning... 8 files. | ||
660 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target all | ||
661 | [7/7] Linking CXX executable cmake-example | ||
662 | |||
663 | # Run the cross-compiled unit tests with QEMU user-mode | ||
664 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target test | ||
665 | [0/1] Running tests... | ||
666 | Test project .../build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
667 | Start 1: test-cmake-example | ||
668 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
669 | |||
670 | 100% tests passed, 0 tests failed out of 1 | ||
671 | |||
672 | Total Test time (real) = 0.03 sec | ||
673 | |||
674 | # Using CTest directly is possible as well | ||
675 | CTEST_NATIVE="$(dirname "$CMAKE_NATIVE")/ctest" | ||
676 | |||
677 | # List available CMake presets | ||
678 | "$CTEST_NATIVE" --list-presets | ||
679 | Available test presets: | ||
680 | |||
681 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
682 | |||
683 | # Run the cross-compiled unit tests with QEMU user-mode | ||
684 | "$CTEST_NATIVE" --preset "cmake-example-cortexa57" | ||
685 | Test project ...build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
686 | Start 1: test-cmake-example | ||
687 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
688 | |||
689 | 100% tests passed, 0 tests failed out of 1 | ||
690 | |||
691 | Total Test time (real) = 0.03 sec | ||
692 | |||
693 | # Deploying the new build to the target device (default is QEUM at 192.168.7.2) | ||
694 | oe-scripts/install_and_deploy_cmake-example-cortexa57 | ||
695 | |||
696 | # Start a remote debugging session with gdbserver on the target and GDB on the host | ||
697 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m | ||
698 | oe-scripts/gdb_1234_usr-bin-cmake-example | ||
699 | break main | ||
700 | run | ||
701 | step | ||
702 | stepi | ||
703 | continue | ||
704 | quit | ||
705 | |||
706 | # Stop gdbserver on the target device | ||
707 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m stop | ||
708 | |||
709 | #. *Shared sysroots mode* | ||
710 | |||
711 | Creating an SDK with shared :term:`Sysroots <Sysroot>` that contains all the | ||
712 | dependencies needed to work with ``my-recipe`` is possible with the following | ||
713 | example command:: | ||
714 | |||
715 | $ devtool ide-sdk --mode=shared my-recipe | ||
716 | |||
717 | For VSCode the cross-toolchain is exposed as a CMake kit. CMake kits are | ||
718 | defined in ``~/.local/share/CMakeTools/cmake-tools-kits.json``. | ||
719 | The following example shows how the cross-toolchain can be selected in | ||
720 | VSCode. First of all we need a folder containing a CMake project. | ||
721 | For this example, let's create a CMake project and start VSCode:: | ||
722 | |||
723 | mkdir kit-test | ||
724 | echo "project(foo VERSION 1.0)" > kit-test/CMakeLists.txt | ||
725 | code kit-test | ||
726 | |||
727 | If there is a CMake project in the workspace, cross-compilation is | ||
728 | supported: | ||
729 | |||
730 | - Press ``Ctrl + Shift + P``, type ``CMake: Scan for Kits`` | ||
731 | - Press ``Ctrl + Shift + P``, type ``CMake: Select a Kit`` | ||
732 | |||
733 | Finally most of the features provided by CMake and the IDE should be | ||
734 | available. | ||
735 | |||
736 | Other IDEs than VSCode are supported as well. However, | ||
737 | ``devtool ide-sdk --mode=shared --ide=none my-recipe`` is currently | ||
738 | just a simple wrapper for the setup of the extensible SDK, as described in | ||
739 | :ref:`setting_up_ext_sdk_in_build`. | ||
740 | |||
741 | Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software | ||
742 | ======================================================================================================= | ||
743 | |||
744 | The ``devtool upgrade`` command upgrades an existing recipe to that of a | ||
745 | more up-to-date version found upstream. Throughout the life of software, | ||
746 | recipes continually undergo version upgrades by their upstream | ||
747 | publishers. You can use the ``devtool upgrade`` workflow to make sure | ||
748 | your recipes you are using for builds are up-to-date with their upstream | ||
749 | counterparts. | ||
750 | |||
751 | .. note:: | ||
752 | |||
753 | Several methods exist by which you can upgrade recipes --- | ||
754 | ``devtool upgrade`` happens to be one. You can read about all the methods by | ||
755 | which you can upgrade recipes in the | ||
756 | :ref:`dev-manual/upgrading-recipes:upgrading recipes` section of the Yocto | ||
757 | Project Development Tasks Manual. | ||
758 | |||
759 | The ``devtool upgrade`` command is flexible enough to allow you to specify | ||
760 | source code revision and versioning schemes, extract code into or out of the | ||
761 | ``devtool`` :ref:`devtool-the-workspace-layer-structure`, and work with any | ||
762 | source file forms that the | ||
763 | :ref:`bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support. | ||
764 | |||
765 | The following diagram shows the common development flow used with the | ||
766 | ``devtool upgrade`` command: | ||
767 | |||
768 | .. image:: figures/devtool-upgrade-flow.png | ||
769 | :width: 100% | ||
770 | |||
771 | #. *Initiate the Upgrade*: The top part of the flow shows the typical | ||
772 | scenario by which you use the ``devtool upgrade`` command. The | ||
773 | following conditions exist: | ||
774 | |||
775 | - The recipe exists in a local layer external to the ``devtool`` | ||
776 | workspace. | ||
777 | |||
778 | - The source files for the new release exist in the same location | ||
779 | pointed to by :term:`SRC_URI` | ||
780 | in the recipe (e.g. a tarball with the new version number in the | ||
781 | name, or as a different revision in the upstream Git repository). | ||
782 | |||
783 | A common situation is where third-party software has undergone a | ||
784 | revision so that it has been upgraded. The recipe you have access to | ||
785 | is likely in your own layer. Thus, you need to upgrade the recipe to | ||
786 | use the newer version of the software:: | ||
787 | |||
788 | $ devtool upgrade -V version recipe | ||
789 | |||
790 | By default, the ``devtool upgrade`` command extracts source | ||
791 | code into the ``sources`` directory in the | ||
792 | :ref:`devtool-the-workspace-layer-structure`. | ||
793 | If you want the code extracted to any other location, you need to | ||
794 | provide the ``srctree`` positional argument with the command as follows:: | ||
795 | |||
796 | $ devtool upgrade -V version recipe srctree | ||
797 | |||
798 | .. note:: | ||
799 | |||
800 | In this example, the "-V" option specifies the new version. If you | ||
801 | don't use "-V", the command upgrades the recipe to the latest | ||
802 | version. | ||
803 | |||
804 | If the source files pointed to by the :term:`SRC_URI` statement in the | ||
805 | recipe are in a Git repository, you must provide the "-S" option and | ||
806 | specify a revision for the software. | ||
807 | |||
808 | Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable | ||
809 | to locate the source code and any local patch files from other | ||
810 | developers. The result is that the command sets up the source code, | ||
811 | the new version of the recipe, and an append file all within the | ||
812 | workspace. | ||
813 | |||
814 | Additionally, if you have any non-patch local files (i.e. files | ||
815 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
816 | excluding ``*.patch/`` or ``*.diff``), these files are copied to an | ||
817 | ``oe-local-files`` folder under the newly created source tree. | ||
818 | Copying the files here gives you a convenient area from which you can | ||
819 | modify the files. Any changes or additions you make to those files | ||
820 | are incorporated into the build the next time you build the software | ||
821 | just as are other changes you might have made to the source. | ||
822 | |||
823 | #. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen | ||
824 | after upgrading the software to a new version. Conflicts occur | ||
825 | if your recipe specifies some patch files in :term:`SRC_URI` that | ||
826 | conflict with changes made in the new version of the software. For | ||
827 | such cases, you need to resolve the conflicts by editing the source | ||
828 | and following the normal ``git rebase`` conflict resolution process. | ||
829 | |||
830 | Before moving onto the next step, be sure to resolve any such | ||
831 | conflicts created through use of a newer or different version of the | ||
832 | software. | ||
833 | |||
834 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
835 | depends on what you are going to do with the new code. | ||
836 | |||
837 | If you need to eventually move the build output to the target | ||
838 | hardware, use the following ``devtool`` command:: | ||
839 | |||
840 | $ devtool build recipe | ||
841 | |||
842 | On the other hand, if you want an image to contain the recipe's | ||
843 | packages from the workspace for immediate deployment onto a device | ||
844 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
845 | command:: | ||
846 | |||
847 | $ devtool build-image image | ||
848 | |||
849 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
850 | or ``bitbake`` to build your recipe, you probably want to see if the | ||
851 | resulting build output works as expected on target hardware. | ||
852 | |||
853 | .. note:: | ||
854 | |||
855 | This step assumes you have a previously built image that is | ||
856 | already either running in QEMU or running on actual hardware. | ||
857 | Also, it is assumed that for deployment of the image to the | ||
858 | target, SSH is installed in the image and if the image is running | ||
859 | on real hardware that you have network access to and from your | ||
860 | development machine. | ||
861 | |||
862 | You can deploy your build output to that target hardware by using the | ||
863 | ``devtool deploy-target`` command:: | ||
864 | |||
865 | $ devtool deploy-target recipe target | ||
866 | |||
867 | The target is a live target machine running as an SSH server. | ||
868 | |||
869 | You can, of course, also deploy the image you build using the | ||
870 | ``devtool build-image`` command to actual hardware. However, | ||
871 | ``devtool`` does not provide a specific command that allows you to do | ||
872 | this. | ||
873 | |||
874 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
875 | creates any patches corresponding to commits in the local Git | ||
876 | repository, moves the new recipe to a more permanent layer, and then | ||
877 | resets the recipe so that the recipe is built normally rather than | ||
878 | from the workspace. | ||
879 | |||
880 | Any work you did in the ``oe-local-files`` directory is preserved in | ||
881 | the original files next to the recipe during the ``devtool finish`` | ||
882 | command. | ||
883 | |||
884 | If you specify a destination layer that is the same as the original | ||
885 | source, then the old version of the recipe and associated files are | ||
886 | removed prior to adding the new version:: | ||
887 | |||
888 | $ devtool finish recipe layer | ||
889 | |||
890 | .. note:: | ||
891 | |||
892 | Any changes you want to turn into patches must be committed to the | ||
893 | Git repository in the source tree. | ||
894 | |||
895 | As a final process of the ``devtool finish`` command, the state of | ||
896 | the standard layers and the upstream source is restored so that you | ||
897 | can build the recipe from those areas rather than the workspace. | ||
898 | |||
899 | .. note:: | ||
900 | |||
901 | You can use the ``devtool reset`` command to put things back should you | ||
902 | decide you do not want to proceed with your work. If you do use this | ||
903 | command, realize that the source tree is preserved. | ||
904 | |||
905 | A Closer Look at ``devtool add`` | ||
906 | ================================ | ||
907 | |||
908 | The ``devtool add`` command automatically creates a recipe based on the | ||
909 | source tree you provide with the command. Currently, the command has | ||
910 | support for the following: | ||
911 | |||
912 | - Autotools (``autoconf`` and ``automake``) | ||
913 | |||
914 | - CMake | ||
915 | |||
916 | - Scons | ||
917 | |||
918 | - ``qmake`` | ||
919 | |||
920 | - Plain ``Makefile`` | ||
921 | |||
922 | - Out-of-tree kernel module | ||
923 | |||
924 | - Binary package (i.e. "-b" option) | ||
925 | |||
926 | - Node.js module | ||
927 | |||
928 | - Python modules that use ``setuptools`` or ``distutils`` | ||
929 | |||
930 | Apart from binary packages, the determination of how a source tree | ||
931 | should be treated is automatic based on the files present within that | ||
932 | source tree. For example, if a ``CMakeLists.txt`` file is found, then | ||
933 | the source tree is assumed to be using CMake and is treated accordingly. | ||
934 | |||
935 | .. note:: | ||
936 | |||
937 | In most cases, you need to edit the automatically generated recipe in | ||
938 | order to make it build properly. Typically, you would go through | ||
939 | several edit and build cycles until the recipe successfully builds. | ||
940 | Once the recipe builds, you could use possible further iterations to | ||
941 | test the recipe on the target device. | ||
942 | |||
943 | The remainder of this section covers specifics regarding how parts of | ||
944 | the recipe are generated. | ||
945 | |||
946 | Name and Version | ||
947 | ---------------- | ||
948 | |||
949 | If you do not specify a name and version on the command line, | ||
950 | ``devtool add`` uses various metadata within the source tree in an | ||
951 | attempt to determine the name and version of the software being built. | ||
952 | Based on what the tool determines, ``devtool`` sets the name of the | ||
953 | created recipe file accordingly. | ||
954 | |||
955 | If ``devtool`` cannot determine the name and version, the command prints | ||
956 | an error. For such cases, you must re-run the command and provide the | ||
957 | name and version, just the name, or just the version as part of the | ||
958 | command line. | ||
959 | |||
960 | Sometimes the name or version determined from the source tree might be | ||
961 | incorrect. For such a case, you must reset the recipe:: | ||
962 | |||
963 | $ devtool reset -n recipename | ||
964 | |||
965 | After running the ``devtool reset`` command, you need to | ||
966 | run ``devtool add`` again and provide the name or the version. | ||
967 | |||
968 | Dependency Detection and Mapping | ||
969 | -------------------------------- | ||
970 | |||
971 | The ``devtool add`` command attempts to detect build-time dependencies and map | ||
972 | them to other recipes in the system. During this mapping, the command fills in | ||
973 | the names of those recipes as part of the :term:`DEPENDS` variable within the | ||
974 | recipe. If a dependency cannot be mapped, ``devtool`` places a comment | ||
975 | in the recipe indicating such. The inability to map a dependency can | ||
976 | result from naming not being recognized or because the dependency simply | ||
977 | is not available. For cases where the dependency is not available, you | ||
978 | must use the ``devtool add`` command to add an additional recipe that | ||
979 | satisfies the dependency. Once you add that recipe, you need to update | ||
980 | the :term:`DEPENDS` variable in the original recipe to include the new | ||
981 | recipe. | ||
982 | |||
983 | If you need to add runtime dependencies, you can do so by adding the | ||
984 | following to your recipe:: | ||
985 | |||
986 | RDEPENDS:${PN} += "dependency1 dependency2 ..." | ||
987 | |||
988 | .. note:: | ||
989 | |||
990 | The ``devtool add`` command often cannot distinguish between mandatory and | ||
991 | optional dependencies. Consequently, some of the detected dependencies might | ||
992 | in fact be optional. When in doubt, consult the documentation or the | ||
993 | configure script for the software the recipe is building for further | ||
994 | details. In some cases, you might find you can substitute the | ||
995 | dependency with an option that disables the associated functionality | ||
996 | passed to the configure script. | ||
997 | |||
998 | License Detection | ||
999 | ----------------- | ||
1000 | |||
1001 | The ``devtool add`` command attempts to determine if the software you are | ||
1002 | adding is able to be distributed under a common, open-source license. If | ||
1003 | so, the command sets the :term:`LICENSE` value accordingly. | ||
1004 | You should double-check the value added by the command against the | ||
1005 | documentation or source files for the software you are building and, if | ||
1006 | necessary, update that :term:`LICENSE` value. | ||
1007 | |||
1008 | The ``devtool add`` command also sets the :term:`LIC_FILES_CHKSUM` | ||
1009 | value to point to all files that appear to be license-related. Realize | ||
1010 | that license statements often appear in comments at the top of source | ||
1011 | files or within the documentation. In such cases, the command does not | ||
1012 | recognize those license statements. Consequently, you might need to | ||
1013 | amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those | ||
1014 | comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly | ||
1015 | important for third-party software. The mechanism attempts to ensure | ||
1016 | correct licensing should you upgrade the recipe to a newer upstream | ||
1017 | version in future. Any change in licensing is detected and you receive | ||
1018 | an error prompting you to check the license text again. | ||
1019 | |||
1020 | If the ``devtool add`` command cannot determine licensing information, | ||
1021 | ``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the | ||
1022 | :term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue | ||
1023 | with development even though the settings are unlikely to be correct in | ||
1024 | all cases. You should check the documentation or source files for the | ||
1025 | software you are building to determine the actual license. | ||
1026 | |||
1027 | Adding Makefile-Only Software | ||
1028 | ----------------------------- | ||
1029 | |||
1030 | The use of Make by itself is very common in both proprietary and | ||
1031 | open-source software. Unfortunately, Makefiles are often not written | ||
1032 | with cross-compilation in mind. Thus, ``devtool add`` often cannot do | ||
1033 | very much to ensure that these Makefiles build correctly. It is very | ||
1034 | common, for example, to explicitly call ``gcc`` instead of using the | ||
1035 | :term:`CC` variable. Usually, in a | ||
1036 | cross-compilation environment, ``gcc`` is the compiler for the build | ||
1037 | host and the cross-compiler is named something similar to | ||
1038 | ``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to | ||
1039 | point to the associated sysroot for the target machine). | ||
1040 | |||
1041 | When writing a recipe for Makefile-only software, keep the following in | ||
1042 | mind: | ||
1043 | |||
1044 | - You probably need to patch the Makefile to use variables instead of | ||
1045 | hardcoding tools within the toolchain such as ``gcc`` and ``g++``. | ||
1046 | |||
1047 | - The environment in which Make runs is set up with various standard | ||
1048 | variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a | ||
1049 | similar manner to the environment set up by an :ref:`SDK | ||
1050 | <overview-manual/concepts:Application Development SDK>`'s environment | ||
1051 | setup script. One easy way to see these variables is to run the | ||
1052 | ``devtool build`` command on the recipe and then look in | ||
1053 | ``oe-logs/run.do_compile``. Towards the top of this file, there is | ||
1054 | a list of environment variables that are set. You can take | ||
1055 | advantage of these variables within the Makefile. | ||
1056 | |||
1057 | - If the Makefile sets a default for a variable using "=", that default | ||
1058 | overrides the value set in the environment, which is usually not | ||
1059 | desirable. For this case, you can either patch the Makefile so it | ||
1060 | sets the default using the "?=" operator, or you can alternatively | ||
1061 | force the value on the ``make`` command line. To force the value on | ||
1062 | the command line, add the variable setting to | ||
1063 | :term:`EXTRA_OEMAKE` or | ||
1064 | :term:`PACKAGECONFIG_CONFARGS` | ||
1065 | within the recipe. Here is an example using :term:`EXTRA_OEMAKE`:: | ||
1066 | |||
1067 | EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" | ||
1068 | |||
1069 | In the above example, | ||
1070 | single quotes are used around the variable settings as the values are | ||
1071 | likely to contain spaces because required default options are passed | ||
1072 | to the compiler. | ||
1073 | |||
1074 | - Hardcoding paths inside Makefiles is often problematic in a | ||
1075 | cross-compilation environment. This is particularly true because | ||
1076 | those hardcoded paths often point to locations on the build host and | ||
1077 | thus will either be read-only or will introduce contamination into | ||
1078 | the cross-compilation because they are specific to the build host | ||
1079 | rather than the target. Patching the Makefile to use prefix variables | ||
1080 | or other path variables is usually the way to handle this situation. | ||
1081 | |||
1082 | - Sometimes a Makefile runs target-specific commands such as | ||
1083 | ``ldconfig``. For such cases, you might be able to apply patches that | ||
1084 | remove these commands from the Makefile. | ||
1085 | |||
1086 | Adding Native Tools | ||
1087 | ------------------- | ||
1088 | |||
1089 | Often, you need to build additional tools that run on the :term:`Build Host` | ||
1090 | as opposed to the target. You should indicate this requirement by using one of | ||
1091 | the following methods when you run ``devtool add``: | ||
1092 | |||
1093 | - Specify the name of the recipe such that it ends with "-native". | ||
1094 | Specifying the name like this produces a recipe that only builds for | ||
1095 | the build host. | ||
1096 | |||
1097 | - Specify the "--also-native" option with the ``devtool add`` | ||
1098 | command. Specifying this option creates a recipe file that still | ||
1099 | builds for the target but also creates a variant with a "-native" | ||
1100 | suffix that builds for the build host. | ||
1101 | |||
1102 | .. note:: | ||
1103 | |||
1104 | If you need to add a tool that is shipped as part of a source tree | ||
1105 | that builds code for the target, you can typically accomplish this by | ||
1106 | building the native and target parts separately rather than within | ||
1107 | the same compilation process. Realize though that with the | ||
1108 | "--also-native" option, you can add the tool using just one | ||
1109 | recipe file. | ||
1110 | |||
1111 | Adding Node.js Modules | ||
1112 | ---------------------- | ||
1113 | |||
1114 | You can use the ``devtool add`` command two different ways to add | ||
1115 | Node.js modules: through ``npm`` or from a repository or local source. | ||
1116 | |||
1117 | Use the following form to add Node.js modules through ``npm``:: | ||
1118 | |||
1119 | $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" | ||
1120 | |||
1121 | The name and | ||
1122 | version parameters are mandatory. Lockdown and shrinkwrap files are | ||
1123 | generated and pointed to by the recipe in order to freeze the version | ||
1124 | that is fetched for the dependencies according to the first time. This | ||
1125 | also saves checksums that are verified on future fetches. Together, | ||
1126 | these behaviors ensure the reproducibility and integrity of the build. | ||
1127 | |||
1128 | .. note:: | ||
1129 | |||
1130 | - You must use quotes around the URL. ``devtool add`` does not | ||
1131 | require the quotes, but the shell considers ";" as a splitter | ||
1132 | between multiple commands. Thus, without the quotes, | ||
1133 | ``devtool add`` does not receive the other parts, which results in | ||
1134 | several "command not found" errors. | ||
1135 | |||
1136 | As mentioned earlier, you can also add Node.js modules directly from a | ||
1137 | repository or local source tree. To add modules this way, use | ||
1138 | ``devtool add`` in the following form:: | ||
1139 | |||
1140 | $ devtool add https://github.com/diversario/node-ssdp | ||
1141 | |||
1142 | In this example, ``devtool`` fetches the specified Git repository, detects the | ||
1143 | code as Node.js code, fetches dependencies using ``npm``, and sets | ||
1144 | :term:`SRC_URI` accordingly. | ||
1145 | |||
1146 | Working With Recipes | ||
1147 | ==================== | ||
1148 | |||
1149 | When building a recipe using the ``devtool build`` command, the typical | ||
1150 | build progresses as follows: | ||
1151 | |||
1152 | #. Fetch the source | ||
1153 | |||
1154 | #. Unpack the source | ||
1155 | |||
1156 | #. Configure the source | ||
1157 | |||
1158 | #. Compile the source | ||
1159 | |||
1160 | #. Install the build output | ||
1161 | |||
1162 | #. Package the installed output | ||
1163 | |||
1164 | For recipes in the workspace, fetching and unpacking is disabled as the | ||
1165 | source tree has already been prepared and is persistent. Each of these | ||
1166 | build steps is defined as a function (task), usually with a "do\_" prefix | ||
1167 | (e.g. :ref:`ref-tasks-fetch`, | ||
1168 | :ref:`ref-tasks-unpack`, and so | ||
1169 | forth). These functions are typically shell scripts but can instead be | ||
1170 | written in Python. | ||
1171 | |||
1172 | If you look at the contents of a recipe, you will see that the recipe | ||
1173 | does not include complete instructions for building the software. | ||
1174 | Instead, common functionality is encapsulated in classes inherited with | ||
1175 | the ``inherit`` directive. This technique leaves the recipe to describe | ||
1176 | just the things that are specific to the software being built. There is | ||
1177 | a :ref:`ref-classes-base` class that is implicitly inherited by all recipes | ||
1178 | and provides the functionality that most recipes typically need. | ||
1179 | |||
1180 | The remainder of this section presents information useful when working | ||
1181 | with recipes. | ||
1182 | |||
1183 | Finding Logs and Work Files | ||
1184 | --------------------------- | ||
1185 | |||
1186 | After the first run of the ``devtool build`` command, recipes that were | ||
1187 | previously created using the ``devtool add`` command or whose sources | ||
1188 | were modified using the ``devtool modify`` command contain symbolic | ||
1189 | links created within the source tree: | ||
1190 | |||
1191 | - ``oe-logs``: This link points to the directory in which log files and | ||
1192 | run scripts for each build step are created. | ||
1193 | |||
1194 | - ``oe-workdir``: This link points to the temporary work area for the | ||
1195 | recipe. The following locations under ``oe-workdir`` are particularly | ||
1196 | useful: | ||
1197 | |||
1198 | - ``image/``: Contains all of the files installed during the | ||
1199 | :ref:`ref-tasks-install` stage. | ||
1200 | Within a recipe, this directory is referred to by the expression | ||
1201 | ``${``\ :term:`D`\ ``}``. | ||
1202 | |||
1203 | - ``sysroot-destdir/``: Contains a subset of files installed within | ||
1204 | :ref:`ref-tasks-install` that have been put into the shared sysroot. For | ||
1205 | more information, see the | ||
1206 | ":ref:`dev-manual/new-recipe:sharing files between recipes`" section. | ||
1207 | |||
1208 | - ``packages-split/``: Contains subdirectories for each package | ||
1209 | produced by the recipe. For more information, see the | ||
1210 | ":ref:`dev-manual/devtool:packaging`" section. | ||
1211 | |||
1212 | You can use these links to get more information on what is happening at | ||
1213 | each build step. | ||
1214 | |||
1215 | Setting Configure Arguments | ||
1216 | --------------------------- | ||
1217 | |||
1218 | If the software your recipe is building uses GNU autoconf, then a fixed | ||
1219 | set of arguments is passed to it to enable cross-compilation plus any | ||
1220 | extras specified by :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS` | ||
1221 | set within the recipe. If you wish to pass additional options, add them | ||
1222 | to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build | ||
1223 | tools have similar variables (e.g. :term:`EXTRA_OECMAKE` for CMake, | ||
1224 | :term:`EXTRA_OESCONS` for Scons, and so forth). If you need to pass anything on | ||
1225 | the ``make`` command line, you can use :term:`EXTRA_OEMAKE` or the | ||
1226 | :term:`PACKAGECONFIG_CONFARGS` variables to do so. | ||
1227 | |||
1228 | You can use the ``devtool configure-help`` command to help you set the | ||
1229 | arguments listed in the previous paragraph. The command determines the | ||
1230 | exact options being passed, and shows them to you along with any custom | ||
1231 | arguments specified through :term:`EXTRA_OECONF` or | ||
1232 | :term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you | ||
1233 | the output of the configure script's "--help" option as a | ||
1234 | reference. | ||
1235 | |||
1236 | Sharing Files Between Recipes | ||
1237 | ----------------------------- | ||
1238 | |||
1239 | Recipes often need to use files provided by other recipes on the | ||
1240 | :term:`Build Host`. For example, | ||
1241 | an application linking to a common library needs access to the library | ||
1242 | itself and its associated headers. The way this access is accomplished | ||
1243 | is through the :term:`Sysroot`. There is a sysroot for the target machine, and a | ||
1244 | sysroot for the build host. | ||
1245 | |||
1246 | Recipes should never write files directly into the sysroot. Instead, | ||
1247 | files should be installed into standard locations during the | ||
1248 | :ref:`ref-tasks-install` task within the ``${``\ :term:`D`\ ``}`` directory. A | ||
1249 | subset of these files automatically goes into the sysroot. The reason | ||
1250 | for this limitation is that almost all files that go into the sysroot | ||
1251 | are cataloged in manifests in order to ensure they can be removed later | ||
1252 | when a recipe is modified or removed. Thus, the sysroot is able to | ||
1253 | remain free from stale files. | ||
1254 | |||
1255 | Packaging | ||
1256 | --------- | ||
1257 | |||
1258 | If you examine how build output gets into the final image on | ||
1259 | the target device, it is important to understand packaging because the | ||
1260 | contents of the image are expressed in terms of packages and not | ||
1261 | recipes. | ||
1262 | |||
1263 | During the :ref:`ref-tasks-package` task, files installed during the | ||
1264 | :ref:`ref-tasks-install` task are split into one main package, which is almost | ||
1265 | always named the same as the recipe, and into several other packages. This | ||
1266 | separation exists because not all of those installed files are useful in every | ||
1267 | image. For example, you probably do not need any of the documentation installed | ||
1268 | in a production image. Consequently, for each recipe the documentation | ||
1269 | files are separated into a ``-doc`` package. Recipes that package | ||
1270 | software containing optional modules or plugins might undergo additional | ||
1271 | package splitting as well. | ||
1272 | |||
1273 | After building a recipe, you can see where files have gone by looking in | ||
1274 | the ``oe-workdir/packages-split`` directory, which contains a | ||
1275 | subdirectory for each package. Apart from some advanced cases, the | ||
1276 | :term:`PACKAGES` and :term:`FILES` variables controls | ||
1277 | splitting. The :term:`PACKAGES` variable lists all of the packages to be | ||
1278 | produced, while the :term:`FILES` variable specifies which files to include | ||
1279 | in each package by using an override to specify the package. For | ||
1280 | example, ``FILES:${PN}`` specifies the files to go into the main package | ||
1281 | (i.e. the main package has the same name as the recipe and | ||
1282 | ``${``\ :term:`PN`\ ``}`` evaluates to the | ||
1283 | recipe name). The order of the :term:`PACKAGES` value is significant. For | ||
1284 | each installed file, the first package whose :term:`FILES` value matches the | ||
1285 | file is the package into which the file goes. Both the :term:`PACKAGES` and | ||
1286 | :term:`FILES` variables have default values. Consequently, you might find | ||
1287 | you do not even need to set these variables in your recipe unless the | ||
1288 | software the recipe is building installs files into non-standard | ||
1289 | locations. | ||
1290 | |||
1291 | Restoring the Target Device to its Original State | ||
1292 | ================================================= | ||
1293 | |||
1294 | If you use the ``devtool deploy-target`` command to write a recipe's | ||
1295 | build output to the target, and you are working on an existing component | ||
1296 | of the system, then you might find yourself in a situation where you | ||
1297 | need to restore the original files that existed prior to running the | ||
1298 | ``devtool deploy-target`` command. Because the ``devtool deploy-target`` | ||
1299 | command backs up any files it overwrites, you can use the | ||
1300 | ``devtool undeploy-target`` command to restore those files and remove | ||
1301 | any other files the recipe deployed. Consider the following example:: | ||
1302 | |||
1303 | $ devtool undeploy-target lighttpd root@192.168.7.2 | ||
1304 | |||
1305 | If you have deployed | ||
1306 | multiple applications, you can remove them all using the "-a" option | ||
1307 | thus restoring the target device to its original state:: | ||
1308 | |||
1309 | $ devtool undeploy-target -a root@192.168.7.2 | ||
1310 | |||
1311 | Information about files deployed to | ||
1312 | the target as well as any backed up files are stored on the target | ||
1313 | itself. This storage, of course, requires some additional space on the | ||
1314 | target machine. | ||
1315 | |||
1316 | .. note:: | ||
1317 | |||
1318 | The ``devtool deploy-target`` and ``devtool undeploy-target`` commands do | ||
1319 | not currently interact with any package management system on the target | ||
1320 | device (e.g. RPM or OPKG). Consequently, you should not intermingle | ||
1321 | ``devtool deploy-target`` and package manager operations on the target | ||
1322 | device. Doing so could result in a conflicting set of files. | ||
diff --git a/documentation/dev-manual/disk-space.rst b/documentation/dev-manual/disk-space.rst index 6d1638a302..ba3afa5a2c 100644 --- a/documentation/dev-manual/disk-space.rst +++ b/documentation/dev-manual/disk-space.rst | |||
@@ -49,10 +49,10 @@ requires a full build environment to be available and doesn't work well | |||
49 | covering multiple releases. It won't work either on limited environments | 49 | covering multiple releases. It won't work either on limited environments |
50 | such as BSD based NAS:: | 50 | such as BSD based NAS:: |
51 | 51 | ||
52 | sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache | 52 | sstate-cache-management.py --remove-duplicated --cache-dir=sstate-cache |
53 | 53 | ||
54 | This command will ask you to confirm the deletions it identifies. | 54 | This command will ask you to confirm the deletions it identifies. |
55 | Run ``sstate-cache-management.sh`` for more details about this script. | 55 | Run ``sstate-cache-management.py`` for more details about this script. |
56 | 56 | ||
57 | .. note:: | 57 | .. note:: |
58 | 58 | ||
diff --git a/documentation/dev-manual/efficiently-fetching-sources.rst b/documentation/dev-manual/efficiently-fetching-sources.rst index a15f0a92ce..a3366226c0 100644 --- a/documentation/dev-manual/efficiently-fetching-sources.rst +++ b/documentation/dev-manual/efficiently-fetching-sources.rst | |||
@@ -6,7 +6,7 @@ Efficiently Fetching Source Files During a Build | |||
6 | The OpenEmbedded build system works with source files located through | 6 | The OpenEmbedded build system works with source files located through |
7 | the :term:`SRC_URI` variable. When | 7 | the :term:`SRC_URI` variable. When |
8 | you build something using BitBake, a big part of the operation is | 8 | you build something using BitBake, a big part of the operation is |
9 | locating and downloading all the source tarballs. For images, | 9 | locating and downloading all of the source code. For images, |
10 | downloading all the source for various packages can take a significant | 10 | downloading all the source for various packages can take a significant |
11 | amount of time. | 11 | amount of time. |
12 | 12 | ||
@@ -18,7 +18,7 @@ Setting up Effective Mirrors | |||
18 | ============================ | 18 | ============================ |
19 | 19 | ||
20 | A good deal that goes into a Yocto Project build is simply downloading | 20 | A good deal that goes into a Yocto Project build is simply downloading |
21 | all of the source tarballs. Maybe you have been working with another | 21 | source code. Maybe you have been working with another |
22 | build system for which you have built up a | 22 | build system for which you have built up a |
23 | sizable directory of source tarballs. Or, perhaps someone else has such | 23 | sizable directory of source tarballs. Or, perhaps someone else has such |
24 | a directory for which you have read access. If so, you can save time by | 24 | a directory for which you have read access. If so, you can save time by |
diff --git a/documentation/dev-manual/external-scm.rst b/documentation/dev-manual/external-scm.rst index 97a7e63e36..896b1b5ac7 100644 --- a/documentation/dev-manual/external-scm.rst +++ b/documentation/dev-manual/external-scm.rst | |||
@@ -12,10 +12,13 @@ revision number for changes. Currently, you can do this with Apache | |||
12 | Subversion (SVN), Git, and Bazaar (BZR) repositories. | 12 | Subversion (SVN), Git, and Bazaar (BZR) repositories. |
13 | 13 | ||
14 | To enable this behavior, the :term:`PV` of | 14 | To enable this behavior, the :term:`PV` of |
15 | the recipe needs to reference | 15 | the recipe needs to include a ``+`` sign in its assignment. |
16 | :term:`SRCPV`. Here is an example:: | 16 | Here is an example:: |
17 | 17 | ||
18 | PV = "1.2.3+git${SRCPV}" | 18 | PV = "1.2.3+git" |
19 | |||
20 | :term:`Bitbake` later includes the source control information in :term:`PKGV` | ||
21 | during the packaging phase. | ||
19 | 22 | ||
20 | Then, you can add the following to your | 23 | Then, you can add the following to your |
21 | ``local.conf``:: | 24 | ``local.conf``:: |
diff --git a/documentation/dev-manual/figures/devtool-add-flow.png b/documentation/dev-manual/figures/devtool-add-flow.png new file mode 100644 index 0000000000..e7d6173d2d --- /dev/null +++ b/documentation/dev-manual/figures/devtool-add-flow.png | |||
Binary files differ | |||
diff --git a/documentation/dev-manual/figures/devtool-modify-flow.png b/documentation/dev-manual/figures/devtool-modify-flow.png new file mode 100644 index 0000000000..18ba8b7e65 --- /dev/null +++ b/documentation/dev-manual/figures/devtool-modify-flow.png | |||
Binary files differ | |||
diff --git a/documentation/dev-manual/figures/devtool-upgrade-flow.png b/documentation/dev-manual/figures/devtool-upgrade-flow.png new file mode 100644 index 0000000000..7d4f395e24 --- /dev/null +++ b/documentation/dev-manual/figures/devtool-upgrade-flow.png | |||
Binary files differ | |||
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst index 9ccf60f701..63736e0abf 100644 --- a/documentation/dev-manual/index.rst +++ b/documentation/dev-manual/index.rst | |||
@@ -20,9 +20,11 @@ Yocto Project Development Tasks Manual | |||
20 | development-shell | 20 | development-shell |
21 | python-development-shell | 21 | python-development-shell |
22 | building | 22 | building |
23 | multiconfig | ||
23 | speeding-up-build | 24 | speeding-up-build |
24 | libraries | 25 | libraries |
25 | prebuilt-libraries | 26 | prebuilt-libraries |
27 | devtool | ||
26 | x32-psabi | 28 | x32-psabi |
27 | gobject-introspection | 29 | gobject-introspection |
28 | external-toolchain | 30 | external-toolchain |
@@ -39,7 +41,6 @@ Yocto Project Development Tasks Manual | |||
39 | external-scm | 41 | external-scm |
40 | read-only-rootfs | 42 | read-only-rootfs |
41 | build-quality | 43 | build-quality |
42 | runtime-testing | ||
43 | debugging | 44 | debugging |
44 | licenses | 45 | licenses |
45 | security-subjects | 46 | security-subjects |
@@ -48,5 +49,6 @@ Yocto Project Development Tasks Manual | |||
48 | error-reporting-tool | 49 | error-reporting-tool |
49 | wayland | 50 | wayland |
50 | qemu | 51 | qemu |
52 | bblock | ||
51 | 53 | ||
52 | .. include:: /boilerplate.rst | 54 | .. include:: /boilerplate.rst |
diff --git a/documentation/dev-manual/init-manager.rst b/documentation/dev-manual/init-manager.rst index 20d61ea830..d0fdfdf773 100644 --- a/documentation/dev-manual/init-manager.rst +++ b/documentation/dev-manual/init-manager.rst | |||
@@ -44,7 +44,7 @@ therefore increasing the total system boot time. systemd also substantially | |||
44 | increases system size because of its multiple components and the extra | 44 | increases system size because of its multiple components and the extra |
45 | dependencies it pulls. | 45 | dependencies it pulls. |
46 | 46 | ||
47 | On the contrary, BusyBox init is the simplest and the lightest solution and | 47 | By contrast, BusyBox init is the simplest and the lightest solution and |
48 | also comes with BusyBox mdev as device manager, a lighter replacement to | 48 | also comes with BusyBox mdev as device manager, a lighter replacement to |
49 | :wikipedia:`udev <Udev>`, which SysVinit and systemd both use. | 49 | :wikipedia:`udev <Udev>`, which SysVinit and systemd both use. |
50 | 50 | ||
@@ -121,7 +121,7 @@ increasing levels of complexity and functionality: | |||
121 | :widths: 40 20 20 20 | 121 | :widths: 40 20 20 20 |
122 | :header-rows: 1 | 122 | :header-rows: 1 |
123 | 123 | ||
124 | * - | 124 | * - |
125 | - BusyBox init | 125 | - BusyBox init |
126 | - SysVinit | 126 | - SysVinit |
127 | - systemd | 127 | - systemd |
diff --git a/documentation/dev-manual/layers.rst b/documentation/dev-manual/layers.rst index f7929e630e..67482bf544 100644 --- a/documentation/dev-manual/layers.rst +++ b/documentation/dev-manual/layers.rst | |||
@@ -80,7 +80,7 @@ Follow these general steps to create your layer without using tools: | |||
80 | BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" | 80 | BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" |
81 | BBFILE_PRIORITY_yoctobsp = "5" | 81 | BBFILE_PRIORITY_yoctobsp = "5" |
82 | LAYERVERSION_yoctobsp = "4" | 82 | LAYERVERSION_yoctobsp = "4" |
83 | LAYERSERIES_COMPAT_yoctobsp = "dunfell" | 83 | LAYERSERIES_COMPAT_yoctobsp = "walnascar" |
84 | 84 | ||
85 | Here is an explanation of the layer configuration file: | 85 | Here is an explanation of the layer configuration file: |
86 | 86 | ||
@@ -306,7 +306,7 @@ The Yocto Project Compatibility Program consists of a layer application | |||
306 | process that requests permission to use the Yocto Project Compatibility | 306 | process that requests permission to use the Yocto Project Compatibility |
307 | Logo for your layer and application. The process consists of two parts: | 307 | Logo for your layer and application. The process consists of two parts: |
308 | 308 | ||
309 | #. Successfully passing a script (``yocto-check-layer``) that when run | 309 | #. Successfully passing a script (``yocto-check-layer``) that, when run |
310 | against your layer, tests it against constraints based on experiences | 310 | against your layer, tests it against constraints based on experiences |
311 | of how layers have worked in the real world and where pitfalls have | 311 | of how layers have worked in the real world and where pitfalls have |
312 | been found. Getting a "PASS" result from the script is required for | 312 | been found. Getting a "PASS" result from the script is required for |
@@ -470,11 +470,23 @@ corresponding recipe file. For example, the append file | |||
470 | means the original recipe and append filenames are version | 470 | means the original recipe and append filenames are version |
471 | number-specific. If the corresponding recipe is renamed to update to a | 471 | number-specific. If the corresponding recipe is renamed to update to a |
472 | newer version, you must also rename and possibly update the | 472 | newer version, you must also rename and possibly update the |
473 | corresponding ``.bbappend`` as well. During the build process, BitBake | 473 | corresponding ``.bbappend`` as well. |
474 | displays an error on starting if it detects a ``.bbappend`` file that | 474 | |
475 | does not have a corresponding recipe with a matching name. See the | 475 | During the build process, BitBake displays an error on startup if it detects a |
476 | :term:`BB_DANGLINGAPPENDS_WARNONLY` | 476 | ``.bbappend`` file that does not have a corresponding recipe with a matching |
477 | variable for information on how to handle this error. | 477 | name. To handle these errors, the best practice is to rename the ``.bbappend`` |
478 | to match the original recipe version. This also gives you the opportunity to see | ||
479 | if the ``.bbappend`` is still relevant for the new version of the recipe. | ||
480 | |||
481 | Another method is to use the character ``%`` in the ``.bbappend`` filename. For | ||
482 | example, to append information to every ``6.*`` minor versions of the recipe | ||
483 | ``someapp``, the ``someapp_6.%.bbappend`` file can be created. This way, an | ||
484 | error will only be triggered if the ``someapp`` recipe has a major version | ||
485 | update. | ||
486 | |||
487 | Finally, another method to deal with these errors is to use the variable | ||
488 | :term:`BBMASK`, especially in cases where modifying the ``.bbappend`` is not | ||
489 | possible. | ||
478 | 490 | ||
479 | Overlaying a File Using Your Layer | 491 | Overlaying a File Using Your Layer |
480 | ---------------------------------- | 492 | ---------------------------------- |
@@ -492,21 +504,20 @@ the "meta" layer at ``meta/recipes-bsp/formfactor``:: | |||
492 | SECTION = "base" | 504 | SECTION = "base" |
493 | LICENSE = "MIT" | 505 | LICENSE = "MIT" |
494 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" | 506 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" |
495 | PR = "r45" | ||
496 | 507 | ||
497 | SRC_URI = "file://config file://machconfig" | 508 | SRC_URI = "file://config file://machconfig" |
498 | S = "${WORKDIR}" | 509 | S = "${UNPACKDIR}" |
499 | 510 | ||
500 | PACKAGE_ARCH = "${MACHINE_ARCH}" | 511 | PACKAGE_ARCH = "${MACHINE_ARCH}" |
501 | INHIBIT_DEFAULT_DEPS = "1" | 512 | INHIBIT_DEFAULT_DEPS = "1" |
502 | 513 | ||
503 | do_install() { | 514 | do_install() { |
504 | # Install file only if it has contents | 515 | # Install file only if it has contents |
505 | install -d ${D}${sysconfdir}/formfactor/ | 516 | install -d ${D}${sysconfdir}/formfactor/ |
506 | install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ | 517 | install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ |
507 | if [ -s "${S}/machconfig" ]; then | 518 | if [ -s "${S}/machconfig" ]; then |
508 | install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ | 519 | install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ |
509 | fi | 520 | fi |
510 | } | 521 | } |
511 | 522 | ||
512 | In the main recipe, note the :term:`SRC_URI` | 523 | In the main recipe, note the :term:`SRC_URI` |
@@ -570,11 +581,10 @@ Directory`. Here is the main ``xserver-xf86-config`` recipe, which is named | |||
570 | SECTION = "x11/base" | 581 | SECTION = "x11/base" |
571 | LICENSE = "MIT" | 582 | LICENSE = "MIT" |
572 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" | 583 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" |
573 | PR = "r33" | ||
574 | 584 | ||
575 | SRC_URI = "file://xorg.conf" | 585 | SRC_URI = "file://xorg.conf" |
576 | 586 | ||
577 | S = "${WORKDIR}" | 587 | S = "${UNPACKDIR}" |
578 | 588 | ||
579 | CONFFILES:${PN} = "${sysconfdir}/X11/xorg.conf" | 589 | CONFFILES:${PN} = "${sysconfdir}/X11/xorg.conf" |
580 | 590 | ||
@@ -582,10 +592,10 @@ Directory`. Here is the main ``xserver-xf86-config`` recipe, which is named | |||
582 | ALLOW_EMPTY:${PN} = "1" | 592 | ALLOW_EMPTY:${PN} = "1" |
583 | 593 | ||
584 | do_install () { | 594 | do_install () { |
585 | if test -s ${WORKDIR}/xorg.conf; then | 595 | if test -s ${UNPACKDIR}/xorg.conf; then |
586 | install -d ${D}/${sysconfdir}/X11 | 596 | install -d ${D}/${sysconfdir}/X11 |
587 | install -m 0644 ${WORKDIR}/xorg.conf ${D}/${sysconfdir}/X11/ | 597 | install -m 0644 ${UNPACKDIR}/xorg.conf ${D}/${sysconfdir}/X11/ |
588 | fi | 598 | fi |
589 | } | 599 | } |
590 | 600 | ||
591 | Here is the append file, which is named ``xserver-xf86-config_%.bbappend`` | 601 | Here is the append file, which is named ``xserver-xf86-config_%.bbappend`` |
@@ -602,8 +612,8 @@ file is in the layer at ``recipes-graphics/xorg-xserver``:: | |||
602 | PITFT="${@bb.utils.contains("MACHINE_FEATURES", "pitft", "1", "0", d)}" | 612 | PITFT="${@bb.utils.contains("MACHINE_FEATURES", "pitft", "1", "0", d)}" |
603 | if [ "${PITFT}" = "1" ]; then | 613 | if [ "${PITFT}" = "1" ]; then |
604 | install -d ${D}/${sysconfdir}/X11/xorg.conf.d/ | 614 | install -d ${D}/${sysconfdir}/X11/xorg.conf.d/ |
605 | install -m 0644 ${WORKDIR}/xorg.conf.d/98-pitft.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ | 615 | install -m 0644 ${UNPACKDIR}/xorg.conf.d/98-pitft.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ |
606 | install -m 0644 ${WORKDIR}/xorg.conf.d/99-calibration.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ | 616 | install -m 0644 ${UNPACKDIR}/xorg.conf.d/99-calibration.conf ${D}/${sysconfdir}/X11/xorg.conf.d/ |
607 | fi | 617 | fi |
608 | } | 618 | } |
609 | 619 | ||
@@ -644,6 +654,96 @@ variable and append the layer's root name:: | |||
644 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake | 654 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake |
645 | might address this. | 655 | might address this. |
646 | 656 | ||
657 | Providing Global-level Configurations With Your Layer | ||
658 | ----------------------------------------------------- | ||
659 | |||
660 | When creating a layer, you may need to define configurations that should take | ||
661 | effect globally in your build environment when the layer is part of the build. | ||
662 | The ``layer.conf`` file is a :term:`configuration file` that affects the build | ||
663 | system globally, so it is a candidate for this use-case. | ||
664 | |||
665 | .. warning:: | ||
666 | |||
667 | Providing unconditional global level configuration from the ``layer.conf`` | ||
668 | file is *not* a good practice, and should be avoided. For this reason, the | ||
669 | section :ref:`ref-conditional-layer-confs` below shows how the ``layer.conf`` | ||
670 | file can be used to provide configurations only if a certain condition is | ||
671 | met. | ||
672 | |||
673 | For example, if your layer provides a Linux kernel recipe named | ||
674 | ``linux-custom``, you may want to make :term:`PREFERRED_PROVIDER_virtual/kernel | ||
675 | <PREFERRED_PROVIDER>` point to ``linux-custom``:: | ||
676 | |||
677 | PREFERRED_PROVIDER_virtual/kernel = "linux-custom" | ||
678 | |||
679 | This can be defined in the ``layer.conf`` file. If your layer is at the last | ||
680 | position in the :term:`BBLAYERS` list, it will take precedence over previous | ||
681 | ``PREFERRED_PROVIDER_virtual/kernel`` assignments (unless one is set from a | ||
682 | :term:`configuration file` that is parsed later, such as machine or distro | ||
683 | configuration files). | ||
684 | |||
685 | .. _ref-conditional-layer-confs: | ||
686 | |||
687 | Conditionally Provide Global-level Configurations With Your Layer | ||
688 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
689 | |||
690 | In some cases, your layer may provide global configurations only if some | ||
691 | features it provides are enabled. Since the ``layer.conf`` file is parsed at an | ||
692 | earlier stage in the parsing process, the :term:`DISTRO_FEATURES` and | ||
693 | :term:`MACHINE_FEATURES` variables are not yet available to ``layer.conf``, and | ||
694 | declaring conditional assignments based on these variables is not possible. The | ||
695 | following technique shows a way to bypass this limitation by using the | ||
696 | :term:`USER_CLASSES` variable and a conditional ``require`` command. | ||
697 | |||
698 | In the following steps, let's assume our layer is named ``meta-mylayer`` and | ||
699 | that this layer defines a custom :ref:`distro feature <ref-features-distro>` | ||
700 | named ``mylayer-kernel``. We will set the :term:`PREFERRED_PROVIDER` variable | ||
701 | for the kernel only if our feature ``mylayer-kernel`` is part of the | ||
702 | :term:`DISTRO_FEATURES`: | ||
703 | |||
704 | #. Create an include file in the directory | ||
705 | ``meta-mylayer/conf/distro/include/``, for example a file named | ||
706 | ``mylayer-kernel-provider.inc`` that sets the kernel provider to | ||
707 | ``linux-custom``:: | ||
708 | |||
709 | PREFERRED_PROVIDER_virtual/kernel = "linux-custom" | ||
710 | |||
711 | #. Provide a path to this include file in your ``layer.conf``:: | ||
712 | |||
713 | META_MYLAYER_KERNEL_PROVIDER_PATH = "${LAYERDIR}/conf/distro/include/mylayer-kernel-provider.inc" | ||
714 | |||
715 | #. Create a new class in ``meta-mylayer/classes-global/``, for example a class | ||
716 | ``meta-mylayer-cfg.bbclass``. Make it conditionally require the file | ||
717 | ``mylayer-kernel-provider.inc`` defined above, using the variable | ||
718 | ``META_MYLAYER_KERNEL_PROVIDER_PATH`` defined in ``layer.conf``:: | ||
719 | |||
720 | require ${@bb.utils.contains('DISTRO_FEATURES', 'mylayer-kernel', '${META_MYLAYER_KERNEL_PROVIDER_PATH}', '', d)} | ||
721 | |||
722 | For details on the ``bb.utils.contains`` function, see its definition in | ||
723 | :bitbake_git:`lib/bb/utils.py </tree/lib/bb/utils.py>`. | ||
724 | |||
725 | .. note:: | ||
726 | |||
727 | The ``require`` command is designed to not fail if the function | ||
728 | ``bb.utils.contains`` returns an empty string. | ||
729 | |||
730 | #. Back to your ``layer.conf`` file, add the class ``meta-mylayer-cfg`` class to | ||
731 | the :term:`USER_CLASSES` variable:: | ||
732 | |||
733 | USER_CLASSES:append = " meta-mylayer-cfg" | ||
734 | |||
735 | This will add the class ``meta-mylayer-cfg`` to the list of classes to | ||
736 | globally inherit. Since the ``require`` command is conditional in | ||
737 | ``meta-mylayer-cfg.bbclass``, even though inherited the class will have no | ||
738 | effect unless the feature ``mylayer-kernel`` is enabled through | ||
739 | :term:`DISTRO_FEATURES`. | ||
740 | |||
741 | This technique can also be used for :ref:`Machine features | ||
742 | <ref-features-machine>` by following the same steps. Though not mandatory, it is | ||
743 | recommended to put include files for :term:`DISTRO_FEATURES` in your layer's | ||
744 | ``conf/distro/include`` and the ones for :term:`MACHINE_FEATURES` in your | ||
745 | layer's ``conf/machine/include``. | ||
746 | |||
647 | Managing Layers | 747 | Managing Layers |
648 | =============== | 748 | =============== |
649 | 749 | ||
@@ -732,7 +832,7 @@ The following list describes the available commands: | |||
732 | 832 | ||
733 | - ``save-build-conf``: Saves the currently active build configuration | 833 | - ``save-build-conf``: Saves the currently active build configuration |
734 | (``conf/local.conf``, ``conf/bblayers.conf``) as a template into a layer. | 834 | (``conf/local.conf``, ``conf/bblayers.conf``) as a template into a layer. |
735 | This template can later be used for setting up builds via :term:``TEMPLATECONF``. | 835 | This template can later be used for setting up builds via :term:`TEMPLATECONF`. |
736 | For information about saving and using configuration templates, see | 836 | For information about saving and using configuration templates, see |
737 | ":ref:`dev-manual/custom-template-configuration-directory:creating a custom template configuration directory`". | 837 | ":ref:`dev-manual/custom-template-configuration-directory:creating a custom template configuration directory`". |
738 | 838 | ||
diff --git a/documentation/dev-manual/libraries.rst b/documentation/dev-manual/libraries.rst index 521dbb9a7c..a8c38128ea 100644 --- a/documentation/dev-manual/libraries.rst +++ b/documentation/dev-manual/libraries.rst | |||
@@ -37,40 +37,10 @@ library files. | |||
37 | Some previously released versions of the Yocto Project defined the | 37 | Some previously released versions of the Yocto Project defined the |
38 | static library files through ``${PN}-dev``. | 38 | static library files through ``${PN}-dev``. |
39 | 39 | ||
40 | Here is the part of the BitBake configuration file, where you can see | 40 | Here is a small part of the BitBake configuration file, where you can see |
41 | how the static library files are defined:: | 41 | how the static library files are defined:: |
42 | 42 | ||
43 | PACKAGE_BEFORE_PN ?= "" | ||
44 | PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" | 43 | PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" |
45 | PACKAGES_DYNAMIC = "^${PN}-locale-.*" | ||
46 | FILES = "" | ||
47 | |||
48 | FILES:${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \ | ||
49 | ${sysconfdir} ${sharedstatedir} ${localstatedir} \ | ||
50 | ${base_bindir}/* ${base_sbindir}/* \ | ||
51 | ${base_libdir}/*${SOLIBS} \ | ||
52 | ${base_prefix}/lib/udev ${prefix}/lib/udev \ | ||
53 | ${base_libdir}/udev ${libdir}/udev \ | ||
54 | ${datadir}/${BPN} ${libdir}/${BPN}/* \ | ||
55 | ${datadir}/pixmaps ${datadir}/applications \ | ||
56 | ${datadir}/idl ${datadir}/omf ${datadir}/sounds \ | ||
57 | ${libdir}/bonobo/servers" | ||
58 | |||
59 | FILES:${PN}-bin = "${bindir}/* ${sbindir}/*" | ||
60 | |||
61 | FILES:${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \ | ||
62 | ${datadir}/gnome/help" | ||
63 | SECTION:${PN}-doc = "doc" | ||
64 | |||
65 | FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}" | ||
66 | FILES:${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \ | ||
67 | ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \ | ||
68 | ${datadir}/aclocal ${base_libdir}/*.o \ | ||
69 | ${libdir}/${BPN}/*.la ${base_libdir}/*.la \ | ||
70 | ${libdir}/cmake ${datadir}/cmake" | ||
71 | SECTION:${PN}-dev = "devel" | ||
72 | ALLOW_EMPTY:${PN}-dev = "1" | ||
73 | RDEPENDS:${PN}-dev = "${PN} (= ${EXTENDPKGV})" | ||
74 | 44 | ||
75 | FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" | 45 | FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" |
76 | SECTION:${PN}-staticdev = "devel" | 46 | SECTION:${PN}-staticdev = "devel" |
diff --git a/documentation/dev-manual/licenses.rst b/documentation/dev-manual/licenses.rst index bffff3675f..7d6636eeff 100644 --- a/documentation/dev-manual/licenses.rst +++ b/documentation/dev-manual/licenses.rst | |||
@@ -55,11 +55,11 @@ Consider this next example:: | |||
55 | 55 | ||
56 | LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ | 56 | LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ |
57 | md5=bb14ed3c4cda583abc85401304b5cd4e" | 57 | md5=bb14ed3c4cda583abc85401304b5cd4e" |
58 | LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" | 58 | LIC_FILES_CHKSUM = "file://${UNPACKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" |
59 | 59 | ||
60 | The first line locates a file in ``${S}/src/ls.c`` and isolates lines | 60 | The first line locates a file in ``${S}/src/ls.c`` and isolates lines |
61 | five through 16 as license text. The second line refers to a file in | 61 | five through 16 as license text. The second line refers to a file in |
62 | :term:`WORKDIR`. | 62 | :term:`UNPACKDIR`. |
63 | 63 | ||
64 | Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes, | 64 | Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes, |
65 | unless the :term:`LICENSE` variable is set to "CLOSED". | 65 | unless the :term:`LICENSE` variable is set to "CLOSED". |
diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst new file mode 100644 index 0000000000..71fe542efb --- /dev/null +++ b/documentation/dev-manual/multiconfig.rst | |||
@@ -0,0 +1,312 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Building Images for Multiple Targets With Multiconfig | ||
4 | ***************************************************** | ||
5 | |||
6 | You can use a single ``bitbake`` command to build multiple images or | ||
7 | packages for different targets where each image or package requires a | ||
8 | different configuration (multiple configuration builds). The builds, in | ||
9 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
10 | section uses that term throughout. | ||
11 | |||
12 | This section describes how to set up for multiple configuration builds | ||
13 | and how to account for cross-build dependencies between the | ||
14 | multiconfigs. | ||
15 | |||
16 | Setting Up and Running a Multiple Configuration Build | ||
17 | ===================================================== | ||
18 | |||
19 | To accomplish a multiple configuration build, you must define each | ||
20 | target's configuration separately using a parallel :term:`configuration file` in | ||
21 | the :term:`Build Directory` or configuration directory within a layer, and you | ||
22 | must follow a required file hierarchy. Additionally, you must enable the | ||
23 | multiple configuration builds in your ``local.conf`` file. | ||
24 | |||
25 | Follow these steps to set up and execute multiple configuration builds: | ||
26 | |||
27 | - *Create Separate Configuration Files*: You need to create a single | ||
28 | :term:`Configuration File` for each build target (each multiconfig). | ||
29 | The configuration definitions are implementation dependent but often | ||
30 | each configuration file will define the :term:`MACHINE` and the | ||
31 | temporary directory (:term:`TMPDIR`) BitBake uses for the build. | ||
32 | |||
33 | .. note:: | ||
34 | |||
35 | Whether the same temporary directory (:term:`TMPDIR`) can be shared will | ||
36 | depend on what is similar and what is different between the | ||
37 | configurations. Multiple :term:`MACHINE` targets can share the same | ||
38 | :term:`TMPDIR` as long as the rest of the configuration is the same, | ||
39 | multiple :term:`DISTRO` settings would need separate :term:`TMPDIR` | ||
40 | directories. | ||
41 | |||
42 | For example, consider a scenario with two different multiconfigs for the same | ||
43 | :term:`MACHINE`: "qemux86" built for two distributions such as "poky" and | ||
44 | "poky-lsb". In this case, you would need to use two different :term:`TMPDIR`. | ||
45 | |||
46 | In the general case, using separate :term:`TMPDIR` for the different | ||
47 | multiconfigs is strongly recommended. | ||
48 | |||
49 | The location for these multiconfig configuration files is specific. | ||
50 | They must reside in the current :term:`Build Directory` in a sub-directory of | ||
51 | ``conf`` named ``multiconfig`` or within a :term:`Layer`'s ``conf`` directory | ||
52 | under a directory named ``multiconfig``. Here is an example that defines | ||
53 | two configuration files for the "x86" and "arm" multiconfigs: | ||
54 | |||
55 | .. image:: figures/multiconfig_files.png | ||
56 | :align: center | ||
57 | :width: 50% | ||
58 | |||
59 | The usual :term:`BBPATH` search path is used to locate multiconfig files in | ||
60 | a similar way to other configuration files. | ||
61 | |||
62 | Here is an example showing the minimal statements needed in a | ||
63 | :term:`configuration file` named ``qemux86.conf`` for a ``qemux86`` target | ||
64 | whose temporary build directory is ``tmp-qemux86``:: | ||
65 | |||
66 | MACHINE = "qemux86" | ||
67 | TMPDIR .= "-${BB_CURRENT_MC}" | ||
68 | |||
69 | BitBake will expand the :term:`BB_CURRENT_MC` variable to the value of the | ||
70 | current multiconfig in use. We append this value to :term:`TMPDIR` so that | ||
71 | any change on the definition of :term:`TMPDIR` will automatically affect the | ||
72 | value of :term:`TMPDIR` for each multiconfig. | ||
73 | |||
74 | - *Add the BitBake Multi-configuration Variable to the Local | ||
75 | Configuration File*: Use the | ||
76 | :term:`BBMULTICONFIG` | ||
77 | variable in your ``conf/local.conf`` configuration file to specify | ||
78 | each multiconfig. Continuing with the example from the previous | ||
79 | figure, the :term:`BBMULTICONFIG` variable needs to enable two | ||
80 | multiconfigs: "x86" and "arm" by specifying each configuration file:: | ||
81 | |||
82 | BBMULTICONFIG = "x86 arm" | ||
83 | |||
84 | .. note:: | ||
85 | |||
86 | A "default" configuration already exists by definition. This | ||
87 | configuration is named: "" (i.e. empty string) and is defined by | ||
88 | the variables coming from your ``local.conf`` | ||
89 | file. Consequently, the previous example actually adds two | ||
90 | additional configurations to your build: "arm" and "x86" along | ||
91 | with "". | ||
92 | |||
93 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
94 | the multiple configuration build:: | ||
95 | |||
96 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
97 | |||
98 | For the example in this section, the following command applies:: | ||
99 | |||
100 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
101 | |||
102 | The previous BitBake command builds several components: | ||
103 | |||
104 | - A ``core-image-minimal`` image that is configured through the ``x86.conf`` | ||
105 | configuration file | ||
106 | |||
107 | - A ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
108 | configuration file | ||
109 | |||
110 | - A ``core-image-base`` that is configured through your ``local.conf`` | ||
111 | configuration file | ||
112 | |||
113 | .. note:: | ||
114 | |||
115 | Support for multiple configuration builds in the Yocto Project &DISTRO; | ||
116 | (&DISTRO_NAME;) Release does not include Shared State (sstate) | ||
117 | optimizations. Consequently, if a build uses the same object twice | ||
118 | in, for example, two different :term:`TMPDIR` | ||
119 | directories, the build either loads from an existing sstate cache for | ||
120 | that build at the start or builds the object fresh. | ||
121 | |||
122 | Enabling Multiple Configuration Build Dependencies | ||
123 | ================================================== | ||
124 | |||
125 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
126 | multiple configuration build. For example, suppose that in order to | ||
127 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
128 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
129 | essentially that the | ||
130 | :ref:`ref-tasks-image` task in the | ||
131 | ``core-image-sato`` recipe depends on the completion of the | ||
132 | :ref:`ref-tasks-rootfs` task of the | ||
133 | ``core-image-minimal`` recipe. | ||
134 | |||
135 | To enable dependencies in a multiple configuration build, you must | ||
136 | declare the dependencies in the recipe using the following statement | ||
137 | form:: | ||
138 | |||
139 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
140 | |||
141 | To better show how to use this statement, consider the example scenario | ||
142 | from the first paragraph of this section. The following statement needs | ||
143 | to be added to the recipe that builds the ``core-image-sato`` image:: | ||
144 | |||
145 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
146 | |||
147 | In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The | ||
148 | task on which the :ref:`ref-tasks-image` task in the recipe depends is the | ||
149 | :ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated | ||
150 | with the "arm" multiconfig. | ||
151 | |||
152 | Once you set up this dependency, you can build the "x86" multiconfig | ||
153 | using a BitBake command as follows:: | ||
154 | |||
155 | $ bitbake mc:x86:core-image-sato | ||
156 | |||
157 | This command executes all the tasks needed to create the | ||
158 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
159 | dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the | ||
160 | "arm" multiconfig build. | ||
161 | |||
162 | Having a recipe depend on the root filesystem of another build might not | ||
163 | seem that useful. Consider this change to the statement in the | ||
164 | ``core-image-sato`` recipe:: | ||
165 | |||
166 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
167 | |||
168 | In this case, BitBake must | ||
169 | create the ``core-image-minimal`` image for the "arm" build since the | ||
170 | "x86" build depends on it. | ||
171 | |||
172 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
173 | and have separate configuration files, BitBake places the artifacts for | ||
174 | each build in the respective temporary build directories (i.e. | ||
175 | :term:`TMPDIR`). | ||
176 | |||
177 | Suggested best practices | ||
178 | ======================== | ||
179 | |||
180 | - :term:`TMPDIR` (other than the default set in bitbake.conf) is only set in | ||
181 | ``local.conf`` by the user. This means that we should **not** manipulate | ||
182 | :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration | ||
183 | file`. | ||
184 | |||
185 | - A multiconfig should specify a :term:`TMPDIR`, and should specify it by | ||
186 | appending the multiconfig name with :term:`BB_CURRENT_MC`. | ||
187 | |||
188 | - Recipes that are used to transfer the output from a multiconfig build to | ||
189 | another should use ``do_task[mcdepends]`` to trigger the build of the | ||
190 | component, and then transfer the item to the current configuration in | ||
191 | :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy`, assuming the value of | ||
192 | the deployed item based on :term:`TMPDIR`. | ||
193 | |||
194 | The :ref:`ref-tasks-install` and :ref:`ref-tasks-deploy` tasks should look | ||
195 | like this:: | ||
196 | |||
197 | do_install() { | ||
198 | install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${D}/some/path | ||
199 | } | ||
200 | |||
201 | do_deploy() { | ||
202 | install -m 0644 ${TMPDIR}-<multiconfig>/tmp/deploy/images/<machine>/somefile ${DEPLOYDIR}/somefile | ||
203 | } | ||
204 | |||
205 | In the example above: | ||
206 | |||
207 | - ``<multiconfig>`` is the multiconfig name as set by the multiconfig | ||
208 | :term:`configuration file` (see the :ref:`dev-manual/multiconfig:Setting | ||
209 | Up and Running a Multiple Configuration Build` section above). | ||
210 | |||
211 | - ``<machine>`` must be the :term:`MACHINE` for which ``somefile`` was built | ||
212 | and deployed. This value may differ from the current :term:`MACHINE` if | ||
213 | the multiconfig :term:`configuration file` overrides it. | ||
214 | |||
215 | - Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1`` | ||
216 | if they don't rely on default dependencies such as the standard C library. | ||
217 | |||
218 | Common use case: building baremetal firmware alongside a Linux build | ||
219 | ==================================================================== | ||
220 | |||
221 | A common use case for multiconfig is to use the default configuration as the | ||
222 | regular Linux build, while one or more multiconfigs can be used to build special | ||
223 | components, such as baremetal firmware. It would also apply to a scenario where | ||
224 | a microcontroller, for example, is companion to a main processor where Linux is | ||
225 | running. This section details how one can achieve these kinds of scenarios with | ||
226 | a multiconfig build. | ||
227 | |||
228 | Adding a multiconfig configuration file and recipe for a baremetal firmware | ||
229 | --------------------------------------------------------------------------- | ||
230 | |||
231 | As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple | ||
232 | Configuration Build`, each multiconfig will require a separate | ||
233 | :term:`Configuration File`. In addition, we will define a separate | ||
234 | :term:`TMPDIR` for our baremetal firmware build configuration. | ||
235 | |||
236 | For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf`` | ||
237 | as follows:: | ||
238 | |||
239 | TMPDIR .= "-${BB_CURRENT_MC}" | ||
240 | TCLIBC = "newlib" | ||
241 | |||
242 | The ``baremetal-firmware.conf`` file configures a separate :term:`TMPDIR` for | ||
243 | holding binaries compiled with the `newlib <https://sourceware.org/newlib/>`__ | ||
244 | toolchain (see :term:`TCLIBC`). | ||
245 | |||
246 | .. note:: | ||
247 | |||
248 | Here, the default :term:`MACHINE` is not overridden by the multiconfig | ||
249 | configuration file. As a consequence, the architecture of the built baremetal | ||
250 | binaries will be the same. In other cases, where the firmware runs on a | ||
251 | completely different architecture, the :term:`MACHINE` must be overridden. | ||
252 | |||
253 | We then create a recipe ``my-firmware.bb`` that defines how the baremetal | ||
254 | firmware is built. The recipe should contain enough information for the | ||
255 | :term:`OpenEmbedded build system` to properly compile the firmware with our | ||
256 | toolchain. The building tasks may vary depending on the nature of the firmware. | ||
257 | However, the recipe should define a :ref:`ref-classes-deploy` task that deploys | ||
258 | the output into the :term:`DEPLOYDIR` directory. We will consider in the | ||
259 | following that the file is named ``my-firmware.elf``. | ||
260 | |||
261 | Building the firmware | ||
262 | --------------------- | ||
263 | |||
264 | The firmware can be built with BitBake with the following command:: | ||
265 | |||
266 | $ bitbake mc:baremetal-firmware:my-firmware | ||
267 | |||
268 | However, we would prefer for ``my-firmware`` to be automatically built when | ||
269 | triggering a normal Linux build. | ||
270 | |||
271 | Using a ``mcdepend``, a recipe belonging to the Linux build can trigger the | ||
272 | build of ``my-firmware``. For example, let's consider that our Linux build needs | ||
273 | to assemble a "special" firmware that uses the output of our ``my-firmware`` | ||
274 | recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this | ||
275 | dependency in ``my-parent-firmware.bb`` with:: | ||
276 | |||
277 | do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy" | ||
278 | |||
279 | The above will ensure that when the :ref:`ref-tasks-compile` task of | ||
280 | ``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of | ||
281 | ``my-firmware`` will already have run successfully. | ||
282 | |||
283 | Using the output of ``my-firmware`` | ||
284 | ----------------------------------- | ||
285 | |||
286 | After ``my-firmware`` recipe has deployed ``my-firmware.elf``, we need to use | ||
287 | the output in some way. We can make a series of assumptions, based on the | ||
288 | default Yocto Project variables in order to get the binary for packaging. | ||
289 | |||
290 | First, we can set the following in ``my-parent-firmware.bb``:: | ||
291 | |||
292 | FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf" | ||
293 | FIRMWARE_FILE[vardepsexclude] += "TMPDIR" | ||
294 | |||
295 | The first assignment stores the value of the path to the firmware built and | ||
296 | deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the | ||
297 | :term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies --- | ||
298 | meaning that changing the value of :term:`TMPDIR` (for example, changing the | ||
299 | host on which the firmware is built) will not invalidate the :ref:`shared state | ||
300 | cache <overview-manual/concepts:shared state cache>`. | ||
301 | |||
302 | Additionally, ``<machine>`` should be replaced by the :term:`MACHINE` for which | ||
303 | we are building in the baremetal-firmware context. | ||
304 | |||
305 | We can then add a :ref:`ref-tasks-install` task to ``my-parent-firmware``:: | ||
306 | |||
307 | do_install() { | ||
308 | install -Dm 0644 ${FIRMWARE_FILE} ${D}/lib/firmware/my-firmware.elf | ||
309 | } | ||
310 | |||
311 | Doing the above will allow the firmware binary to be transferred and packaged | ||
312 | into the Linux context and rootfs. | ||
diff --git a/documentation/dev-manual/new-recipe.rst b/documentation/dev-manual/new-recipe.rst index 61fc2eb122..832aa300e1 100644 --- a/documentation/dev-manual/new-recipe.rst +++ b/documentation/dev-manual/new-recipe.rst | |||
@@ -56,7 +56,7 @@ necessary when adding a recipe to build a new piece of software to be | |||
56 | included in a build. | 56 | included in a build. |
57 | 57 | ||
58 | You can find a complete description of the ``devtool add`` command in | 58 | You can find a complete description of the ``devtool add`` command in |
59 | the ":ref:`sdk-manual/extensible:a closer look at \`\`devtool add\`\``" section | 59 | the ":ref:`dev-manual/devtool:a closer look at ``devtool add```" section |
60 | in the Yocto Project Application Development and the Extensible Software | 60 | in the Yocto Project Application Development and the Extensible Software |
61 | Development Kit (eSDK) manual. | 61 | Development Kit (eSDK) manual. |
62 | 62 | ||
@@ -188,13 +188,14 @@ the recipe. | |||
188 | Use lower-cased characters and do not include the reserved suffixes | 188 | Use lower-cased characters and do not include the reserved suffixes |
189 | ``-native``, ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use | 189 | ``-native``, ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use |
190 | them as part of your recipe name unless the string applies). Here are some | 190 | them as part of your recipe name unless the string applies). Here are some |
191 | examples: | 191 | examples (which includes the use of the string "git" as a special case of a |
192 | version identifier): | ||
192 | 193 | ||
193 | .. code-block:: none | 194 | .. code-block:: none |
194 | 195 | ||
195 | cups_1.7.0.bb | 196 | cups_2.4.12.bb |
196 | gawk_4.0.2.bb | 197 | gawk_5.3.2.bb |
197 | irssi_0.8.16-rc1.bb | 198 | psplash_git.bb |
198 | 199 | ||
199 | Running a Build on the Recipe | 200 | Running a Build on the Recipe |
200 | ============================= | 201 | ============================= |
@@ -276,11 +277,11 @@ upgrading the recipe to a future version is as simple as renaming the | |||
276 | recipe to match the new version. | 277 | recipe to match the new version. |
277 | 278 | ||
278 | Here is a simple example from the | 279 | Here is a simple example from the |
279 | ``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source | 280 | :oe_git:`strace recipe </openembedded-core/tree/meta/recipes-devtools/strace>` |
280 | comes from a single tarball. Notice the use of the | 281 | where the source comes from a single tarball. Notice the use of the |
281 | :term:`PV` variable:: | 282 | :term:`PV` variable:: |
282 | 283 | ||
283 | SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \ | 284 | SRC_URI = "${GITHUB_BASE_URI}/download/v${PV}/strace-${PV}.tar.xz \ |
284 | 285 | ||
285 | Files mentioned in :term:`SRC_URI` whose names end in a typical archive | 286 | Files mentioned in :term:`SRC_URI` whose names end in a typical archive |
286 | extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so | 287 | extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so |
@@ -291,13 +292,13 @@ another example that specifies these types of files, see the | |||
291 | 292 | ||
292 | Another way of specifying source is from an SCM. For Git repositories, | 293 | Another way of specifying source is from an SCM. For Git repositories, |
293 | you must specify :term:`SRCREV` and you should specify :term:`PV` to include | 294 | you must specify :term:`SRCREV` and you should specify :term:`PV` to include |
294 | the revision with :term:`SRCPV`. Here is an example from the recipe | 295 | a ``+`` sign in its definition. Here is an example from the recipe |
295 | ``meta/recipes-core/musl/gcompat_git.bb``:: | 296 | :oe_git:`l3afpad_git.bb </openembedded-core/tree/meta/recipes-sato/l3afpad/l3afpad_git.bb>`:: |
296 | 297 | ||
297 | SRC_URI = "git://git.adelielinux.org/adelie/gcompat.git;protocol=https;branch=current" | 298 | SRC_URI = "git://github.com/stevenhoneyman/l3afpad.git;branch=master;protocol=https" |
298 | 299 | ||
299 | PV = "1.0.0+1.1+git${SRCPV}" | 300 | PV = "0.8.18.1.11+git" |
300 | SRCREV = "af5a49e489fdc04b9cf02547650d7aeaccd43793" | 301 | SRCREV ="3cdccdc9505643e50f8208171d9eee5de11a42ff" |
301 | 302 | ||
302 | If your :term:`SRC_URI` statement includes URLs pointing to individual files | 303 | If your :term:`SRC_URI` statement includes URLs pointing to individual files |
303 | fetched from a remote server other than a version control system, | 304 | fetched from a remote server other than a version control system, |
@@ -347,8 +348,8 @@ paste them into your recipe and then run the build again to continue. | |||
347 | continuing with the build. | 348 | continuing with the build. |
348 | 349 | ||
349 | This final example is a bit more complicated and is from the | 350 | This final example is a bit more complicated and is from the |
350 | ``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The | 351 | :oe_git:`rxvt-unicode </openembedded-core/tree/meta/recipes-sato/rxvt-unicode>` |
351 | example's :term:`SRC_URI` statement identifies multiple files as the source | 352 | recipe. The example's :term:`SRC_URI` statement identifies multiple files as the source |
352 | files for the recipe: a tarball, a patch file, a desktop file, and an icon:: | 353 | files for the recipe: a tarball, a patch file, a desktop file, and an icon:: |
353 | 354 | ||
354 | SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ | 355 | SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ |
@@ -706,7 +707,7 @@ hierarchy to locations that would mirror their locations on the target | |||
706 | device. The installation process copies files from the | 707 | device. The installation process copies files from the |
707 | ``${``\ :term:`S`\ ``}``, | 708 | ``${``\ :term:`S`\ ``}``, |
708 | ``${``\ :term:`B`\ ``}``, and | 709 | ``${``\ :term:`B`\ ``}``, and |
709 | ``${``\ :term:`WORKDIR`\ ``}`` | 710 | ``${``\ :term:`UNPACKDIR`\ ``}`` |
710 | directories to the ``${``\ :term:`D`\ ``}`` | 711 | directories to the ``${``\ :term:`D`\ ``}`` |
711 | directory to create the structure as it should appear on the target | 712 | directory to create the structure as it should appear on the target |
712 | system. | 713 | system. |
@@ -1145,7 +1146,7 @@ Building an application from a single file that is stored locally (e.g. under | |||
1145 | ``files``) requires a recipe that has the file listed in the :term:`SRC_URI` | 1146 | ``files``) requires a recipe that has the file listed in the :term:`SRC_URI` |
1146 | variable. Additionally, you need to manually write the :ref:`ref-tasks-compile` | 1147 | variable. Additionally, you need to manually write the :ref:`ref-tasks-compile` |
1147 | and :ref:`ref-tasks-install` tasks. The :term:`S` variable defines the | 1148 | and :ref:`ref-tasks-install` tasks. The :term:`S` variable defines the |
1148 | directory containing the source code, which is set to :term:`WORKDIR` in this | 1149 | directory containing the source code, which is set to :term:`UNPACKDIR` in this |
1149 | case --- the directory BitBake uses for the build:: | 1150 | case --- the directory BitBake uses for the build:: |
1150 | 1151 | ||
1151 | SUMMARY = "Simple helloworld application" | 1152 | SUMMARY = "Simple helloworld application" |
@@ -1155,7 +1156,7 @@ case --- the directory BitBake uses for the build:: | |||
1155 | 1156 | ||
1156 | SRC_URI = "file://helloworld.c" | 1157 | SRC_URI = "file://helloworld.c" |
1157 | 1158 | ||
1158 | S = "${WORKDIR}" | 1159 | S = "${UNPACKDIR}" |
1159 | 1160 | ||
1160 | do_compile() { | 1161 | do_compile() { |
1161 | ${CC} ${LDFLAGS} helloworld.c -o helloworld | 1162 | ${CC} ${LDFLAGS} helloworld.c -o helloworld |
@@ -1211,8 +1212,6 @@ In the following example, ``lz4`` is a makefile-based package:: | |||
1211 | " | 1212 | " |
1212 | UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)" | 1213 | UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)" |
1213 | 1214 | ||
1214 | S = "${WORKDIR}/git" | ||
1215 | |||
1216 | CVE_STATUS[CVE-2014-4715] = "fixed-version: Fixed in r118, which is larger than the current version" | 1215 | CVE_STATUS[CVE-2014-4715] = "fixed-version: Fixed in r118, which is larger than the current version" |
1217 | 1216 | ||
1218 | EXTRA_OEMAKE = "PREFIX=${prefix} CC='${CC}' CFLAGS='${CFLAGS}' DESTDIR=${D} LIBDIR=${libdir} INCLUDEDIR=${includedir} BUILD_STATIC=no" | 1217 | EXTRA_OEMAKE = "PREFIX=${prefix} CC='${CC}' CFLAGS='${CFLAGS}' DESTDIR=${D} LIBDIR=${libdir} INCLUDEDIR=${includedir} BUILD_STATIC=no" |
@@ -1271,8 +1270,6 @@ is a simple example of an application without dependencies:: | |||
1271 | SRC_URI = "git://gitlab.com/ipcalc/ipcalc.git;protocol=https;branch=master" | 1270 | SRC_URI = "git://gitlab.com/ipcalc/ipcalc.git;protocol=https;branch=master" |
1272 | SRCREV = "4c4261a47f355946ee74013d4f5d0494487cc2d6" | 1271 | SRCREV = "4c4261a47f355946ee74013d4f5d0494487cc2d6" |
1273 | 1272 | ||
1274 | S = "${WORKDIR}/git" | ||
1275 | |||
1276 | inherit meson | 1273 | inherit meson |
1277 | 1274 | ||
1278 | Applications with dependencies are likely to inherit the | 1275 | Applications with dependencies are likely to inherit the |
@@ -1397,11 +1394,31 @@ doing the following: | |||
1397 | where you have installed them and whether those files are in | 1394 | where you have installed them and whether those files are in |
1398 | different locations than the defaults. | 1395 | different locations than the defaults. |
1399 | 1396 | ||
1397 | As a basic example of a :ref:`ref-classes-bin-package`-style recipe, consider | ||
1398 | this snippet from the | ||
1399 | :oe_git:`wireless-regdb </openembedded-core/tree/meta/recipes-kernel/wireless-regdb>` | ||
1400 | recipe file, which fetches a single tarball of binary content and manually | ||
1401 | installs with no need for any configuration or compilation:: | ||
1402 | |||
1403 | SRC_URI = "https://www.kernel.org/pub/software/network/${BPN}/${BP}.tar.xz" | ||
1404 | SRC_URI[sha256sum] = "57f8e7721cf5a880c13ae0c202edbb21092a060d45f9e9c59bcd2a8272bfa456" | ||
1405 | |||
1406 | inherit bin_package allarch | ||
1407 | |||
1408 | do_install() { | ||
1409 | install -d -m0755 ${D}${nonarch_libdir}/crda | ||
1410 | install -d -m0755 ${D}${sysconfdir}/wireless-regdb/pubkeys | ||
1411 | install -m 0644 regulatory.bin ${D}${nonarch_libdir}/crda/regulatory.bin | ||
1412 | install -m 0644 wens.key.pub.pem ${D}${sysconfdir}/wireless-regdb/pubkeys/wens.key.pub.pem | ||
1413 | install -m 0644 -D regulatory.db ${D}${nonarch_base_libdir}/firmware/regulatory.db | ||
1414 | install -m 0644 regulatory.db.p7s ${D}${nonarch_base_libdir}/firmware/regulatory.db.p7s | ||
1415 | } | ||
1416 | |||
1400 | Following Recipe Style Guidelines | 1417 | Following Recipe Style Guidelines |
1401 | ================================= | 1418 | ================================= |
1402 | 1419 | ||
1403 | When writing recipes, it is good to conform to existing style guidelines. | 1420 | When writing recipes, it is good to conform to existing style guidelines. |
1404 | See the ":doc:`../contributor-guide/recipe-style-guide`" in the Yocto Project | 1421 | See the ":doc:`/contributor-guide/recipe-style-guide`" in the Yocto Project |
1405 | and OpenEmbedded Contributor Guide for reference. | 1422 | and OpenEmbedded Contributor Guide for reference. |
1406 | 1423 | ||
1407 | It is common for existing recipes to deviate a bit from this style. | 1424 | It is common for existing recipes to deviate a bit from this style. |
@@ -1428,7 +1445,7 @@ chapter of the BitBake User Manual. | |||
1428 | The following example shows some of the ways you can use variables in | 1445 | The following example shows some of the ways you can use variables in |
1429 | recipes:: | 1446 | recipes:: |
1430 | 1447 | ||
1431 | S = "${WORKDIR}/postfix-${PV}" | 1448 | S = "${UNPACKDIR}/postfix-${PV}" |
1432 | CFLAGS += "-DNO_ASM" | 1449 | CFLAGS += "-DNO_ASM" |
1433 | CFLAGS:append = " --enable-important-feature" | 1450 | CFLAGS:append = " --enable-important-feature" |
1434 | 1451 | ||
diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index e5028fffdc..8bd48c8e8f 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst | |||
@@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: | |||
16 | - :ref:`dev-manual/packages:generating and using signed packages` | 16 | - :ref:`dev-manual/packages:generating and using signed packages` |
17 | 17 | ||
18 | - :ref:`Setting up and running package test | 18 | - :ref:`Setting up and running package test |
19 | (ptest) <dev-manual/packages:testing packages with ptest>` | 19 | (ptest) <test-manual/ptest:testing packages with ptest>` |
20 | 20 | ||
21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` | 21 | - :ref:`dev-manual/packages:creating node package manager (npm) packages` |
22 | 22 | ||
@@ -84,10 +84,6 @@ the following: | |||
84 | 84 | ||
85 | - :term:`PR`: The recipe revision. | 85 | - :term:`PR`: The recipe revision. |
86 | 86 | ||
87 | - :term:`SRCPV`: The OpenEmbedded | ||
88 | build system uses this string to help define the value of :term:`PV` when | ||
89 | the source code revision needs to be included in it. | ||
90 | |||
91 | - :yocto_wiki:`PR Service </PR_Service>`: A | 87 | - :yocto_wiki:`PR Service </PR_Service>`: A |
92 | network-based service that helps automate keeping package feeds | 88 | network-based service that helps automate keeping package feeds |
93 | compatible with existing package manager applications such as RPM, | 89 | compatible with existing package manager applications such as RPM, |
@@ -256,15 +252,14 @@ the software:: | |||
256 | 252 | ||
257 | SRCREV = "${AUTOREV}" | 253 | SRCREV = "${AUTOREV}" |
258 | 254 | ||
259 | Furthermore, you need to reference :term:`SRCPV` in :term:`PV` in order to | 255 | Furthermore, you need to include a ``+`` sign in :term:`PV` in order to |
260 | automatically update the version whenever the revision of the source | 256 | automatically update the version whenever the revision of the source |
261 | code changes. Here is an example:: | 257 | code changes. Here is an example:: |
262 | 258 | ||
263 | PV = "1.0+git${SRCPV}" | 259 | PV = "1.0+git" |
264 | |||
265 | The OpenEmbedded build system substitutes :term:`SRCPV` with the following: | ||
266 | 260 | ||
267 | .. code-block:: none | 261 | The OpenEmbedded build system will automatically add the source control |
262 | information to the end of the variable :term:`PKGV`, in this format:: | ||
268 | 263 | ||
269 | AUTOINC+source_code_revision | 264 | AUTOINC+source_code_revision |
270 | 265 | ||
@@ -824,9 +819,19 @@ to use signed package feeds (repositories) when doing a build. | |||
824 | Signing RPM Packages | 819 | Signing RPM Packages |
825 | -------------------- | 820 | -------------------- |
826 | 821 | ||
827 | To enable signing RPM packages, you must set up the following | 822 | To enable signing RPM packages, you must modify the ``rpm`` |
828 | configurations in either your ``local.config`` or ``distro.config`` | 823 | recipe configuration to include support for OpenPGP signing. |
829 | file:: | 824 | That may be done either in a ``.bbappend`` for the ``rpm`` recipe:: |
825 | |||
826 | PACKAGECONFIG:append = " sequoia" | ||
827 | |||
828 | or in a :term:`Configuration File`:: | ||
829 | |||
830 | PACKAGECONFIG:append:pn-rpm-native = " sequoia" | ||
831 | PACKAGECONFIG:append:pn-rpm = " sequoia" | ||
832 | |||
833 | You must also set up the following settings in a | ||
834 | :term:`Configuration File`:: | ||
830 | 835 | ||
831 | # Inherit sign_rpm.bbclass to enable signing functionality | 836 | # Inherit sign_rpm.bbclass to enable signing functionality |
832 | INHERIT += " sign_rpm" | 837 | INHERIT += " sign_rpm" |
@@ -887,114 +892,8 @@ related to signed package feeds are available: | |||
887 | Testing Packages With ptest | 892 | Testing Packages With ptest |
888 | =========================== | 893 | =========================== |
889 | 894 | ||
890 | A Package Test (ptest) runs tests against packages built by the | 895 | See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the |
891 | OpenEmbedded build system on the target machine. A ptest contains at | 896 | Yocto Project Test Environment Manual. |
892 | least two items: the actual test, and a shell script (``run-ptest``) | ||
893 | that starts the test. The shell script that starts the test must not | ||
894 | contain the actual test --- the script only starts the test. On the other | ||
895 | hand, the test can be anything from a simple shell script that runs a | ||
896 | binary and checks the output to an elaborate system of test binaries and | ||
897 | data files. | ||
898 | |||
899 | The test generates output in the format used by Automake:: | ||
900 | |||
901 | result: testname | ||
902 | |||
903 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
904 | the testname can be any identifying string. | ||
905 | |||
906 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
907 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
908 | |||
909 | .. note:: | ||
910 | |||
911 | A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` | ||
912 | class. | ||
913 | |||
914 | Adding ptest to Your Build | ||
915 | -------------------------- | ||
916 | |||
917 | To add package testing to your build, add the :term:`DISTRO_FEATURES` and | ||
918 | :term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which | ||
919 | is found in the :term:`Build Directory`:: | ||
920 | |||
921 | DISTRO_FEATURES:append = " ptest" | ||
922 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
923 | |||
924 | Once your build is complete, the ptest files are installed into the | ||
925 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
926 | is the name of the package. | ||
927 | |||
928 | Running ptest | ||
929 | ------------- | ||
930 | |||
931 | The ``ptest-runner`` package installs a shell script that loops through | ||
932 | all installed ptest test suites and runs them in sequence. Consequently, | ||
933 | you might want to add this package to your image. | ||
934 | |||
935 | Getting Your Package Ready | ||
936 | -------------------------- | ||
937 | |||
938 | In order to enable a recipe to run installed ptests on target hardware, | ||
939 | you need to prepare the recipes that build the packages you want to | ||
940 | test. Here is what you have to do for each recipe: | ||
941 | |||
942 | - *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* | ||
943 | Include the following line in each recipe:: | ||
944 | |||
945 | inherit ptest | ||
946 | |||
947 | - *Create run-ptest:* This script starts your test. Locate the | ||
948 | script where you will refer to it using | ||
949 | :term:`SRC_URI`. Here is an | ||
950 | example that starts a test for ``dbus``:: | ||
951 | |||
952 | #!/bin/sh | ||
953 | cd test | ||
954 | make -k runtest-TESTS | ||
955 | |||
956 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
957 | dependencies that normally do not exist for the package (such as | ||
958 | requiring "make" to run the test suite), use the | ||
959 | :term:`DEPENDS` and | ||
960 | :term:`RDEPENDS` variables in | ||
961 | your recipe in order for the package to meet the dependencies. Here | ||
962 | is an example where the package has a runtime dependency on "make":: | ||
963 | |||
964 | RDEPENDS:${PN}-ptest += "make" | ||
965 | |||
966 | - *Add a function to build the test suite:* Not many packages support | ||
967 | cross-compilation of their test suites. Consequently, you usually | ||
968 | need to add a cross-compilation function to the package. | ||
969 | |||
970 | Many packages based on Automake compile and run the test suite by | ||
971 | using a single command such as ``make check``. However, the host | ||
972 | ``make check`` builds and runs on the same computer, while | ||
973 | cross-compiling requires that the package is built on the host but | ||
974 | executed for the target architecture (though often, as in the case | ||
975 | for ptest, the execution occurs on the host). The built version of | ||
976 | Automake that ships with the Yocto Project includes a patch that | ||
977 | separates building and execution. Consequently, packages that use the | ||
978 | unaltered, patched version of ``make check`` automatically | ||
979 | cross-compiles. | ||
980 | |||
981 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
982 | build the test suite. Add a function similar to the following to your | ||
983 | recipe:: | ||
984 | |||
985 | do_compile_ptest() { | ||
986 | oe_runmake buildtest-TESTS | ||
987 | } | ||
988 | |||
989 | - *Ensure special configurations are set:* If the package requires | ||
990 | special configurations prior to compiling the test code, you must | ||
991 | insert a ``do_configure_ptest`` function into the recipe. | ||
992 | |||
993 | - *Install the test suite:* The :ref:`ref-classes-ptest` class | ||
994 | automatically copies the file ``run-ptest`` to the target and then runs make | ||
995 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
996 | to create a ``do_install_ptest`` function and make sure it gets | ||
997 | called after the "make install-ptest" completes. | ||
998 | 897 | ||
999 | Creating Node Package Manager (NPM) Packages | 898 | Creating Node Package Manager (NPM) Packages |
1000 | ============================================ | 899 | ============================================ |
@@ -1125,7 +1024,7 @@ The ``devtool edit-recipe`` command lets you take a look at the recipe:: | |||
1125 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ | 1024 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ |
1126 | " | 1025 | " |
1127 | 1026 | ||
1128 | S = "${WORKDIR}/npm" | 1027 | S = "${UNPACKDIR}/npm" |
1129 | 1028 | ||
1130 | inherit npm | 1029 | inherit npm |
1131 | 1030 | ||
diff --git a/documentation/dev-manual/prebuilt-libraries.rst b/documentation/dev-manual/prebuilt-libraries.rst index a05f39ca1e..59621ca16d 100644 --- a/documentation/dev-manual/prebuilt-libraries.rst +++ b/documentation/dev-manual/prebuilt-libraries.rst | |||
@@ -97,7 +97,7 @@ The complete recipe would look like this:: | |||
97 | # we use a local link. | 97 | # we use a local link. |
98 | SRC_URI = "file://libft4222-linux-${PV}.tgz" | 98 | SRC_URI = "file://libft4222-linux-${PV}.tgz" |
99 | 99 | ||
100 | S = "${WORKDIR}" | 100 | S = "${UNPACKDIR}" |
101 | 101 | ||
102 | ARCH_DIR:x86-64 = "build-x86_64" | 102 | ARCH_DIR:x86-64 = "build-x86_64" |
103 | ARCH_DIR:i586 = "build-i386" | 103 | ARCH_DIR:i586 = "build-i386" |
@@ -170,7 +170,7 @@ as follows:: | |||
170 | The modifications cause the ``.so`` file to be the real library | 170 | The modifications cause the ``.so`` file to be the real library |
171 | and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into | 171 | and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into |
172 | ``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed, | 172 | ``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed, |
173 | ``${PN}-dev`` collects files before `${PN}`. ``${PN}-dev`` must not collect any of | 173 | ``${PN}-dev`` collects files before ``${PN}``. ``${PN}-dev`` must not collect any of |
174 | the files you want in ``${PN}``. | 174 | the files you want in ``${PN}``. |
175 | 175 | ||
176 | Finally, loadable modules, essentially unversioned libraries that are linked | 176 | Finally, loadable modules, essentially unversioned libraries that are linked |
@@ -204,6 +204,6 @@ versioned library example. The "magic" is setting the :term:`SOLIBS` and | |||
204 | 204 | ||
205 | do_install () { | 205 | do_install () { |
206 | install -d ${D}${libdir} | 206 | install -d ${D}${libdir} |
207 | install -m 0755 ${WORKDIR}/libfoo.so ${D}${libdir} | 207 | install -m 0755 ${UNPACKDIR}/libfoo.so ${D}${libdir} |
208 | } | 208 | } |
209 | 209 | ||
diff --git a/documentation/dev-manual/qemu.rst b/documentation/dev-manual/qemu.rst index 19f3e40d63..92c93a82ab 100644 --- a/documentation/dev-manual/qemu.rst +++ b/documentation/dev-manual/qemu.rst | |||
@@ -75,7 +75,7 @@ available. Follow these general steps to run QEMU: | |||
75 | your :term:`Build Directory`. | 75 | your :term:`Build Directory`. |
76 | 76 | ||
77 | - If you have not built an image, you can go to the | 77 | - If you have not built an image, you can go to the |
78 | :yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu/>` area and download a | 78 | :yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu/>` area and download a |
79 | pre-built image that matches your architecture and can be run on | 79 | pre-built image that matches your architecture and can be run on |
80 | QEMU. | 80 | QEMU. |
81 | 81 | ||
@@ -280,12 +280,11 @@ present, the toolchain is also automatically used. | |||
280 | networking. | 280 | networking. |
281 | 281 | ||
282 | - SSH servers are available in some QEMU images. The ``core-image-sato`` | 282 | - SSH servers are available in some QEMU images. The ``core-image-sato`` |
283 | QEMU image has a Dropbear secure shell (SSH) server that runs with | 283 | QEMU image has a Dropbear secure shell (SSH) server that runs with the |
284 | the root password disabled. The ``core-image-full-cmdline`` and | 284 | root password disabled. The ``core-image-full-cmdline`` QEMU image has |
285 | ``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear. | 285 | OpenSSH instead of Dropbear. Including these SSH servers allow you to use |
286 | Including these SSH servers allow you to use standard ``ssh`` and | 286 | standard ``ssh`` and ``scp`` commands. The ``core-image-minimal`` QEMU |
287 | ``scp`` commands. The ``core-image-minimal`` QEMU image, however, | 287 | image, however, contains no SSH server. |
288 | contains no SSH server. | ||
289 | 288 | ||
290 | - You can use a provided, user-space NFS server to boot the QEMU | 289 | - You can use a provided, user-space NFS server to boot the QEMU |
291 | session using a local copy of the root filesystem on the host. In | 290 | session using a local copy of the root filesystem on the host. In |
@@ -368,9 +367,6 @@ command line: | |||
368 | 367 | ||
369 | - `VM`: The virtual machine image, which must be a ``.wic.vmdk`` | 368 | - `VM`: The virtual machine image, which must be a ``.wic.vmdk`` |
370 | file. Use this option when you want to boot a ``.wic.vmdk`` image. | 369 | file. Use this option when you want to boot a ``.wic.vmdk`` image. |
371 | The image filename you provide must contain one of the following | ||
372 | strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64", | ||
373 | "qemumips", "qemuppc", or "qemush4". | ||
374 | 370 | ||
375 | - `ROOTFS`: A root filesystem that has one of the following filetype | 371 | - `ROOTFS`: A root filesystem that has one of the following filetype |
376 | extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If | 372 | extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If |
diff --git a/documentation/dev-manual/runtime-testing.rst b/documentation/dev-manual/runtime-testing.rst deleted file mode 100644 index 7a2b42f25a..0000000000 --- a/documentation/dev-manual/runtime-testing.rst +++ /dev/null | |||
@@ -1,594 +0,0 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Performing Automated Runtime Testing | ||
4 | ************************************ | ||
5 | |||
6 | The OpenEmbedded build system makes available a series of automated | ||
7 | tests for images to verify runtime functionality. You can run these | ||
8 | tests on either QEMU or actual target hardware. Tests are written in | ||
9 | Python making use of the ``unittest`` module, and the majority of them | ||
10 | run commands on the target system over SSH. This section describes how | ||
11 | you set up the environment to use these tests, run available tests, and | ||
12 | write and add your own tests. | ||
13 | |||
14 | For information on the test and QA infrastructure available within the | ||
15 | Yocto Project, see the ":ref:`ref-manual/release-process:testing and quality assurance`" | ||
16 | section in the Yocto Project Reference Manual. | ||
17 | |||
18 | Enabling Tests | ||
19 | ============== | ||
20 | |||
21 | Depending on whether you are planning to run tests using QEMU or on the | ||
22 | hardware, you have to take different steps to enable the tests. See the | ||
23 | following subsections for information on how to enable both types of | ||
24 | tests. | ||
25 | |||
26 | Enabling Runtime Tests on QEMU | ||
27 | ------------------------------ | ||
28 | |||
29 | In order to run tests, you need to do the following: | ||
30 | |||
31 | - *Set up to avoid interaction with sudo for networking:* To | ||
32 | accomplish this, you must do one of the following: | ||
33 | |||
34 | - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all | ||
35 | commands or just for ``runqemu-ifup``. You must provide the full | ||
36 | path as that can change if you are using multiple clones of the | ||
37 | source repository. | ||
38 | |||
39 | .. note:: | ||
40 | |||
41 | On some distributions, you also need to comment out "Defaults | ||
42 | requiretty" in ``/etc/sudoers``. | ||
43 | |||
44 | - Manually configure a tap interface for your system. | ||
45 | |||
46 | - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which | ||
47 | should generate a list of tap devices. This is the option | ||
48 | typically chosen for Autobuilder-type environments. | ||
49 | |||
50 | .. note:: | ||
51 | |||
52 | - Be sure to use an absolute path when calling this script | ||
53 | with sudo. | ||
54 | |||
55 | - Ensure that your host has the package ``iptables`` installed. | ||
56 | |||
57 | - The package recipe ``qemu-helper-native`` is required to run | ||
58 | this script. Build the package using the following command:: | ||
59 | |||
60 | $ bitbake qemu-helper-native | ||
61 | |||
62 | - *Set the DISPLAY variable:* You need to set this variable so that | ||
63 | you have an X server available (e.g. start ``vncserver`` for a | ||
64 | headless machine). | ||
65 | |||
66 | - *Be sure your host's firewall accepts incoming connections from | ||
67 | 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an | ||
68 | HTTP server on a random high number port, which is used to serve | ||
69 | files to the target. The DNF module serves | ||
70 | ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands. | ||
71 | That means your host's firewall must accept incoming connections from | ||
72 | 192.168.7.0/24, which is the default IP range used for tap devices by | ||
73 | ``runqemu``. | ||
74 | |||
75 | - *Be sure your host has the correct packages installed:* Depending | ||
76 | your host's distribution, you need to have the following packages | ||
77 | installed: | ||
78 | |||
79 | - Ubuntu and Debian: ``sysstat`` and ``iproute2`` | ||
80 | |||
81 | - openSUSE: ``sysstat`` and ``iproute2`` | ||
82 | |||
83 | - Fedora: ``sysstat`` and ``iproute`` | ||
84 | |||
85 | - CentOS: ``sysstat`` and ``iproute`` | ||
86 | |||
87 | Once you start running the tests, the following happens: | ||
88 | |||
89 | #. A copy of the root filesystem is written to ``${WORKDIR}/testimage``. | ||
90 | |||
91 | #. The image is booted under QEMU using the standard ``runqemu`` script. | ||
92 | |||
93 | #. A default timeout of 500 seconds occurs to allow for the boot process | ||
94 | to reach the login prompt. You can change the timeout period by | ||
95 | setting | ||
96 | :term:`TEST_QEMUBOOT_TIMEOUT` | ||
97 | in the ``local.conf`` file. | ||
98 | |||
99 | #. Once the boot process is reached and the login prompt appears, the | ||
100 | tests run. The full boot log is written to | ||
101 | ``${WORKDIR}/testimage/qemu_boot_log``. | ||
102 | |||
103 | #. Each test module loads in the order found in :term:`TEST_SUITES`. You can | ||
104 | find the full output of the commands run over SSH in | ||
105 | ``${WORKDIR}/testimgage/ssh_target_log``. | ||
106 | |||
107 | #. If no failures occur, the task running the tests ends successfully. | ||
108 | You can find the output from the ``unittest`` in the task log at | ||
109 | ``${WORKDIR}/temp/log.do_testimage``. | ||
110 | |||
111 | Enabling Runtime Tests on Hardware | ||
112 | ---------------------------------- | ||
113 | |||
114 | The OpenEmbedded build system can run tests on real hardware, and for | ||
115 | certain devices it can also deploy the image to be tested onto the | ||
116 | device beforehand. | ||
117 | |||
118 | For automated deployment, a "controller image" is installed onto the | ||
119 | hardware once as part of setup. Then, each time tests are to be run, the | ||
120 | following occurs: | ||
121 | |||
122 | #. The controller image is booted into and used to write the image to be | ||
123 | tested to a second partition. | ||
124 | |||
125 | #. The device is then rebooted using an external script that you need to | ||
126 | provide. | ||
127 | |||
128 | #. The device boots into the image to be tested. | ||
129 | |||
130 | When running tests (independent of whether the image has been deployed | ||
131 | automatically or not), the device is expected to be connected to a | ||
132 | network on a pre-determined IP address. You can either use static IP | ||
133 | addresses written into the image, or set the image to use DHCP and have | ||
134 | your DHCP server on the test network assign a known IP address based on | ||
135 | the MAC address of the device. | ||
136 | |||
137 | In order to run tests on hardware, you need to set :term:`TEST_TARGET` to an | ||
138 | appropriate value. For QEMU, you do not have to change anything, the | ||
139 | default value is "qemu". For running tests on hardware, the following | ||
140 | options are available: | ||
141 | |||
142 | - *"simpleremote":* Choose "simpleremote" if you are going to run tests | ||
143 | on a target system that is already running the image to be tested and | ||
144 | is available on the network. You can use "simpleremote" in | ||
145 | conjunction with either real hardware or an image running within a | ||
146 | separately started QEMU or any other virtual machine manager. | ||
147 | |||
148 | - *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is | ||
149 | an EFI-based machine with ``systemd-boot`` as bootloader and | ||
150 | ``core-image-testmaster`` (or something similar) is installed. Also, | ||
151 | your hardware under test must be in a DHCP-enabled network that gives | ||
152 | it the same IP address for each reboot. | ||
153 | |||
154 | If you choose "SystemdbootTarget", there are additional requirements | ||
155 | and considerations. See the | ||
156 | ":ref:`dev-manual/runtime-testing:selecting systemdboottarget`" section, which | ||
157 | follows, for more information. | ||
158 | |||
159 | - *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying | ||
160 | images and running tests on the BeagleBone "Black" or original | ||
161 | "White" hardware. For information on how to use these tests, see the | ||
162 | comments at the top of the BeagleBoneTarget | ||
163 | ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file. | ||
164 | |||
165 | - *"GrubTarget":* Choose "GrubTarget" if you are deploying images and running | ||
166 | tests on any generic PC that boots using GRUB. For information on how | ||
167 | to use these tests, see the comments at the top of the GrubTarget | ||
168 | ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file. | ||
169 | |||
170 | - *"your-target":* Create your own custom target if you want to run | ||
171 | tests when you are deploying images and running tests on a custom | ||
172 | machine within your BSP layer. To do this, you need to add a Python | ||
173 | unit that defines the target class under ``lib/oeqa/controllers/`` | ||
174 | within your layer. You must also provide an empty ``__init__.py``. | ||
175 | For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``. | ||
176 | |||
177 | Selecting SystemdbootTarget | ||
178 | --------------------------- | ||
179 | |||
180 | If you did not set :term:`TEST_TARGET` to "SystemdbootTarget", then you do | ||
181 | not need any information in this section. You can skip down to the | ||
182 | ":ref:`dev-manual/runtime-testing:running tests`" section. | ||
183 | |||
184 | If you did set :term:`TEST_TARGET` to "SystemdbootTarget", you also need to | ||
185 | perform a one-time setup of your controller image by doing the following: | ||
186 | |||
187 | #. *Set EFI_PROVIDER:* Be sure that :term:`EFI_PROVIDER` is as follows:: | ||
188 | |||
189 | EFI_PROVIDER = "systemd-boot" | ||
190 | |||
191 | #. *Build the controller image:* Build the ``core-image-testmaster`` image. | ||
192 | The ``core-image-testmaster`` recipe is provided as an example for a | ||
193 | "controller" image and you can customize the image recipe as you would | ||
194 | any other recipe. | ||
195 | |||
196 | Image recipe requirements are: | ||
197 | |||
198 | - Inherits ``core-image`` so that kernel modules are installed. | ||
199 | |||
200 | - Installs normal linux utilities not BusyBox ones (e.g. ``bash``, | ||
201 | ``coreutils``, ``tar``, ``gzip``, and ``kmod``). | ||
202 | |||
203 | - Uses a custom :term:`Initramfs` image with a custom | ||
204 | installer. A normal image that you can install usually creates a | ||
205 | single root filesystem partition. This image uses another installer that | ||
206 | creates a specific partition layout. Not all Board Support | ||
207 | Packages (BSPs) can use an installer. For such cases, you need to | ||
208 | manually create the following partition layout on the target: | ||
209 | |||
210 | - First partition mounted under ``/boot``, labeled "boot". | ||
211 | |||
212 | - The main root filesystem partition where this image gets installed, | ||
213 | which is mounted under ``/``. | ||
214 | |||
215 | - Another partition labeled "testrootfs" where test images get | ||
216 | deployed. | ||
217 | |||
218 | #. *Install image:* Install the image that you just built on the target | ||
219 | system. | ||
220 | |||
221 | The final thing you need to do when setting :term:`TEST_TARGET` to | ||
222 | "SystemdbootTarget" is to set up the test image: | ||
223 | |||
224 | #. *Set up your local.conf file:* Make sure you have the following | ||
225 | statements in your ``local.conf`` file:: | ||
226 | |||
227 | IMAGE_FSTYPES += "tar.gz" | ||
228 | IMAGE_CLASSES += "testimage" | ||
229 | TEST_TARGET = "SystemdbootTarget" | ||
230 | TEST_TARGET_IP = "192.168.2.3" | ||
231 | |||
232 | #. *Build your test image:* Use BitBake to build the image:: | ||
233 | |||
234 | $ bitbake core-image-sato | ||
235 | |||
236 | Power Control | ||
237 | ------------- | ||
238 | |||
239 | For most hardware targets other than "simpleremote", you can control | ||
240 | power: | ||
241 | |||
242 | - You can use :term:`TEST_POWERCONTROL_CMD` together with | ||
243 | :term:`TEST_POWERCONTROL_EXTRA_ARGS` as a command that runs on the host | ||
244 | and does power cycling. The test code passes one argument to that | ||
245 | command: off, on or cycle (off then on). Here is an example that | ||
246 | could appear in your ``local.conf`` file:: | ||
247 | |||
248 | TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" | ||
249 | |||
250 | In this example, the expect | ||
251 | script does the following: | ||
252 | |||
253 | .. code-block:: shell | ||
254 | |||
255 | ssh test@10.11.12.1 "pyctl nuc1 arg" | ||
256 | |||
257 | It then runs a Python script that controls power for a label called | ||
258 | ``nuc1``. | ||
259 | |||
260 | .. note:: | ||
261 | |||
262 | You need to customize :term:`TEST_POWERCONTROL_CMD` and | ||
263 | :term:`TEST_POWERCONTROL_EXTRA_ARGS` for your own setup. The one requirement | ||
264 | is that it accepts "on", "off", and "cycle" as the last argument. | ||
265 | |||
266 | - When no command is defined, it connects to the device over SSH and | ||
267 | uses the classic reboot command to reboot the device. Classic reboot | ||
268 | is fine as long as the machine actually reboots (i.e. the SSH test | ||
269 | has not failed). It is useful for scenarios where you have a simple | ||
270 | setup, typically with a single board, and where some manual | ||
271 | interaction is okay from time to time. | ||
272 | |||
273 | If you have no hardware to automatically perform power control but still | ||
274 | wish to experiment with automated hardware testing, you can use the | ||
275 | ``dialog-power-control`` script that shows a dialog prompting you to perform | ||
276 | the required power action. This script requires either KDialog or Zenity | ||
277 | to be installed. To use this script, set the | ||
278 | :term:`TEST_POWERCONTROL_CMD` | ||
279 | variable as follows:: | ||
280 | |||
281 | TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" | ||
282 | |||
283 | Serial Console Connection | ||
284 | ------------------------- | ||
285 | |||
286 | For test target classes requiring a serial console to interact with the | ||
287 | bootloader (e.g. BeagleBoneTarget and GrubTarget), | ||
288 | you need to specify a command to use to connect to the serial console of | ||
289 | the target machine by using the | ||
290 | :term:`TEST_SERIALCONTROL_CMD` | ||
291 | variable and optionally the | ||
292 | :term:`TEST_SERIALCONTROL_EXTRA_ARGS` | ||
293 | variable. | ||
294 | |||
295 | These cases could be a serial terminal program if the machine is | ||
296 | connected to a local serial port, or a ``telnet`` or ``ssh`` command | ||
297 | connecting to a remote console server. Regardless of the case, the | ||
298 | command simply needs to connect to the serial console and forward that | ||
299 | connection to standard input and output as any normal terminal program | ||
300 | does. For example, to use the picocom terminal program on serial device | ||
301 | ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:: | ||
302 | |||
303 | TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" | ||
304 | |||
305 | For local | ||
306 | devices where the serial port device disappears when the device reboots, | ||
307 | an additional "serdevtry" wrapper script is provided. To use this | ||
308 | wrapper, simply prefix the terminal command with | ||
309 | ``${COREBASE}/scripts/contrib/serdevtry``:: | ||
310 | |||
311 | TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0" | ||
312 | |||
313 | Running Tests | ||
314 | ============= | ||
315 | |||
316 | You can start the tests automatically or manually: | ||
317 | |||
318 | - *Automatically running tests:* To run the tests automatically after the | ||
319 | OpenEmbedded build system successfully creates an image, first set the | ||
320 | :term:`TESTIMAGE_AUTO` variable to "1" in your ``local.conf`` file in the | ||
321 | :term:`Build Directory`:: | ||
322 | |||
323 | TESTIMAGE_AUTO = "1" | ||
324 | |||
325 | Next, build your image. If the image successfully builds, the | ||
326 | tests run:: | ||
327 | |||
328 | bitbake core-image-sato | ||
329 | |||
330 | - *Manually running tests:* To manually run the tests, first globally | ||
331 | inherit the :ref:`ref-classes-testimage` class by editing your | ||
332 | ``local.conf`` file:: | ||
333 | |||
334 | IMAGE_CLASSES += "testimage" | ||
335 | |||
336 | Next, use BitBake to run the tests:: | ||
337 | |||
338 | bitbake -c testimage image | ||
339 | |||
340 | All test files reside in ``meta/lib/oeqa/runtime/cases`` in the | ||
341 | :term:`Source Directory`. A test name maps | ||
342 | directly to a Python module. Each test module may contain a number of | ||
343 | individual tests. Tests are usually grouped together by the area tested | ||
344 | (e.g tests for systemd reside in ``meta/lib/oeqa/runtime/cases/systemd.py``). | ||
345 | |||
346 | You can add tests to any layer provided you place them in the proper | ||
347 | area and you extend :term:`BBPATH` in | ||
348 | the ``local.conf`` file as normal. Be sure that tests reside in | ||
349 | ``layer/lib/oeqa/runtime/cases``. | ||
350 | |||
351 | .. note:: | ||
352 | |||
353 | Be sure that module names do not collide with module names used in | ||
354 | the default set of test modules in ``meta/lib/oeqa/runtime/cases``. | ||
355 | |||
356 | You can change the set of tests run by appending or overriding | ||
357 | :term:`TEST_SUITES` variable in | ||
358 | ``local.conf``. Each name in :term:`TEST_SUITES` represents a required test | ||
359 | for the image. Test modules named within :term:`TEST_SUITES` cannot be | ||
360 | skipped even if a test is not suitable for an image (e.g. running the | ||
361 | RPM tests on an image without ``rpm``). Appending "auto" to | ||
362 | :term:`TEST_SUITES` causes the build system to try to run all tests that are | ||
363 | suitable for the image (i.e. each test module may elect to skip itself). | ||
364 | |||
365 | The order you list tests in :term:`TEST_SUITES` is important and influences | ||
366 | test dependencies. Consequently, tests that depend on other tests should | ||
367 | be added after the test on which they depend. For example, since the | ||
368 | ``ssh`` test depends on the ``ping`` test, "ssh" needs to come after | ||
369 | "ping" in the list. The test class provides no re-ordering or dependency | ||
370 | handling. | ||
371 | |||
372 | .. note:: | ||
373 | |||
374 | Each module can have multiple classes with multiple test methods. | ||
375 | And, Python ``unittest`` rules apply. | ||
376 | |||
377 | Here are some things to keep in mind when running tests: | ||
378 | |||
379 | - The default tests for the image are defined as:: | ||
380 | |||
381 | DEFAULT_TEST_SUITES:pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg" | ||
382 | |||
383 | - Add your own test to the list of the by using the following:: | ||
384 | |||
385 | TEST_SUITES:append = " mytest" | ||
386 | |||
387 | - Run a specific list of tests as follows:: | ||
388 | |||
389 | TEST_SUITES = "test1 test2 test3" | ||
390 | |||
391 | Remember, order is important. Be sure to place a test that is | ||
392 | dependent on another test later in the order. | ||
393 | |||
394 | Exporting Tests | ||
395 | =============== | ||
396 | |||
397 | You can export tests so that they can run independently of the build | ||
398 | system. Exporting tests is required if you want to be able to hand the | ||
399 | test execution off to a scheduler. You can only export tests that are | ||
400 | defined in :term:`TEST_SUITES`. | ||
401 | |||
402 | If your image is already built, make sure the following are set in your | ||
403 | ``local.conf`` file:: | ||
404 | |||
405 | INHERIT += "testexport" | ||
406 | TEST_TARGET_IP = "IP-address-for-the-test-target" | ||
407 | TEST_SERVER_IP = "IP-address-for-the-test-server" | ||
408 | |||
409 | You can then export the tests with the | ||
410 | following BitBake command form:: | ||
411 | |||
412 | $ bitbake image -c testexport | ||
413 | |||
414 | Exporting the tests places them in the :term:`Build Directory` in | ||
415 | ``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR` | ||
416 | variable. | ||
417 | |||
418 | You can now run the tests outside of the build environment:: | ||
419 | |||
420 | $ cd tmp/testexport/image | ||
421 | $ ./runexported.py testdata.json | ||
422 | |||
423 | Here is a complete example that shows IP addresses and uses the | ||
424 | ``core-image-sato`` image:: | ||
425 | |||
426 | INHERIT += "testexport" | ||
427 | TEST_TARGET_IP = "192.168.7.2" | ||
428 | TEST_SERVER_IP = "192.168.7.1" | ||
429 | |||
430 | Use BitBake to export the tests:: | ||
431 | |||
432 | $ bitbake core-image-sato -c testexport | ||
433 | |||
434 | Run the tests outside of | ||
435 | the build environment using the following:: | ||
436 | |||
437 | $ cd tmp/testexport/core-image-sato | ||
438 | $ ./runexported.py testdata.json | ||
439 | |||
440 | Writing New Tests | ||
441 | ================= | ||
442 | |||
443 | As mentioned previously, all new test files need to be in the proper | ||
444 | place for the build system to find them. New tests for additional | ||
445 | functionality outside of the core should be added to the layer that adds | ||
446 | the functionality, in ``layer/lib/oeqa/runtime/cases`` (as long as | ||
447 | :term:`BBPATH` is extended in the | ||
448 | layer's ``layer.conf`` file as normal). Just remember the following: | ||
449 | |||
450 | - Filenames need to map directly to test (module) names. | ||
451 | |||
452 | - Do not use module names that collide with existing core tests. | ||
453 | |||
454 | - Minimally, an empty ``__init__.py`` file must be present in the runtime | ||
455 | directory. | ||
456 | |||
457 | To create a new test, start by copying an existing module (e.g. | ||
458 | ``oe_syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use | ||
459 | code from ``meta/lib/oeqa/utils``, which are helper classes. | ||
460 | |||
461 | .. note:: | ||
462 | |||
463 | Structure shell commands such that you rely on them and they return a | ||
464 | single code for success. Be aware that sometimes you will need to | ||
465 | parse the output. See the ``df.py`` and ``date.py`` modules for examples. | ||
466 | |||
467 | You will notice that all test classes inherit ``oeRuntimeTest``, which | ||
468 | is found in ``meta/lib/oetest.py``. This base class offers some helper | ||
469 | attributes, which are described in the following sections: | ||
470 | |||
471 | Class Methods | ||
472 | ------------- | ||
473 | |||
474 | Class methods are as follows: | ||
475 | |||
476 | - *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed | ||
477 | package list of the image, which is based on the manifest file that | ||
478 | is generated during the :ref:`ref-tasks-rootfs` task. | ||
479 | |||
480 | - *hasFeature(feature):* Returns "True" if the feature is in | ||
481 | :term:`IMAGE_FEATURES` or | ||
482 | :term:`DISTRO_FEATURES`. | ||
483 | |||
484 | Class Attributes | ||
485 | ---------------- | ||
486 | |||
487 | Class attributes are as follows: | ||
488 | |||
489 | - *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image. | ||
490 | Otherwise, ``pscmd`` equals "ps" (busybox). | ||
491 | |||
492 | - *tc:* The called test context, which gives access to the | ||
493 | following attributes: | ||
494 | |||
495 | - *d:* The BitBake datastore, which allows you to use stuff such | ||
496 | as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``. | ||
497 | |||
498 | - *testslist and testsrequired:* Used internally. The tests | ||
499 | do not need these. | ||
500 | |||
501 | - *filesdir:* The absolute path to | ||
502 | ``meta/lib/oeqa/runtime/files``, which contains helper files for | ||
503 | tests meant for copying on the target such as small files written | ||
504 | in C for compilation. | ||
505 | |||
506 | - *target:* The target controller object used to deploy and | ||
507 | start an image on a particular target (e.g. Qemu, SimpleRemote, | ||
508 | and SystemdbootTarget). Tests usually use the following: | ||
509 | |||
510 | - *ip:* The target's IP address. | ||
511 | |||
512 | - *server_ip:* The host's IP address, which is usually used | ||
513 | by the DNF test suite. | ||
514 | |||
515 | - *run(cmd, timeout=None):* The single, most used method. | ||
516 | This command is a wrapper for: ``ssh root@host "cmd"``. The | ||
517 | command returns a tuple: (status, output), which are what their | ||
518 | names imply - the return code of "cmd" and whatever output it | ||
519 | produces. The optional timeout argument represents the number | ||
520 | of seconds the test should wait for "cmd" to return. If the | ||
521 | argument is "None", the test uses the default instance's | ||
522 | timeout period, which is 300 seconds. If the argument is "0", | ||
523 | the test runs until the command returns. | ||
524 | |||
525 | - *copy_to(localpath, remotepath):* | ||
526 | ``scp localpath root@ip:remotepath``. | ||
527 | |||
528 | - *copy_from(remotepath, localpath):* | ||
529 | ``scp root@host:remotepath localpath``. | ||
530 | |||
531 | Instance Attributes | ||
532 | ------------------- | ||
533 | |||
534 | There is a single instance attribute, which is ``target``. The ``target`` | ||
535 | instance attribute is identical to the class attribute of the same name, | ||
536 | which is described in the previous section. This attribute exists as | ||
537 | both an instance and class attribute so tests can use | ||
538 | ``self.target.run(cmd)`` in instance methods instead of | ||
539 | ``oeRuntimeTest.tc.target.run(cmd)``. | ||
540 | |||
541 | Installing Packages in the DUT Without the Package Manager | ||
542 | ========================================================== | ||
543 | |||
544 | When a test requires a package built by BitBake, it is possible to | ||
545 | install that package. Installing the package does not require a package | ||
546 | manager be installed in the device under test (DUT). It does, however, | ||
547 | require an SSH connection and the target must be using the | ||
548 | ``sshcontrol`` class. | ||
549 | |||
550 | .. note:: | ||
551 | |||
552 | This method uses ``scp`` to copy files from the host to the target, which | ||
553 | causes permissions and special attributes to be lost. | ||
554 | |||
555 | A JSON file is used to define the packages needed by a test. This file | ||
556 | must be in the same path as the file used to define the tests. | ||
557 | Furthermore, the filename must map directly to the test module name with | ||
558 | a ``.json`` extension. | ||
559 | |||
560 | The JSON file must include an object with the test name as keys of an | ||
561 | object or an array. This object (or array of objects) uses the following | ||
562 | data: | ||
563 | |||
564 | - "pkg" --- a mandatory string that is the name of the package to be | ||
565 | installed. | ||
566 | |||
567 | - "rm" --- an optional boolean, which defaults to "false", that specifies | ||
568 | to remove the package after the test. | ||
569 | |||
570 | - "extract" --- an optional boolean, which defaults to "false", that | ||
571 | specifies if the package must be extracted from the package format. | ||
572 | When set to "true", the package is not automatically installed into | ||
573 | the DUT. | ||
574 | |||
575 | Here is an example JSON file that handles test "foo" installing | ||
576 | package "bar" and test "foobar" installing packages "foo" and "bar". | ||
577 | Once the test is complete, the packages are removed from the DUT:: | ||
578 | |||
579 | { | ||
580 | "foo": { | ||
581 | "pkg": "bar" | ||
582 | }, | ||
583 | "foobar": [ | ||
584 | { | ||
585 | "pkg": "foo", | ||
586 | "rm": true | ||
587 | }, | ||
588 | { | ||
589 | "pkg": "bar", | ||
590 | "rm": true | ||
591 | } | ||
592 | ] | ||
593 | } | ||
594 | |||
diff --git a/documentation/dev-manual/sbom.rst b/documentation/dev-manual/sbom.rst index b72bad1554..ca0fc8b9d6 100644 --- a/documentation/dev-manual/sbom.rst +++ b/documentation/dev-manual/sbom.rst | |||
@@ -24,12 +24,20 @@ users can read in standardized format. | |||
24 | :term:`SBOM` information is also critical to performing vulnerability exposure | 24 | :term:`SBOM` information is also critical to performing vulnerability exposure |
25 | assessments, as all the components used in the Software Supply Chain are listed. | 25 | assessments, as all the components used in the Software Supply Chain are listed. |
26 | 26 | ||
27 | The OpenEmbedded build system doesn't generate such information by default. | 27 | The OpenEmbedded build system doesn't generate such information by default, |
28 | To make this happen, you must inherit the | 28 | though the :term:`Poky` reference distribution has it enabled out of the box. |
29 | :ref:`ref-classes-create-spdx` class from a configuration file:: | 29 | |
30 | To enable it, inherit the :ref:`ref-classes-create-spdx` class from a | ||
31 | configuration file:: | ||
30 | 32 | ||
31 | INHERIT += "create-spdx" | 33 | INHERIT += "create-spdx" |
32 | 34 | ||
35 | In the :term:`Poky` reference distribution, :term:`SPDX` generation does | ||
36 | consume some build time resources and thus if needed it can be disabled from a | ||
37 | :term:`configuration file`:: | ||
38 | |||
39 | INHERIT:remove = "create-spdx" | ||
40 | |||
33 | Upon building an image, you will then get: | 41 | Upon building an image, you will then get: |
34 | 42 | ||
35 | - :term:`SPDX` output in JSON format as an ``IMAGE-MACHINE.spdx.json`` file in | 43 | - :term:`SPDX` output in JSON format as an ``IMAGE-MACHINE.spdx.json`` file in |
@@ -52,6 +60,9 @@ more information in the output :term:`SPDX` data: | |||
52 | - Add a description of the source files used to generate host tools and target | 60 | - Add a description of the source files used to generate host tools and target |
53 | packages (:term:`SPDX_INCLUDE_SOURCES`) | 61 | packages (:term:`SPDX_INCLUDE_SOURCES`) |
54 | 62 | ||
63 | - Add a description of the **compiled** source files used to generate host tools | ||
64 | and target packages (:term:`SPDX_INCLUDE_COMPILED_SOURCES`) | ||
65 | |||
55 | - Add archives of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`). | 66 | - Add archives of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`). |
56 | 67 | ||
57 | Though the toplevel :term:`SPDX` output is available in | 68 | Though the toplevel :term:`SPDX` output is available in |
diff --git a/documentation/dev-manual/securing-images.rst b/documentation/dev-manual/securing-images.rst index e5791d3d6d..f4b528e559 100644 --- a/documentation/dev-manual/securing-images.rst +++ b/documentation/dev-manual/securing-images.rst | |||
@@ -107,18 +107,18 @@ Considerations Specific to the OpenEmbedded Build System | |||
107 | You can take some steps that are specific to the OpenEmbedded build | 107 | You can take some steps that are specific to the OpenEmbedded build |
108 | system to make your images more secure: | 108 | system to make your images more secure: |
109 | 109 | ||
110 | - Ensure "debug-tweaks" is not one of your selected | 110 | - Ensure that "allow-empty-password", "allow-root-login", or |
111 | :term:`IMAGE_FEATURES`. | 111 | "empty-root-password" are not one of your selected :term:`IMAGE_FEATURES`. |
112 | When creating a new project, the default is to provide you with an | 112 | When creating a new project, the default is to provide you with an |
113 | initial ``local.conf`` file that enables this feature using the | 113 | initial ``local.conf`` file that enables these features using the |
114 | :term:`EXTRA_IMAGE_FEATURES` | 114 | :term:`EXTRA_IMAGE_FEATURES` |
115 | variable with the line:: | 115 | variable with the line:: |
116 | 116 | ||
117 | EXTRA_IMAGE_FEATURES = "debug-tweaks" | 117 | EXTRA_IMAGE_FEATURES = "allow-empty-password empty-root-password allow-root-login" |
118 | 118 | ||
119 | To disable that feature, simply comment out that line in your | 119 | To disable these features, simply comment out that line in your |
120 | ``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain | 120 | ``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain |
121 | "debug-tweaks" before producing your final image. Among other things, | 121 | any of these features before producing your final image. Among other things, |
122 | leaving this in place sets the root password as blank, which makes | 122 | leaving this in place sets the root password as blank, which makes |
123 | logging in for debugging or inspection easy during development but | 123 | logging in for debugging or inspection easy during development but |
124 | also means anyone can easily log in during production. | 124 | also means anyone can easily log in during production. |
diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst index 8539bc0889..44bd2de137 100644 --- a/documentation/dev-manual/start.rst +++ b/documentation/dev-manual/start.rst | |||
@@ -109,7 +109,7 @@ particular working environment and set of practices. | |||
109 | 109 | ||
110 | - Keep your cross-development toolchains updated. You can do this | 110 | - Keep your cross-development toolchains updated. You can do this |
111 | through provisioning either as new toolchain downloads or as | 111 | through provisioning either as new toolchain downloads or as |
112 | updates through a package update mechanism using ``opkg`` to | 112 | updates through a package update mechanism to |
113 | provide updates to an existing toolchain. The exact mechanics of | 113 | provide updates to an existing toolchain. The exact mechanics of |
114 | how and when to do this depend on local policy. | 114 | how and when to do this depend on local policy. |
115 | 115 | ||
@@ -159,7 +159,7 @@ particular working environment and set of practices. | |||
159 | are made. | 159 | are made. |
160 | 160 | ||
161 | - Allows triggering of automated image booting and testing under | 161 | - Allows triggering of automated image booting and testing under |
162 | the QuickEMUlator (QEMU). | 162 | the Quick EMUlator (QEMU). |
163 | 163 | ||
164 | - Supports incremental build testing and from-scratch builds. | 164 | - Supports incremental build testing and from-scratch builds. |
165 | 165 | ||
@@ -234,7 +234,7 @@ particular working environment and set of practices. | |||
234 | - The Yocto Project community encourages you to send patches to the | 234 | - The Yocto Project community encourages you to send patches to the |
235 | project to fix bugs or add features. If you do submit patches, | 235 | project to fix bugs or add features. If you do submit patches, |
236 | follow the project commit guidelines for writing good commit | 236 | follow the project commit guidelines for writing good commit |
237 | messages. See the ":doc:`../contributor-guide/submit-changes`" | 237 | messages. See the ":doc:`/contributor-guide/submit-changes`" |
238 | section in the Yocto Project and OpenEmbedded Contributor Guide. | 238 | section in the Yocto Project and OpenEmbedded Contributor Guide. |
239 | 239 | ||
240 | - Send changes to the core sooner than later as others are likely | 240 | - Send changes to the core sooner than later as others are likely |
@@ -310,7 +310,7 @@ Project Build Host: | |||
310 | 310 | ||
311 | - GNU make &MIN_MAKE_VERSION; or greater | 311 | - GNU make &MIN_MAKE_VERSION; or greater |
312 | 312 | ||
313 | If your build host does not meet any of these listed version | 313 | If your build host does not satisfy all of these listed version |
314 | requirements, you can take steps to prepare the system so that you | 314 | requirements, you can take steps to prepare the system so that you |
315 | can still use the Yocto Project. See the | 315 | can still use the Yocto Project. See the |
316 | ":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`" | 316 | ":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`" |
@@ -568,7 +568,7 @@ extension accordingly. | |||
568 | Locating Yocto Project Source Files | 568 | Locating Yocto Project Source Files |
569 | =================================== | 569 | =================================== |
570 | 570 | ||
571 | This section shows you how to locate, fetch and configure the source | 571 | This section shows you how to locate, fetch, unpack, patch and configure the source |
572 | files you'll need to work with the Yocto Project. | 572 | files you'll need to work with the Yocto Project. |
573 | 573 | ||
574 | .. note:: | 574 | .. note:: |
@@ -615,11 +615,11 @@ Accessing Source Archives | |||
615 | The Yocto Project also provides source archives of its releases, which | 615 | The Yocto Project also provides source archives of its releases, which |
616 | are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory | 616 | are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory |
617 | containing the release you wish to use, for example | 617 | containing the release you wish to use, for example |
618 | :yocto_dl:`yocto-&DISTRO; </releases/yocto/yocto-&DISTRO;/>`. | 618 | :yocto_dl:`&DISTRO_REL_LATEST_TAG; </releases/yocto/&DISTRO_REL_LATEST_TAG;/>`. |
619 | 619 | ||
620 | You will find there source archives of individual components (if you wish | 620 | You will find there source archives of individual components (if you wish |
621 | to use them individually), and of the corresponding Poky release bundling | 621 | to use them individually), and of the corresponding Poky release bundling |
622 | a selection of these components. | 622 | a selection of these components. |
623 | 623 | ||
624 | .. note:: | 624 | .. note:: |
625 | 625 | ||
@@ -720,11 +720,11 @@ Follow these steps to create a local version of the upstream | |||
720 | $ git branch | 720 | $ git branch |
721 | * master | 721 | * master |
722 | 722 | ||
723 | Your local repository of poky is identical to the | 723 | Your local repository of poky is initially identical to the |
724 | upstream poky repository at the time from which it was cloned. As you | 724 | upstream poky repository from which it was cloned. As you |
725 | work with the local branch, you can periodically use the | 725 | work with the local branch, you can periodically use the |
726 | ``git pull --rebase`` command to be sure you are up-to-date | 726 | ``git pull`` command to be sure you stay up-to-date |
727 | with the upstream branch. | 727 | with the upstream poky branch. |
728 | 728 | ||
729 | Checking Out by Branch in Poky | 729 | Checking Out by Branch in Poky |
730 | ------------------------------ | 730 | ------------------------------ |
@@ -853,3 +853,14 @@ similar to checking out by branch name except you use tag names. | |||
853 | ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;" | 853 | ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;" |
854 | development branch at the point where Yocto Project &DISTRO; was | 854 | development branch at the point where Yocto Project &DISTRO; was |
855 | released. | 855 | released. |
856 | |||
857 | Initializing the Build Environment | ||
858 | ================================== | ||
859 | |||
860 | Before you can use Yocto you need to setup the build environment. | ||
861 | From within the ``poky`` directory, source the :ref:`ref-manual/structure:``oe-init-build-env``` environment | ||
862 | setup script to define Yocto Project's build environment on your build host:: | ||
863 | |||
864 | $ source oe-init-build-env | ||
865 | |||
866 | Note, that this step will have to be repeated every time you open a new shell. | ||
diff --git a/documentation/dev-manual/temporary-source-code.rst b/documentation/dev-manual/temporary-source-code.rst index 08bf68d982..9a7cd0f771 100644 --- a/documentation/dev-manual/temporary-source-code.rst +++ b/documentation/dev-manual/temporary-source-code.rst | |||
@@ -18,11 +18,10 @@ build packages is available in the :term:`Build Directory` as defined by the | |||
18 | defined in the ``meta/conf/bitbake.conf`` configuration file in the | 18 | defined in the ``meta/conf/bitbake.conf`` configuration file in the |
19 | :term:`Source Directory`:: | 19 | :term:`Source Directory`:: |
20 | 20 | ||
21 | S = "${WORKDIR}/${BP}" | 21 | S = "${UNPACKDIR}/${BP}" |
22 | 22 | ||
23 | You should be aware that many recipes override the | 23 | You should be aware that many recipes override the |
24 | :term:`S` variable. For example, recipes that fetch their source from Git | 24 | :term:`S` variable when the default isn't accurate. |
25 | usually set :term:`S` to ``${WORKDIR}/git``. | ||
26 | 25 | ||
27 | .. note:: | 26 | .. note:: |
28 | 27 | ||
@@ -31,8 +30,16 @@ usually set :term:`S` to ``${WORKDIR}/git``. | |||
31 | 30 | ||
32 | BP = "${BPN}-${PV}" | 31 | BP = "${BPN}-${PV}" |
33 | 32 | ||
33 | This matches the location that the git fetcher unpacks to, and usually | ||
34 | matches unpacked content of release tarballs (e.g. they contain a single | ||
35 | directory which matches value of ${BP} exactly). | ||
34 | 36 | ||
35 | The path to the work directory for the recipe | 37 | The path to the unpack directory for the recipe |
38 | (:term:`UNPACKDIR`) is defined as follows:: | ||
39 | |||
40 | ${WORKDIR}/sources | ||
41 | |||
42 | In turn, the path to the work directory for the recipe | ||
36 | (:term:`WORKDIR`) is defined as | 43 | (:term:`WORKDIR`) is defined as |
37 | follows:: | 44 | follows:: |
38 | 45 | ||
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst index 4fac78bdfb..2e65678679 100644 --- a/documentation/dev-manual/upgrading-recipes.rst +++ b/documentation/dev-manual/upgrading-recipes.rst | |||
@@ -203,7 +203,7 @@ As mentioned earlier, an alternative method for upgrading recipes to | |||
203 | newer versions is to use | 203 | newer versions is to use |
204 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. | 204 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. |
205 | You can read about ``devtool upgrade`` in general in the | 205 | You can read about ``devtool upgrade`` in general in the |
206 | ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" | 206 | ":ref:`dev-manual/devtool:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" |
207 | section in the Yocto Project Application Development and the Extensible | 207 | section in the Yocto Project Application Development and the Extensible |
208 | Software Development Kit (eSDK) Manual. | 208 | Software Development Kit (eSDK) Manual. |
209 | 209 | ||
@@ -333,7 +333,7 @@ Manually Upgrading a Recipe | |||
333 | 333 | ||
334 | If for some reason you choose not to upgrade recipes using | 334 | If for some reason you choose not to upgrade recipes using |
335 | :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or | 335 | :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or |
336 | by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, | 336 | by :ref:`dev-manual/upgrading-recipes:Using ``devtool upgrade```, |
337 | you can manually edit the recipe files to upgrade the versions. | 337 | you can manually edit the recipe files to upgrade the versions. |
338 | 338 | ||
339 | .. note:: | 339 | .. note:: |
diff --git a/documentation/dev-manual/vulnerabilities.rst b/documentation/dev-manual/vulnerabilities.rst index 1bc2a85929..5331a63991 100644 --- a/documentation/dev-manual/vulnerabilities.rst +++ b/documentation/dev-manual/vulnerabilities.rst | |||
@@ -22,7 +22,7 @@ issues may be impacting Poky and OE-Core. It is up to the maintainers, users, | |||
22 | contributors and anyone interested in the issues to investigate and possibly fix them by | 22 | contributors and anyone interested in the issues to investigate and possibly fix them by |
23 | updating software components to newer versions or by applying patches to address them. | 23 | updating software components to newer versions or by applying patches to address them. |
24 | It is recommended to work with Poky and OE-Core upstream maintainers and submit | 24 | It is recommended to work with Poky and OE-Core upstream maintainers and submit |
25 | patches to fix them, see ":doc:`../contributor-guide/submit-changes`" for details. | 25 | patches to fix them, see ":doc:`/contributor-guide/submit-changes`" for details. |
26 | 26 | ||
27 | Vulnerability check at build time | 27 | Vulnerability check at build time |
28 | ================================= | 28 | ================================= |
@@ -57,42 +57,86 @@ applied and that the issue needs to be investigated. ``Ignored`` means that afte | |||
57 | analysis, it has been deemed to ignore the issue as it for example affects | 57 | analysis, it has been deemed to ignore the issue as it for example affects |
58 | the software component on a different operating system platform. | 58 | the software component on a different operating system platform. |
59 | 59 | ||
60 | By default, no NVD API key is used to retrieve data from the CVE database, which | ||
61 | results in larger delays between NVD API requests. See the :term:`NVDCVE_API_KEY` | ||
62 | documentation on how to request and set a NVD API key. | ||
63 | |||
60 | After a build with CVE check enabled, reports for each compiled source recipe will be | 64 | After a build with CVE check enabled, reports for each compiled source recipe will be |
61 | found in ``build/tmp/deploy/cve``. | 65 | found in ``build/tmp/deploy/cve``. |
62 | 66 | ||
63 | For example the CVE check report for the ``flex-native`` recipe looks like:: | 67 | For example the CVE check report for the ``flex-native`` recipe looks like:: |
64 | 68 | ||
65 | $ cat poky/build/tmp/deploy/cve/flex-native | 69 | $ cat ./tmp/deploy/cve/flex-native_cve.json |
66 | LAYER: meta | 70 | { |
67 | PACKAGE NAME: flex-native | 71 | "version": "1", |
68 | PACKAGE VERSION: 2.6.4 | 72 | "package": [ |
69 | CVE: CVE-2016-6354 | 73 | { |
70 | CVE STATUS: Patched | 74 | "name": "flex-native", |
71 | CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read. | 75 | "layer": "meta", |
72 | CVSS v2 BASE SCORE: 7.5 | 76 | "version": "2.6.4", |
73 | CVSS v3 BASE SCORE: 9.8 | 77 | "products": [ |
74 | VECTOR: NETWORK | 78 | { |
75 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354 | 79 | "product": "flex", |
76 | 80 | "cvesInRecord": "No" | |
77 | LAYER: meta | 81 | }, |
78 | PACKAGE NAME: flex-native | 82 | { |
79 | PACKAGE VERSION: 2.6.4 | 83 | "product": "flex", |
80 | CVE: CVE-2019-6293 | 84 | "cvesInRecord": "Yes" |
81 | CVE STATUS: Ignored | 85 | } |
82 | CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service. | 86 | ], |
83 | CVSS v2 BASE SCORE: 4.3 | 87 | "issue": [ |
84 | CVSS v3 BASE SCORE: 5.5 | 88 | { |
85 | VECTOR: NETWORK | 89 | "id": "CVE-2006-0459", |
86 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293 | 90 | "status": "Patched", |
91 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2006-0459", | ||
92 | "summary": "flex.skl in Will Estes and John Millaway Fast Lexical Analyzer Generator (flex) before 2.5.33 does not allocate enough memory for grammars containing (1) REJECT statements or (2) trailing context rules, which causes flex to generate code that contains a buffer overflow that might allow context-dependent attackers to execute arbitrary code.", | ||
93 | "scorev2": "7.5", | ||
94 | "scorev3": "0.0", | ||
95 | "scorev4": "0.0", | ||
96 | "modified": "2024-11-21T00:06Z", | ||
97 | "vector": "NETWORK", | ||
98 | "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P", | ||
99 | "detail": "version-not-in-range" | ||
100 | }, | ||
101 | { | ||
102 | "id": "CVE-2016-6354", | ||
103 | "status": "Patched", | ||
104 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2016-6354", | ||
105 | "summary": "Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.", | ||
106 | "scorev2": "7.5", | ||
107 | "scorev3": "9.8", | ||
108 | "scorev4": "0.0", | ||
109 | "modified": "2024-11-21T02:55Z", | ||
110 | "vector": "NETWORK", | ||
111 | "vectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P", | ||
112 | "detail": "version-not-in-range" | ||
113 | }, | ||
114 | { | ||
115 | "id": "CVE-2019-6293", | ||
116 | "status": "Ignored", | ||
117 | "link": "https://nvd.nist.gov/vuln/detail/CVE-2019-6293", | ||
118 | "summary": "An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.", | ||
119 | "scorev2": "4.3", | ||
120 | "scorev3": "5.5", | ||
121 | "scorev4": "0.0", | ||
122 | "modified": "2024-11-21T04:46Z", | ||
123 | "vector": "NETWORK", | ||
124 | "vectorString": "AV:N/AC:M/Au:N/C:N/I:N/A:P", | ||
125 | "detail": "upstream-wontfix", | ||
126 | "description": "there is stack exhaustion but no bug and it is building the parser, not running it, effectively similar to a compiler ICE. Upstream no plans to address this." | ||
127 | } | ||
128 | ] | ||
129 | } | ||
130 | ] | ||
131 | } | ||
87 | 132 | ||
88 | For images, a summary of all recipes included in the image and their CVEs is also | 133 | For images, a summary of all recipes included in the image and their CVEs is also |
89 | generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found | 134 | generated in the JSON format. These ``.json`` reports can be found |
90 | in the ``tmp/deploy/images`` directory for each compiled image. | 135 | in the ``tmp/deploy/images`` directory for each compiled image. |
91 | 136 | ||
92 | At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: | 137 | At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: |
93 | 138 | ||
94 | WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log | 139 | WARNING: qemu-native-9.2.0-r0 do_cve_check: Found unpatched CVE (CVE-2023-1386) |
95 | WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log | ||
96 | 140 | ||
97 | It is also possible to check the CVE status of individual packages as follows:: | 141 | It is also possible to check the CVE status of individual packages as follows:: |
98 | 142 | ||
@@ -111,10 +155,10 @@ upstream `NIST CVE database <https://nvd.nist.gov/>`__. | |||
111 | 155 | ||
112 | The variable supports using vendor and product names like this:: | 156 | The variable supports using vendor and product names like this:: |
113 | 157 | ||
114 | CVE_PRODUCT = "flex_project:flex" | 158 | CVE_PRODUCT = "flex_project:flex westes:flex" |
115 | 159 | ||
116 | In this example the vendor name used in the CVE database is ``flex_project`` and the | 160 | In this example we have two possible vendors names, ``flex_project`` and ``westes``, |
117 | product is ``flex``. With this setting the ``flex`` recipe only maps to this specific | 161 | with the product name ``flex``. With this setting the ``flex`` recipe only maps to this specific |
118 | product and not products from other vendors with same name ``flex``. | 162 | product and not products from other vendors with same name ``flex``. |
119 | 163 | ||
120 | Similarly, when the recipe version :term:`PV` is not compatible with software versions used by | 164 | Similarly, when the recipe version :term:`PV` is not compatible with software versions used by |
diff --git a/documentation/dev-manual/wic.rst b/documentation/dev-manual/wic.rst index 05e9cb381b..fced0e170c 100644 --- a/documentation/dev-manual/wic.rst +++ b/documentation/dev-manual/wic.rst | |||
@@ -139,20 +139,20 @@ individual images through the ``list`` command. You can use the ``list`` | |||
139 | command to return the available Wic images as follows:: | 139 | command to return the available Wic images as follows:: |
140 | 140 | ||
141 | $ wic list images | 141 | $ wic list images |
142 | genericx86 Create an EFI disk image for genericx86* | 142 | genericx86 Create an EFI disk image for genericx86* |
143 | beaglebone-yocto Create SD card image for Beaglebone | 143 | beaglebone-yocto Create SD card image for Beaglebone |
144 | qemuriscv Create qcow2 image for RISC-V QEMU machines | 144 | qemuriscv Create qcow2 image for RISC-V QEMU machines |
145 | mkefidisk Create an EFI disk image | 145 | mkefidisk Create an EFI disk image |
146 | qemuloongarch Create qcow2 image for LoongArch QEMU machines | 146 | qemuloongarch Create qcow2 image for LoongArch QEMU machines |
147 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | 147 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin |
148 | directdisk Create a 'pcbios' direct disk image | 148 | directdisk Create a 'pcbios' direct disk image |
149 | efi-bootdisk | 149 | efi-bootdisk |
150 | mkhybridiso Create a hybrid ISO image | 150 | mkhybridiso Create a hybrid ISO image |
151 | directdisk-gpt Create a 'pcbios' direct disk image | 151 | directdisk-gpt Create a 'pcbios' direct disk image |
152 | systemd-bootdisk Create an EFI disk image with systemd-boot | 152 | systemd-bootdisk Create an EFI disk image with systemd-boot |
153 | sdimage-bootpart Create SD card image with a boot partition | 153 | sdimage-bootpart Create SD card image with a boot partition |
154 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | 154 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image |
155 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | 155 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config |
156 | 156 | ||
157 | Once you know the list of available | 157 | Once you know the list of available |
158 | Wic images, you can use ``help`` with the command to get help on a | 158 | Wic images, you can use ``help`` with the command to get help on a |
@@ -282,20 +282,20 @@ following two locations:: | |||
282 | Use the following command to list the available kickstart files:: | 282 | Use the following command to list the available kickstart files:: |
283 | 283 | ||
284 | $ wic list images | 284 | $ wic list images |
285 | genericx86 Create an EFI disk image for genericx86* | 285 | genericx86 Create an EFI disk image for genericx86* |
286 | beaglebone-yocto Create SD card image for Beaglebone | 286 | beaglebone-yocto Create SD card image for Beaglebone |
287 | qemuriscv Create qcow2 image for RISC-V QEMU machines | 287 | qemuriscv Create qcow2 image for RISC-V QEMU machines |
288 | mkefidisk Create an EFI disk image | 288 | mkefidisk Create an EFI disk image |
289 | qemuloongarch Create qcow2 image for LoongArch QEMU machines | 289 | qemuloongarch Create qcow2 image for LoongArch QEMU machines |
290 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | 290 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin |
291 | directdisk Create a 'pcbios' direct disk image | 291 | directdisk Create a 'pcbios' direct disk image |
292 | efi-bootdisk | 292 | efi-bootdisk |
293 | mkhybridiso Create a hybrid ISO image | 293 | mkhybridiso Create a hybrid ISO image |
294 | directdisk-gpt Create a 'pcbios' direct disk image | 294 | directdisk-gpt Create a 'pcbios' direct disk image |
295 | systemd-bootdisk Create an EFI disk image with systemd-boot | 295 | systemd-bootdisk Create an EFI disk image with systemd-boot |
296 | sdimage-bootpart Create SD card image with a boot partition | 296 | sdimage-bootpart Create SD card image with a boot partition |
297 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | 297 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image |
298 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | 298 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config |
299 | 299 | ||
300 | When you use an existing file, you | 300 | When you use an existing file, you |
301 | do not have to use the ``.wks`` extension. Here is an example in Raw | 301 | do not have to use the ``.wks`` extension. Here is an example in Raw |
@@ -513,7 +513,7 @@ or :: | |||
513 | 513 | ||
514 | For more information on how to use the ``bmaptool`` | 514 | For more information on how to use the ``bmaptool`` |
515 | to flash a device with an image, see the | 515 | to flash a device with an image, see the |
516 | ":ref:`dev-manual/bmaptool:flashing images using \`\`bmaptool\`\``" | 516 | ":ref:`dev-manual/bmaptool:flashing images using \`bmaptool\``" |
517 | section. | 517 | section. |
518 | 518 | ||
519 | Using a Modified Kickstart File | 519 | Using a Modified Kickstart File |
@@ -721,7 +721,7 @@ the existing kernel, and then inserts a new kernel: | |||
721 | 721 | ||
722 | Once the new kernel is added back into the image, you can use the | 722 | Once the new kernel is added back into the image, you can use the |
723 | ``dd`` command or :ref:`bmaptool | 723 | ``dd`` command or :ref:`bmaptool |
724 | <dev-manual/bmaptool:flashing images using \`\`bmaptool\`\`>` | 724 | <dev-manual/bmaptool:flashing images using \`bmaptool\`>` commands |
725 | to flash your wic image onto an SD card or USB stick and test your | 725 | to flash your wic image onto an SD card or USB stick and test your |
726 | target. | 726 | target. |
727 | 727 | ||