diff options
Diffstat (limited to 'documentation/dev-manual/common-tasks.rst')
-rw-r--r-- | documentation/dev-manual/common-tasks.rst | 11587 |
1 files changed, 0 insertions, 11587 deletions
diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst deleted file mode 100644 index 65db4aed33..0000000000 --- a/documentation/dev-manual/common-tasks.rst +++ /dev/null | |||
@@ -1,11587 +0,0 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | ************ | ||
4 | Common Tasks | ||
5 | ************ | ||
6 | |||
7 | This chapter describes fundamental procedures such as creating layers, | ||
8 | adding new software packages, extending or customizing images, porting | ||
9 | work to new hardware (adding a new machine), and so forth. You will find | ||
10 | that the procedures documented here occur often in the development cycle | ||
11 | using the Yocto Project. | ||
12 | |||
13 | Understanding and Creating Layers | ||
14 | ================================= | ||
15 | |||
16 | The OpenEmbedded build system supports organizing | ||
17 | :term:`Metadata` into multiple layers. | ||
18 | Layers allow you to isolate different types of customizations from each | ||
19 | other. For introductory information on the Yocto Project Layer Model, | ||
20 | see the | ||
21 | ":ref:`overview-manual/yp-intro:the yocto project layer model`" | ||
22 | section in the Yocto Project Overview and Concepts Manual. | ||
23 | |||
24 | Creating Your Own Layer | ||
25 | ----------------------- | ||
26 | |||
27 | It is very easy to create your own layers to use with the OpenEmbedded | ||
28 | build system. The Yocto Project ships with tools that speed up creating | ||
29 | layers. This section describes the steps you perform by hand to create | ||
30 | layers so that you can better understand them. For information about the | ||
31 | layer-creation tools, see the | ||
32 | ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`" | ||
33 | section in the Yocto Project Board Support Package (BSP) Developer's | ||
34 | Guide and the ":ref:`dev-manual/common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`" | ||
35 | section further down in this manual. | ||
36 | |||
37 | Follow these general steps to create your layer without using tools: | ||
38 | |||
39 | 1. *Check Existing Layers:* Before creating a new layer, you should be | ||
40 | sure someone has not already created a layer containing the Metadata | ||
41 | you need. You can see the :oe_layerindex:`OpenEmbedded Metadata Index <>` | ||
42 | for a list of layers from the OpenEmbedded community that can be used in | ||
43 | the Yocto Project. You could find a layer that is identical or close | ||
44 | to what you need. | ||
45 | |||
46 | 2. *Create a Directory:* Create the directory for your layer. When you | ||
47 | create the layer, be sure to create the directory in an area not | ||
48 | associated with the Yocto Project :term:`Source Directory` | ||
49 | (e.g. the cloned ``poky`` repository). | ||
50 | |||
51 | While not strictly required, prepend the name of the directory with | ||
52 | the string "meta-". For example: | ||
53 | :: | ||
54 | |||
55 | meta-mylayer | ||
56 | meta-GUI_xyz | ||
57 | meta-mymachine | ||
58 | |||
59 | With rare exceptions, a layer's name follows this form: | ||
60 | :: | ||
61 | |||
62 | meta-root_name | ||
63 | |||
64 | Following this layer naming convention can save | ||
65 | you trouble later when tools, components, or variables "assume" your | ||
66 | layer name begins with "meta-". A notable example is in configuration | ||
67 | files as shown in the following step where layer names without the | ||
68 | "meta-" string are appended to several variables used in the | ||
69 | configuration. | ||
70 | |||
71 | 3. *Create a Layer Configuration File:* Inside your new layer folder, | ||
72 | you need to create a ``conf/layer.conf`` file. It is easiest to take | ||
73 | an existing layer configuration file and copy that to your layer's | ||
74 | ``conf`` directory and then modify the file as needed. | ||
75 | |||
76 | The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project | ||
77 | :yocto_git:`Source Repositories </poky/tree/meta-yocto-bsp/conf>` | ||
78 | demonstrates the required syntax. For your layer, you need to replace | ||
79 | "yoctobsp" with a unique identifier for your layer (e.g. "machinexyz" | ||
80 | for a layer named "meta-machinexyz"): | ||
81 | :: | ||
82 | |||
83 | # We have a conf and classes directory, add to BBPATH | ||
84 | BBPATH .= ":${LAYERDIR}" | ||
85 | |||
86 | # We have recipes-* directories, add to BBFILES | ||
87 | BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ | ||
88 | ${LAYERDIR}/recipes-*/*/*.bbappend" | ||
89 | |||
90 | BBFILE_COLLECTIONS += "yoctobsp" | ||
91 | BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" | ||
92 | BBFILE_PRIORITY_yoctobsp = "5" | ||
93 | LAYERVERSION_yoctobsp = "4" | ||
94 | LAYERSERIES_COMPAT_yoctobsp = "dunfell" | ||
95 | |||
96 | Following is an explanation of the layer configuration file: | ||
97 | |||
98 | - :term:`BBPATH`: Adds the layer's | ||
99 | root directory to BitBake's search path. Through the use of the | ||
100 | ``BBPATH`` variable, BitBake locates class files (``.bbclass``), | ||
101 | configuration files, and files that are included with ``include`` | ||
102 | and ``require`` statements. For these cases, BitBake uses the | ||
103 | first file that matches the name found in ``BBPATH``. This is | ||
104 | similar to the way the ``PATH`` variable is used for binaries. It | ||
105 | is recommended, therefore, that you use unique class and | ||
106 | configuration filenames in your custom layer. | ||
107 | |||
108 | - :term:`BBFILES`: Defines the | ||
109 | location for all recipes in the layer. | ||
110 | |||
111 | - :term:`BBFILE_COLLECTIONS`: | ||
112 | Establishes the current layer through a unique identifier that is | ||
113 | used throughout the OpenEmbedded build system to refer to the | ||
114 | layer. In this example, the identifier "yoctobsp" is the | ||
115 | representation for the container layer named "meta-yocto-bsp". | ||
116 | |||
117 | - :term:`BBFILE_PATTERN`: | ||
118 | Expands immediately during parsing to provide the directory of the | ||
119 | layer. | ||
120 | |||
121 | - :term:`BBFILE_PRIORITY`: | ||
122 | Establishes a priority to use for recipes in the layer when the | ||
123 | OpenEmbedded build finds recipes of the same name in different | ||
124 | layers. | ||
125 | |||
126 | - :term:`LAYERVERSION`: | ||
127 | Establishes a version number for the layer. You can use this | ||
128 | version number to specify this exact version of the layer as a | ||
129 | dependency when using the | ||
130 | :term:`LAYERDEPENDS` | ||
131 | variable. | ||
132 | |||
133 | - :term:`LAYERDEPENDS`: | ||
134 | Lists all layers on which this layer depends (if any). | ||
135 | |||
136 | - :term:`LAYERSERIES_COMPAT`: | ||
137 | Lists the :yocto_wiki:`Yocto Project </Releases>` | ||
138 | releases for which the current version is compatible. This | ||
139 | variable is a good way to indicate if your particular layer is | ||
140 | current. | ||
141 | |||
142 | 4. *Add Content:* Depending on the type of layer, add the content. If | ||
143 | the layer adds support for a machine, add the machine configuration | ||
144 | in a ``conf/machine/`` file within the layer. If the layer adds | ||
145 | distro policy, add the distro configuration in a ``conf/distro/`` | ||
146 | file within the layer. If the layer introduces new recipes, put the | ||
147 | recipes you need in ``recipes-*`` subdirectories within the layer. | ||
148 | |||
149 | .. note:: | ||
150 | |||
151 | For an explanation of layer hierarchy that is compliant with the | ||
152 | Yocto Project, see the ":ref:`bsp-guide/bsp:example filesystem layout`" | ||
153 | section in the Yocto Project Board Support Package (BSP) Developer's Guide. | ||
154 | |||
155 | 5. *Optionally Test for Compatibility:* If you want permission to use | ||
156 | the Yocto Project Compatibility logo with your layer or application | ||
157 | that uses your layer, perform the steps to apply for compatibility. | ||
158 | See the "`Making Sure Your Layer is Compatible With Yocto | ||
159 | Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__" | ||
160 | section for more information. | ||
161 | |||
162 | Following Best Practices When Creating Layers | ||
163 | --------------------------------------------- | ||
164 | |||
165 | To create layers that are easier to maintain and that will not impact | ||
166 | builds for other machines, you should consider the information in the | ||
167 | following list: | ||
168 | |||
169 | - *Avoid "Overlaying" Entire Recipes from Other Layers in Your | ||
170 | Configuration:* In other words, do not copy an entire recipe into | ||
171 | your layer and then modify it. Rather, use an append file | ||
172 | (``.bbappend``) to override only those parts of the original recipe | ||
173 | you need to modify. | ||
174 | |||
175 | - *Avoid Duplicating Include Files:* Use append files (``.bbappend``) | ||
176 | for each recipe that uses an include file. Or, if you are introducing | ||
177 | a new recipe that requires the included file, use the path relative | ||
178 | to the original layer directory to refer to the file. For example, | ||
179 | use ``require recipes-core/``\ `package`\ ``/``\ `file`\ ``.inc`` instead | ||
180 | of ``require`` `file`\ ``.inc``. If you're finding you have to overlay | ||
181 | the include file, it could indicate a deficiency in the include file | ||
182 | in the layer to which it originally belongs. If this is the case, you | ||
183 | should try to address that deficiency instead of overlaying the | ||
184 | include file. For example, you could address this by getting the | ||
185 | maintainer of the include file to add a variable or variables to make | ||
186 | it easy to override the parts needing to be overridden. | ||
187 | |||
188 | - *Structure Your Layers:* Proper use of overrides within append files | ||
189 | and placement of machine-specific files within your layer can ensure | ||
190 | that a build is not using the wrong Metadata and negatively impacting | ||
191 | a build for a different machine. Following are some examples: | ||
192 | |||
193 | - *Modify Variables to Support a Different Machine:* Suppose you | ||
194 | have a layer named ``meta-one`` that adds support for building | ||
195 | machine "one". To do so, you use an append file named | ||
196 | ``base-files.bbappend`` and create a dependency on "foo" by | ||
197 | altering the :term:`DEPENDS` | ||
198 | variable: | ||
199 | :: | ||
200 | |||
201 | DEPENDS = "foo" | ||
202 | |||
203 | The dependency is created during any | ||
204 | build that includes the layer ``meta-one``. However, you might not | ||
205 | want this dependency for all machines. For example, suppose you | ||
206 | are building for machine "two" but your ``bblayers.conf`` file has | ||
207 | the ``meta-one`` layer included. During the build, the | ||
208 | ``base-files`` for machine "two" will also have the dependency on | ||
209 | ``foo``. | ||
210 | |||
211 | To make sure your changes apply only when building machine "one", | ||
212 | use a machine override with the ``DEPENDS`` statement: | ||
213 | :: | ||
214 | |||
215 | DEPENDS_one = "foo" | ||
216 | |||
217 | You should follow the same strategy when using ``_append`` | ||
218 | and ``_prepend`` operations: | ||
219 | :: | ||
220 | |||
221 | DEPENDS_append_one = " foo" | ||
222 | DEPENDS_prepend_one = "foo " | ||
223 | |||
224 | As an actual example, here's a | ||
225 | snippet from the generic kernel include file ``linux-yocto.inc``, | ||
226 | wherein the kernel compile and link options are adjusted in the | ||
227 | case of a subset of the supported architectures: | ||
228 | :: | ||
229 | |||
230 | DEPENDS_append_aarch64 = " libgcc" | ||
231 | KERNEL_CC_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" | ||
232 | KERNEL_LD_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" | ||
233 | |||
234 | DEPENDS_append_nios2 = " libgcc" | ||
235 | KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}" | ||
236 | KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}" | ||
237 | |||
238 | DEPENDS_append_arc = " libgcc" | ||
239 | KERNEL_CC_append_arc = " ${TOOLCHAIN_OPTIONS}" | ||
240 | KERNEL_LD_append_arc = " ${TOOLCHAIN_OPTIONS}" | ||
241 | |||
242 | KERNEL_FEATURES_append_qemuall=" features/debug/printk.scc" | ||
243 | |||
244 | .. note:: | ||
245 | |||
246 | Avoiding "+=" and "=+" and using machine-specific ``_append`` | ||
247 | and ``_prepend`` operations is recommended as well. | ||
248 | |||
249 | - *Place Machine-Specific Files in Machine-Specific Locations:* When | ||
250 | you have a base recipe, such as ``base-files.bb``, that contains a | ||
251 | :term:`SRC_URI` statement to a | ||
252 | file, you can use an append file to cause the build to use your | ||
253 | own version of the file. For example, an append file in your layer | ||
254 | at ``meta-one/recipes-core/base-files/base-files.bbappend`` could | ||
255 | extend :term:`FILESPATH` using :term:`FILESEXTRAPATHS` as follows: | ||
256 | :: | ||
257 | |||
258 | FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" | ||
259 | |||
260 | The build for machine "one" will pick up your machine-specific file as | ||
261 | long as you have the file in | ||
262 | ``meta-one/recipes-core/base-files/base-files/``. However, if you | ||
263 | are building for a different machine and the ``bblayers.conf`` | ||
264 | file includes the ``meta-one`` layer and the location of your | ||
265 | machine-specific file is the first location where that file is | ||
266 | found according to ``FILESPATH``, builds for all machines will | ||
267 | also use that machine-specific file. | ||
268 | |||
269 | You can make sure that a machine-specific file is used for a | ||
270 | particular machine by putting the file in a subdirectory specific | ||
271 | to the machine. For example, rather than placing the file in | ||
272 | ``meta-one/recipes-core/base-files/base-files/`` as shown above, | ||
273 | put it in ``meta-one/recipes-core/base-files/base-files/one/``. | ||
274 | Not only does this make sure the file is used only when building | ||
275 | for machine "one", but the build process locates the file more | ||
276 | quickly. | ||
277 | |||
278 | In summary, you need to place all files referenced from | ||
279 | ``SRC_URI`` in a machine-specific subdirectory within the layer in | ||
280 | order to restrict those files to machine-specific builds. | ||
281 | |||
282 | - *Perform Steps to Apply for Yocto Project Compatibility:* If you want | ||
283 | permission to use the Yocto Project Compatibility logo with your | ||
284 | layer or application that uses your layer, perform the steps to apply | ||
285 | for compatibility. See the "`Making Sure Your Layer is Compatible | ||
286 | With Yocto | ||
287 | Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__" | ||
288 | section for more information. | ||
289 | |||
290 | - *Follow the Layer Naming Convention:* Store custom layers in a Git | ||
291 | repository that use the ``meta-layer_name`` format. | ||
292 | |||
293 | - *Group Your Layers Locally:* Clone your repository alongside other | ||
294 | cloned ``meta`` directories from the :term:`Source Directory`. | ||
295 | |||
296 | Making Sure Your Layer is Compatible With Yocto Project | ||
297 | ------------------------------------------------------- | ||
298 | |||
299 | When you create a layer used with the Yocto Project, it is advantageous | ||
300 | to make sure that the layer interacts well with existing Yocto Project | ||
301 | layers (i.e. the layer is compatible with the Yocto Project). Ensuring | ||
302 | compatibility makes the layer easy to be consumed by others in the Yocto | ||
303 | Project community and could allow you permission to use the Yocto | ||
304 | Project Compatible Logo. | ||
305 | |||
306 | .. note:: | ||
307 | |||
308 | Only Yocto Project member organizations are permitted to use the | ||
309 | Yocto Project Compatible Logo. The logo is not available for general | ||
310 | use. For information on how to become a Yocto Project member | ||
311 | organization, see the :yocto_home:`Yocto Project Website <>`. | ||
312 | |||
313 | The Yocto Project Compatibility Program consists of a layer application | ||
314 | process that requests permission to use the Yocto Project Compatibility | ||
315 | Logo for your layer and application. The process consists of two parts: | ||
316 | |||
317 | 1. Successfully passing a script (``yocto-check-layer``) that when run | ||
318 | against your layer, tests it against constraints based on experiences | ||
319 | of how layers have worked in the real world and where pitfalls have | ||
320 | been found. Getting a "PASS" result from the script is required for | ||
321 | successful compatibility registration. | ||
322 | |||
323 | 2. Completion of an application acceptance form, which you can find at | ||
324 | :yocto_home:`/webform/yocto-project-compatible-registration`. | ||
325 | |||
326 | To be granted permission to use the logo, you need to satisfy the | ||
327 | following: | ||
328 | |||
329 | - Be able to check the box indicating that you got a "PASS" when | ||
330 | running the script against your layer. | ||
331 | |||
332 | - Answer "Yes" to the questions on the form or have an acceptable | ||
333 | explanation for any questions answered "No". | ||
334 | |||
335 | - Be a Yocto Project Member Organization. | ||
336 | |||
337 | The remainder of this section presents information on the registration | ||
338 | form and on the ``yocto-check-layer`` script. | ||
339 | |||
340 | Yocto Project Compatible Program Application | ||
341 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
342 | |||
343 | Use the form to apply for your layer's approval. Upon successful | ||
344 | application, you can use the Yocto Project Compatibility Logo with your | ||
345 | layer and the application that uses your layer. | ||
346 | |||
347 | To access the form, use this link: | ||
348 | :yocto_home:`/webform/yocto-project-compatible-registration`. | ||
349 | Follow the instructions on the form to complete your application. | ||
350 | |||
351 | The application consists of the following sections: | ||
352 | |||
353 | - *Contact Information:* Provide your contact information as the fields | ||
354 | require. Along with your information, provide the released versions | ||
355 | of the Yocto Project for which your layer is compatible. | ||
356 | |||
357 | - *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the | ||
358 | items in the checklist. Space exists at the bottom of the form for | ||
359 | any explanations for items for which you answered "No". | ||
360 | |||
361 | - *Recommendations:* Provide answers for the questions regarding Linux | ||
362 | kernel use and build success. | ||
363 | |||
364 | ``yocto-check-layer`` Script | ||
365 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
366 | |||
367 | The ``yocto-check-layer`` script provides you a way to assess how | ||
368 | compatible your layer is with the Yocto Project. You should run this | ||
369 | script prior to using the form to apply for compatibility as described | ||
370 | in the previous section. You need to achieve a "PASS" result in order to | ||
371 | have your application form successfully processed. | ||
372 | |||
373 | The script divides tests into three areas: COMMON, BSP, and DISTRO. For | ||
374 | example, given a distribution layer (DISTRO), the layer must pass both | ||
375 | the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP | ||
376 | layer, the layer must pass the COMMON and BSP set of tests. | ||
377 | |||
378 | To execute the script, enter the following commands from your build | ||
379 | directory: | ||
380 | :: | ||
381 | |||
382 | $ source oe-init-build-env | ||
383 | $ yocto-check-layer your_layer_directory | ||
384 | |||
385 | Be sure to provide the actual directory for your | ||
386 | layer as part of the command. | ||
387 | |||
388 | Entering the command causes the script to determine the type of layer | ||
389 | and then to execute a set of specific tests against the layer. The | ||
390 | following list overviews the test: | ||
391 | |||
392 | - ``common.test_readme``: Tests if a ``README`` file exists in the | ||
393 | layer and the file is not empty. | ||
394 | |||
395 | - ``common.test_parse``: Tests to make sure that BitBake can parse the | ||
396 | files without error (i.e. ``bitbake -p``). | ||
397 | |||
398 | - ``common.test_show_environment``: Tests that the global or per-recipe | ||
399 | environment is in order without errors (i.e. ``bitbake -e``). | ||
400 | |||
401 | - ``common.test_world``: Verifies that ``bitbake world`` works. | ||
402 | |||
403 | - ``common.test_signatures``: Tests to be sure that BSP and DISTRO | ||
404 | layers do not come with recipes that change signatures. | ||
405 | |||
406 | - ``common.test_layerseries_compat``: Verifies layer compatibility is | ||
407 | set properly. | ||
408 | |||
409 | - ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine | ||
410 | configurations. | ||
411 | |||
412 | - ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not | ||
413 | set the machine when the layer is added. | ||
414 | |||
415 | - ``bsp.test_machine_world``: Verifies that ``bitbake world`` works | ||
416 | regardless of which machine is selected. | ||
417 | |||
418 | - ``bsp.test_machine_signatures``: Verifies that building for a | ||
419 | particular machine affects only the signature of tasks specific to | ||
420 | that machine. | ||
421 | |||
422 | - ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has | ||
423 | distro configurations. | ||
424 | |||
425 | - ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer | ||
426 | does not set the distribution when the layer is added. | ||
427 | |||
428 | Enabling Your Layer | ||
429 | ------------------- | ||
430 | |||
431 | Before the OpenEmbedded build system can use your new layer, you need to | ||
432 | enable it. To enable your layer, simply add your layer's path to the | ||
433 | ``BBLAYERS`` variable in your ``conf/bblayers.conf`` file, which is | ||
434 | found in the :term:`Build Directory`. | ||
435 | The following example shows how to enable a layer named | ||
436 | ``meta-mylayer``: | ||
437 | :: | ||
438 | |||
439 | # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf | ||
440 | # changes incompatibly | ||
441 | POKY_BBLAYERS_CONF_VERSION = "2" | ||
442 | BBPATH = "${TOPDIR}" | ||
443 | BBFILES ?= "" | ||
444 | BBLAYERS ?= " \ | ||
445 | /home/user/poky/meta \ | ||
446 | /home/user/poky/meta-poky \ | ||
447 | /home/user/poky/meta-yocto-bsp \ | ||
448 | /home/user/poky/meta-mylayer \ | ||
449 | " | ||
450 | |||
451 | BitBake parses each ``conf/layer.conf`` file from the top down as | ||
452 | specified in the ``BBLAYERS`` variable within the ``conf/bblayers.conf`` | ||
453 | file. During the processing of each ``conf/layer.conf`` file, BitBake | ||
454 | adds the recipes, classes and configurations contained within the | ||
455 | particular layer to the source directory. | ||
456 | |||
457 | Using .bbappend Files in Your Layer | ||
458 | ----------------------------------- | ||
459 | |||
460 | A recipe that appends Metadata to another recipe is called a BitBake | ||
461 | append file. A BitBake append file uses the ``.bbappend`` file type | ||
462 | suffix, while the corresponding recipe to which Metadata is being | ||
463 | appended uses the ``.bb`` file type suffix. | ||
464 | |||
465 | You can use a ``.bbappend`` file in your layer to make additions or | ||
466 | changes to the content of another layer's recipe without having to copy | ||
467 | the other layer's recipe into your layer. Your ``.bbappend`` file | ||
468 | resides in your layer, while the main ``.bb`` recipe file to which you | ||
469 | are appending Metadata resides in a different layer. | ||
470 | |||
471 | Being able to append information to an existing recipe not only avoids | ||
472 | duplication, but also automatically applies recipe changes from a | ||
473 | different layer into your layer. If you were copying recipes, you would | ||
474 | have to manually merge changes as they occur. | ||
475 | |||
476 | When you create an append file, you must use the same root name as the | ||
477 | corresponding recipe file. For example, the append file | ||
478 | ``someapp_3.1.bbappend`` must apply to ``someapp_3.1.bb``. This | ||
479 | means the original recipe and append file names are version | ||
480 | number-specific. If the corresponding recipe is renamed to update to a | ||
481 | newer version, you must also rename and possibly update the | ||
482 | corresponding ``.bbappend`` as well. During the build process, BitBake | ||
483 | displays an error on starting if it detects a ``.bbappend`` file that | ||
484 | does not have a corresponding recipe with a matching name. See the | ||
485 | :term:`BB_DANGLINGAPPENDS_WARNONLY` | ||
486 | variable for information on how to handle this error. | ||
487 | |||
488 | As an example, consider the main formfactor recipe and a corresponding | ||
489 | formfactor append file both from the :term:`Source Directory`. | ||
490 | Here is the main | ||
491 | formfactor recipe, which is named ``formfactor_0.0.bb`` and located in | ||
492 | the "meta" layer at ``meta/recipes-bsp/formfactor``: | ||
493 | :: | ||
494 | |||
495 | SUMMARY = "Device formfactor information" | ||
496 | DESCRIPTION = "A formfactor configuration file provides information about the \ | ||
497 | target hardware for which the image is being built and information that the \ | ||
498 | build system cannot obtain from other sources such as the kernel." | ||
499 | SECTION = "base" | ||
500 | LICENSE = "MIT" | ||
501 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" | ||
502 | PR = "r45" | ||
503 | |||
504 | SRC_URI = "file://config file://machconfig" | ||
505 | S = "${WORKDIR}" | ||
506 | |||
507 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
508 | INHIBIT_DEFAULT_DEPS = "1" | ||
509 | |||
510 | do_install() { | ||
511 | # Install file only if it has contents | ||
512 | install -d ${D}${sysconfdir}/formfactor/ | ||
513 | install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ | ||
514 | if [ -s "${S}/machconfig" ]; then | ||
515 | install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ | ||
516 | fi | ||
517 | } | ||
518 | |||
519 | In the main recipe, note the :term:`SRC_URI` | ||
520 | variable, which tells the OpenEmbedded build system where to find files | ||
521 | during the build. | ||
522 | |||
523 | Following is the append file, which is named ``formfactor_0.0.bbappend`` | ||
524 | and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The | ||
525 | file is in the layer at ``recipes-bsp/formfactor``: | ||
526 | :: | ||
527 | |||
528 | FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" | ||
529 | |||
530 | By default, the build system uses the | ||
531 | :term:`FILESPATH` variable to | ||
532 | locate files. This append file extends the locations by setting the | ||
533 | :term:`FILESEXTRAPATHS` | ||
534 | variable. Setting this variable in the ``.bbappend`` file is the most | ||
535 | reliable and recommended method for adding directories to the search | ||
536 | path used by the build system to find files. | ||
537 | |||
538 | The statement in this example extends the directories to include | ||
539 | ``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``, | ||
540 | which resolves to a directory named ``formfactor`` in the same directory | ||
541 | in which the append file resides (i.e. | ||
542 | ``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must | ||
543 | have the supporting directory structure set up that will contain any | ||
544 | files or patches you will be including from the layer. | ||
545 | |||
546 | Using the immediate expansion assignment operator ``:=`` is important | ||
547 | because of the reference to ``THISDIR``. The trailing colon character is | ||
548 | important as it ensures that items in the list remain colon-separated. | ||
549 | |||
550 | .. note:: | ||
551 | |||
552 | BitBake automatically defines the ``THISDIR`` variable. You should | ||
553 | never set this variable yourself. Using "_prepend" as part of the | ||
554 | ``FILESEXTRAPATHS`` ensures your path will be searched prior to other | ||
555 | paths in the final list. | ||
556 | |||
557 | Also, not all append files add extra files. Many append files simply | ||
558 | exist to add build options (e.g. ``systemd``). For these cases, your | ||
559 | append file would not even use the ``FILESEXTRAPATHS`` statement. | ||
560 | |||
561 | Prioritizing Your Layer | ||
562 | ----------------------- | ||
563 | |||
564 | Each layer is assigned a priority value. Priority values control which | ||
565 | layer takes precedence if there are recipe files with the same name in | ||
566 | multiple layers. For these cases, the recipe file from the layer with a | ||
567 | higher priority number takes precedence. Priority values also affect the | ||
568 | order in which multiple ``.bbappend`` files for the same recipe are | ||
569 | applied. You can either specify the priority manually, or allow the | ||
570 | build system to calculate it based on the layer's dependencies. | ||
571 | |||
572 | To specify the layer's priority manually, use the | ||
573 | :term:`BBFILE_PRIORITY` | ||
574 | variable and append the layer's root name: | ||
575 | :: | ||
576 | |||
577 | BBFILE_PRIORITY_mylayer = "1" | ||
578 | |||
579 | .. note:: | ||
580 | |||
581 | It is possible for a recipe with a lower version number | ||
582 | :term:`PV` in a layer that has a higher | ||
583 | priority to take precedence. | ||
584 | |||
585 | Also, the layer priority does not currently affect the precedence | ||
586 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake | ||
587 | might address this. | ||
588 | |||
589 | Managing Layers | ||
590 | --------------- | ||
591 | |||
592 | You can use the BitBake layer management tool ``bitbake-layers`` to | ||
593 | provide a view into the structure of recipes across a multi-layer | ||
594 | project. Being able to generate output that reports on configured layers | ||
595 | with their paths and priorities and on ``.bbappend`` files and their | ||
596 | applicable recipes can help to reveal potential problems. | ||
597 | |||
598 | For help on the BitBake layer management tool, use the following | ||
599 | command: | ||
600 | :: | ||
601 | |||
602 | $ bitbake-layers --help | ||
603 | NOTE: Starting bitbake server... | ||
604 | usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ... | ||
605 | |||
606 | BitBake layers utility | ||
607 | |||
608 | optional arguments: | ||
609 | -d, --debug Enable debug output | ||
610 | -q, --quiet Print only errors | ||
611 | -F, --force Force add without recipe parse verification | ||
612 | --color COLOR Colorize output (where COLOR is auto, always, never) | ||
613 | -h, --help show this help message and exit | ||
614 | |||
615 | subcommands: | ||
616 | <subcommand> | ||
617 | layerindex-fetch Fetches a layer from a layer index along with its | ||
618 | dependent layers, and adds them to conf/bblayers.conf. | ||
619 | layerindex-show-depends | ||
620 | Find layer dependencies from layer index. | ||
621 | add-layer Add one or more layers to bblayers.conf. | ||
622 | remove-layer Remove one or more layers from bblayers.conf. | ||
623 | flatten flatten layer configuration into a separate output | ||
624 | directory. | ||
625 | show-layers show current configured layers. | ||
626 | show-overlayed list overlayed recipes (where the same recipe exists | ||
627 | in another layer) | ||
628 | show-recipes list available recipes, showing the layer they are | ||
629 | provided by | ||
630 | show-appends list bbappend files and recipe files they apply to | ||
631 | show-cross-depends Show dependencies between recipes that cross layer | ||
632 | boundaries. | ||
633 | create-layer Create a basic layer | ||
634 | |||
635 | Use bitbake-layers <subcommand> --help to get help on a specific command | ||
636 | |||
637 | The following list describes the available commands: | ||
638 | |||
639 | - ``help:`` Displays general help or help on a specified command. | ||
640 | |||
641 | - ``show-layers:`` Shows the current configured layers. | ||
642 | |||
643 | - ``show-overlayed:`` Lists overlayed recipes. A recipe is overlayed | ||
644 | when a recipe with the same name exists in another layer that has a | ||
645 | higher layer priority. | ||
646 | |||
647 | - ``show-recipes:`` Lists available recipes and the layers that | ||
648 | provide them. | ||
649 | |||
650 | - ``show-appends:`` Lists ``.bbappend`` files and the recipe files to | ||
651 | which they apply. | ||
652 | |||
653 | - ``show-cross-depends:`` Lists dependency relationships between | ||
654 | recipes that cross layer boundaries. | ||
655 | |||
656 | - ``add-layer:`` Adds a layer to ``bblayers.conf``. | ||
657 | |||
658 | - ``remove-layer:`` Removes a layer from ``bblayers.conf`` | ||
659 | |||
660 | - ``flatten:`` Flattens the layer configuration into a separate | ||
661 | output directory. Flattening your layer configuration builds a | ||
662 | "flattened" directory that contains the contents of all layers, with | ||
663 | any overlayed recipes removed and any ``.bbappend`` files appended to | ||
664 | the corresponding recipes. You might have to perform some manual | ||
665 | cleanup of the flattened layer as follows: | ||
666 | |||
667 | - Non-recipe files (such as patches) are overwritten. The flatten | ||
668 | command shows a warning for these files. | ||
669 | |||
670 | - Anything beyond the normal layer setup has been added to the | ||
671 | ``layer.conf`` file. Only the lowest priority layer's | ||
672 | ``layer.conf`` is used. | ||
673 | |||
674 | - Overridden and appended items from ``.bbappend`` files need to be | ||
675 | cleaned up. The contents of each ``.bbappend`` end up in the | ||
676 | flattened recipe. However, if there are appended or changed | ||
677 | variable values, you need to tidy these up yourself. Consider the | ||
678 | following example. Here, the ``bitbake-layers`` command adds the | ||
679 | line ``#### bbappended ...`` so that you know where the following | ||
680 | lines originate: | ||
681 | :: | ||
682 | |||
683 | ... | ||
684 | DESCRIPTION = "A useful utility" | ||
685 | ... | ||
686 | EXTRA_OECONF = "--enable-something" | ||
687 | ... | ||
688 | |||
689 | #### bbappended from meta-anotherlayer #### | ||
690 | |||
691 | DESCRIPTION = "Customized utility" | ||
692 | EXTRA_OECONF += "--enable-somethingelse" | ||
693 | |||
694 | |||
695 | Ideally, you would tidy up these utilities as follows: | ||
696 | :: | ||
697 | |||
698 | ... | ||
699 | DESCRIPTION = "Customized utility" | ||
700 | ... | ||
701 | EXTRA_OECONF = "--enable-something --enable-somethingelse" | ||
702 | ... | ||
703 | |||
704 | - ``layerindex-fetch``: Fetches a layer from a layer index, along | ||
705 | with its dependent layers, and adds the layers to the | ||
706 | ``conf/bblayers.conf`` file. | ||
707 | |||
708 | - ``layerindex-show-depends``: Finds layer dependencies from the | ||
709 | layer index. | ||
710 | |||
711 | - ``create-layer``: Creates a basic layer. | ||
712 | |||
713 | Creating a General Layer Using the ``bitbake-layers`` Script | ||
714 | ------------------------------------------------------------ | ||
715 | |||
716 | The ``bitbake-layers`` script with the ``create-layer`` subcommand | ||
717 | simplifies creating a new general layer. | ||
718 | |||
719 | .. note:: | ||
720 | |||
721 | - For information on BSP layers, see the ":ref:`bsp-guide/bsp:bsp layers`" | ||
722 | section in the Yocto | ||
723 | Project Board Specific (BSP) Developer's Guide. | ||
724 | |||
725 | - In order to use a layer with the OpenEmbedded build system, you | ||
726 | need to add the layer to your ``bblayers.conf`` configuration | ||
727 | file. See the ":ref:`dev-manual/common-tasks:adding a layer using the \`\`bitbake-layers\`\` script`" | ||
728 | section for more information. | ||
729 | |||
730 | The default mode of the script's operation with this subcommand is to | ||
731 | create a layer with the following: | ||
732 | |||
733 | - A layer priority of 6. | ||
734 | |||
735 | - A ``conf`` subdirectory that contains a ``layer.conf`` file. | ||
736 | |||
737 | - A ``recipes-example`` subdirectory that contains a further | ||
738 | subdirectory named ``example``, which contains an ``example.bb`` | ||
739 | recipe file. | ||
740 | |||
741 | - A ``COPYING.MIT``, which is the license statement for the layer. The | ||
742 | script assumes you want to use the MIT license, which is typical for | ||
743 | most layers, for the contents of the layer itself. | ||
744 | |||
745 | - A ``README`` file, which is a file describing the contents of your | ||
746 | new layer. | ||
747 | |||
748 | In its simplest form, you can use the following command form to create a | ||
749 | layer. The command creates a layer whose name corresponds to | ||
750 | "your_layer_name" in the current directory: | ||
751 | :: | ||
752 | |||
753 | $ bitbake-layers create-layer your_layer_name | ||
754 | |||
755 | As an example, the following command creates a layer named ``meta-scottrif`` | ||
756 | in your home directory: | ||
757 | :: | ||
758 | |||
759 | $ cd /usr/home | ||
760 | $ bitbake-layers create-layer meta-scottrif | ||
761 | NOTE: Starting bitbake server... | ||
762 | Add your new layer with 'bitbake-layers add-layer meta-scottrif' | ||
763 | |||
764 | If you want to set the priority of the layer to other than the default | ||
765 | value of "6", you can either use the ``--priority`` option or you | ||
766 | can edit the | ||
767 | :term:`BBFILE_PRIORITY` value | ||
768 | in the ``conf/layer.conf`` after the script creates it. Furthermore, if | ||
769 | you want to give the example recipe file some name other than the | ||
770 | default, you can use the ``--example-recipe-name`` option. | ||
771 | |||
772 | The easiest way to see how the ``bitbake-layers create-layer`` command | ||
773 | works is to experiment with the script. You can also read the usage | ||
774 | information by entering the following: | ||
775 | :: | ||
776 | |||
777 | $ bitbake-layers create-layer --help | ||
778 | NOTE: Starting bitbake server... | ||
779 | usage: bitbake-layers create-layer [-h] [--priority PRIORITY] | ||
780 | [--example-recipe-name EXAMPLERECIPE] | ||
781 | layerdir | ||
782 | |||
783 | Create a basic layer | ||
784 | |||
785 | positional arguments: | ||
786 | layerdir Layer directory to create | ||
787 | |||
788 | optional arguments: | ||
789 | -h, --help show this help message and exit | ||
790 | --priority PRIORITY, -p PRIORITY | ||
791 | Layer directory to create | ||
792 | --example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE | ||
793 | Filename of the example recipe | ||
794 | |||
795 | Adding a Layer Using the ``bitbake-layers`` Script | ||
796 | -------------------------------------------------- | ||
797 | |||
798 | Once you create your general layer, you must add it to your | ||
799 | ``bblayers.conf`` file. Adding the layer to this configuration file | ||
800 | makes the OpenEmbedded build system aware of your layer so that it can | ||
801 | search it for metadata. | ||
802 | |||
803 | Add your layer by using the ``bitbake-layers add-layer`` command: | ||
804 | :: | ||
805 | |||
806 | $ bitbake-layers add-layer your_layer_name | ||
807 | |||
808 | Here is an example that adds a | ||
809 | layer named ``meta-scottrif`` to the configuration file. Following the | ||
810 | command that adds the layer is another ``bitbake-layers`` command that | ||
811 | shows the layers that are in your ``bblayers.conf`` file: | ||
812 | :: | ||
813 | |||
814 | $ bitbake-layers add-layer meta-scottrif | ||
815 | NOTE: Starting bitbake server... | ||
816 | Parsing recipes: 100% |##########################################################| Time: 0:00:49 | ||
817 | Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors. | ||
818 | $ bitbake-layers show-layers | ||
819 | NOTE: Starting bitbake server... | ||
820 | layer path priority | ||
821 | ========================================================================== | ||
822 | meta /home/scottrif/poky/meta 5 | ||
823 | meta-poky /home/scottrif/poky/meta-poky 5 | ||
824 | meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5 | ||
825 | workspace /home/scottrif/poky/build/workspace 99 | ||
826 | meta-scottrif /home/scottrif/poky/build/meta-scottrif 6 | ||
827 | |||
828 | |||
829 | Adding the layer to this file | ||
830 | enables the build system to locate the layer during the build. | ||
831 | |||
832 | .. note:: | ||
833 | |||
834 | During a build, the OpenEmbedded build system looks in the layers | ||
835 | from the top of the list down to the bottom in that order. | ||
836 | |||
837 | Customizing Images | ||
838 | ================== | ||
839 | |||
840 | You can customize images to satisfy particular requirements. This | ||
841 | section describes several methods and provides guidelines for each. | ||
842 | |||
843 | Customizing Images Using ``local.conf`` | ||
844 | --------------------------------------- | ||
845 | |||
846 | Probably the easiest way to customize an image is to add a package by | ||
847 | way of the ``local.conf`` configuration file. Because it is limited to | ||
848 | local use, this method generally only allows you to add packages and is | ||
849 | not as flexible as creating your own customized image. When you add | ||
850 | packages using local variables this way, you need to realize that these | ||
851 | variable changes are in effect for every build and consequently affect | ||
852 | all images, which might not be what you require. | ||
853 | |||
854 | To add a package to your image using the local configuration file, use | ||
855 | the ``IMAGE_INSTALL`` variable with the ``_append`` operator: | ||
856 | :: | ||
857 | |||
858 | IMAGE_INSTALL_append = " strace" | ||
859 | |||
860 | Use of the syntax is important - | ||
861 | specifically, the space between the quote and the package name, which is | ||
862 | ``strace`` in this example. This space is required since the ``_append`` | ||
863 | operator does not add the space. | ||
864 | |||
865 | Furthermore, you must use ``_append`` instead of the ``+=`` operator if | ||
866 | you want to avoid ordering issues. The reason for this is because doing | ||
867 | so unconditionally appends to the variable and avoids ordering problems | ||
868 | due to the variable being set in image recipes and ``.bbclass`` files | ||
869 | with operators like ``?=``. Using ``_append`` ensures the operation | ||
870 | takes effect. | ||
871 | |||
872 | As shown in its simplest use, ``IMAGE_INSTALL_append`` affects all | ||
873 | images. It is possible to extend the syntax so that the variable applies | ||
874 | to a specific image only. Here is an example: | ||
875 | :: | ||
876 | |||
877 | IMAGE_INSTALL_append_pn-core-image-minimal = " strace" | ||
878 | |||
879 | This example adds ``strace`` to the ``core-image-minimal`` image only. | ||
880 | |||
881 | You can add packages using a similar approach through the | ||
882 | ``CORE_IMAGE_EXTRA_INSTALL`` variable. If you use this variable, only | ||
883 | ``core-image-*`` images are affected. | ||
884 | |||
885 | Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES`` | ||
886 | ------------------------------------------------------------------------------- | ||
887 | |||
888 | Another method for customizing your image is to enable or disable | ||
889 | high-level image features by using the | ||
890 | :term:`IMAGE_FEATURES` and | ||
891 | :term:`EXTRA_IMAGE_FEATURES` | ||
892 | variables. Although the functions for both variables are nearly | ||
893 | equivalent, best practices dictate using ``IMAGE_FEATURES`` from within | ||
894 | a recipe and using ``EXTRA_IMAGE_FEATURES`` from within your | ||
895 | ``local.conf`` file, which is found in the | ||
896 | :term:`Build Directory`. | ||
897 | |||
898 | To understand how these features work, the best reference is | ||
899 | ``meta/classes/core-image.bbclass``. This class lists out the available | ||
900 | ``IMAGE_FEATURES`` of which most map to package groups while some, such | ||
901 | as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general | ||
902 | configuration settings. | ||
903 | |||
904 | In summary, the file looks at the contents of the ``IMAGE_FEATURES`` | ||
905 | variable and then maps or configures the feature accordingly. Based on | ||
906 | this information, the build system automatically adds the appropriate | ||
907 | packages or configurations to the | ||
908 | :term:`IMAGE_INSTALL` variable. | ||
909 | Effectively, you are enabling extra features by extending the class or | ||
910 | creating a custom class for use with specialized image ``.bb`` files. | ||
911 | |||
912 | Use the ``EXTRA_IMAGE_FEATURES`` variable from within your local | ||
913 | configuration file. Using a separate area from which to enable features | ||
914 | with this variable helps you avoid overwriting the features in the image | ||
915 | recipe that are enabled with ``IMAGE_FEATURES``. The value of | ||
916 | ``EXTRA_IMAGE_FEATURES`` is added to ``IMAGE_FEATURES`` within | ||
917 | ``meta/conf/bitbake.conf``. | ||
918 | |||
919 | To illustrate how you can use these variables to modify your image, | ||
920 | consider an example that selects the SSH server. The Yocto Project ships | ||
921 | with two SSH servers you can use with your images: Dropbear and OpenSSH. | ||
922 | Dropbear is a minimal SSH server appropriate for resource-constrained | ||
923 | environments, while OpenSSH is a well-known standard SSH server | ||
924 | implementation. By default, the ``core-image-sato`` image is configured | ||
925 | to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb`` | ||
926 | images both include OpenSSH. The ``core-image-minimal`` image does not | ||
927 | contain an SSH server. | ||
928 | |||
929 | You can customize your image and change these defaults. Edit the | ||
930 | ``IMAGE_FEATURES`` variable in your recipe or use the | ||
931 | ``EXTRA_IMAGE_FEATURES`` in your ``local.conf`` file so that it | ||
932 | configures the image you are working with to include | ||
933 | ``ssh-server-dropbear`` or ``ssh-server-openssh``. | ||
934 | |||
935 | .. note:: | ||
936 | |||
937 | See the ":ref:`ref-manual/features:image features`" section in the Yocto | ||
938 | Project Reference Manual for a complete list of image features that ship | ||
939 | with the Yocto Project. | ||
940 | |||
941 | Customizing Images Using Custom .bb Files | ||
942 | ----------------------------------------- | ||
943 | |||
944 | You can also customize an image by creating a custom recipe that defines | ||
945 | additional software as part of the image. The following example shows | ||
946 | the form for the two lines you need: | ||
947 | :: | ||
948 | |||
949 | IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2" | ||
950 | inherit core-image | ||
951 | |||
952 | Defining the software using a custom recipe gives you total control over | ||
953 | the contents of the image. It is important to use the correct names of | ||
954 | packages in the ``IMAGE_INSTALL`` variable. You must use the | ||
955 | OpenEmbedded notation and not the Debian notation for the names (e.g. | ||
956 | ``glibc-dev`` instead of ``libc6-dev``). | ||
957 | |||
958 | The other method for creating a custom image is to base it on an | ||
959 | existing image. For example, if you want to create an image based on | ||
960 | ``core-image-sato`` but add the additional package ``strace`` to the | ||
961 | image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new | ||
962 | ``.bb`` and add the following line to the end of the copy: | ||
963 | :: | ||
964 | |||
965 | IMAGE_INSTALL += "strace" | ||
966 | |||
967 | Customizing Images Using Custom Package Groups | ||
968 | ---------------------------------------------- | ||
969 | |||
970 | For complex custom images, the best approach for customizing an image is | ||
971 | to create a custom package group recipe that is used to build the image | ||
972 | or images. A good example of a package group recipe is | ||
973 | ``meta/recipes-core/packagegroups/packagegroup-base.bb``. | ||
974 | |||
975 | If you examine that recipe, you see that the ``PACKAGES`` variable lists | ||
976 | the package group packages to produce. The ``inherit packagegroup`` | ||
977 | statement sets appropriate default values and automatically adds | ||
978 | ``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each | ||
979 | package specified in the ``PACKAGES`` statement. | ||
980 | |||
981 | .. note:: | ||
982 | |||
983 | The ``inherit packagegroup`` line should be located near the top of the | ||
984 | recipe, certainly before the ``PACKAGES`` statement. | ||
985 | |||
986 | For each package you specify in ``PACKAGES``, you can use ``RDEPENDS`` | ||
987 | and ``RRECOMMENDS`` entries to provide a list of packages the parent | ||
988 | task package should contain. You can see examples of these further down | ||
989 | in the ``packagegroup-base.bb`` recipe. | ||
990 | |||
991 | Here is a short, fabricated example showing the same basic pieces for a | ||
992 | hypothetical packagegroup defined in ``packagegroup-custom.bb``, where | ||
993 | the variable ``PN`` is the standard way to abbreviate the reference to | ||
994 | the full packagegroup name ``packagegroup-custom``: | ||
995 | :: | ||
996 | |||
997 | DESCRIPTION = "My Custom Package Groups" | ||
998 | |||
999 | inherit packagegroup | ||
1000 | |||
1001 | PACKAGES = "\ | ||
1002 | ${PN}-apps \ | ||
1003 | ${PN}-tools \ | ||
1004 | " | ||
1005 | |||
1006 | RDEPENDS_${PN}-apps = "\ | ||
1007 | dropbear \ | ||
1008 | portmap \ | ||
1009 | psplash" | ||
1010 | |||
1011 | RDEPENDS_${PN}-tools = "\ | ||
1012 | oprofile \ | ||
1013 | oprofileui-server \ | ||
1014 | lttng-tools" | ||
1015 | |||
1016 | RRECOMMENDS_${PN}-tools = "\ | ||
1017 | kernel-module-oprofile" | ||
1018 | |||
1019 | In the previous example, two package group packages are created with | ||
1020 | their dependencies and their recommended package dependencies listed: | ||
1021 | ``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To | ||
1022 | build an image using these package group packages, you need to add | ||
1023 | ``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to | ||
1024 | ``IMAGE_INSTALL``. For other forms of image dependencies see the other | ||
1025 | areas of this section. | ||
1026 | |||
1027 | Customizing an Image Hostname | ||
1028 | ----------------------------- | ||
1029 | |||
1030 | By default, the configured hostname (i.e. ``/etc/hostname``) in an image | ||
1031 | is the same as the machine name. For example, if | ||
1032 | :term:`MACHINE` equals "qemux86", the | ||
1033 | configured hostname written to ``/etc/hostname`` is "qemux86". | ||
1034 | |||
1035 | You can customize this name by altering the value of the "hostname" | ||
1036 | variable in the ``base-files`` recipe using either an append file or a | ||
1037 | configuration file. Use the following in an append file: | ||
1038 | :: | ||
1039 | |||
1040 | hostname = "myhostname" | ||
1041 | |||
1042 | Use the following in a configuration file: | ||
1043 | :: | ||
1044 | |||
1045 | hostname_pn-base-files = "myhostname" | ||
1046 | |||
1047 | Changing the default value of the variable "hostname" can be useful in | ||
1048 | certain situations. For example, suppose you need to do extensive | ||
1049 | testing on an image and you would like to easily identify the image | ||
1050 | under test from existing images with typical default hostnames. In this | ||
1051 | situation, you could change the default hostname to "testme", which | ||
1052 | results in all the images using the name "testme". Once testing is | ||
1053 | complete and you do not need to rebuild the image for test any longer, | ||
1054 | you can easily reset the default hostname. | ||
1055 | |||
1056 | Another point of interest is that if you unset the variable, the image | ||
1057 | will have no default hostname in the filesystem. Here is an example that | ||
1058 | unsets the variable in a configuration file: | ||
1059 | :: | ||
1060 | |||
1061 | hostname_pn-base-files = "" | ||
1062 | |||
1063 | Having no default hostname in the filesystem is suitable for | ||
1064 | environments that use dynamic hostnames such as virtual machines. | ||
1065 | |||
1066 | Writing a New Recipe | ||
1067 | ==================== | ||
1068 | |||
1069 | Recipes (``.bb`` files) are fundamental components in the Yocto Project | ||
1070 | environment. Each software component built by the OpenEmbedded build | ||
1071 | system requires a recipe to define the component. This section describes | ||
1072 | how to create, write, and test a new recipe. | ||
1073 | |||
1074 | .. note:: | ||
1075 | |||
1076 | For information on variables that are useful for recipes and for | ||
1077 | information about recipe naming issues, see the | ||
1078 | ":ref:`ref-manual/varlocality:recipes`" section of the Yocto Project | ||
1079 | Reference Manual. | ||
1080 | |||
1081 | Overview | ||
1082 | -------- | ||
1083 | |||
1084 | The following figure shows the basic process for creating a new recipe. | ||
1085 | The remainder of the section provides details for the steps. | ||
1086 | |||
1087 | .. image:: figures/recipe-workflow.png | ||
1088 | :align: center | ||
1089 | |||
1090 | Locate or Automatically Create a Base Recipe | ||
1091 | -------------------------------------------- | ||
1092 | |||
1093 | You can always write a recipe from scratch. However, three choices exist | ||
1094 | that can help you quickly get a start on a new recipe: | ||
1095 | |||
1096 | - ``devtool add``: A command that assists in creating a recipe and an | ||
1097 | environment conducive to development. | ||
1098 | |||
1099 | - ``recipetool create``: A command provided by the Yocto Project that | ||
1100 | automates creation of a base recipe based on the source files. | ||
1101 | |||
1102 | - *Existing Recipes:* Location and modification of an existing recipe | ||
1103 | that is similar in function to the recipe you need. | ||
1104 | |||
1105 | .. note:: | ||
1106 | |||
1107 | For information on recipe syntax, see the | ||
1108 | ":ref:`dev-manual/common-tasks:recipe syntax`" section. | ||
1109 | |||
1110 | Creating the Base Recipe Using ``devtool add`` | ||
1111 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1112 | |||
1113 | The ``devtool add`` command uses the same logic for auto-creating the | ||
1114 | recipe as ``recipetool create``, which is listed below. Additionally, | ||
1115 | however, ``devtool add`` sets up an environment that makes it easy for | ||
1116 | you to patch the source and to make changes to the recipe as is often | ||
1117 | necessary when adding a recipe to build a new piece of software to be | ||
1118 | included in a build. | ||
1119 | |||
1120 | You can find a complete description of the ``devtool add`` command in | ||
1121 | the ":ref:`sdk-manual/extensible:a closer look at \`\`devtool add\`\``" section | ||
1122 | in the Yocto Project Application Development and the Extensible Software | ||
1123 | Development Kit (eSDK) manual. | ||
1124 | |||
1125 | Creating the Base Recipe Using ``recipetool create`` | ||
1126 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1127 | |||
1128 | ``recipetool create`` automates creation of a base recipe given a set of | ||
1129 | source code files. As long as you can extract or point to the source | ||
1130 | files, the tool will construct a recipe and automatically configure all | ||
1131 | pre-build information into the recipe. For example, suppose you have an | ||
1132 | application that builds using Autotools. Creating the base recipe using | ||
1133 | ``recipetool`` results in a recipe that has the pre-build dependencies, | ||
1134 | license requirements, and checksums configured. | ||
1135 | |||
1136 | To run the tool, you just need to be in your | ||
1137 | :term:`Build Directory` and have sourced the | ||
1138 | build environment setup script (i.e. | ||
1139 | :ref:`structure-core-script`). | ||
1140 | To get help on the tool, use the following command: | ||
1141 | :: | ||
1142 | |||
1143 | $ recipetool -h | ||
1144 | NOTE: Starting bitbake server... | ||
1145 | usage: recipetool [-d] [-q] [--color COLOR] [-h] <subcommand> ... | ||
1146 | |||
1147 | OpenEmbedded recipe tool | ||
1148 | |||
1149 | options: | ||
1150 | -d, --debug Enable debug output | ||
1151 | -q, --quiet Print only errors | ||
1152 | --color COLOR Colorize output (where COLOR is auto, always, never) | ||
1153 | -h, --help show this help message and exit | ||
1154 | |||
1155 | subcommands: | ||
1156 | create Create a new recipe | ||
1157 | newappend Create a bbappend for the specified target in the specified | ||
1158 | layer | ||
1159 | setvar Set a variable within a recipe | ||
1160 | appendfile Create/update a bbappend to replace a target file | ||
1161 | appendsrcfiles Create/update a bbappend to add or replace source files | ||
1162 | appendsrcfile Create/update a bbappend to add or replace a source file | ||
1163 | Use recipetool <subcommand> --help to get help on a specific command | ||
1164 | |||
1165 | Running ``recipetool create -o OUTFILE`` creates the base recipe and | ||
1166 | locates it properly in the layer that contains your source files. | ||
1167 | Following are some syntax examples: | ||
1168 | |||
1169 | - Use this syntax to generate a recipe based on source. Once generated, | ||
1170 | the recipe resides in the existing source code layer: | ||
1171 | :: | ||
1172 | |||
1173 | recipetool create -o OUTFILE source | ||
1174 | |||
1175 | - Use this syntax to generate a recipe using code that | ||
1176 | you extract from source. The extracted code is placed in its own layer | ||
1177 | defined by ``EXTERNALSRC``. | ||
1178 | :: | ||
1179 | |||
1180 | recipetool create -o OUTFILE -x EXTERNALSRC source | ||
1181 | |||
1182 | - Use this syntax to generate a recipe based on source. The options | ||
1183 | direct ``recipetool`` to generate debugging information. Once generated, | ||
1184 | the recipe resides in the existing source code layer: | ||
1185 | :: | ||
1186 | |||
1187 | recipetool create -d -o OUTFILE source | ||
1188 | |||
1189 | Locating and Using a Similar Recipe | ||
1190 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1191 | |||
1192 | Before writing a recipe from scratch, it is often useful to discover | ||
1193 | whether someone else has already written one that meets (or comes close | ||
1194 | to meeting) your needs. The Yocto Project and OpenEmbedded communities | ||
1195 | maintain many recipes that might be candidates for what you are doing. | ||
1196 | You can find a good central index of these recipes in the | ||
1197 | :oe_layerindex:`OpenEmbedded Layer Index <>`. | ||
1198 | |||
1199 | Working from an existing recipe or a skeleton recipe is the best way to | ||
1200 | get started. Here are some points on both methods: | ||
1201 | |||
1202 | - *Locate and modify a recipe that is close to what you want to do:* | ||
1203 | This method works when you are familiar with the current recipe | ||
1204 | space. The method does not work so well for those new to the Yocto | ||
1205 | Project or writing recipes. | ||
1206 | |||
1207 | Some risks associated with this method are using a recipe that has | ||
1208 | areas totally unrelated to what you are trying to accomplish with | ||
1209 | your recipe, not recognizing areas of the recipe that you might have | ||
1210 | to add from scratch, and so forth. All these risks stem from | ||
1211 | unfamiliarity with the existing recipe space. | ||
1212 | |||
1213 | - *Use and modify the following skeleton recipe:* If for some reason | ||
1214 | you do not want to use ``recipetool`` and you cannot find an existing | ||
1215 | recipe that is close to meeting your needs, you can use the following | ||
1216 | structure to provide the fundamental areas of a new recipe. | ||
1217 | :: | ||
1218 | |||
1219 | DESCRIPTION = "" | ||
1220 | HOMEPAGE = "" | ||
1221 | LICENSE = "" | ||
1222 | SECTION = "" | ||
1223 | DEPENDS = "" | ||
1224 | LIC_FILES_CHKSUM = "" | ||
1225 | |||
1226 | SRC_URI = "" | ||
1227 | |||
1228 | Storing and Naming the Recipe | ||
1229 | ----------------------------- | ||
1230 | |||
1231 | Once you have your base recipe, you should put it in your own layer and | ||
1232 | name it appropriately. Locating it correctly ensures that the | ||
1233 | OpenEmbedded build system can find it when you use BitBake to process | ||
1234 | the recipe. | ||
1235 | |||
1236 | - *Storing Your Recipe:* The OpenEmbedded build system locates your | ||
1237 | recipe through the layer's ``conf/layer.conf`` file and the | ||
1238 | :term:`BBFILES` variable. This | ||
1239 | variable sets up a path from which the build system can locate | ||
1240 | recipes. Here is the typical use: | ||
1241 | :: | ||
1242 | |||
1243 | BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ | ||
1244 | ${LAYERDIR}/recipes-*/*/*.bbappend" | ||
1245 | |||
1246 | Consequently, you need to be sure you locate your new recipe inside | ||
1247 | your layer such that it can be found. | ||
1248 | |||
1249 | You can find more information on how layers are structured in the | ||
1250 | "`Understanding and Creating | ||
1251 | Layers <#understanding-and-creating-layers>`__" section. | ||
1252 | |||
1253 | - *Naming Your Recipe:* When you name your recipe, you need to follow | ||
1254 | this naming convention: | ||
1255 | :: | ||
1256 | |||
1257 | basename_version.bb | ||
1258 | |||
1259 | Use lower-cased characters and do not include the reserved suffixes | ||
1260 | ``-native``, ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use | ||
1261 | them as part of your recipe name unless the string applies). Here are some | ||
1262 | examples: | ||
1263 | |||
1264 | .. code-block:: none | ||
1265 | |||
1266 | cups_1.7.0.bb | ||
1267 | gawk_4.0.2.bb | ||
1268 | irssi_0.8.16-rc1.bb | ||
1269 | |||
1270 | Running a Build on the Recipe | ||
1271 | ----------------------------- | ||
1272 | |||
1273 | Creating a new recipe is usually an iterative process that requires | ||
1274 | using BitBake to process the recipe multiple times in order to | ||
1275 | progressively discover and add information to the recipe file. | ||
1276 | |||
1277 | Assuming you have sourced the build environment setup script (i.e. | ||
1278 | :ref:`structure-core-script`) and you are in | ||
1279 | the :term:`Build Directory`, use | ||
1280 | BitBake to process your recipe. All you need to provide is the | ||
1281 | ``basename`` of the recipe as described in the previous section: | ||
1282 | :: | ||
1283 | |||
1284 | $ bitbake basename | ||
1285 | |||
1286 | During the build, the OpenEmbedded build system creates a temporary work | ||
1287 | directory for each recipe | ||
1288 | (``${``\ :term:`WORKDIR`\ ``}``) | ||
1289 | where it keeps extracted source files, log files, intermediate | ||
1290 | compilation and packaging files, and so forth. | ||
1291 | |||
1292 | The path to the per-recipe temporary work directory depends on the | ||
1293 | context in which it is being built. The quickest way to find this path | ||
1294 | is to have BitBake return it by running the following: | ||
1295 | :: | ||
1296 | |||
1297 | $ bitbake -e basename | grep ^WORKDIR= | ||
1298 | |||
1299 | As an example, assume a Source Directory | ||
1300 | top-level folder named ``poky``, a default Build Directory at | ||
1301 | ``poky/build``, and a ``qemux86-poky-linux`` machine target system. | ||
1302 | Furthermore, suppose your recipe is named ``foo_1.3.0.bb``. In this | ||
1303 | case, the work directory the build system uses to build the package | ||
1304 | would be as follows: | ||
1305 | :: | ||
1306 | |||
1307 | poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 | ||
1308 | |||
1309 | Inside this directory you can find sub-directories such as ``image``, | ||
1310 | ``packages-split``, and ``temp``. After the build, you can examine these | ||
1311 | to determine how well the build went. | ||
1312 | |||
1313 | .. note:: | ||
1314 | |||
1315 | You can find log files for each task in the recipe's ``temp`` | ||
1316 | directory (e.g. ``poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp``). | ||
1317 | Log files are named ``log.taskname`` (e.g. ``log.do_configure``, | ||
1318 | ``log.do_fetch``, and ``log.do_compile``). | ||
1319 | |||
1320 | You can find more information about the build process in | ||
1321 | ":doc:`/overview-manual/development-environment`" | ||
1322 | chapter of the Yocto Project Overview and Concepts Manual. | ||
1323 | |||
1324 | Fetching Code | ||
1325 | ------------- | ||
1326 | |||
1327 | The first thing your recipe must do is specify how to fetch the source | ||
1328 | files. Fetching is controlled mainly through the | ||
1329 | :term:`SRC_URI` variable. Your recipe | ||
1330 | must have a ``SRC_URI`` variable that points to where the source is | ||
1331 | located. For a graphical representation of source locations, see the | ||
1332 | ":ref:`overview-manual/concepts:sources`" section in | ||
1333 | the Yocto Project Overview and Concepts Manual. | ||
1334 | |||
1335 | The :ref:`ref-tasks-fetch` task uses | ||
1336 | the prefix of each entry in the ``SRC_URI`` variable value to determine | ||
1337 | which :ref:`fetcher <bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers>` to use to get your | ||
1338 | source files. It is the ``SRC_URI`` variable that triggers the fetcher. | ||
1339 | The :ref:`ref-tasks-patch` task uses | ||
1340 | the variable after source is fetched to apply patches. The OpenEmbedded | ||
1341 | build system uses | ||
1342 | :term:`FILESOVERRIDES` for | ||
1343 | scanning directory locations for local files in ``SRC_URI``. | ||
1344 | |||
1345 | The ``SRC_URI`` variable in your recipe must define each unique location | ||
1346 | for your source files. It is good practice to not hard-code version | ||
1347 | numbers in a URL used in ``SRC_URI``. Rather than hard-code these | ||
1348 | values, use ``${``\ :term:`PV`\ ``}``, | ||
1349 | which causes the fetch process to use the version specified in the | ||
1350 | recipe filename. Specifying the version in this manner means that | ||
1351 | upgrading the recipe to a future version is as simple as renaming the | ||
1352 | recipe to match the new version. | ||
1353 | |||
1354 | Here is a simple example from the | ||
1355 | ``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source | ||
1356 | comes from a single tarball. Notice the use of the | ||
1357 | :term:`PV` variable: | ||
1358 | :: | ||
1359 | |||
1360 | SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \ | ||
1361 | |||
1362 | Files mentioned in ``SRC_URI`` whose names end in a typical archive | ||
1363 | extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so | ||
1364 | forth), are automatically extracted during the | ||
1365 | :ref:`ref-tasks-unpack` task. For | ||
1366 | another example that specifies these types of files, see the | ||
1367 | "`Autotooled Package <#new-recipe-autotooled-package>`__" section. | ||
1368 | |||
1369 | Another way of specifying source is from an SCM. For Git repositories, | ||
1370 | you must specify :term:`SRCREV` and | ||
1371 | you should specify :term:`PV` to include | ||
1372 | the revision with :term:`SRCPV`. Here | ||
1373 | is an example from the recipe | ||
1374 | ``meta/recipes-kernel/blktrace/blktrace_git.bb``: | ||
1375 | :: | ||
1376 | |||
1377 | SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385" | ||
1378 | |||
1379 | PR = "r6" | ||
1380 | PV = "1.0.5+git${SRCPV}" | ||
1381 | |||
1382 | SRC_URI = "git://git.kernel.dk/blktrace.git \ | ||
1383 | file://ldflags.patch" | ||
1384 | |||
1385 | If your ``SRC_URI`` statement includes URLs pointing to individual files | ||
1386 | fetched from a remote server other than a version control system, | ||
1387 | BitBake attempts to verify the files against checksums defined in your | ||
1388 | recipe to ensure they have not been tampered with or otherwise modified | ||
1389 | since the recipe was written. Two checksums are used: | ||
1390 | ``SRC_URI[md5sum]`` and ``SRC_URI[sha256sum]``. | ||
1391 | |||
1392 | If your ``SRC_URI`` variable points to more than a single URL (excluding | ||
1393 | SCM URLs), you need to provide the ``md5`` and ``sha256`` checksums for | ||
1394 | each URL. For these cases, you provide a name for each URL as part of | ||
1395 | the ``SRC_URI`` and then reference that name in the subsequent checksum | ||
1396 | statements. Here is an example combining lines from the files | ||
1397 | ``git.inc`` and ``git_2.24.1.bb``: | ||
1398 | :: | ||
1399 | |||
1400 | SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \ | ||
1401 | ${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages" | ||
1402 | |||
1403 | SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b" | ||
1404 | SRC_URI[tarball.sha256sum] = "ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02" | ||
1405 | SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d" | ||
1406 | SRC_URI[manpages.sha256sum] = "9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230" | ||
1407 | |||
1408 | Proper values for ``md5`` and ``sha256`` checksums might be available | ||
1409 | with other signatures on the download page for the upstream source (e.g. | ||
1410 | ``md5``, ``sha1``, ``sha256``, ``GPG``, and so forth). Because the | ||
1411 | OpenEmbedded build system only deals with ``sha256sum`` and ``md5sum``, | ||
1412 | you should verify all the signatures you find by hand. | ||
1413 | |||
1414 | If no ``SRC_URI`` checksums are specified when you attempt to build the | ||
1415 | recipe, or you provide an incorrect checksum, the build will produce an | ||
1416 | error for each missing or incorrect checksum. As part of the error | ||
1417 | message, the build system provides the checksum string corresponding to | ||
1418 | the fetched file. Once you have the correct checksums, you can copy and | ||
1419 | paste them into your recipe and then run the build again to continue. | ||
1420 | |||
1421 | .. note:: | ||
1422 | |||
1423 | As mentioned, if the upstream source provides signatures for | ||
1424 | verifying the downloaded source code, you should verify those | ||
1425 | manually before setting the checksum values in the recipe and | ||
1426 | continuing with the build. | ||
1427 | |||
1428 | This final example is a bit more complicated and is from the | ||
1429 | ``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The | ||
1430 | example's ``SRC_URI`` statement identifies multiple files as the source | ||
1431 | files for the recipe: a tarball, a patch file, a desktop file, and an | ||
1432 | icon. | ||
1433 | :: | ||
1434 | |||
1435 | SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ | ||
1436 | file://xwc.patch \ | ||
1437 | file://rxvt.desktop \ | ||
1438 | file://rxvt.png" | ||
1439 | |||
1440 | When you specify local files using the ``file://`` URI protocol, the | ||
1441 | build system fetches files from the local machine. The path is relative | ||
1442 | to the :term:`FILESPATH` variable | ||
1443 | and searches specific directories in a certain order: | ||
1444 | ``${``\ :term:`BP`\ ``}``, | ||
1445 | ``${``\ :term:`BPN`\ ``}``, and | ||
1446 | ``files``. The directories are assumed to be subdirectories of the | ||
1447 | directory in which the recipe or append file resides. For another | ||
1448 | example that specifies these types of files, see the "`Single .c File | ||
1449 | Package (Hello | ||
1450 | World!) <#new-recipe-single-c-file-package-hello-world>`__" section. | ||
1451 | |||
1452 | The previous example also specifies a patch file. Patch files are files | ||
1453 | whose names usually end in ``.patch`` or ``.diff`` but can end with | ||
1454 | compressed suffixes such as ``diff.gz`` and ``patch.bz2``, for example. | ||
1455 | The build system automatically applies patches as described in the | ||
1456 | "`Patching Code <#new-recipe-patching-code>`__" section. | ||
1457 | |||
1458 | Unpacking Code | ||
1459 | -------------- | ||
1460 | |||
1461 | During the build, the | ||
1462 | :ref:`ref-tasks-unpack` task unpacks | ||
1463 | the source with ``${``\ :term:`S`\ ``}`` | ||
1464 | pointing to where it is unpacked. | ||
1465 | |||
1466 | If you are fetching your source files from an upstream source archived | ||
1467 | tarball and the tarball's internal structure matches the common | ||
1468 | convention of a top-level subdirectory named | ||
1469 | ``${``\ :term:`BPN`\ ``}-${``\ :term:`PV`\ ``}``, | ||
1470 | then you do not need to set ``S``. However, if ``SRC_URI`` specifies to | ||
1471 | fetch source from an archive that does not use this convention, or from | ||
1472 | an SCM like Git or Subversion, your recipe needs to define ``S``. | ||
1473 | |||
1474 | If processing your recipe using BitBake successfully unpacks the source | ||
1475 | files, you need to be sure that the directory pointed to by ``${S}`` | ||
1476 | matches the structure of the source. | ||
1477 | |||
1478 | Patching Code | ||
1479 | ------------- | ||
1480 | |||
1481 | Sometimes it is necessary to patch code after it has been fetched. Any | ||
1482 | files mentioned in ``SRC_URI`` whose names end in ``.patch`` or | ||
1483 | ``.diff`` or compressed versions of these suffixes (e.g. ``diff.gz`` are | ||
1484 | treated as patches. The | ||
1485 | :ref:`ref-tasks-patch` task | ||
1486 | automatically applies these patches. | ||
1487 | |||
1488 | The build system should be able to apply patches with the "-p1" option | ||
1489 | (i.e. one directory level in the path will be stripped off). If your | ||
1490 | patch needs to have more directory levels stripped off, specify the | ||
1491 | number of levels using the "striplevel" option in the ``SRC_URI`` entry | ||
1492 | for the patch. Alternatively, if your patch needs to be applied in a | ||
1493 | specific subdirectory that is not specified in the patch file, use the | ||
1494 | "patchdir" option in the entry. | ||
1495 | |||
1496 | As with all local files referenced in | ||
1497 | :term:`SRC_URI` using ``file://``, | ||
1498 | you should place patch files in a directory next to the recipe either | ||
1499 | named the same as the base name of the recipe | ||
1500 | (:term:`BP` and | ||
1501 | :term:`BPN`) or "files". | ||
1502 | |||
1503 | Licensing | ||
1504 | --------- | ||
1505 | |||
1506 | Your recipe needs to have both the | ||
1507 | :term:`LICENSE` and | ||
1508 | :term:`LIC_FILES_CHKSUM` | ||
1509 | variables: | ||
1510 | |||
1511 | - ``LICENSE``: This variable specifies the license for the software. | ||
1512 | If you do not know the license under which the software you are | ||
1513 | building is distributed, you should go to the source code and look | ||
1514 | for that information. Typical files containing this information | ||
1515 | include ``COPYING``, ``LICENSE``, and ``README`` files. You could | ||
1516 | also find the information near the top of a source file. For example, | ||
1517 | given a piece of software licensed under the GNU General Public | ||
1518 | License version 2, you would set ``LICENSE`` as follows: | ||
1519 | :: | ||
1520 | |||
1521 | LICENSE = "GPLv2" | ||
1522 | |||
1523 | The licenses you specify within ``LICENSE`` can have any name as long | ||
1524 | as you do not use spaces, since spaces are used as separators between | ||
1525 | license names. For standard licenses, use the names of the files in | ||
1526 | ``meta/files/common-licenses/`` or the ``SPDXLICENSEMAP`` flag names | ||
1527 | defined in ``meta/conf/licenses.conf``. | ||
1528 | |||
1529 | - ``LIC_FILES_CHKSUM``: The OpenEmbedded build system uses this | ||
1530 | variable to make sure the license text has not changed. If it has, | ||
1531 | the build produces an error and it affords you the chance to figure | ||
1532 | it out and correct the problem. | ||
1533 | |||
1534 | You need to specify all applicable licensing files for the software. | ||
1535 | At the end of the configuration step, the build process will compare | ||
1536 | the checksums of the files to be sure the text has not changed. Any | ||
1537 | differences result in an error with the message containing the | ||
1538 | current checksum. For more explanation and examples of how to set the | ||
1539 | ``LIC_FILES_CHKSUM`` variable, see the | ||
1540 | ":ref:`dev-manual/common-tasks:tracking license changes`" section. | ||
1541 | |||
1542 | To determine the correct checksum string, you can list the | ||
1543 | appropriate files in the ``LIC_FILES_CHKSUM`` variable with incorrect | ||
1544 | md5 strings, attempt to build the software, and then note the | ||
1545 | resulting error messages that will report the correct md5 strings. | ||
1546 | See the "`Fetching Code <#new-recipe-fetching-code>`__" section for | ||
1547 | additional information. | ||
1548 | |||
1549 | Here is an example that assumes the software has a ``COPYING`` file: | ||
1550 | :: | ||
1551 | |||
1552 | LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" | ||
1553 | |||
1554 | When you try to build the | ||
1555 | software, the build system will produce an error and give you the | ||
1556 | correct string that you can substitute into the recipe file for a | ||
1557 | subsequent build. | ||
1558 | |||
1559 | Dependencies | ||
1560 | ------------ | ||
1561 | |||
1562 | Most software packages have a short list of other packages that they | ||
1563 | require, which are called dependencies. These dependencies fall into two | ||
1564 | main categories: build-time dependencies, which are required when the | ||
1565 | software is built; and runtime dependencies, which are required to be | ||
1566 | installed on the target in order for the software to run. | ||
1567 | |||
1568 | Within a recipe, you specify build-time dependencies using the | ||
1569 | :term:`DEPENDS` variable. Although | ||
1570 | nuances exist, items specified in ``DEPENDS`` should be names of other | ||
1571 | recipes. It is important that you specify all build-time dependencies | ||
1572 | explicitly. | ||
1573 | |||
1574 | Another consideration is that configure scripts might automatically | ||
1575 | check for optional dependencies and enable corresponding functionality | ||
1576 | if those dependencies are found. If you wish to make a recipe that is | ||
1577 | more generally useful (e.g. publish the recipe in a layer for others to | ||
1578 | use), instead of hard-disabling the functionality, you can use the | ||
1579 | :term:`PACKAGECONFIG` variable to allow functionality and the | ||
1580 | corresponding dependencies to be enabled and disabled easily by other | ||
1581 | users of the recipe. | ||
1582 | |||
1583 | Similar to build-time dependencies, you specify runtime dependencies | ||
1584 | through a variable - | ||
1585 | :term:`RDEPENDS`, which is | ||
1586 | package-specific. All variables that are package-specific need to have | ||
1587 | the name of the package added to the end as an override. Since the main | ||
1588 | package for a recipe has the same name as the recipe, and the recipe's | ||
1589 | name can be found through the | ||
1590 | ``${``\ :term:`PN`\ ``}`` variable, then | ||
1591 | you specify the dependencies for the main package by setting | ||
1592 | ``RDEPENDS_${PN}``. If the package were named ``${PN}-tools``, then you | ||
1593 | would set ``RDEPENDS_${PN}-tools``, and so forth. | ||
1594 | |||
1595 | Some runtime dependencies will be set automatically at packaging time. | ||
1596 | These dependencies include any shared library dependencies (i.e. if a | ||
1597 | package "example" contains "libexample" and another package "mypackage" | ||
1598 | contains a binary that links to "libexample" then the OpenEmbedded build | ||
1599 | system will automatically add a runtime dependency to "mypackage" on | ||
1600 | "example"). See the | ||
1601 | ":ref:`overview-manual/concepts:automatically added runtime dependencies`" | ||
1602 | section in the Yocto Project Overview and Concepts Manual for further | ||
1603 | details. | ||
1604 | |||
1605 | Configuring the Recipe | ||
1606 | ---------------------- | ||
1607 | |||
1608 | Most software provides some means of setting build-time configuration | ||
1609 | options before compilation. Typically, setting these options is | ||
1610 | accomplished by running a configure script with options, or by modifying | ||
1611 | a build configuration file. | ||
1612 | |||
1613 | .. note:: | ||
1614 | |||
1615 | As of Yocto Project Release 1.7, some of the core recipes that | ||
1616 | package binary configuration scripts now disable the scripts due to | ||
1617 | the scripts previously requiring error-prone path substitution. The | ||
1618 | OpenEmbedded build system uses ``pkg-config`` now, which is much more | ||
1619 | robust. You can find a list of the ``*-config`` scripts that are disabled | ||
1620 | in the ":ref:`migration-1.7-binary-configuration-scripts-disabled`" section | ||
1621 | in the Yocto Project Reference Manual. | ||
1622 | |||
1623 | A major part of build-time configuration is about checking for | ||
1624 | build-time dependencies and possibly enabling optional functionality as | ||
1625 | a result. You need to specify any build-time dependencies for the | ||
1626 | software you are building in your recipe's | ||
1627 | :term:`DEPENDS` value, in terms of | ||
1628 | other recipes that satisfy those dependencies. You can often find | ||
1629 | build-time or runtime dependencies described in the software's | ||
1630 | documentation. | ||
1631 | |||
1632 | The following list provides configuration items of note based on how | ||
1633 | your software is built: | ||
1634 | |||
1635 | - *Autotools:* If your source files have a ``configure.ac`` file, then | ||
1636 | your software is built using Autotools. If this is the case, you just | ||
1637 | need to worry about modifying the configuration. | ||
1638 | |||
1639 | When using Autotools, your recipe needs to inherit the | ||
1640 | :ref:`autotools <ref-classes-autotools>` class | ||
1641 | and your recipe does not have to contain a | ||
1642 | :ref:`ref-tasks-configure` task. | ||
1643 | However, you might still want to make some adjustments. For example, | ||
1644 | you can set | ||
1645 | :term:`EXTRA_OECONF` or | ||
1646 | :term:`PACKAGECONFIG_CONFARGS` | ||
1647 | to pass any needed configure options that are specific to the recipe. | ||
1648 | |||
1649 | - *CMake:* If your source files have a ``CMakeLists.txt`` file, then | ||
1650 | your software is built using CMake. If this is the case, you just | ||
1651 | need to worry about modifying the configuration. | ||
1652 | |||
1653 | When you use CMake, your recipe needs to inherit the | ||
1654 | :ref:`cmake <ref-classes-cmake>` class and your | ||
1655 | recipe does not have to contain a | ||
1656 | :ref:`ref-tasks-configure` task. | ||
1657 | You can make some adjustments by setting | ||
1658 | :term:`EXTRA_OECMAKE` to | ||
1659 | pass any needed configure options that are specific to the recipe. | ||
1660 | |||
1661 | .. note:: | ||
1662 | |||
1663 | If you need to install one or more custom CMake toolchain files | ||
1664 | that are supplied by the application you are building, install the | ||
1665 | files to ``${D}${datadir}/cmake/Modules`` during ``do_install``. | ||
1666 | |||
1667 | - *Other:* If your source files do not have a ``configure.ac`` or | ||
1668 | ``CMakeLists.txt`` file, then your software is built using some | ||
1669 | method other than Autotools or CMake. If this is the case, you | ||
1670 | normally need to provide a | ||
1671 | :ref:`ref-tasks-configure` task | ||
1672 | in your recipe unless, of course, there is nothing to configure. | ||
1673 | |||
1674 | Even if your software is not being built by Autotools or CMake, you | ||
1675 | still might not need to deal with any configuration issues. You need | ||
1676 | to determine if configuration is even a required step. You might need | ||
1677 | to modify a Makefile or some configuration file used for the build to | ||
1678 | specify necessary build options. Or, perhaps you might need to run a | ||
1679 | provided, custom configure script with the appropriate options. | ||
1680 | |||
1681 | For the case involving a custom configure script, you would run | ||
1682 | ``./configure --help`` and look for the options you need to set. | ||
1683 | |||
1684 | Once configuration succeeds, it is always good practice to look at the | ||
1685 | ``log.do_configure`` file to ensure that the appropriate options have | ||
1686 | been enabled and no additional build-time dependencies need to be added | ||
1687 | to ``DEPENDS``. For example, if the configure script reports that it | ||
1688 | found something not mentioned in ``DEPENDS``, or that it did not find | ||
1689 | something that it needed for some desired optional functionality, then | ||
1690 | you would need to add those to ``DEPENDS``. Looking at the log might | ||
1691 | also reveal items being checked for, enabled, or both that you do not | ||
1692 | want, or items not being found that are in ``DEPENDS``, in which case | ||
1693 | you would need to look at passing extra options to the configure script | ||
1694 | as needed. For reference information on configure options specific to | ||
1695 | the software you are building, you can consult the output of the | ||
1696 | ``./configure --help`` command within ``${S}`` or consult the software's | ||
1697 | upstream documentation. | ||
1698 | |||
1699 | Using Headers to Interface with Devices | ||
1700 | --------------------------------------- | ||
1701 | |||
1702 | If your recipe builds an application that needs to communicate with some | ||
1703 | device or needs an API into a custom kernel, you will need to provide | ||
1704 | appropriate header files. Under no circumstances should you ever modify | ||
1705 | the existing | ||
1706 | ``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc`` file. | ||
1707 | These headers are used to build ``libc`` and must not be compromised | ||
1708 | with custom or machine-specific header information. If you customize | ||
1709 | ``libc`` through modified headers all other applications that use | ||
1710 | ``libc`` thus become affected. | ||
1711 | |||
1712 | .. note:: | ||
1713 | |||
1714 | Never copy and customize the ``libc`` header file (i.e. | ||
1715 | ``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc``). | ||
1716 | |||
1717 | The correct way to interface to a device or custom kernel is to use a | ||
1718 | separate package that provides the additional headers for the driver or | ||
1719 | other unique interfaces. When doing so, your application also becomes | ||
1720 | responsible for creating a dependency on that specific provider. | ||
1721 | |||
1722 | Consider the following: | ||
1723 | |||
1724 | - Never modify ``linux-libc-headers.inc``. Consider that file to be | ||
1725 | part of the ``libc`` system, and not something you use to access the | ||
1726 | kernel directly. You should access ``libc`` through specific ``libc`` | ||
1727 | calls. | ||
1728 | |||
1729 | - Applications that must talk directly to devices should either provide | ||
1730 | necessary headers themselves, or establish a dependency on a special | ||
1731 | headers package that is specific to that driver. | ||
1732 | |||
1733 | For example, suppose you want to modify an existing header that adds I/O | ||
1734 | control or network support. If the modifications are used by a small | ||
1735 | number programs, providing a unique version of a header is easy and has | ||
1736 | little impact. When doing so, bear in mind the guidelines in the | ||
1737 | previous list. | ||
1738 | |||
1739 | .. note:: | ||
1740 | |||
1741 | If for some reason your changes need to modify the behavior of the ``libc``, | ||
1742 | and subsequently all other applications on the system, use a ``.bbappend`` | ||
1743 | to modify the ``linux-kernel-headers.inc`` file. However, take care to not | ||
1744 | make the changes machine specific. | ||
1745 | |||
1746 | Consider a case where your kernel is older and you need an older | ||
1747 | ``libc`` ABI. The headers installed by your recipe should still be a | ||
1748 | standard mainline kernel, not your own custom one. | ||
1749 | |||
1750 | When you use custom kernel headers you need to get them from | ||
1751 | :term:`STAGING_KERNEL_DIR`, | ||
1752 | which is the directory with kernel headers that are required to build | ||
1753 | out-of-tree modules. Your recipe will also need the following: | ||
1754 | :: | ||
1755 | |||
1756 | do_configure[depends] += "virtual/kernel:do_shared_workdir" | ||
1757 | |||
1758 | Compilation | ||
1759 | ----------- | ||
1760 | |||
1761 | During a build, the ``do_compile`` task happens after source is fetched, | ||
1762 | unpacked, and configured. If the recipe passes through ``do_compile`` | ||
1763 | successfully, nothing needs to be done. | ||
1764 | |||
1765 | However, if the compile step fails, you need to diagnose the failure. | ||
1766 | Here are some common issues that cause failures. | ||
1767 | |||
1768 | .. note:: | ||
1769 | |||
1770 | For cases where improper paths are detected for configuration files | ||
1771 | or for when libraries/headers cannot be found, be sure you are using | ||
1772 | the more robust ``pkg-config``. See the note in section | ||
1773 | ":ref:`dev-manual/common-tasks:Configuring the Recipe`" for additional information. | ||
1774 | |||
1775 | - *Parallel build failures:* These failures manifest themselves as | ||
1776 | intermittent errors, or errors reporting that a file or directory | ||
1777 | that should be created by some other part of the build process could | ||
1778 | not be found. This type of failure can occur even if, upon | ||
1779 | inspection, the file or directory does exist after the build has | ||
1780 | failed, because that part of the build process happened in the wrong | ||
1781 | order. | ||
1782 | |||
1783 | To fix the problem, you need to either satisfy the missing dependency | ||
1784 | in the Makefile or whatever script produced the Makefile, or (as a | ||
1785 | workaround) set :term:`PARALLEL_MAKE` to an empty string: | ||
1786 | :: | ||
1787 | |||
1788 | PARALLEL_MAKE = "" | ||
1789 | |||
1790 | For information on parallel Makefile issues, see the "`Debugging | ||
1791 | Parallel Make Races <#debugging-parallel-make-races>`__" section. | ||
1792 | |||
1793 | - *Improper host path usage:* This failure applies to recipes building | ||
1794 | for the target or ``nativesdk`` only. The failure occurs when the | ||
1795 | compilation process uses improper headers, libraries, or other files | ||
1796 | from the host system when cross-compiling for the target. | ||
1797 | |||
1798 | To fix the problem, examine the ``log.do_compile`` file to identify | ||
1799 | the host paths being used (e.g. ``/usr/include``, ``/usr/lib``, and | ||
1800 | so forth) and then either add configure options, apply a patch, or do | ||
1801 | both. | ||
1802 | |||
1803 | - *Failure to find required libraries/headers:* If a build-time | ||
1804 | dependency is missing because it has not been declared in | ||
1805 | :term:`DEPENDS`, or because the | ||
1806 | dependency exists but the path used by the build process to find the | ||
1807 | file is incorrect and the configure step did not detect it, the | ||
1808 | compilation process could fail. For either of these failures, the | ||
1809 | compilation process notes that files could not be found. In these | ||
1810 | cases, you need to go back and add additional options to the | ||
1811 | configure script as well as possibly add additional build-time | ||
1812 | dependencies to ``DEPENDS``. | ||
1813 | |||
1814 | Occasionally, it is necessary to apply a patch to the source to | ||
1815 | ensure the correct paths are used. If you need to specify paths to | ||
1816 | find files staged into the sysroot from other recipes, use the | ||
1817 | variables that the OpenEmbedded build system provides (e.g. | ||
1818 | ``STAGING_BINDIR``, ``STAGING_INCDIR``, ``STAGING_DATADIR``, and so | ||
1819 | forth). | ||
1820 | |||
1821 | Installing | ||
1822 | ---------- | ||
1823 | |||
1824 | During ``do_install``, the task copies the built files along with their | ||
1825 | hierarchy to locations that would mirror their locations on the target | ||
1826 | device. The installation process copies files from the | ||
1827 | ``${``\ :term:`S`\ ``}``, | ||
1828 | ``${``\ :term:`B`\ ``}``, and | ||
1829 | ``${``\ :term:`WORKDIR`\ ``}`` | ||
1830 | directories to the ``${``\ :term:`D`\ ``}`` | ||
1831 | directory to create the structure as it should appear on the target | ||
1832 | system. | ||
1833 | |||
1834 | How your software is built affects what you must do to be sure your | ||
1835 | software is installed correctly. The following list describes what you | ||
1836 | must do for installation depending on the type of build system used by | ||
1837 | the software being built: | ||
1838 | |||
1839 | - *Autotools and CMake:* If the software your recipe is building uses | ||
1840 | Autotools or CMake, the OpenEmbedded build system understands how to | ||
1841 | install the software. Consequently, you do not have to have a | ||
1842 | ``do_install`` task as part of your recipe. You just need to make | ||
1843 | sure the install portion of the build completes with no issues. | ||
1844 | However, if you wish to install additional files not already being | ||
1845 | installed by ``make install``, you should do this using a | ||
1846 | ``do_install_append`` function using the install command as described | ||
1847 | in the "Manual" bulleted item later in this list. | ||
1848 | |||
1849 | - *Other (using* ``make install``\ *)*: You need to define a ``do_install`` | ||
1850 | function in your recipe. The function should call | ||
1851 | ``oe_runmake install`` and will likely need to pass in the | ||
1852 | destination directory as well. How you pass that path is dependent on | ||
1853 | how the ``Makefile`` being run is written (e.g. ``DESTDIR=${D}``, | ||
1854 | ``PREFIX=${D}``, ``INSTALLROOT=${D}``, and so forth). | ||
1855 | |||
1856 | For an example recipe using ``make install``, see the | ||
1857 | "`Makefile-Based Package <#new-recipe-makefile-based-package>`__" | ||
1858 | section. | ||
1859 | |||
1860 | - *Manual:* You need to define a ``do_install`` function in your | ||
1861 | recipe. The function must first use ``install -d`` to create the | ||
1862 | directories under | ||
1863 | ``${``\ :term:`D`\ ``}``. Once the | ||
1864 | directories exist, your function can use ``install`` to manually | ||
1865 | install the built software into the directories. | ||
1866 | |||
1867 | You can find more information on ``install`` at | ||
1868 | https://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html. | ||
1869 | |||
1870 | For the scenarios that do not use Autotools or CMake, you need to track | ||
1871 | the installation and diagnose and fix any issues until everything | ||
1872 | installs correctly. You need to look in the default location of | ||
1873 | ``${D}``, which is ``${WORKDIR}/image``, to be sure your files have been | ||
1874 | installed correctly. | ||
1875 | |||
1876 | .. note:: | ||
1877 | |||
1878 | - During the installation process, you might need to modify some of | ||
1879 | the installed files to suit the target layout. For example, you | ||
1880 | might need to replace hard-coded paths in an initscript with | ||
1881 | values of variables provided by the build system, such as | ||
1882 | replacing ``/usr/bin/`` with ``${bindir}``. If you do perform such | ||
1883 | modifications during ``do_install``, be sure to modify the | ||
1884 | destination file after copying rather than before copying. | ||
1885 | Modifying after copying ensures that the build system can | ||
1886 | re-execute ``do_install`` if needed. | ||
1887 | |||
1888 | - ``oe_runmake install``, which can be run directly or can be run | ||
1889 | indirectly by the | ||
1890 | :ref:`autotools <ref-classes-autotools>` and | ||
1891 | :ref:`cmake <ref-classes-cmake>` classes, | ||
1892 | runs ``make install`` in parallel. Sometimes, a Makefile can have | ||
1893 | missing dependencies between targets that can result in race | ||
1894 | conditions. If you experience intermittent failures during | ||
1895 | ``do_install``, you might be able to work around them by disabling | ||
1896 | parallel Makefile installs by adding the following to the recipe: | ||
1897 | :: | ||
1898 | |||
1899 | PARALLEL_MAKEINST = "" | ||
1900 | |||
1901 | See :term:`PARALLEL_MAKEINST` for additional information. | ||
1902 | |||
1903 | - If you need to install one or more custom CMake toolchain files | ||
1904 | that are supplied by the application you are building, install the | ||
1905 | files to ``${D}${datadir}/cmake/Modules`` during | ||
1906 | :ref:`ref-tasks-install`. | ||
1907 | |||
1908 | Enabling System Services | ||
1909 | ------------------------ | ||
1910 | |||
1911 | If you want to install a service, which is a process that usually starts | ||
1912 | on boot and runs in the background, then you must include some | ||
1913 | additional definitions in your recipe. | ||
1914 | |||
1915 | If you are adding services and the service initialization script or the | ||
1916 | service file itself is not installed, you must provide for that | ||
1917 | installation in your recipe using a ``do_install_append`` function. If | ||
1918 | your recipe already has a ``do_install`` function, update the function | ||
1919 | near its end rather than adding an additional ``do_install_append`` | ||
1920 | function. | ||
1921 | |||
1922 | When you create the installation for your services, you need to | ||
1923 | accomplish what is normally done by ``make install``. In other words, | ||
1924 | make sure your installation arranges the output similar to how it is | ||
1925 | arranged on the target system. | ||
1926 | |||
1927 | The OpenEmbedded build system provides support for starting services two | ||
1928 | different ways: | ||
1929 | |||
1930 | - *SysVinit:* SysVinit is a system and service manager that manages the | ||
1931 | init system used to control the very basic functions of your system. | ||
1932 | The init program is the first program started by the Linux kernel | ||
1933 | when the system boots. Init then controls the startup, running and | ||
1934 | shutdown of all other programs. | ||
1935 | |||
1936 | To enable a service using SysVinit, your recipe needs to inherit the | ||
1937 | :ref:`update-rc.d <ref-classes-update-rc.d>` | ||
1938 | class. The class helps facilitate safely installing the package on | ||
1939 | the target. | ||
1940 | |||
1941 | You will need to set the | ||
1942 | :term:`INITSCRIPT_PACKAGES`, | ||
1943 | :term:`INITSCRIPT_NAME`, | ||
1944 | and | ||
1945 | :term:`INITSCRIPT_PARAMS` | ||
1946 | variables within your recipe. | ||
1947 | |||
1948 | - *systemd:* System Management Daemon (systemd) was designed to replace | ||
1949 | SysVinit and to provide enhanced management of services. For more | ||
1950 | information on systemd, see the systemd homepage at | ||
1951 | https://freedesktop.org/wiki/Software/systemd/. | ||
1952 | |||
1953 | To enable a service using systemd, your recipe needs to inherit the | ||
1954 | :ref:`systemd <ref-classes-systemd>` class. See | ||
1955 | the ``systemd.bbclass`` file located in your :term:`Source Directory` | ||
1956 | section for | ||
1957 | more information. | ||
1958 | |||
1959 | Packaging | ||
1960 | --------- | ||
1961 | |||
1962 | Successful packaging is a combination of automated processes performed | ||
1963 | by the OpenEmbedded build system and some specific steps you need to | ||
1964 | take. The following list describes the process: | ||
1965 | |||
1966 | - *Splitting Files*: The ``do_package`` task splits the files produced | ||
1967 | by the recipe into logical components. Even software that produces a | ||
1968 | single binary might still have debug symbols, documentation, and | ||
1969 | other logical components that should be split out. The ``do_package`` | ||
1970 | task ensures that files are split up and packaged correctly. | ||
1971 | |||
1972 | - *Running QA Checks*: The | ||
1973 | :ref:`insane <ref-classes-insane>` class adds a | ||
1974 | step to the package generation process so that output quality | ||
1975 | assurance checks are generated by the OpenEmbedded build system. This | ||
1976 | step performs a range of checks to be sure the build's output is free | ||
1977 | of common problems that show up during runtime. For information on | ||
1978 | these checks, see the | ||
1979 | :ref:`insane <ref-classes-insane>` class and | ||
1980 | the ":ref:`ref-manual/qa-checks:qa error and warning messages`" | ||
1981 | chapter in the Yocto Project Reference Manual. | ||
1982 | |||
1983 | - *Hand-Checking Your Packages*: After you build your software, you | ||
1984 | need to be sure your packages are correct. Examine the | ||
1985 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
1986 | directory and make sure files are where you expect them to be. If you | ||
1987 | discover problems, you can set | ||
1988 | :term:`PACKAGES`, | ||
1989 | :term:`FILES`, | ||
1990 | ``do_install(_append)``, and so forth as needed. | ||
1991 | |||
1992 | - *Splitting an Application into Multiple Packages*: If you need to | ||
1993 | split an application into several packages, see the "`Splitting an | ||
1994 | Application into Multiple | ||
1995 | Packages <#splitting-an-application-into-multiple-packages>`__" | ||
1996 | section for an example. | ||
1997 | |||
1998 | - *Installing a Post-Installation Script*: For an example showing how | ||
1999 | to install a post-installation script, see the "`Post-Installation | ||
2000 | Scripts <#new-recipe-post-installation-scripts>`__" section. | ||
2001 | |||
2002 | - *Marking Package Architecture*: Depending on what your recipe is | ||
2003 | building and how it is configured, it might be important to mark the | ||
2004 | packages produced as being specific to a particular machine, or to | ||
2005 | mark them as not being specific to a particular machine or | ||
2006 | architecture at all. | ||
2007 | |||
2008 | By default, packages apply to any machine with the same architecture | ||
2009 | as the target machine. When a recipe produces packages that are | ||
2010 | machine-specific (e.g. the | ||
2011 | :term:`MACHINE` value is passed | ||
2012 | into the configure script or a patch is applied only for a particular | ||
2013 | machine), you should mark them as such by adding the following to the | ||
2014 | recipe: | ||
2015 | :: | ||
2016 | |||
2017 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
2018 | |||
2019 | On the other hand, if the recipe produces packages that do not | ||
2020 | contain anything specific to the target machine or architecture at | ||
2021 | all (e.g. recipes that simply package script files or configuration | ||
2022 | files), you should use the | ||
2023 | :ref:`allarch <ref-classes-allarch>` class to | ||
2024 | do this for you by adding this to your recipe: | ||
2025 | :: | ||
2026 | |||
2027 | inherit allarch | ||
2028 | |||
2029 | Ensuring that the package architecture is correct is not critical | ||
2030 | while you are doing the first few builds of your recipe. However, it | ||
2031 | is important in order to ensure that your recipe rebuilds (or does | ||
2032 | not rebuild) appropriately in response to changes in configuration, | ||
2033 | and to ensure that you get the appropriate packages installed on the | ||
2034 | target machine, particularly if you run separate builds for more than | ||
2035 | one target machine. | ||
2036 | |||
2037 | Sharing Files Between Recipes | ||
2038 | ----------------------------- | ||
2039 | |||
2040 | Recipes often need to use files provided by other recipes on the build | ||
2041 | host. For example, an application linking to a common library needs | ||
2042 | access to the library itself and its associated headers. The way this | ||
2043 | access is accomplished is by populating a sysroot with files. Each | ||
2044 | recipe has two sysroots in its work directory, one for target files | ||
2045 | (``recipe-sysroot``) and one for files that are native to the build host | ||
2046 | (``recipe-sysroot-native``). | ||
2047 | |||
2048 | .. note:: | ||
2049 | |||
2050 | You could find the term "staging" used within the Yocto project | ||
2051 | regarding files populating sysroots (e.g. the :term:`STAGING_DIR` | ||
2052 | variable). | ||
2053 | |||
2054 | Recipes should never populate the sysroot directly (i.e. write files | ||
2055 | into sysroot). Instead, files should be installed into standard | ||
2056 | locations during the | ||
2057 | :ref:`ref-tasks-install` task within | ||
2058 | the ``${``\ :term:`D`\ ``}`` directory. The | ||
2059 | reason for this limitation is that almost all files that populate the | ||
2060 | sysroot are cataloged in manifests in order to ensure the files can be | ||
2061 | removed later when a recipe is either modified or removed. Thus, the | ||
2062 | sysroot is able to remain free from stale files. | ||
2063 | |||
2064 | A subset of the files installed by the | ||
2065 | :ref:`ref-tasks-install` task are | ||
2066 | used by the | ||
2067 | :ref:`ref-tasks-populate_sysroot` | ||
2068 | task as defined by the the | ||
2069 | :term:`SYSROOT_DIRS` variable to | ||
2070 | automatically populate the sysroot. It is possible to modify the list of | ||
2071 | directories that populate the sysroot. The following example shows how | ||
2072 | you could add the ``/opt`` directory to the list of directories within a | ||
2073 | recipe: | ||
2074 | :: | ||
2075 | |||
2076 | SYSROOT_DIRS += "/opt" | ||
2077 | |||
2078 | .. note:: | ||
2079 | |||
2080 | The `/sysroot-only` is to be used by recipes that generate artifacts | ||
2081 | that are not included in the target filesystem, allowing them to share | ||
2082 | these artifacts without needing to use the ``DEPLOY_DIR``. | ||
2083 | |||
2084 | For a more complete description of the | ||
2085 | :ref:`ref-tasks-populate_sysroot` | ||
2086 | task and its associated functions, see the | ||
2087 | :ref:`staging <ref-classes-staging>` class. | ||
2088 | |||
2089 | Using Virtual Providers | ||
2090 | ----------------------- | ||
2091 | |||
2092 | Prior to a build, if you know that several different recipes provide the | ||
2093 | same functionality, you can use a virtual provider (i.e. ``virtual/*``) | ||
2094 | as a placeholder for the actual provider. The actual provider is | ||
2095 | determined at build-time. | ||
2096 | |||
2097 | A common scenario where a virtual provider is used would be for the | ||
2098 | kernel recipe. Suppose you have three kernel recipes whose | ||
2099 | :term:`PN` values map to ``kernel-big``, | ||
2100 | ``kernel-mid``, and ``kernel-small``. Furthermore, each of these recipes | ||
2101 | in some way uses a :term:`PROVIDES` | ||
2102 | statement that essentially identifies itself as being able to provide | ||
2103 | ``virtual/kernel``. Here is one way through the | ||
2104 | :ref:`kernel <ref-classes-kernel>` class: | ||
2105 | :: | ||
2106 | |||
2107 | PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }" | ||
2108 | |||
2109 | Any recipe that inherits the ``kernel`` class is | ||
2110 | going to utilize a ``PROVIDES`` statement that identifies that recipe as | ||
2111 | being able to provide the ``virtual/kernel`` item. | ||
2112 | |||
2113 | Now comes the time to actually build an image and you need a kernel | ||
2114 | recipe, but which one? You can configure your build to call out the | ||
2115 | kernel recipe you want by using the :term:`PREFERRED_PROVIDER` variable. As | ||
2116 | an example, consider the :yocto_git:`x86-base.inc | ||
2117 | </poky/tree/meta/conf/machine/include/x86-base.inc>` include file, which is a | ||
2118 | machine (i.e. :term:`MACHINE`) configuration file. This include file is the | ||
2119 | reason all x86-based machines use the ``linux-yocto`` kernel. Here are the | ||
2120 | relevant lines from the include file: | ||
2121 | :: | ||
2122 | |||
2123 | PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto" | ||
2124 | PREFERRED_VERSION_linux-yocto ??= "4.15%" | ||
2125 | |||
2126 | When you use a virtual provider, you do not have to "hard code" a recipe | ||
2127 | name as a build dependency. You can use the | ||
2128 | :term:`DEPENDS` variable to state the | ||
2129 | build is dependent on ``virtual/kernel`` for example: | ||
2130 | :: | ||
2131 | |||
2132 | DEPENDS = "virtual/kernel" | ||
2133 | |||
2134 | During the build, the OpenEmbedded build system picks | ||
2135 | the correct recipe needed for the ``virtual/kernel`` dependency based on | ||
2136 | the ``PREFERRED_PROVIDER`` variable. If you want to use the small kernel | ||
2137 | mentioned at the beginning of this section, configure your build as | ||
2138 | follows: | ||
2139 | :: | ||
2140 | |||
2141 | PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small" | ||
2142 | |||
2143 | .. note:: | ||
2144 | |||
2145 | Any recipe that ``PROVIDES`` a ``virtual/*`` item that is ultimately not | ||
2146 | selected through ``PREFERRED_PROVIDER`` does not get built. Preventing these | ||
2147 | recipes from building is usually the desired behavior since this mechanism's | ||
2148 | purpose is to select between mutually exclusive alternative providers. | ||
2149 | |||
2150 | The following lists specific examples of virtual providers: | ||
2151 | |||
2152 | - ``virtual/kernel``: Provides the name of the kernel recipe to use | ||
2153 | when building a kernel image. | ||
2154 | |||
2155 | - ``virtual/bootloader``: Provides the name of the bootloader to use | ||
2156 | when building an image. | ||
2157 | |||
2158 | - ``virtual/libgbm``: Provides ``gbm.pc``. | ||
2159 | |||
2160 | - ``virtual/egl``: Provides ``egl.pc`` and possibly ``wayland-egl.pc``. | ||
2161 | |||
2162 | - ``virtual/libgl``: Provides ``gl.pc`` (i.e. libGL). | ||
2163 | |||
2164 | - ``virtual/libgles1``: Provides ``glesv1_cm.pc`` (i.e. libGLESv1_CM). | ||
2165 | |||
2166 | - ``virtual/libgles2``: Provides ``glesv2.pc`` (i.e. libGLESv2). | ||
2167 | |||
2168 | .. note:: | ||
2169 | |||
2170 | Virtual providers only apply to build time dependencies specified with | ||
2171 | :term:`PROVIDES` and :term:`DEPENDS`. They do not apply to runtime | ||
2172 | dependencies specified with :term:`RPROVIDES` and :term:`RDEPENDS`. | ||
2173 | |||
2174 | Properly Versioning Pre-Release Recipes | ||
2175 | --------------------------------------- | ||
2176 | |||
2177 | Sometimes the name of a recipe can lead to versioning problems when the | ||
2178 | recipe is upgraded to a final release. For example, consider the | ||
2179 | ``irssi_0.8.16-rc1.bb`` recipe file in the list of example recipes in | ||
2180 | the "`Storing and Naming the | ||
2181 | Recipe <#new-recipe-storing-and-naming-the-recipe>`__" section. This | ||
2182 | recipe is at a release candidate stage (i.e. "rc1"). When the recipe is | ||
2183 | released, the recipe filename becomes ``irssi_0.8.16.bb``. The version | ||
2184 | change from ``0.8.16-rc1`` to ``0.8.16`` is seen as a decrease by the | ||
2185 | build system and package managers, so the resulting packages will not | ||
2186 | correctly trigger an upgrade. | ||
2187 | |||
2188 | In order to ensure the versions compare properly, the recommended | ||
2189 | convention is to set :term:`PV` within the | ||
2190 | recipe to "previous_version+current_version". You can use an additional | ||
2191 | variable so that you can use the current version elsewhere. Here is an | ||
2192 | example: | ||
2193 | :: | ||
2194 | |||
2195 | REALPV = "0.8.16-rc1" | ||
2196 | PV = "0.8.15+${REALPV}" | ||
2197 | |||
2198 | Post-Installation Scripts | ||
2199 | ------------------------- | ||
2200 | |||
2201 | Post-installation scripts run immediately after installing a package on | ||
2202 | the target or during image creation when a package is included in an | ||
2203 | image. To add a post-installation script to a package, add a | ||
2204 | ``pkg_postinst_``\ `PACKAGENAME`\ ``()`` function to the recipe file | ||
2205 | (``.bb``) and replace `PACKAGENAME` with the name of the package you want | ||
2206 | to attach to the ``postinst`` script. To apply the post-installation | ||
2207 | script to the main package for the recipe, which is usually what is | ||
2208 | required, specify | ||
2209 | ``${``\ :term:`PN`\ ``}`` in place of | ||
2210 | PACKAGENAME. | ||
2211 | |||
2212 | A post-installation function has the following structure: | ||
2213 | :: | ||
2214 | |||
2215 | pkg_postinst_PACKAGENAME() { | ||
2216 | # Commands to carry out | ||
2217 | } | ||
2218 | |||
2219 | The script defined in the post-installation function is called when the | ||
2220 | root filesystem is created. If the script succeeds, the package is | ||
2221 | marked as installed. | ||
2222 | |||
2223 | .. note:: | ||
2224 | |||
2225 | Any RPM post-installation script that runs on the target should | ||
2226 | return a 0 exit code. RPM does not allow non-zero exit codes for | ||
2227 | these scripts, and the RPM package manager will cause the package to | ||
2228 | fail installation on the target. | ||
2229 | |||
2230 | Sometimes it is necessary for the execution of a post-installation | ||
2231 | script to be delayed until the first boot. For example, the script might | ||
2232 | need to be executed on the device itself. To delay script execution | ||
2233 | until boot time, you must explicitly mark post installs to defer to the | ||
2234 | target. You can use ``pkg_postinst_ontarget()`` or call | ||
2235 | ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``. Any | ||
2236 | failure of a ``pkg_postinst()`` script (including exit 1) triggers an | ||
2237 | error during the | ||
2238 | :ref:`ref-tasks-rootfs` task. | ||
2239 | |||
2240 | If you have recipes that use ``pkg_postinst`` function and they require | ||
2241 | the use of non-standard native tools that have dependencies during | ||
2242 | rootfs construction, you need to use the | ||
2243 | :term:`PACKAGE_WRITE_DEPS` | ||
2244 | variable in your recipe to list these tools. If you do not use this | ||
2245 | variable, the tools might be missing and execution of the | ||
2246 | post-installation script is deferred until first boot. Deferring the | ||
2247 | script to first boot is undesirable and for read-only rootfs impossible. | ||
2248 | |||
2249 | .. note:: | ||
2250 | |||
2251 | Equivalent support for pre-install, pre-uninstall, and post-uninstall | ||
2252 | scripts exist by way of ``pkg_preinst``, ``pkg_prerm``, and ``pkg_postrm``, | ||
2253 | respectively. These scrips work in exactly the same way as does | ||
2254 | ``pkg_postinst`` with the exception that they run at different times. Also, | ||
2255 | because of when they run, they are not applicable to being run at image | ||
2256 | creation time like ``pkg_postinst``. | ||
2257 | |||
2258 | Testing | ||
2259 | ------- | ||
2260 | |||
2261 | The final step for completing your recipe is to be sure that the | ||
2262 | software you built runs correctly. To accomplish runtime testing, add | ||
2263 | the build's output packages to your image and test them on the target. | ||
2264 | |||
2265 | For information on how to customize your image by adding specific | ||
2266 | packages, see the "`Customizing | ||
2267 | Images <#usingpoky-extend-customimage>`__" section. | ||
2268 | |||
2269 | Examples | ||
2270 | -------- | ||
2271 | |||
2272 | To help summarize how to write a recipe, this section provides some | ||
2273 | examples given various scenarios: | ||
2274 | |||
2275 | - Recipes that use local files | ||
2276 | |||
2277 | - Using an Autotooled package | ||
2278 | |||
2279 | - Using a Makefile-based package | ||
2280 | |||
2281 | - Splitting an application into multiple packages | ||
2282 | |||
2283 | - Adding binaries to an image | ||
2284 | |||
2285 | Single .c File Package (Hello World!) | ||
2286 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2287 | |||
2288 | Building an application from a single file that is stored locally (e.g. | ||
2289 | under ``files``) requires a recipe that has the file listed in the | ||
2290 | ``SRC_URI`` variable. Additionally, you need to manually write the | ||
2291 | ``do_compile`` and ``do_install`` tasks. The ``S`` variable defines the | ||
2292 | directory containing the source code, which is set to | ||
2293 | :term:`WORKDIR` in this case - the | ||
2294 | directory BitBake uses for the build. | ||
2295 | :: | ||
2296 | |||
2297 | SUMMARY = "Simple helloworld application" | ||
2298 | SECTION = "examples" | ||
2299 | LICENSE = "MIT" | ||
2300 | LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" | ||
2301 | |||
2302 | SRC_URI = "file://helloworld.c" | ||
2303 | |||
2304 | S = "${WORKDIR}" | ||
2305 | |||
2306 | do_compile() { | ||
2307 | ${CC} ${LDFLAGS} helloworld.c -o helloworld | ||
2308 | } | ||
2309 | |||
2310 | do_install() { | ||
2311 | install -d ${D}${bindir} | ||
2312 | install -m 0755 helloworld ${D}${bindir} | ||
2313 | } | ||
2314 | |||
2315 | By default, the ``helloworld``, ``helloworld-dbg``, and | ||
2316 | ``helloworld-dev`` packages are built. For information on how to | ||
2317 | customize the packaging process, see the "`Splitting an Application into | ||
2318 | Multiple Packages <#splitting-an-application-into-multiple-packages>`__" | ||
2319 | section. | ||
2320 | |||
2321 | Autotooled Package | ||
2322 | ~~~~~~~~~~~~~~~~~~ | ||
2323 | |||
2324 | Applications that use Autotools such as ``autoconf`` and ``automake`` | ||
2325 | require a recipe that has a source archive listed in ``SRC_URI`` and | ||
2326 | also inherit the | ||
2327 | :ref:`autotools <ref-classes-autotools>` class, | ||
2328 | which contains the definitions of all the steps needed to build an | ||
2329 | Autotool-based application. The result of the build is automatically | ||
2330 | packaged. And, if the application uses NLS for localization, packages | ||
2331 | with local information are generated (one package per language). | ||
2332 | Following is one example: (``hello_2.3.bb``) | ||
2333 | :: | ||
2334 | |||
2335 | SUMMARY = "GNU Helloworld application" | ||
2336 | SECTION = "examples" | ||
2337 | LICENSE = "GPLv2+" | ||
2338 | LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" | ||
2339 | |||
2340 | SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz" | ||
2341 | |||
2342 | inherit autotools gettext | ||
2343 | |||
2344 | The variable ``LIC_FILES_CHKSUM`` is used to track source license | ||
2345 | changes as described in the | ||
2346 | ":ref:`dev-manual/common-tasks:tracking license changes`" section in | ||
2347 | the Yocto Project Overview and Concepts Manual. You can quickly create | ||
2348 | Autotool-based recipes in a manner similar to the previous example. | ||
2349 | |||
2350 | Makefile-Based Package | ||
2351 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
2352 | |||
2353 | Applications that use GNU ``make`` also require a recipe that has the | ||
2354 | source archive listed in ``SRC_URI``. You do not need to add a | ||
2355 | ``do_compile`` step since by default BitBake starts the ``make`` command | ||
2356 | to compile the application. If you need additional ``make`` options, you | ||
2357 | should store them in the | ||
2358 | :term:`EXTRA_OEMAKE` or | ||
2359 | :term:`PACKAGECONFIG_CONFARGS` | ||
2360 | variables. BitBake passes these options into the GNU ``make`` | ||
2361 | invocation. Note that a ``do_install`` task is still required. | ||
2362 | Otherwise, BitBake runs an empty ``do_install`` task by default. | ||
2363 | |||
2364 | Some applications might require extra parameters to be passed to the | ||
2365 | compiler. For example, the application might need an additional header | ||
2366 | path. You can accomplish this by adding to the ``CFLAGS`` variable. The | ||
2367 | following example shows this: | ||
2368 | :: | ||
2369 | |||
2370 | CFLAGS_prepend = "-I ${S}/include " | ||
2371 | |||
2372 | In the following example, ``mtd-utils`` is a makefile-based package: | ||
2373 | :: | ||
2374 | |||
2375 | SUMMARY = "Tools for managing memory technology devices" | ||
2376 | SECTION = "base" | ||
2377 | DEPENDS = "zlib lzo e2fsprogs util-linux" | ||
2378 | HOMEPAGE = "http://www.linux-mtd.infradead.org/" | ||
2379 | LICENSE = "GPLv2+" | ||
2380 | LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \ | ||
2381 | file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c" | ||
2382 | |||
2383 | # Use the latest version at 26 Oct, 2013 | ||
2384 | SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b" | ||
2385 | SRC_URI = "git://git.infradead.org/mtd-utils.git \ | ||
2386 | file://add-exclusion-to-mkfs-jffs2-git-2.patch \ | ||
2387 | " | ||
2388 | |||
2389 | PV = "1.5.1+git${SRCPV}" | ||
2390 | |||
2391 | S = "${WORKDIR}/git" | ||
2392 | |||
2393 | EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'" | ||
2394 | |||
2395 | do_install () { | ||
2396 | oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir} | ||
2397 | } | ||
2398 | |||
2399 | PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc" | ||
2400 | |||
2401 | FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool" | ||
2402 | FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*" | ||
2403 | FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image" | ||
2404 | |||
2405 | PARALLEL_MAKE = "" | ||
2406 | |||
2407 | BBCLASSEXTEND = "native" | ||
2408 | |||
2409 | Splitting an Application into Multiple Packages | ||
2410 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2411 | |||
2412 | You can use the variables ``PACKAGES`` and ``FILES`` to split an | ||
2413 | application into multiple packages. | ||
2414 | |||
2415 | Following is an example that uses the ``libxpm`` recipe. By default, | ||
2416 | this recipe generates a single package that contains the library along | ||
2417 | with a few binaries. You can modify the recipe to split the binaries | ||
2418 | into separate packages: | ||
2419 | :: | ||
2420 | |||
2421 | require xorg-lib-common.inc | ||
2422 | |||
2423 | SUMMARY = "Xpm: X Pixmap extension library" | ||
2424 | LICENSE = "BSD" | ||
2425 | LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" | ||
2426 | DEPENDS += "libxext libsm libxt" | ||
2427 | PE = "1" | ||
2428 | |||
2429 | XORG_PN = "libXpm" | ||
2430 | |||
2431 | PACKAGES =+ "sxpm cxpm" | ||
2432 | FILES_cxpm = "${bindir}/cxpm" | ||
2433 | FILES_sxpm = "${bindir}/sxpm" | ||
2434 | |||
2435 | In the previous example, we want to ship the ``sxpm`` and ``cxpm`` | ||
2436 | binaries in separate packages. Since ``bindir`` would be packaged into | ||
2437 | the main ``PN`` package by default, we prepend the ``PACKAGES`` variable | ||
2438 | so additional package names are added to the start of list. This results | ||
2439 | in the extra ``FILES_*`` variables then containing information that | ||
2440 | define which files and directories go into which packages. Files | ||
2441 | included by earlier packages are skipped by latter packages. Thus, the | ||
2442 | main ``PN`` package does not include the above listed files. | ||
2443 | |||
2444 | Packaging Externally Produced Binaries | ||
2445 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2446 | |||
2447 | Sometimes, you need to add pre-compiled binaries to an image. For | ||
2448 | example, suppose that binaries for proprietary code exist, which are | ||
2449 | created by a particular division of a company. Your part of the company | ||
2450 | needs to use those binaries as part of an image that you are building | ||
2451 | using the OpenEmbedded build system. Since you only have the binaries | ||
2452 | and not the source code, you cannot use a typical recipe that expects to | ||
2453 | fetch the source specified in | ||
2454 | :term:`SRC_URI` and then compile it. | ||
2455 | |||
2456 | One method is to package the binaries and then install them as part of | ||
2457 | the image. Generally, it is not a good idea to package binaries since, | ||
2458 | among other things, it can hinder the ability to reproduce builds and | ||
2459 | could lead to compatibility problems with ABI in the future. However, | ||
2460 | sometimes you have no choice. | ||
2461 | |||
2462 | The easiest solution is to create a recipe that uses the | ||
2463 | :ref:`bin_package <ref-classes-bin-package>` class | ||
2464 | and to be sure that you are using default locations for build artifacts. | ||
2465 | In most cases, the ``bin_package`` class handles "skipping" the | ||
2466 | configure and compile steps as well as sets things up to grab packages | ||
2467 | from the appropriate area. In particular, this class sets ``noexec`` on | ||
2468 | both the :ref:`ref-tasks-configure` | ||
2469 | and :ref:`ref-tasks-compile` tasks, | ||
2470 | sets ``FILES_${PN}`` to "/" so that it picks up all files, and sets up a | ||
2471 | :ref:`ref-tasks-install` task, which | ||
2472 | effectively copies all files from ``${S}`` to ``${D}``. The | ||
2473 | ``bin_package`` class works well when the files extracted into ``${S}`` | ||
2474 | are already laid out in the way they should be laid out on the target. | ||
2475 | For more information on these variables, see the | ||
2476 | :term:`FILES`, | ||
2477 | :term:`PN`, | ||
2478 | :term:`S`, and | ||
2479 | :term:`D` variables in the Yocto Project | ||
2480 | Reference Manual's variable glossary. | ||
2481 | |||
2482 | .. note:: | ||
2483 | |||
2484 | - Using :term:`DEPENDS` is a good | ||
2485 | idea even for components distributed in binary form, and is often | ||
2486 | necessary for shared libraries. For a shared library, listing the | ||
2487 | library dependencies in ``DEPENDS`` makes sure that the libraries | ||
2488 | are available in the staging sysroot when other recipes link | ||
2489 | against the library, which might be necessary for successful | ||
2490 | linking. | ||
2491 | |||
2492 | - Using ``DEPENDS`` also allows runtime dependencies between | ||
2493 | packages to be added automatically. See the | ||
2494 | ":ref:`overview-manual/concepts:automatically added runtime dependencies`" | ||
2495 | section in the Yocto Project Overview and Concepts Manual for more | ||
2496 | information. | ||
2497 | |||
2498 | If you cannot use the ``bin_package`` class, you need to be sure you are | ||
2499 | doing the following: | ||
2500 | |||
2501 | - Create a recipe where the | ||
2502 | :ref:`ref-tasks-configure` and | ||
2503 | :ref:`ref-tasks-compile` tasks do | ||
2504 | nothing: It is usually sufficient to just not define these tasks in | ||
2505 | the recipe, because the default implementations do nothing unless a | ||
2506 | Makefile is found in | ||
2507 | ``${``\ :term:`S`\ ``}``. | ||
2508 | |||
2509 | If ``${S}`` might contain a Makefile, or if you inherit some class | ||
2510 | that replaces ``do_configure`` and ``do_compile`` with custom | ||
2511 | versions, then you can use the | ||
2512 | ``[``\ :ref:`noexec <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]`` | ||
2513 | flag to turn the tasks into no-ops, as follows: | ||
2514 | :: | ||
2515 | |||
2516 | do_configure[noexec] = "1" | ||
2517 | do_compile[noexec] = "1" | ||
2518 | |||
2519 | Unlike | ||
2520 | :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:deleting a task`, | ||
2521 | using the flag preserves the dependency chain from the | ||
2522 | :ref:`ref-tasks-fetch`, | ||
2523 | :ref:`ref-tasks-unpack`, and | ||
2524 | :ref:`ref-tasks-patch` tasks to the | ||
2525 | :ref:`ref-tasks-install` task. | ||
2526 | |||
2527 | - Make sure your ``do_install`` task installs the binaries | ||
2528 | appropriately. | ||
2529 | |||
2530 | - Ensure that you set up :term:`FILES` | ||
2531 | (usually | ||
2532 | ``FILES_${``\ :term:`PN`\ ``}``) to | ||
2533 | point to the files you have installed, which of course depends on | ||
2534 | where you have installed them and whether those files are in | ||
2535 | different locations than the defaults. | ||
2536 | |||
2537 | .. note:: | ||
2538 | |||
2539 | If image prelinking is enabled (e.g. "image-prelink" is in :term:`USER_CLASSES` | ||
2540 | which it is by default), prelink will change the binaries in the generated images | ||
2541 | and this often catches people out. Remove that class to ensure binaries are | ||
2542 | preserved exactly if that is necessary. | ||
2543 | |||
2544 | Following Recipe Style Guidelines | ||
2545 | --------------------------------- | ||
2546 | |||
2547 | When writing recipes, it is good to conform to existing style | ||
2548 | guidelines. The :oe_wiki:`OpenEmbedded Styleguide </Styleguide>` wiki page | ||
2549 | provides rough guidelines for preferred recipe style. | ||
2550 | |||
2551 | It is common for existing recipes to deviate a bit from this style. | ||
2552 | However, aiming for at least a consistent style is a good idea. Some | ||
2553 | practices, such as omitting spaces around ``=`` operators in assignments | ||
2554 | or ordering recipe components in an erratic way, are widely seen as poor | ||
2555 | style. | ||
2556 | |||
2557 | Recipe Syntax | ||
2558 | ------------- | ||
2559 | |||
2560 | Understanding recipe file syntax is important for writing recipes. The | ||
2561 | following list overviews the basic items that make up a BitBake recipe | ||
2562 | file. For more complete BitBake syntax descriptions, see the | ||
2563 | ":doc:`bitbake-user-manual/bitbake-user-manual-metadata`" | ||
2564 | chapter of the BitBake User Manual. | ||
2565 | |||
2566 | - *Variable Assignments and Manipulations:* Variable assignments allow | ||
2567 | a value to be assigned to a variable. The assignment can be static | ||
2568 | text or might include the contents of other variables. In addition to | ||
2569 | the assignment, appending and prepending operations are also | ||
2570 | supported. | ||
2571 | |||
2572 | The following example shows some of the ways you can use variables in | ||
2573 | recipes: | ||
2574 | :: | ||
2575 | |||
2576 | S = "${WORKDIR}/postfix-${PV}" | ||
2577 | CFLAGS += "-DNO_ASM" | ||
2578 | SRC_URI_append = " file://fixup.patch" | ||
2579 | |||
2580 | - *Functions:* Functions provide a series of actions to be performed. | ||
2581 | You usually use functions to override the default implementation of a | ||
2582 | task function or to complement a default function (i.e. append or | ||
2583 | prepend to an existing function). Standard functions use ``sh`` shell | ||
2584 | syntax, although access to OpenEmbedded variables and internal | ||
2585 | methods are also available. | ||
2586 | |||
2587 | The following is an example function from the ``sed`` recipe: | ||
2588 | :: | ||
2589 | |||
2590 | do_install () { | ||
2591 | autotools_do_install | ||
2592 | install -d ${D}${base_bindir} | ||
2593 | mv ${D}${bindir}/sed ${D}${base_bindir}/sed | ||
2594 | rmdir ${D}${bindir}/ | ||
2595 | } | ||
2596 | |||
2597 | It is | ||
2598 | also possible to implement new functions that are called between | ||
2599 | existing tasks as long as the new functions are not replacing or | ||
2600 | complementing the default functions. You can implement functions in | ||
2601 | Python instead of shell. Both of these options are not seen in the | ||
2602 | majority of recipes. | ||
2603 | |||
2604 | - *Keywords:* BitBake recipes use only a few keywords. You use keywords | ||
2605 | to include common functions (``inherit``), load parts of a recipe | ||
2606 | from other files (``include`` and ``require``) and export variables | ||
2607 | to the environment (``export``). | ||
2608 | |||
2609 | The following example shows the use of some of these keywords: | ||
2610 | :: | ||
2611 | |||
2612 | export POSTCONF = "${STAGING_BINDIR}/postconf" | ||
2613 | inherit autoconf | ||
2614 | require otherfile.inc | ||
2615 | |||
2616 | - *Comments (#):* Any lines that begin with the hash character (``#``) | ||
2617 | are treated as comment lines and are ignored: | ||
2618 | :: | ||
2619 | |||
2620 | # This is a comment | ||
2621 | |||
2622 | This next list summarizes the most important and most commonly used | ||
2623 | parts of the recipe syntax. For more information on these parts of the | ||
2624 | syntax, you can reference the | ||
2625 | :doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata` chapter | ||
2626 | in the BitBake User Manual. | ||
2627 | |||
2628 | - *Line Continuation (\\):* Use the backward slash (``\``) character to | ||
2629 | split a statement over multiple lines. Place the slash character at | ||
2630 | the end of the line that is to be continued on the next line: | ||
2631 | :: | ||
2632 | |||
2633 | VAR = "A really long \ | ||
2634 | line" | ||
2635 | |||
2636 | .. note:: | ||
2637 | |||
2638 | You cannot have any characters including spaces or tabs after the | ||
2639 | slash character. | ||
2640 | |||
2641 | - *Using Variables (${VARNAME}):* Use the ``${VARNAME}`` syntax to | ||
2642 | access the contents of a variable: | ||
2643 | :: | ||
2644 | |||
2645 | SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz" | ||
2646 | |||
2647 | .. note:: | ||
2648 | |||
2649 | It is important to understand that the value of a variable | ||
2650 | expressed in this form does not get substituted automatically. The | ||
2651 | expansion of these expressions happens on-demand later (e.g. | ||
2652 | usually when a function that makes reference to the variable | ||
2653 | executes). This behavior ensures that the values are most | ||
2654 | appropriate for the context in which they are finally used. On the | ||
2655 | rare occasion that you do need the variable expression to be | ||
2656 | expanded immediately, you can use the | ||
2657 | := | ||
2658 | operator instead of | ||
2659 | = | ||
2660 | when you make the assignment, but this is not generally needed. | ||
2661 | |||
2662 | - *Quote All Assignments ("value"):* Use double quotes around values in | ||
2663 | all variable assignments (e.g. ``"value"``). Following is an example: | ||
2664 | :: | ||
2665 | |||
2666 | VAR1 = "${OTHERVAR}" | ||
2667 | VAR2 = "The version is ${PV}" | ||
2668 | |||
2669 | - *Conditional Assignment (?=):* Conditional assignment is used to | ||
2670 | assign a value to a variable, but only when the variable is currently | ||
2671 | unset. Use the question mark followed by the equal sign (``?=``) to | ||
2672 | make a "soft" assignment used for conditional assignment. Typically, | ||
2673 | "soft" assignments are used in the ``local.conf`` file for variables | ||
2674 | that are allowed to come through from the external environment. | ||
2675 | |||
2676 | Here is an example where ``VAR1`` is set to "New value" if it is | ||
2677 | currently empty. However, if ``VAR1`` has already been set, it | ||
2678 | remains unchanged: | ||
2679 | :: | ||
2680 | |||
2681 | VAR1 ?= "New value" | ||
2682 | |||
2683 | In this next example, ``VAR1`` is left with the value "Original value": | ||
2684 | :: | ||
2685 | |||
2686 | VAR1 = "Original value" | ||
2687 | VAR1 ?= "New value" | ||
2688 | |||
2689 | - *Appending (+=):* Use the plus character followed by the equals sign | ||
2690 | (``+=``) to append values to existing variables. | ||
2691 | |||
2692 | .. note:: | ||
2693 | |||
2694 | This operator adds a space between the existing content of the | ||
2695 | variable and the new content. | ||
2696 | |||
2697 | Here is an example: | ||
2698 | :: | ||
2699 | |||
2700 | SRC_URI += "file://fix-makefile.patch" | ||
2701 | |||
2702 | - *Prepending (=+):* Use the equals sign followed by the plus character | ||
2703 | (``=+``) to prepend values to existing variables. | ||
2704 | |||
2705 | .. note:: | ||
2706 | |||
2707 | This operator adds a space between the new content and the | ||
2708 | existing content of the variable. | ||
2709 | |||
2710 | Here is an example: | ||
2711 | :: | ||
2712 | |||
2713 | VAR =+ "Starts" | ||
2714 | |||
2715 | - *Appending (_append):* Use the ``_append`` operator to append values | ||
2716 | to existing variables. This operator does not add any additional | ||
2717 | space. Also, the operator is applied after all the ``+=``, and ``=+`` | ||
2718 | operators have been applied and after all ``=`` assignments have | ||
2719 | occurred. | ||
2720 | |||
2721 | The following example shows the space being explicitly added to the | ||
2722 | start to ensure the appended value is not merged with the existing | ||
2723 | value: | ||
2724 | :: | ||
2725 | |||
2726 | SRC_URI_append = " file://fix-makefile.patch" | ||
2727 | |||
2728 | You can also use | ||
2729 | the ``_append`` operator with overrides, which results in the actions | ||
2730 | only being performed for the specified target or machine: | ||
2731 | :: | ||
2732 | |||
2733 | SRC_URI_append_sh4 = " file://fix-makefile.patch" | ||
2734 | |||
2735 | - *Prepending (_prepend):* Use the ``_prepend`` operator to prepend | ||
2736 | values to existing variables. This operator does not add any | ||
2737 | additional space. Also, the operator is applied after all the ``+=``, | ||
2738 | and ``=+`` operators have been applied and after all ``=`` | ||
2739 | assignments have occurred. | ||
2740 | |||
2741 | The following example shows the space being explicitly added to the | ||
2742 | end to ensure the prepended value is not merged with the existing | ||
2743 | value: | ||
2744 | :: | ||
2745 | |||
2746 | CFLAGS_prepend = "-I${S}/myincludes " | ||
2747 | |||
2748 | You can also use the | ||
2749 | ``_prepend`` operator with overrides, which results in the actions | ||
2750 | only being performed for the specified target or machine: | ||
2751 | :: | ||
2752 | |||
2753 | CFLAGS_prepend_sh4 = "-I${S}/myincludes " | ||
2754 | |||
2755 | - *Overrides:* You can use overrides to set a value conditionally, | ||
2756 | typically based on how the recipe is being built. For example, to set | ||
2757 | the :term:`KBRANCH` variable's | ||
2758 | value to "standard/base" for any target | ||
2759 | :term:`MACHINE`, except for | ||
2760 | qemuarm where it should be set to "standard/arm-versatile-926ejs", | ||
2761 | you would do the following: | ||
2762 | :: | ||
2763 | |||
2764 | KBRANCH = "standard/base" | ||
2765 | KBRANCH_qemuarm = "standard/arm-versatile-926ejs" | ||
2766 | |||
2767 | Overrides are also used to separate | ||
2768 | alternate values of a variable in other situations. For example, when | ||
2769 | setting variables such as | ||
2770 | :term:`FILES` and | ||
2771 | :term:`RDEPENDS` that are | ||
2772 | specific to individual packages produced by a recipe, you should | ||
2773 | always use an override that specifies the name of the package. | ||
2774 | |||
2775 | - *Indentation:* Use spaces for indentation rather than tabs. For | ||
2776 | shell functions, both currently work. However, it is a policy | ||
2777 | decision of the Yocto Project to use tabs in shell functions. Realize | ||
2778 | that some layers have a policy to use spaces for all indentation. | ||
2779 | |||
2780 | - *Using Python for Complex Operations:* For more advanced processing, | ||
2781 | it is possible to use Python code during variable assignments (e.g. | ||
2782 | search and replacement on a variable). | ||
2783 | |||
2784 | You indicate Python code using the ``${@python_code}`` syntax for the | ||
2785 | variable assignment: | ||
2786 | :: | ||
2787 | |||
2788 | SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz | ||
2789 | |||
2790 | - *Shell Function Syntax:* Write shell functions as if you were writing | ||
2791 | a shell script when you describe a list of actions to take. You | ||
2792 | should ensure that your script works with a generic ``sh`` and that | ||
2793 | it does not require any ``bash`` or other shell-specific | ||
2794 | functionality. The same considerations apply to various system | ||
2795 | utilities (e.g. ``sed``, ``grep``, ``awk``, and so forth) that you | ||
2796 | might wish to use. If in doubt, you should check with multiple | ||
2797 | implementations - including those from BusyBox. | ||
2798 | |||
2799 | Adding a New Machine | ||
2800 | ==================== | ||
2801 | |||
2802 | Adding a new machine to the Yocto Project is a straightforward process. | ||
2803 | This section describes how to add machines that are similar to those | ||
2804 | that the Yocto Project already supports. | ||
2805 | |||
2806 | .. note:: | ||
2807 | |||
2808 | Although well within the capabilities of the Yocto Project, adding a | ||
2809 | totally new architecture might require changes to ``gcc``/``glibc`` | ||
2810 | and to the site information, which is beyond the scope of this | ||
2811 | manual. | ||
2812 | |||
2813 | For a complete example that shows how to add a new machine, see the | ||
2814 | ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`" | ||
2815 | section in the Yocto Project Board Support Package (BSP) Developer's | ||
2816 | Guide. | ||
2817 | |||
2818 | Adding the Machine Configuration File | ||
2819 | ------------------------------------- | ||
2820 | |||
2821 | To add a new machine, you need to add a new machine configuration file | ||
2822 | to the layer's ``conf/machine`` directory. This configuration file | ||
2823 | provides details about the device you are adding. | ||
2824 | |||
2825 | The OpenEmbedded build system uses the root name of the machine | ||
2826 | configuration file to reference the new machine. For example, given a | ||
2827 | machine configuration file named ``crownbay.conf``, the build system | ||
2828 | recognizes the machine as "crownbay". | ||
2829 | |||
2830 | The most important variables you must set in your machine configuration | ||
2831 | file or include from a lower-level configuration file are as follows: | ||
2832 | |||
2833 | - ``TARGET_ARCH`` (e.g. "arm") | ||
2834 | |||
2835 | - ``PREFERRED_PROVIDER_virtual/kernel`` | ||
2836 | |||
2837 | - ``MACHINE_FEATURES`` (e.g. "apm screen wifi") | ||
2838 | |||
2839 | You might also need these variables: | ||
2840 | |||
2841 | - ``SERIAL_CONSOLES`` (e.g. "115200;ttyS0 115200;ttyS1") | ||
2842 | |||
2843 | - ``KERNEL_IMAGETYPE`` (e.g. "zImage") | ||
2844 | |||
2845 | - ``IMAGE_FSTYPES`` (e.g. "tar.gz jffs2") | ||
2846 | |||
2847 | You can find full details on these variables in the reference section. | ||
2848 | You can leverage existing machine ``.conf`` files from | ||
2849 | ``meta-yocto-bsp/conf/machine/``. | ||
2850 | |||
2851 | Adding a Kernel for the Machine | ||
2852 | ------------------------------- | ||
2853 | |||
2854 | The OpenEmbedded build system needs to be able to build a kernel for the | ||
2855 | machine. You need to either create a new kernel recipe for this machine, | ||
2856 | or extend an existing kernel recipe. You can find several kernel recipe | ||
2857 | examples in the Source Directory at ``meta/recipes-kernel/linux`` that | ||
2858 | you can use as references. | ||
2859 | |||
2860 | If you are creating a new kernel recipe, normal recipe-writing rules | ||
2861 | apply for setting up a ``SRC_URI``. Thus, you need to specify any | ||
2862 | necessary patches and set ``S`` to point at the source code. You need to | ||
2863 | create a ``do_configure`` task that configures the unpacked kernel with | ||
2864 | a ``defconfig`` file. You can do this by using a ``make defconfig`` | ||
2865 | command or, more commonly, by copying in a suitable ``defconfig`` file | ||
2866 | and then running ``make oldconfig``. By making use of ``inherit kernel`` | ||
2867 | and potentially some of the ``linux-*.inc`` files, most other | ||
2868 | functionality is centralized and the defaults of the class normally work | ||
2869 | well. | ||
2870 | |||
2871 | If you are extending an existing kernel recipe, it is usually a matter | ||
2872 | of adding a suitable ``defconfig`` file. The file needs to be added into | ||
2873 | a location similar to ``defconfig`` files used for other machines in a | ||
2874 | given kernel recipe. A possible way to do this is by listing the file in | ||
2875 | the ``SRC_URI`` and adding the machine to the expression in | ||
2876 | ``COMPATIBLE_MACHINE``: | ||
2877 | :: | ||
2878 | |||
2879 | COMPATIBLE_MACHINE = '(qemux86|qemumips)' | ||
2880 | |||
2881 | For more information on ``defconfig`` files, see the | ||
2882 | ":ref:`kernel-dev/common:changing the configuration`" | ||
2883 | section in the Yocto Project Linux Kernel Development Manual. | ||
2884 | |||
2885 | Adding a Formfactor Configuration File | ||
2886 | -------------------------------------- | ||
2887 | |||
2888 | A formfactor configuration file provides information about the target | ||
2889 | hardware for which the image is being built and information that the | ||
2890 | build system cannot obtain from other sources such as the kernel. Some | ||
2891 | examples of information contained in a formfactor configuration file | ||
2892 | include framebuffer orientation, whether or not the system has a | ||
2893 | keyboard, the positioning of the keyboard in relation to the screen, and | ||
2894 | the screen resolution. | ||
2895 | |||
2896 | The build system uses reasonable defaults in most cases. However, if | ||
2897 | customization is necessary, you need to create a ``machconfig`` file in | ||
2898 | the ``meta/recipes-bsp/formfactor/files`` directory. This directory | ||
2899 | contains directories for specific machines such as ``qemuarm`` and | ||
2900 | ``qemux86``. For information about the settings available and the | ||
2901 | defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file | ||
2902 | found in the same area. | ||
2903 | |||
2904 | Following is an example for "qemuarm" machine: | ||
2905 | :: | ||
2906 | |||
2907 | HAVE_TOUCHSCREEN=1 | ||
2908 | HAVE_KEYBOARD=1 | ||
2909 | DISPLAY_CAN_ROTATE=0 | ||
2910 | DISPLAY_ORIENTATION=0 | ||
2911 | #DISPLAY_WIDTH_PIXELS=640 | ||
2912 | #DISPLAY_HEIGHT_PIXELS=480 | ||
2913 | #DISPLAY_BPP=16 | ||
2914 | DISPLAY_DPI=150 | ||
2915 | DISPLAY_SUBPIXEL_ORDER=vrgb | ||
2916 | |||
2917 | Upgrading Recipes | ||
2918 | ================= | ||
2919 | |||
2920 | Over time, upstream developers publish new versions for software built | ||
2921 | by layer recipes. It is recommended to keep recipes up-to-date with | ||
2922 | upstream version releases. | ||
2923 | |||
2924 | While several methods exist that allow you upgrade a recipe, you might | ||
2925 | consider checking on the upgrade status of a recipe first. You can do so | ||
2926 | using the ``devtool check-upgrade-status`` command. See the | ||
2927 | ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" | ||
2928 | section in the Yocto Project Reference Manual for more information. | ||
2929 | |||
2930 | The remainder of this section describes three ways you can upgrade a | ||
2931 | recipe. You can use the Automated Upgrade Helper (AUH) to set up | ||
2932 | automatic version upgrades. Alternatively, you can use | ||
2933 | ``devtool upgrade`` to set up semi-automatic version upgrades. Finally, | ||
2934 | you can manually upgrade a recipe by editing the recipe itself. | ||
2935 | |||
2936 | Using the Auto Upgrade Helper (AUH) | ||
2937 | ----------------------------------- | ||
2938 | |||
2939 | The AUH utility works in conjunction with the OpenEmbedded build system | ||
2940 | in order to automatically generate upgrades for recipes based on new | ||
2941 | versions being published upstream. Use AUH when you want to create a | ||
2942 | service that performs the upgrades automatically and optionally sends | ||
2943 | you an email with the results. | ||
2944 | |||
2945 | AUH allows you to update several recipes with a single use. You can also | ||
2946 | optionally perform build and integration tests using images with the | ||
2947 | results saved to your hard drive and emails of results optionally sent | ||
2948 | to recipe maintainers. Finally, AUH creates Git commits with appropriate | ||
2949 | commit messages in the layer's tree for the changes made to recipes. | ||
2950 | |||
2951 | .. note:: | ||
2952 | |||
2953 | Conditions do exist when you should not use AUH to upgrade recipes | ||
2954 | and you should instead use either ``devtool upgrade`` or upgrade your | ||
2955 | recipes manually: | ||
2956 | |||
2957 | - When AUH cannot complete the upgrade sequence. This situation | ||
2958 | usually results because custom patches carried by the recipe | ||
2959 | cannot be automatically rebased to the new version. In this case, | ||
2960 | ``devtool upgrade`` allows you to manually resolve conflicts. | ||
2961 | |||
2962 | - When for any reason you want fuller control over the upgrade | ||
2963 | process. For example, when you want special arrangements for | ||
2964 | testing. | ||
2965 | |||
2966 | The following steps describe how to set up the AUH utility: | ||
2967 | |||
2968 | 1. *Be Sure the Development Host is Set Up:* You need to be sure that | ||
2969 | your development host is set up to use the Yocto Project. For | ||
2970 | information on how to set up your host, see the | ||
2971 | ":ref:`dev-manual/start:Preparing the Build Host`" section. | ||
2972 | |||
2973 | 2. *Make Sure Git is Configured:* The AUH utility requires Git to be | ||
2974 | configured because AUH uses Git to save upgrades. Thus, you must have | ||
2975 | Git user and email configured. The following command shows your | ||
2976 | configurations: | ||
2977 | :: | ||
2978 | |||
2979 | $ git config --list | ||
2980 | |||
2981 | If you do not have the user and | ||
2982 | email configured, you can use the following commands to do so: | ||
2983 | :: | ||
2984 | |||
2985 | $ git config --global user.name some_name | ||
2986 | $ git config --global user.email username@domain.com | ||
2987 | |||
2988 | 3. *Clone the AUH Repository:* To use AUH, you must clone the repository | ||
2989 | onto your development host. The following command uses Git to create | ||
2990 | a local copy of the repository on your system: | ||
2991 | :: | ||
2992 | |||
2993 | $ git clone git://git.yoctoproject.org/auto-upgrade-helper | ||
2994 | Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. | ||
2995 | remote: Compressing objects: 100% (300/300), done. | ||
2996 | remote: Total 768 (delta 499), reused 703 (delta 434) | ||
2997 | Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. | ||
2998 | Resolving deltas: 100% (499/499), done. | ||
2999 | Checking connectivity... done. | ||
3000 | |||
3001 | AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or | ||
3002 | :term:`Poky` repositories. | ||
3003 | |||
3004 | 4. *Create a Dedicated Build Directory:* Run the | ||
3005 | :ref:`structure-core-script` | ||
3006 | script to create a fresh build directory that you use exclusively for | ||
3007 | running the AUH utility: | ||
3008 | :: | ||
3009 | |||
3010 | $ cd ~/poky | ||
3011 | $ source oe-init-build-env your_AUH_build_directory | ||
3012 | |||
3013 | Re-using an existing build directory and its configurations is not | ||
3014 | recommended as existing settings could cause AUH to fail or behave | ||
3015 | undesirably. | ||
3016 | |||
3017 | 5. *Make Configurations in Your Local Configuration File:* Several | ||
3018 | settings need to exist in the ``local.conf`` file in the build | ||
3019 | directory you just created for AUH. Make these following | ||
3020 | configurations: | ||
3021 | |||
3022 | - If you want to enable :ref:`Build | ||
3023 | History <dev-manual/common-tasks:maintaining build output quality>`, | ||
3024 | which is optional, you need the following lines in the | ||
3025 | ``conf/local.conf`` file: | ||
3026 | :: | ||
3027 | |||
3028 | INHERIT =+ "buildhistory" | ||
3029 | BUILDHISTORY_COMMIT = "1" | ||
3030 | |||
3031 | With this configuration and a successful | ||
3032 | upgrade, a build history "diff" file appears in the | ||
3033 | ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in | ||
3034 | your build directory. | ||
3035 | |||
3036 | - If you want to enable testing through the | ||
3037 | :ref:`testimage <ref-classes-testimage*>` | ||
3038 | class, which is optional, you need to have the following set in | ||
3039 | your ``conf/local.conf`` file: | ||
3040 | :: | ||
3041 | |||
3042 | INHERIT += "testimage" | ||
3043 | |||
3044 | .. note:: | ||
3045 | |||
3046 | If your distro does not enable by default ptest, which Poky | ||
3047 | does, you need the following in your ``local.conf`` file: | ||
3048 | :: | ||
3049 | |||
3050 | DISTRO_FEATURES_append = " ptest" | ||
3051 | |||
3052 | |||
3053 | 6. *Optionally Start a vncserver:* If you are running in a server | ||
3054 | without an X11 session, you need to start a vncserver: | ||
3055 | :: | ||
3056 | |||
3057 | $ vncserver :1 | ||
3058 | $ export DISPLAY=:1 | ||
3059 | |||
3060 | 7. *Create and Edit an AUH Configuration File:* You need to have the | ||
3061 | ``upgrade-helper/upgrade-helper.conf`` configuration file in your | ||
3062 | build directory. You can find a sample configuration file in the | ||
3063 | :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`. | ||
3064 | |||
3065 | Read through the sample file and make configurations as needed. For | ||
3066 | example, if you enabled build history in your ``local.conf`` as | ||
3067 | described earlier, you must enable it in ``upgrade-helper.conf``. | ||
3068 | |||
3069 | Also, if you are using the default ``maintainers.inc`` file supplied | ||
3070 | with Poky and located in ``meta-yocto`` and you do not set a | ||
3071 | "maintainers_whitelist" or "global_maintainer_override" in the | ||
3072 | ``upgrade-helper.conf`` configuration, and you specify "-e all" on | ||
3073 | the AUH command-line, the utility automatically sends out emails to | ||
3074 | all the default maintainers. Please avoid this. | ||
3075 | |||
3076 | This next set of examples describes how to use the AUH: | ||
3077 | |||
3078 | - *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the | ||
3079 | following form: | ||
3080 | :: | ||
3081 | |||
3082 | $ upgrade-helper.py recipe_name | ||
3083 | |||
3084 | For example, this command upgrades the ``xmodmap`` recipe: | ||
3085 | :: | ||
3086 | |||
3087 | $ upgrade-helper.py xmodmap | ||
3088 | |||
3089 | - *Upgrading a Specific Recipe to a Particular Version:* To upgrade a | ||
3090 | specific recipe to a particular version, use the following form: | ||
3091 | :: | ||
3092 | |||
3093 | $ upgrade-helper.py recipe_name -t version | ||
3094 | |||
3095 | For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3: | ||
3096 | :: | ||
3097 | |||
3098 | $ upgrade-helper.py xmodmap -t 1.2.3 | ||
3099 | |||
3100 | - *Upgrading all Recipes to the Latest Versions and Suppressing Email | ||
3101 | Notifications:* To upgrade all recipes to their most recent versions | ||
3102 | and suppress the email notifications, use the following command: | ||
3103 | :: | ||
3104 | |||
3105 | $ upgrade-helper.py all | ||
3106 | |||
3107 | - *Upgrading all Recipes to the Latest Versions and Send Email | ||
3108 | Notifications:* To upgrade all recipes to their most recent versions | ||
3109 | and send email messages to maintainers for each attempted recipe as | ||
3110 | well as a status email, use the following command: | ||
3111 | :: | ||
3112 | |||
3113 | $ upgrade-helper.py -e all | ||
3114 | |||
3115 | Once you have run the AUH utility, you can find the results in the AUH | ||
3116 | build directory: | ||
3117 | :: | ||
3118 | |||
3119 | ${BUILDDIR}/upgrade-helper/timestamp | ||
3120 | |||
3121 | The AUH utility | ||
3122 | also creates recipe update commits from successful upgrade attempts in | ||
3123 | the layer tree. | ||
3124 | |||
3125 | You can easily set up to run the AUH utility on a regular basis by using | ||
3126 | a cron job. See the | ||
3127 | :yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>` | ||
3128 | file distributed with the utility for an example. | ||
3129 | |||
3130 | Using ``devtool upgrade`` | ||
3131 | ------------------------- | ||
3132 | |||
3133 | As mentioned earlier, an alternative method for upgrading recipes to | ||
3134 | newer versions is to use | ||
3135 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. | ||
3136 | You can read about ``devtool upgrade`` in general in the | ||
3137 | ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" | ||
3138 | section in the Yocto Project Application Development and the Extensible | ||
3139 | Software Development Kit (eSDK) Manual. | ||
3140 | |||
3141 | To see all the command-line options available with ``devtool upgrade``, | ||
3142 | use the following help command: | ||
3143 | :: | ||
3144 | |||
3145 | $ devtool upgrade -h | ||
3146 | |||
3147 | If you want to find out what version a recipe is currently at upstream | ||
3148 | without any attempt to upgrade your local version of the recipe, you can | ||
3149 | use the following command: | ||
3150 | :: | ||
3151 | |||
3152 | $ devtool latest-version recipe_name | ||
3153 | |||
3154 | As mentioned in the previous section describing AUH, ``devtool upgrade`` | ||
3155 | works in a less-automated manner than AUH. Specifically, | ||
3156 | ``devtool upgrade`` only works on a single recipe that you name on the | ||
3157 | command line, cannot perform build and integration testing using images, | ||
3158 | and does not automatically generate commits for changes in the source | ||
3159 | tree. Despite all these "limitations", ``devtool upgrade`` updates the | ||
3160 | recipe file to the new upstream version and attempts to rebase custom | ||
3161 | patches contained by the recipe as needed. | ||
3162 | |||
3163 | .. note:: | ||
3164 | |||
3165 | AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat | ||
3166 | of a "wrapper" application for ``devtool upgrade``. | ||
3167 | |||
3168 | A typical scenario involves having used Git to clone an upstream | ||
3169 | repository that you use during build operations. Because you have built the | ||
3170 | recipe in the past, the layer is likely added to your | ||
3171 | configuration already. If for some reason, the layer is not added, you | ||
3172 | could add it easily using the | ||
3173 | ":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" | ||
3174 | script. For example, suppose you use the ``nano.bb`` recipe from the | ||
3175 | ``meta-oe`` layer in the ``meta-openembedded`` repository. For this | ||
3176 | example, assume that the layer has been cloned into following area: | ||
3177 | :: | ||
3178 | |||
3179 | /home/scottrif/meta-openembedded | ||
3180 | |||
3181 | The following command from your | ||
3182 | :term:`Build Directory` adds the layer to | ||
3183 | your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``): | ||
3184 | :: | ||
3185 | |||
3186 | $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe | ||
3187 | NOTE: Starting bitbake server... | ||
3188 | Parsing recipes: 100% |##########################################| Time: 0:00:55 | ||
3189 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
3190 | Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 | ||
3191 | Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 | ||
3192 | Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 | ||
3193 | Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 | ||
3194 | |||
3195 | For this example, assume that the ``nano.bb`` recipe that | ||
3196 | is upstream has a 2.9.3 version number. However, the version in the | ||
3197 | local repository is 2.7.4. The following command from your build | ||
3198 | directory automatically upgrades the recipe for you: | ||
3199 | |||
3200 | .. note:: | ||
3201 | |||
3202 | Using the ``-V`` option is not necessary. Omitting the version number causes | ||
3203 | ``devtool upgrade`` to upgrade the recipe to the most recent version. | ||
3204 | |||
3205 | :: | ||
3206 | |||
3207 | $ devtool upgrade nano -V 2.9.3 | ||
3208 | NOTE: Starting bitbake server... | ||
3209 | NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace | ||
3210 | Parsing recipes: 100% |##########################################| Time: 0:00:46 | ||
3211 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
3212 | NOTE: Extracting current version source... | ||
3213 | NOTE: Resolving any missing task queue dependencies | ||
3214 | . | ||
3215 | . | ||
3216 | . | ||
3217 | NOTE: Executing SetScene Tasks | ||
3218 | NOTE: Executing RunQueue Tasks | ||
3219 | NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. | ||
3220 | Adding changed files: 100% |#####################################| Time: 0:00:00 | ||
3221 | NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano | ||
3222 | NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb | ||
3223 | |||
3224 | Continuing with this example, you can use ``devtool build`` to build the | ||
3225 | newly upgraded recipe: | ||
3226 | :: | ||
3227 | |||
3228 | $ devtool build nano | ||
3229 | NOTE: Starting bitbake server... | ||
3230 | Loading cache: 100% |################################################################################################| Time: 0:00:01 | ||
3231 | Loaded 2040 entries from dependency cache. | ||
3232 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 | ||
3233 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
3234 | NOTE: Resolving any missing task queue dependencies | ||
3235 | . | ||
3236 | . | ||
3237 | . | ||
3238 | NOTE: Executing SetScene Tasks | ||
3239 | NOTE: Executing RunQueue Tasks | ||
3240 | NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano | ||
3241 | NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. | ||
3242 | |||
3243 | Within the ``devtool upgrade`` workflow, opportunity | ||
3244 | exists to deploy and test your rebuilt software. For this example, | ||
3245 | however, running ``devtool finish`` cleans up the workspace once the | ||
3246 | source in your workspace is clean. This usually means using Git to stage | ||
3247 | and submit commits for the changes generated by the upgrade process. | ||
3248 | |||
3249 | Once the tree is clean, you can clean things up in this example with the | ||
3250 | following command from the ``${BUILDDIR}/workspace/sources/nano`` | ||
3251 | directory: | ||
3252 | :: | ||
3253 | |||
3254 | $ devtool finish nano meta-oe | ||
3255 | NOTE: Starting bitbake server... | ||
3256 | Loading cache: 100% |################################################################################################| Time: 0:00:00 | ||
3257 | Loaded 2040 entries from dependency cache. | ||
3258 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 | ||
3259 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
3260 | NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch | ||
3261 | NOTE: Updating recipe nano_2.9.3.bb | ||
3262 | NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb | ||
3263 | NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano | ||
3264 | NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually | ||
3265 | |||
3266 | |||
3267 | Using the ``devtool finish`` command cleans up the workspace and creates a patch | ||
3268 | file based on your commits. The tool puts all patch files back into the | ||
3269 | source directory in a sub-directory named ``nano`` in this case. | ||
3270 | |||
3271 | Manually Upgrading a Recipe | ||
3272 | --------------------------- | ||
3273 | |||
3274 | If for some reason you choose not to upgrade recipes using | ||
3275 | :ref:`dev-manual/common-tasks:Using the Auto Upgrade Helper (AUH)` or | ||
3276 | by :ref:`dev-manual/common-tasks:Using \`\`devtool upgrade\`\``, | ||
3277 | you can manually edit the recipe files to upgrade the versions. | ||
3278 | |||
3279 | .. note:: | ||
3280 | |||
3281 | Manually updating multiple recipes scales poorly and involves many | ||
3282 | steps. The recommendation to upgrade recipe versions is through AUH | ||
3283 | or ``devtool upgrade``, both of which automate some steps and provide | ||
3284 | guidance for others needed for the manual process. | ||
3285 | |||
3286 | To manually upgrade recipe versions, follow these general steps: | ||
3287 | |||
3288 | 1. *Change the Version:* Rename the recipe such that the version (i.e. | ||
3289 | the :term:`PV` part of the recipe name) | ||
3290 | changes appropriately. If the version is not part of the recipe name, | ||
3291 | change the value as it is set for ``PV`` within the recipe itself. | ||
3292 | |||
3293 | 2. *Update* ``SRCREV`` *if Needed*: If the source code your recipe builds | ||
3294 | is fetched from Git or some other version control system, update | ||
3295 | :term:`SRCREV` to point to the | ||
3296 | commit hash that matches the new version. | ||
3297 | |||
3298 | 3. *Build the Software:* Try to build the recipe using BitBake. Typical | ||
3299 | build failures include the following: | ||
3300 | |||
3301 | - License statements were updated for the new version. For this | ||
3302 | case, you need to review any changes to the license and update the | ||
3303 | values of :term:`LICENSE` and | ||
3304 | :term:`LIC_FILES_CHKSUM` | ||
3305 | as needed. | ||
3306 | |||
3307 | .. note:: | ||
3308 | |||
3309 | License changes are often inconsequential. For example, the | ||
3310 | license text's copyright year might have changed. | ||
3311 | |||
3312 | - Custom patches carried by the older version of the recipe might | ||
3313 | fail to apply to the new version. For these cases, you need to | ||
3314 | review the failures. Patches might not be necessary for the new | ||
3315 | version of the software if the upgraded version has fixed those | ||
3316 | issues. If a patch is necessary and failing, you need to rebase it | ||
3317 | into the new version. | ||
3318 | |||
3319 | 4. *Optionally Attempt to Build for Several Architectures:* Once you | ||
3320 | successfully build the new software for a given architecture, you | ||
3321 | could test the build for other architectures by changing the | ||
3322 | :term:`MACHINE` variable and | ||
3323 | rebuilding the software. This optional step is especially important | ||
3324 | if the recipe is to be released publicly. | ||
3325 | |||
3326 | 5. *Check the Upstream Change Log or Release Notes:* Checking both these | ||
3327 | reveals if new features exist that could break | ||
3328 | backwards-compatibility. If so, you need to take steps to mitigate or | ||
3329 | eliminate that situation. | ||
3330 | |||
3331 | 6. *Optionally Create a Bootable Image and Test:* If you want, you can | ||
3332 | test the new software by booting it onto actual hardware. | ||
3333 | |||
3334 | 7. *Create a Commit with the Change in the Layer Repository:* After all | ||
3335 | builds work and any testing is successful, you can create commits for | ||
3336 | any changes in the layer holding your upgraded recipe. | ||
3337 | |||
3338 | Finding Temporary Source Code | ||
3339 | ============================= | ||
3340 | |||
3341 | You might find it helpful during development to modify the temporary | ||
3342 | source code used by recipes to build packages. For example, suppose you | ||
3343 | are developing a patch and you need to experiment a bit to figure out | ||
3344 | your solution. After you have initially built the package, you can | ||
3345 | iteratively tweak the source code, which is located in the | ||
3346 | :term:`Build Directory`, and then you can | ||
3347 | force a re-compile and quickly test your altered code. Once you settle | ||
3348 | on a solution, you can then preserve your changes in the form of | ||
3349 | patches. | ||
3350 | |||
3351 | During a build, the unpacked temporary source code used by recipes to | ||
3352 | build packages is available in the Build Directory as defined by the | ||
3353 | :term:`S` variable. Below is the default | ||
3354 | value for the ``S`` variable as defined in the | ||
3355 | ``meta/conf/bitbake.conf`` configuration file in the | ||
3356 | :term:`Source Directory`: | ||
3357 | :: | ||
3358 | |||
3359 | S = "${WORKDIR}/${BP}" | ||
3360 | |||
3361 | You should be aware that many recipes override the | ||
3362 | ``S`` variable. For example, recipes that fetch their source from Git | ||
3363 | usually set ``S`` to ``${WORKDIR}/git``. | ||
3364 | |||
3365 | .. note:: | ||
3366 | |||
3367 | The :term:`BP` represents the base recipe name, which consists of the name | ||
3368 | and version: | ||
3369 | :: | ||
3370 | |||
3371 | BP = "${BPN}-${PV}" | ||
3372 | |||
3373 | |||
3374 | The path to the work directory for the recipe | ||
3375 | (:term:`WORKDIR`) is defined as | ||
3376 | follows: | ||
3377 | :: | ||
3378 | |||
3379 | ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} | ||
3380 | |||
3381 | The actual directory depends on several things: | ||
3382 | |||
3383 | - :term:`TMPDIR`: The top-level build | ||
3384 | output directory. | ||
3385 | |||
3386 | - :term:`MULTIMACH_TARGET_SYS`: | ||
3387 | The target system identifier. | ||
3388 | |||
3389 | - :term:`PN`: The recipe name. | ||
3390 | |||
3391 | - :term:`EXTENDPE`: The epoch - (if | ||
3392 | :term:`PE` is not specified, which is | ||
3393 | usually the case for most recipes, then ``EXTENDPE`` is blank). | ||
3394 | |||
3395 | - :term:`PV`: The recipe version. | ||
3396 | |||
3397 | - :term:`PR`: The recipe revision. | ||
3398 | |||
3399 | As an example, assume a Source Directory top-level folder named | ||
3400 | ``poky``, a default Build Directory at ``poky/build``, and a | ||
3401 | ``qemux86-poky-linux`` machine target system. Furthermore, suppose your | ||
3402 | recipe is named ``foo_1.3.0.bb``. In this case, the work directory the | ||
3403 | build system uses to build the package would be as follows: | ||
3404 | :: | ||
3405 | |||
3406 | poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 | ||
3407 | |||
3408 | Using Quilt in Your Workflow | ||
3409 | ============================ | ||
3410 | |||
3411 | `Quilt <https://savannah.nongnu.org/projects/quilt>`__ is a powerful tool | ||
3412 | that allows you to capture source code changes without having a clean | ||
3413 | source tree. This section outlines the typical workflow you can use to | ||
3414 | modify source code, test changes, and then preserve the changes in the | ||
3415 | form of a patch all using Quilt. | ||
3416 | |||
3417 | .. note:: | ||
3418 | |||
3419 | With regard to preserving changes to source files, if you clean a | ||
3420 | recipe or have ``rm_work`` enabled, the | ||
3421 | :ref:`devtool workflow <sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow>` | ||
3422 | as described in the Yocto Project Application Development and the | ||
3423 | Extensible Software Development Kit (eSDK) manual is a safer | ||
3424 | development flow than the flow that uses Quilt. | ||
3425 | |||
3426 | Follow these general steps: | ||
3427 | |||
3428 | 1. *Find the Source Code:* Temporary source code used by the | ||
3429 | OpenEmbedded build system is kept in the | ||
3430 | :term:`Build Directory`. See the | ||
3431 | "`Finding Temporary Source | ||
3432 | Code <#finding-the-temporary-source-code>`__" section to learn how to | ||
3433 | locate the directory that has the temporary source code for a | ||
3434 | particular package. | ||
3435 | |||
3436 | 2. *Change Your Working Directory:* You need to be in the directory that | ||
3437 | has the temporary source code. That directory is defined by the | ||
3438 | :term:`S` variable. | ||
3439 | |||
3440 | 3. *Create a New Patch:* Before modifying source code, you need to | ||
3441 | create a new patch. To create a new patch file, use ``quilt new`` as | ||
3442 | below: | ||
3443 | :: | ||
3444 | |||
3445 | $ quilt new my_changes.patch | ||
3446 | |||
3447 | 4. *Notify Quilt and Add Files:* After creating the patch, you need to | ||
3448 | notify Quilt about the files you plan to edit. You notify Quilt by | ||
3449 | adding the files to the patch you just created: | ||
3450 | :: | ||
3451 | |||
3452 | $ quilt add file1.c file2.c file3.c | ||
3453 | |||
3454 | 5. *Edit the Files:* Make your changes in the source code to the files | ||
3455 | you added to the patch. | ||
3456 | |||
3457 | 6. *Test Your Changes:* Once you have modified the source code, the | ||
3458 | easiest way to test your changes is by calling the ``do_compile`` | ||
3459 | task as shown in the following example: | ||
3460 | :: | ||
3461 | |||
3462 | $ bitbake -c compile -f package | ||
3463 | |||
3464 | The ``-f`` or ``--force`` option forces the specified task to | ||
3465 | execute. If you find problems with your code, you can just keep | ||
3466 | editing and re-testing iteratively until things work as expected. | ||
3467 | |||
3468 | .. note:: | ||
3469 | |||
3470 | All the modifications you make to the temporary source code disappear | ||
3471 | once you run the ``do_clean`` or ``do_cleanall`` tasks using BitBake | ||
3472 | (i.e. ``bitbake -c clean package`` and ``bitbake -c cleanall package``). | ||
3473 | Modifications will also disappear if you use the ``rm_work`` feature as | ||
3474 | described in the | ||
3475 | ":ref:`dev-manual/common-tasks:conserving disk space during builds`" | ||
3476 | section. | ||
3477 | |||
3478 | 7. *Generate the Patch:* Once your changes work as expected, you need to | ||
3479 | use Quilt to generate the final patch that contains all your | ||
3480 | modifications. | ||
3481 | :: | ||
3482 | |||
3483 | $ quilt refresh | ||
3484 | |||
3485 | At this point, the | ||
3486 | ``my_changes.patch`` file has all your edits made to the ``file1.c``, | ||
3487 | ``file2.c``, and ``file3.c`` files. | ||
3488 | |||
3489 | You can find the resulting patch file in the ``patches/`` | ||
3490 | subdirectory of the source (``S``) directory. | ||
3491 | |||
3492 | 8. *Copy the Patch File:* For simplicity, copy the patch file into a | ||
3493 | directory named ``files``, which you can create in the same directory | ||
3494 | that holds the recipe (``.bb``) file or the append (``.bbappend``) | ||
3495 | file. Placing the patch here guarantees that the OpenEmbedded build | ||
3496 | system will find the patch. Next, add the patch into the ``SRC_URI`` | ||
3497 | of the recipe. Here is an example: | ||
3498 | :: | ||
3499 | |||
3500 | SRC_URI += "file://my_changes.patch" | ||
3501 | |||
3502 | Using a Development Shell | ||
3503 | ========================= | ||
3504 | |||
3505 | When debugging certain commands or even when just editing packages, | ||
3506 | ``devshell`` can be a useful tool. When you invoke ``devshell``, all | ||
3507 | tasks up to and including | ||
3508 | :ref:`ref-tasks-patch` are run for the | ||
3509 | specified target. Then, a new terminal is opened and you are placed in | ||
3510 | ``${``\ :term:`S`\ ``}``, the source | ||
3511 | directory. In the new terminal, all the OpenEmbedded build-related | ||
3512 | environment variables are still defined so you can use commands such as | ||
3513 | ``configure`` and ``make``. The commands execute just as if the | ||
3514 | OpenEmbedded build system were executing them. Consequently, working | ||
3515 | this way can be helpful when debugging a build or preparing software to | ||
3516 | be used with the OpenEmbedded build system. | ||
3517 | |||
3518 | Following is an example that uses ``devshell`` on a target named | ||
3519 | ``matchbox-desktop``: | ||
3520 | :: | ||
3521 | |||
3522 | $ bitbake matchbox-desktop -c devshell | ||
3523 | |||
3524 | This command spawns a terminal with a shell prompt within the | ||
3525 | OpenEmbedded build environment. The | ||
3526 | :term:`OE_TERMINAL` variable | ||
3527 | controls what type of shell is opened. | ||
3528 | |||
3529 | For spawned terminals, the following occurs: | ||
3530 | |||
3531 | - The ``PATH`` variable includes the cross-toolchain. | ||
3532 | |||
3533 | - The ``pkgconfig`` variables find the correct ``.pc`` files. | ||
3534 | |||
3535 | - The ``configure`` command finds the Yocto Project site files as well | ||
3536 | as any other necessary files. | ||
3537 | |||
3538 | Within this environment, you can run configure or compile commands as if | ||
3539 | they were being run by the OpenEmbedded build system itself. As noted | ||
3540 | earlier, the working directory also automatically changes to the Source | ||
3541 | Directory (:term:`S`). | ||
3542 | |||
3543 | To manually run a specific task using ``devshell``, run the | ||
3544 | corresponding ``run.*`` script in the | ||
3545 | ``${``\ :term:`WORKDIR`\ ``}/temp`` | ||
3546 | directory (e.g., ``run.do_configure.``\ `pid`). If a task's script does | ||
3547 | not exist, which would be the case if the task was skipped by way of the | ||
3548 | sstate cache, you can create the task by first running it outside of the | ||
3549 | ``devshell``: | ||
3550 | :: | ||
3551 | |||
3552 | $ bitbake -c task | ||
3553 | |||
3554 | .. note:: | ||
3555 | |||
3556 | - Execution of a task's ``run.*`` script and BitBake's execution of | ||
3557 | a task are identical. In other words, running the script re-runs | ||
3558 | the task just as it would be run using the ``bitbake -c`` command. | ||
3559 | |||
3560 | - Any ``run.*`` file that does not have a ``.pid`` extension is a | ||
3561 | symbolic link (symlink) to the most recent version of that file. | ||
3562 | |||
3563 | Remember, that the ``devshell`` is a mechanism that allows you to get | ||
3564 | into the BitBake task execution environment. And as such, all commands | ||
3565 | must be called just as BitBake would call them. That means you need to | ||
3566 | provide the appropriate options for cross-compilation and so forth as | ||
3567 | applicable. | ||
3568 | |||
3569 | When you are finished using ``devshell``, exit the shell or close the | ||
3570 | terminal window. | ||
3571 | |||
3572 | .. note:: | ||
3573 | |||
3574 | - It is worth remembering that when using ``devshell`` you need to | ||
3575 | use the full compiler name such as ``arm-poky-linux-gnueabi-gcc`` | ||
3576 | instead of just using ``gcc``. The same applies to other | ||
3577 | applications such as ``binutils``, ``libtool`` and so forth. | ||
3578 | BitBake sets up environment variables such as ``CC`` to assist | ||
3579 | applications, such as ``make`` to find the correct tools. | ||
3580 | |||
3581 | - It is also worth noting that ``devshell`` still works over X11 | ||
3582 | forwarding and similar situations. | ||
3583 | |||
3584 | Using a Development Python Shell | ||
3585 | ================================ | ||
3586 | |||
3587 | Similar to working within a development shell as described in the | ||
3588 | previous section, you can also spawn and work within an interactive | ||
3589 | Python development shell. When debugging certain commands or even when | ||
3590 | just editing packages, ``devpyshell`` can be a useful tool. When you | ||
3591 | invoke ``devpyshell``, all tasks up to and including | ||
3592 | :ref:`ref-tasks-patch` are run for the | ||
3593 | specified target. Then a new terminal is opened. Additionally, key | ||
3594 | Python objects and code are available in the same way they are to | ||
3595 | BitBake tasks, in particular, the data store 'd'. So, commands such as | ||
3596 | the following are useful when exploring the data store and running | ||
3597 | functions: | ||
3598 | :: | ||
3599 | |||
3600 | pydevshell> d.getVar("STAGING_DIR") | ||
3601 | '/media/build1/poky/build/tmp/sysroots' | ||
3602 | pydevshell> d.getVar("STAGING_DIR") | ||
3603 | '${TMPDIR}/sysroots' | ||
3604 | pydevshell> d.setVar("FOO", "bar") | ||
3605 | pydevshell> d.getVar("FOO") | ||
3606 | 'bar' | ||
3607 | pydevshell> d.delVar("FOO") | ||
3608 | pydevshell> d.getVar("FOO") | ||
3609 | pydevshell> bb.build.exec_func("do_unpack", d) | ||
3610 | pydevshell> | ||
3611 | |||
3612 | The commands execute just as if the OpenEmbedded build | ||
3613 | system were executing them. Consequently, working this way can be | ||
3614 | helpful when debugging a build or preparing software to be used with the | ||
3615 | OpenEmbedded build system. | ||
3616 | |||
3617 | Following is an example that uses ``devpyshell`` on a target named | ||
3618 | ``matchbox-desktop``: | ||
3619 | :: | ||
3620 | |||
3621 | $ bitbake matchbox-desktop -c devpyshell | ||
3622 | |||
3623 | This command spawns a terminal and places you in an interactive Python | ||
3624 | interpreter within the OpenEmbedded build environment. The | ||
3625 | :term:`OE_TERMINAL` variable | ||
3626 | controls what type of shell is opened. | ||
3627 | |||
3628 | When you are finished using ``devpyshell``, you can exit the shell | ||
3629 | either by using Ctrl+d or closing the terminal window. | ||
3630 | |||
3631 | Building | ||
3632 | ======== | ||
3633 | |||
3634 | This section describes various build procedures. For example, the steps | ||
3635 | needed for a simple build, a target that uses multiple configurations, | ||
3636 | building an image for more than one machine, and so forth. | ||
3637 | |||
3638 | Building a Simple Image | ||
3639 | ----------------------- | ||
3640 | |||
3641 | In the development environment, you need to build an image whenever you | ||
3642 | change hardware support, add or change system libraries, or add or | ||
3643 | change services that have dependencies. Several methods exist that allow | ||
3644 | you to build an image within the Yocto Project. This section presents | ||
3645 | the basic steps you need to build a simple image using BitBake from a | ||
3646 | build host running Linux. | ||
3647 | |||
3648 | .. note:: | ||
3649 | |||
3650 | - For information on how to build an image using | ||
3651 | :term:`Toaster`, see the | ||
3652 | :doc:`/toaster-manual/index`. | ||
3653 | |||
3654 | - For information on how to use ``devtool`` to build images, see the | ||
3655 | ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
3656 | section in the Yocto Project Application Development and the | ||
3657 | Extensible Software Development Kit (eSDK) manual. | ||
3658 | |||
3659 | - For a quick example on how to build an image using the | ||
3660 | OpenEmbedded build system, see the | ||
3661 | :doc:`/brief-yoctoprojectqs/index` document. | ||
3662 | |||
3663 | The build process creates an entire Linux distribution from source and | ||
3664 | places it in your :term:`Build Directory` under | ||
3665 | ``tmp/deploy/images``. For detailed information on the build process | ||
3666 | using BitBake, see the ":ref:`overview-manual/concepts:images`" section in the | ||
3667 | Yocto Project Overview and Concepts Manual. | ||
3668 | |||
3669 | The following figure and list overviews the build process: | ||
3670 | |||
3671 | .. image:: figures/bitbake-build-flow.png | ||
3672 | :align: center | ||
3673 | |||
3674 | 1. *Set up Your Host Development System to Support Development Using the | ||
3675 | Yocto Project*: See the ":doc:`start`" section for options on how to get a | ||
3676 | build host ready to use the Yocto Project. | ||
3677 | |||
3678 | 2. *Initialize the Build Environment:* Initialize the build environment | ||
3679 | by sourcing the build environment script (i.e. | ||
3680 | :ref:`structure-core-script`): | ||
3681 | :: | ||
3682 | |||
3683 | $ source oe-init-build-env [build_dir] | ||
3684 | |||
3685 | When you use the initialization script, the OpenEmbedded build system | ||
3686 | uses ``build`` as the default :term:`Build Directory` in your current work | ||
3687 | directory. You can use a `build_dir` argument with the script to | ||
3688 | specify a different build directory. | ||
3689 | |||
3690 | .. note:: | ||
3691 | |||
3692 | A common practice is to use a different Build Directory for | ||
3693 | different targets. For example, ``~/build/x86`` for a ``qemux86`` | ||
3694 | target, and ``~/build/arm`` for a ``qemuarm`` target. | ||
3695 | |||
3696 | 3. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the | ||
3697 | ``conf/local.conf`` configuration file, which is found in the Build | ||
3698 | Directory, is set up how you want it. This file defines many aspects | ||
3699 | of the build environment including the target machine architecture | ||
3700 | through the ``MACHINE`` variable, the packaging format used during | ||
3701 | the build | ||
3702 | (:term:`PACKAGE_CLASSES`), | ||
3703 | and a centralized tarball download directory through the | ||
3704 | :term:`DL_DIR` variable. | ||
3705 | |||
3706 | 4. *Build the Image:* Build the image using the ``bitbake`` command: | ||
3707 | :: | ||
3708 | |||
3709 | $ bitbake target | ||
3710 | |||
3711 | .. note:: | ||
3712 | |||
3713 | For information on BitBake, see the :doc:`bitbake:index`. | ||
3714 | |||
3715 | The target is the name of the recipe you want to build. Common | ||
3716 | targets are the images in ``meta/recipes-core/images``, | ||
3717 | ``meta/recipes-sato/images``, and so forth all found in the | ||
3718 | :term:`Source Directory`. Or, the target | ||
3719 | can be the name of a recipe for a specific piece of software such as | ||
3720 | BusyBox. For more details about the images the OpenEmbedded build | ||
3721 | system supports, see the | ||
3722 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
3723 | Project Reference Manual. | ||
3724 | |||
3725 | As an example, the following command builds the | ||
3726 | ``core-image-minimal`` image: | ||
3727 | :: | ||
3728 | |||
3729 | $ bitbake core-image-minimal | ||
3730 | |||
3731 | Once an | ||
3732 | image has been built, it often needs to be installed. The images and | ||
3733 | kernels built by the OpenEmbedded build system are placed in the | ||
3734 | Build Directory in ``tmp/deploy/images``. For information on how to | ||
3735 | run pre-built images such as ``qemux86`` and ``qemuarm``, see the | ||
3736 | :doc:`/sdk-manual/index` manual. For | ||
3737 | information about how to install these images, see the documentation | ||
3738 | for your particular board or machine. | ||
3739 | |||
3740 | Building Images for Multiple Targets Using Multiple Configurations | ||
3741 | ------------------------------------------------------------------ | ||
3742 | |||
3743 | You can use a single ``bitbake`` command to build multiple images or | ||
3744 | packages for different targets where each image or package requires a | ||
3745 | different configuration (multiple configuration builds). The builds, in | ||
3746 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
3747 | section uses that term throughout. | ||
3748 | |||
3749 | This section describes how to set up for multiple configuration builds | ||
3750 | and how to account for cross-build dependencies between the | ||
3751 | multiconfigs. | ||
3752 | |||
3753 | Setting Up and Running a Multiple Configuration Build | ||
3754 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
3755 | |||
3756 | To accomplish a multiple configuration build, you must define each | ||
3757 | target's configuration separately using a parallel configuration file in | ||
3758 | the :term:`Build Directory`, and you | ||
3759 | must follow a required file hierarchy. Additionally, you must enable the | ||
3760 | multiple configuration builds in your ``local.conf`` file. | ||
3761 | |||
3762 | Follow these steps to set up and execute multiple configuration builds: | ||
3763 | |||
3764 | - *Create Separate Configuration Files*: You need to create a single | ||
3765 | configuration file for each build target (each multiconfig). | ||
3766 | Minimally, each configuration file must define the machine and the | ||
3767 | temporary directory BitBake uses for the build. Suggested practice | ||
3768 | dictates that you do not overlap the temporary directories used | ||
3769 | during the builds. However, it is possible that you can share the | ||
3770 | temporary directory | ||
3771 | (:term:`TMPDIR`). For example, | ||
3772 | consider a scenario with two different multiconfigs for the same | ||
3773 | :term:`MACHINE`: "qemux86" built | ||
3774 | for two distributions such as "poky" and "poky-lsb". In this case, | ||
3775 | you might want to use the same ``TMPDIR``. | ||
3776 | |||
3777 | Here is an example showing the minimal statements needed in a | ||
3778 | configuration file for a "qemux86" target whose temporary build | ||
3779 | directory is ``tmpmultix86``: | ||
3780 | :: | ||
3781 | |||
3782 | MACHINE = "qemux86" | ||
3783 | TMPDIR = "${TOPDIR}/tmpmultix86" | ||
3784 | |||
3785 | The location for these multiconfig configuration files is specific. | ||
3786 | They must reside in the current build directory in a sub-directory of | ||
3787 | ``conf`` named ``multiconfig``. Following is an example that defines | ||
3788 | two configuration files for the "x86" and "arm" multiconfigs: | ||
3789 | |||
3790 | .. image:: figures/multiconfig_files.png | ||
3791 | :align: center | ||
3792 | |||
3793 | The reason for this required file hierarchy is because the ``BBPATH`` | ||
3794 | variable is not constructed until the layers are parsed. | ||
3795 | Consequently, using the configuration file as a pre-configuration | ||
3796 | file is not possible unless it is located in the current working | ||
3797 | directory. | ||
3798 | |||
3799 | - *Add the BitBake Multi-configuration Variable to the Local | ||
3800 | Configuration File*: Use the | ||
3801 | :term:`BBMULTICONFIG` | ||
3802 | variable in your ``conf/local.conf`` configuration file to specify | ||
3803 | each multiconfig. Continuing with the example from the previous | ||
3804 | figure, the ``BBMULTICONFIG`` variable needs to enable two | ||
3805 | multiconfigs: "x86" and "arm" by specifying each configuration file: | ||
3806 | :: | ||
3807 | |||
3808 | BBMULTICONFIG = "x86 arm" | ||
3809 | |||
3810 | .. note:: | ||
3811 | |||
3812 | A "default" configuration already exists by definition. This | ||
3813 | configuration is named: "" (i.e. empty string) and is defined by | ||
3814 | the variables coming from your ``local.conf`` | ||
3815 | file. Consequently, the previous example actually adds two | ||
3816 | additional configurations to your build: "arm" and "x86" along | ||
3817 | with "". | ||
3818 | |||
3819 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
3820 | the multiple configuration build: | ||
3821 | :: | ||
3822 | |||
3823 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
3824 | |||
3825 | For the example in this section, the following command applies: | ||
3826 | :: | ||
3827 | |||
3828 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
3829 | |||
3830 | The previous BitBake command builds a ``core-image-minimal`` image | ||
3831 | that is configured through the ``x86.conf`` configuration file, a | ||
3832 | ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
3833 | configuration file and a ``core-image-base`` that is configured | ||
3834 | through your ``local.conf`` configuration file. | ||
3835 | |||
3836 | .. note:: | ||
3837 | |||
3838 | Support for multiple configuration builds in the Yocto Project &DISTRO; | ||
3839 | (&DISTRO_NAME;) Release does not include Shared State (sstate) | ||
3840 | optimizations. Consequently, if a build uses the same object twice | ||
3841 | in, for example, two different ``TMPDIR`` | ||
3842 | directories, the build either loads from an existing sstate cache for | ||
3843 | that build at the start or builds the object fresh. | ||
3844 | |||
3845 | Enabling Multiple Configuration Build Dependencies | ||
3846 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
3847 | |||
3848 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
3849 | multiple configuration build. For example, suppose that in order to | ||
3850 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
3851 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
3852 | essentially that the | ||
3853 | :ref:`ref-tasks-image` task in the | ||
3854 | ``core-image-sato`` recipe depends on the completion of the | ||
3855 | :ref:`ref-tasks-rootfs` task of the | ||
3856 | ``core-image-minimal`` recipe. | ||
3857 | |||
3858 | To enable dependencies in a multiple configuration build, you must | ||
3859 | declare the dependencies in the recipe using the following statement | ||
3860 | form: | ||
3861 | :: | ||
3862 | |||
3863 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
3864 | |||
3865 | To better show how to use this statement, consider the example scenario | ||
3866 | from the first paragraph of this section. The following statement needs | ||
3867 | to be added to the recipe that builds the ``core-image-sato`` image: | ||
3868 | :: | ||
3869 | |||
3870 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
3871 | |||
3872 | In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The | ||
3873 | task on which the ``do_image`` task in the recipe depends is the | ||
3874 | ``do_rootfs`` task from the ``core-image-minimal`` recipe associated | ||
3875 | with the "arm" multiconfig. | ||
3876 | |||
3877 | Once you set up this dependency, you can build the "x86" multiconfig | ||
3878 | using a BitBake command as follows: | ||
3879 | :: | ||
3880 | |||
3881 | $ bitbake mc:x86:core-image-sato | ||
3882 | |||
3883 | This command executes all the tasks needed to create the | ||
3884 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
3885 | dependency, BitBake also executes through the ``do_rootfs`` task for the | ||
3886 | "arm" multiconfig build. | ||
3887 | |||
3888 | Having a recipe depend on the root filesystem of another build might not | ||
3889 | seem that useful. Consider this change to the statement in the | ||
3890 | ``core-image-sato`` recipe: | ||
3891 | :: | ||
3892 | |||
3893 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
3894 | |||
3895 | In this case, BitBake must | ||
3896 | create the ``core-image-minimal`` image for the "arm" build since the | ||
3897 | "x86" build depends on it. | ||
3898 | |||
3899 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
3900 | and have separate configuration files, BitBake places the artifacts for | ||
3901 | each build in the respective temporary build directories (i.e. | ||
3902 | :term:`TMPDIR`). | ||
3903 | |||
3904 | Building an Initial RAM Filesystem (initramfs) Image | ||
3905 | ---------------------------------------------------- | ||
3906 | |||
3907 | An initial RAM filesystem (initramfs) image provides a temporary root | ||
3908 | filesystem used for early system initialization (e.g. loading of modules | ||
3909 | needed to locate and mount the "real" root filesystem). | ||
3910 | |||
3911 | .. note:: | ||
3912 | |||
3913 | The initramfs image is the successor of initial RAM disk (initrd). It | ||
3914 | is a "copy in and out" (cpio) archive of the initial filesystem that | ||
3915 | gets loaded into memory during the Linux startup process. Because | ||
3916 | Linux uses the contents of the archive during initialization, the | ||
3917 | initramfs image needs to contain all of the device drivers and tools | ||
3918 | needed to mount the final root filesystem. | ||
3919 | |||
3920 | Follow these steps to create an initramfs image: | ||
3921 | |||
3922 | 1. *Create the initramfs Image Recipe:* You can reference the | ||
3923 | ``core-image-minimal-initramfs.bb`` recipe found in the | ||
3924 | ``meta/recipes-core`` directory of the :term:`Source Directory` | ||
3925 | as an example | ||
3926 | from which to work. | ||
3927 | |||
3928 | 2. *Decide if You Need to Bundle the initramfs Image Into the Kernel | ||
3929 | Image:* If you want the initramfs image that is built to be bundled | ||
3930 | in with the kernel image, set the | ||
3931 | :term:`INITRAMFS_IMAGE_BUNDLE` | ||
3932 | variable to "1" in your ``local.conf`` configuration file and set the | ||
3933 | :term:`INITRAMFS_IMAGE` | ||
3934 | variable in the recipe that builds the kernel image. | ||
3935 | |||
3936 | .. note:: | ||
3937 | |||
3938 | It is recommended that you do bundle the initramfs image with the | ||
3939 | kernel image to avoid circular dependencies between the kernel | ||
3940 | recipe and the initramfs recipe should the initramfs image include | ||
3941 | kernel modules. | ||
3942 | |||
3943 | Setting the ``INITRAMFS_IMAGE_BUNDLE`` flag causes the initramfs | ||
3944 | image to be unpacked into the ``${B}/usr/`` directory. The unpacked | ||
3945 | initramfs image is then passed to the kernel's ``Makefile`` using the | ||
3946 | :term:`CONFIG_INITRAMFS_SOURCE` | ||
3947 | variable, allowing the initramfs image to be built into the kernel | ||
3948 | normally. | ||
3949 | |||
3950 | .. note:: | ||
3951 | |||
3952 | If you choose to not bundle the initramfs image with the kernel | ||
3953 | image, you are essentially using an | ||
3954 | `Initial RAM Disk (initrd) <https://en.wikipedia.org/wiki/Initrd>`__. | ||
3955 | Creating an initrd is handled primarily through the :term:`INITRD_IMAGE`, | ||
3956 | ``INITRD_LIVE``, and ``INITRD_IMAGE_LIVE`` variables. For more | ||
3957 | information, see the :ref:`ref-classes-image-live` file. | ||
3958 | |||
3959 | 3. *Optionally Add Items to the initramfs Image Through the initramfs | ||
3960 | Image Recipe:* If you add items to the initramfs image by way of its | ||
3961 | recipe, you should use | ||
3962 | :term:`PACKAGE_INSTALL` | ||
3963 | rather than | ||
3964 | :term:`IMAGE_INSTALL`. | ||
3965 | ``PACKAGE_INSTALL`` gives more direct control of what is added to the | ||
3966 | image as compared to the defaults you might not necessarily want that | ||
3967 | are set by the :ref:`image <ref-classes-image>` | ||
3968 | or :ref:`core-image <ref-classes-core-image>` | ||
3969 | classes. | ||
3970 | |||
3971 | 4. *Build the Kernel Image and the initramfs Image:* Build your kernel | ||
3972 | image using BitBake. Because the initramfs image recipe is a | ||
3973 | dependency of the kernel image, the initramfs image is built as well | ||
3974 | and bundled with the kernel image if you used the | ||
3975 | :term:`INITRAMFS_IMAGE_BUNDLE` | ||
3976 | variable described earlier. | ||
3977 | |||
3978 | Building a Tiny System | ||
3979 | ---------------------- | ||
3980 | |||
3981 | Very small distributions have some significant advantages such as | ||
3982 | requiring less on-die or in-package memory (cheaper), better performance | ||
3983 | through efficient cache usage, lower power requirements due to less | ||
3984 | memory, faster boot times, and reduced development overhead. Some | ||
3985 | real-world examples where a very small distribution gives you distinct | ||
3986 | advantages are digital cameras, medical devices, and small headless | ||
3987 | systems. | ||
3988 | |||
3989 | This section presents information that shows you how you can trim your | ||
3990 | distribution to even smaller sizes than the ``poky-tiny`` distribution, | ||
3991 | which is around 5 Mbytes, that can be built out-of-the-box using the | ||
3992 | Yocto Project. | ||
3993 | |||
3994 | Tiny System Overview | ||
3995 | ~~~~~~~~~~~~~~~~~~~~ | ||
3996 | |||
3997 | The following list presents the overall steps you need to consider and | ||
3998 | perform to create distributions with smaller root filesystems, achieve | ||
3999 | faster boot times, maintain your critical functionality, and avoid | ||
4000 | initial RAM disks: | ||
4001 | |||
4002 | - `Determine your goals and guiding | ||
4003 | principles. <#goals-and-guiding-principles>`__ | ||
4004 | |||
4005 | - `Understand what contributes to your image | ||
4006 | size. <#understand-what-gives-your-image-size>`__ | ||
4007 | |||
4008 | - `Reduce the size of the root | ||
4009 | filesystem. <#trim-the-root-filesystem>`__ | ||
4010 | |||
4011 | - `Reduce the size of the kernel. <#trim-the-kernel>`__ | ||
4012 | |||
4013 | - `Eliminate packaging | ||
4014 | requirements. <#remove-package-management-requirements>`__ | ||
4015 | |||
4016 | - `Look for other ways to minimize | ||
4017 | size. <#look-for-other-ways-to-minimize-size>`__ | ||
4018 | |||
4019 | - `Iterate on the process. <#iterate-on-the-process>`__ | ||
4020 | |||
4021 | Goals and Guiding Principles | ||
4022 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4023 | |||
4024 | Before you can reach your destination, you need to know where you are | ||
4025 | going. Here is an example list that you can use as a guide when creating | ||
4026 | very small distributions: | ||
4027 | |||
4028 | - Determine how much space you need (e.g. a kernel that is 1 Mbyte or | ||
4029 | less and a root filesystem that is 3 Mbytes or less). | ||
4030 | |||
4031 | - Find the areas that are currently taking 90% of the space and | ||
4032 | concentrate on reducing those areas. | ||
4033 | |||
4034 | - Do not create any difficult "hacks" to achieve your goals. | ||
4035 | |||
4036 | - Leverage the device-specific options. | ||
4037 | |||
4038 | - Work in a separate layer so that you keep changes isolated. For | ||
4039 | information on how to create layers, see the "`Understanding and | ||
4040 | Creating Layers <#understanding-and-creating-layers>`__" section. | ||
4041 | |||
4042 | Understand What Contributes to Your Image Size | ||
4043 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4044 | |||
4045 | It is easiest to have something to start with when creating your own | ||
4046 | distribution. You can use the Yocto Project out-of-the-box to create the | ||
4047 | ``poky-tiny`` distribution. Ultimately, you will want to make changes in | ||
4048 | your own distribution that are likely modeled after ``poky-tiny``. | ||
4049 | |||
4050 | .. note:: | ||
4051 | |||
4052 | To use ``poky-tiny`` in your build, set the ``DISTRO`` variable in your | ||
4053 | ``local.conf`` file to "poky-tiny" as described in the | ||
4054 | ":ref:`dev-manual/common-tasks:creating your own distribution`" | ||
4055 | section. | ||
4056 | |||
4057 | Understanding some memory concepts will help you reduce the system size. | ||
4058 | Memory consists of static, dynamic, and temporary memory. Static memory | ||
4059 | is the TEXT (code), DATA (initialized data in the code), and BSS | ||
4060 | (uninitialized data) sections. Dynamic memory represents memory that is | ||
4061 | allocated at runtime: stacks, hash tables, and so forth. Temporary | ||
4062 | memory is recovered after the boot process. This memory consists of | ||
4063 | memory used for decompressing the kernel and for the ``__init__`` | ||
4064 | functions. | ||
4065 | |||
4066 | To help you see where you currently are with kernel and root filesystem | ||
4067 | sizes, you can use two tools found in the :term:`Source Directory` | ||
4068 | in the | ||
4069 | ``scripts/tiny/`` directory: | ||
4070 | |||
4071 | - ``ksize.py``: Reports component sizes for the kernel build objects. | ||
4072 | |||
4073 | - ``dirsize.py``: Reports component sizes for the root filesystem. | ||
4074 | |||
4075 | This next tool and command help you organize configuration fragments and | ||
4076 | view file dependencies in a human-readable form: | ||
4077 | |||
4078 | - ``merge_config.sh``: Helps you manage configuration files and | ||
4079 | fragments within the kernel. With this tool, you can merge individual | ||
4080 | configuration fragments together. The tool allows you to make | ||
4081 | overrides and warns you of any missing configuration options. The | ||
4082 | tool is ideal for allowing you to iterate on configurations, create | ||
4083 | minimal configurations, and create configuration files for different | ||
4084 | machines without having to duplicate your process. | ||
4085 | |||
4086 | The ``merge_config.sh`` script is part of the Linux Yocto kernel Git | ||
4087 | repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, | ||
4088 | ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` | ||
4089 | directory. | ||
4090 | |||
4091 | For more information on configuration fragments, see the | ||
4092 | ":ref:`kernel-dev/common:creating configuration fragments`" | ||
4093 | section in the Yocto Project Linux Kernel Development Manual. | ||
4094 | |||
4095 | - ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command | ||
4096 | with these options brings up a Dependency Explorer from which you can | ||
4097 | view file dependencies. Understanding these dependencies allows you | ||
4098 | to make informed decisions when cutting out various pieces of the | ||
4099 | kernel and root filesystem. | ||
4100 | |||
4101 | Trim the Root Filesystem | ||
4102 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4103 | |||
4104 | The root filesystem is made up of packages for booting, libraries, and | ||
4105 | applications. To change things, you can configure how the packaging | ||
4106 | happens, which changes the way you build them. You can also modify the | ||
4107 | filesystem itself or select a different filesystem. | ||
4108 | |||
4109 | First, find out what is hogging your root filesystem by running the | ||
4110 | ``dirsize.py`` script from your root directory: | ||
4111 | :: | ||
4112 | |||
4113 | $ cd root-directory-of-image | ||
4114 | $ dirsize.py 100000 > dirsize-100k.log | ||
4115 | $ cat dirsize-100k.log | ||
4116 | |||
4117 | You can apply a filter to the script to ignore files | ||
4118 | under a certain size. The previous example filters out any files below | ||
4119 | 100 Kbytes. The sizes reported by the tool are uncompressed, and thus | ||
4120 | will be smaller by a relatively constant factor in a compressed root | ||
4121 | filesystem. When you examine your log file, you can focus on areas of | ||
4122 | the root filesystem that take up large amounts of memory. | ||
4123 | |||
4124 | You need to be sure that what you eliminate does not cripple the | ||
4125 | functionality you need. One way to see how packages relate to each other | ||
4126 | is by using the Dependency Explorer UI with the BitBake command: | ||
4127 | :: | ||
4128 | |||
4129 | $ cd image-directory | ||
4130 | $ bitbake -u taskexp -g image | ||
4131 | |||
4132 | Use the interface to | ||
4133 | select potential packages you wish to eliminate and see their dependency | ||
4134 | relationships. | ||
4135 | |||
4136 | When deciding how to reduce the size, get rid of packages that result in | ||
4137 | minimal impact on the feature set. For example, you might not need a VGA | ||
4138 | display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` | ||
4139 | instead of ``udev``. | ||
4140 | |||
4141 | Use your ``local.conf`` file to make changes. For example, to eliminate | ||
4142 | ``udev`` and ``glib``, set the following in the local configuration | ||
4143 | file: | ||
4144 | :: | ||
4145 | |||
4146 | VIRTUAL-RUNTIME_dev_manager = "" | ||
4147 | |||
4148 | Finally, you should consider exactly the type of root filesystem you | ||
4149 | need to meet your needs while also reducing its size. For example, | ||
4150 | consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an | ||
4151 | ``initramfs`` using ``initramfs``. Be aware that ``ext3`` requires a 1 | ||
4152 | Mbyte journal. If you are okay with running read-only, you do not need | ||
4153 | this journal. | ||
4154 | |||
4155 | .. note:: | ||
4156 | |||
4157 | After each round of elimination, you need to rebuild your system and | ||
4158 | then use the tools to see the effects of your reductions. | ||
4159 | |||
4160 | Trim the Kernel | ||
4161 | ~~~~~~~~~~~~~~~ | ||
4162 | |||
4163 | The kernel is built by including policies for hardware-independent | ||
4164 | aspects. What subsystems do you enable? For what architecture are you | ||
4165 | building? Which drivers do you build by default? | ||
4166 | |||
4167 | .. note:: | ||
4168 | |||
4169 | You can modify the kernel source if you want to help with boot time. | ||
4170 | |||
4171 | Run the ``ksize.py`` script from the top-level Linux build directory to | ||
4172 | get an idea of what is making up the kernel: | ||
4173 | :: | ||
4174 | |||
4175 | $ cd top-level-linux-build-directory | ||
4176 | $ ksize.py > ksize.log | ||
4177 | $ cat ksize.log | ||
4178 | |||
4179 | When you examine the log, you will see how much space is taken up with | ||
4180 | the built-in ``.o`` files for drivers, networking, core kernel files, | ||
4181 | filesystem, sound, and so forth. The sizes reported by the tool are | ||
4182 | uncompressed, and thus will be smaller by a relatively constant factor | ||
4183 | in a compressed kernel image. Look to reduce the areas that are large | ||
4184 | and taking up around the "90% rule." | ||
4185 | |||
4186 | To examine, or drill down, into any particular area, use the ``-d`` | ||
4187 | option with the script: | ||
4188 | :: | ||
4189 | |||
4190 | $ ksize.py -d > ksize.log | ||
4191 | |||
4192 | Using this option | ||
4193 | breaks out the individual file information for each area of the kernel | ||
4194 | (e.g. drivers, networking, and so forth). | ||
4195 | |||
4196 | Use your log file to see what you can eliminate from the kernel based on | ||
4197 | features you can let go. For example, if you are not going to need | ||
4198 | sound, you do not need any drivers that support sound. | ||
4199 | |||
4200 | After figuring out what to eliminate, you need to reconfigure the kernel | ||
4201 | to reflect those changes during the next build. You could run | ||
4202 | ``menuconfig`` and make all your changes at once. However, that makes it | ||
4203 | difficult to see the effects of your individual eliminations and also | ||
4204 | makes it difficult to replicate the changes for perhaps another target | ||
4205 | device. A better method is to start with no configurations using | ||
4206 | ``allnoconfig``, create configuration fragments for individual changes, | ||
4207 | and then manage the fragments into a single configuration file using | ||
4208 | ``merge_config.sh``. The tool makes it easy for you to iterate using the | ||
4209 | configuration change and build cycle. | ||
4210 | |||
4211 | Each time you make configuration changes, you need to rebuild the kernel | ||
4212 | and check to see what impact your changes had on the overall size. | ||
4213 | |||
4214 | Remove Package Management Requirements | ||
4215 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4216 | |||
4217 | Packaging requirements add size to the image. One way to reduce the size | ||
4218 | of the image is to remove all the packaging requirements from the image. | ||
4219 | This reduction includes both removing the package manager and its unique | ||
4220 | dependencies as well as removing the package management data itself. | ||
4221 | |||
4222 | To eliminate all the packaging requirements for an image, be sure that | ||
4223 | "package-management" is not part of your | ||
4224 | :term:`IMAGE_FEATURES` | ||
4225 | statement for the image. When you remove this feature, you are removing | ||
4226 | the package manager as well as its dependencies from the root | ||
4227 | filesystem. | ||
4228 | |||
4229 | Look for Other Ways to Minimize Size | ||
4230 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4231 | |||
4232 | Depending on your particular circumstances, other areas that you can | ||
4233 | trim likely exist. The key to finding these areas is through tools and | ||
4234 | methods described here combined with experimentation and iteration. Here | ||
4235 | are a couple of areas to experiment with: | ||
4236 | |||
4237 | - ``glibc``: In general, follow this process: | ||
4238 | |||
4239 | 1. Remove ``glibc`` features from | ||
4240 | :term:`DISTRO_FEATURES` | ||
4241 | that you think you do not need. | ||
4242 | |||
4243 | 2. Build your distribution. | ||
4244 | |||
4245 | 3. If the build fails due to missing symbols in a package, determine | ||
4246 | if you can reconfigure the package to not need those features. For | ||
4247 | example, change the configuration to not support wide character | ||
4248 | support as is done for ``ncurses``. Or, if support for those | ||
4249 | characters is needed, determine what ``glibc`` features provide | ||
4250 | the support and restore the configuration. | ||
4251 | |||
4252 | 4. Rebuild and repeat the process. | ||
4253 | |||
4254 | - ``busybox``: For BusyBox, use a process similar as described for | ||
4255 | ``glibc``. A difference is you will need to boot the resulting system | ||
4256 | to see if you are able to do everything you expect from the running | ||
4257 | system. You need to be sure to integrate configuration fragments into | ||
4258 | Busybox because BusyBox handles its own core features and then allows | ||
4259 | you to add configuration fragments on top. | ||
4260 | |||
4261 | Iterate on the Process | ||
4262 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
4263 | |||
4264 | If you have not reached your goals on system size, you need to iterate | ||
4265 | on the process. The process is the same. Use the tools and see just what | ||
4266 | is taking up 90% of the root filesystem and the kernel. Decide what you | ||
4267 | can eliminate without limiting your device beyond what you need. | ||
4268 | |||
4269 | Depending on your system, a good place to look might be Busybox, which | ||
4270 | provides a stripped down version of Unix tools in a single, executable | ||
4271 | file. You might be able to drop virtual terminal services or perhaps | ||
4272 | ipv6. | ||
4273 | |||
4274 | Building Images for More than One Machine | ||
4275 | ----------------------------------------- | ||
4276 | |||
4277 | A common scenario developers face is creating images for several | ||
4278 | different machines that use the same software environment. In this | ||
4279 | situation, it is tempting to set the tunings and optimization flags for | ||
4280 | each build specifically for the targeted hardware (i.e. "maxing out" the | ||
4281 | tunings). Doing so can considerably add to build times and package feed | ||
4282 | maintenance collectively for the machines. For example, selecting tunes | ||
4283 | that are extremely specific to a CPU core used in a system might enable | ||
4284 | some micro optimizations in GCC for that particular system but would | ||
4285 | otherwise not gain you much of a performance difference across the other | ||
4286 | systems as compared to using a more general tuning across all the builds | ||
4287 | (e.g. setting :term:`DEFAULTTUNE` | ||
4288 | specifically for each machine's build). Rather than "max out" each | ||
4289 | build's tunings, you can take steps that cause the OpenEmbedded build | ||
4290 | system to reuse software across the various machines where it makes | ||
4291 | sense. | ||
4292 | |||
4293 | If build speed and package feed maintenance are considerations, you | ||
4294 | should consider the points in this section that can help you optimize | ||
4295 | your tunings to best consider build times and package feed maintenance. | ||
4296 | |||
4297 | - *Share the Build Directory:* If at all possible, share the | ||
4298 | :term:`TMPDIR` across builds. The | ||
4299 | Yocto Project supports switching between different | ||
4300 | :term:`MACHINE` values in the same | ||
4301 | ``TMPDIR``. This practice is well supported and regularly used by | ||
4302 | developers when building for multiple machines. When you use the same | ||
4303 | ``TMPDIR`` for multiple machine builds, the OpenEmbedded build system | ||
4304 | can reuse the existing native and often cross-recipes for multiple | ||
4305 | machines. Thus, build time decreases. | ||
4306 | |||
4307 | .. note:: | ||
4308 | |||
4309 | If :term:`DISTRO` settings change or fundamental configuration settings | ||
4310 | such as the filesystem layout, you need to work with a clean ``TMPDIR``. | ||
4311 | Sharing ``TMPDIR`` under these circumstances might work but since it is | ||
4312 | not guaranteed, you should use a clean ``TMPDIR``. | ||
4313 | |||
4314 | - *Enable the Appropriate Package Architecture:* By default, the | ||
4315 | OpenEmbedded build system enables three levels of package | ||
4316 | architectures: "all", "tune" or "package", and "machine". Any given | ||
4317 | recipe usually selects one of these package architectures (types) for | ||
4318 | its output. Depending for what a given recipe creates packages, | ||
4319 | making sure you enable the appropriate package architecture can | ||
4320 | directly impact the build time. | ||
4321 | |||
4322 | A recipe that just generates scripts can enable "all" architecture | ||
4323 | because there are no binaries to build. To specifically enable "all" | ||
4324 | architecture, be sure your recipe inherits the | ||
4325 | :ref:`allarch <ref-classes-allarch>` class. | ||
4326 | This class is useful for "all" architectures because it configures | ||
4327 | many variables so packages can be used across multiple architectures. | ||
4328 | |||
4329 | If your recipe needs to generate packages that are machine-specific | ||
4330 | or when one of the build or runtime dependencies is already | ||
4331 | machine-architecture dependent, which makes your recipe also | ||
4332 | machine-architecture dependent, make sure your recipe enables the | ||
4333 | "machine" package architecture through the | ||
4334 | :term:`MACHINE_ARCH` | ||
4335 | variable: | ||
4336 | :: | ||
4337 | |||
4338 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
4339 | |||
4340 | When you do not | ||
4341 | specifically enable a package architecture through the | ||
4342 | :term:`PACKAGE_ARCH`, The | ||
4343 | OpenEmbedded build system defaults to the | ||
4344 | :term:`TUNE_PKGARCH` setting: | ||
4345 | :: | ||
4346 | |||
4347 | PACKAGE_ARCH = "${TUNE_PKGARCH}" | ||
4348 | |||
4349 | - *Choose a Generic Tuning File if Possible:* Some tunes are more | ||
4350 | generic and can run on multiple targets (e.g. an ``armv5`` set of | ||
4351 | packages could run on ``armv6`` and ``armv7`` processors in most | ||
4352 | cases). Similarly, ``i486`` binaries could work on ``i586`` and | ||
4353 | higher processors. You should realize, however, that advances on | ||
4354 | newer processor versions would not be used. | ||
4355 | |||
4356 | If you select the same tune for several different machines, the | ||
4357 | OpenEmbedded build system reuses software previously built, thus | ||
4358 | speeding up the overall build time. Realize that even though a new | ||
4359 | sysroot for each machine is generated, the software is not recompiled | ||
4360 | and only one package feed exists. | ||
4361 | |||
4362 | - *Manage Granular Level Packaging:* Sometimes cases exist where | ||
4363 | injecting another level of package architecture beyond the three | ||
4364 | higher levels noted earlier can be useful. For example, consider how | ||
4365 | NXP (formerly Freescale) allows for the easy reuse of binary packages | ||
4366 | in their layer | ||
4367 | :yocto_git:`meta-freescale </meta-freescale/>`. | ||
4368 | In this example, the | ||
4369 | :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` | ||
4370 | class shares GPU packages for i.MX53 boards because all boards share | ||
4371 | the AMD GPU. The i.MX6-based boards can do the same because all | ||
4372 | boards share the Vivante GPU. This class inspects the BitBake | ||
4373 | datastore to identify if the package provides or depends on one of | ||
4374 | the sub-architecture values. If so, the class sets the | ||
4375 | :term:`PACKAGE_ARCH` value | ||
4376 | based on the ``MACHINE_SUBARCH`` value. If the package does not | ||
4377 | provide or depend on one of the sub-architecture values but it | ||
4378 | matches a value in the machine-specific filter, it sets | ||
4379 | :term:`MACHINE_ARCH`. This | ||
4380 | behavior reduces the number of packages built and saves build time by | ||
4381 | reusing binaries. | ||
4382 | |||
4383 | - *Use Tools to Debug Issues:* Sometimes you can run into situations | ||
4384 | where software is being rebuilt when you think it should not be. For | ||
4385 | example, the OpenEmbedded build system might not be using shared | ||
4386 | state between machines when you think it should be. These types of | ||
4387 | situations are usually due to references to machine-specific | ||
4388 | variables such as :term:`MACHINE`, | ||
4389 | :term:`SERIAL_CONSOLES`, | ||
4390 | :term:`XSERVER`, | ||
4391 | :term:`MACHINE_FEATURES`, | ||
4392 | and so forth in code that is supposed to only be tune-specific or | ||
4393 | when the recipe depends | ||
4394 | (:term:`DEPENDS`, | ||
4395 | :term:`RDEPENDS`, | ||
4396 | :term:`RRECOMMENDS`, | ||
4397 | :term:`RSUGGESTS`, and so forth) | ||
4398 | on some other recipe that already has | ||
4399 | :term:`PACKAGE_ARCH` defined | ||
4400 | as "${MACHINE_ARCH}". | ||
4401 | |||
4402 | .. note:: | ||
4403 | |||
4404 | Patches to fix any issues identified are most welcome as these | ||
4405 | issues occasionally do occur. | ||
4406 | |||
4407 | For such cases, you can use some tools to help you sort out the | ||
4408 | situation: | ||
4409 | |||
4410 | - ``state-diff-machines.sh``*:* You can find this tool in the | ||
4411 | ``scripts`` directory of the Source Repositories. See the comments | ||
4412 | in the script for information on how to use the tool. | ||
4413 | |||
4414 | - *BitBake's "-S printdiff" Option:* Using this option causes | ||
4415 | BitBake to try to establish the closest signature match it can | ||
4416 | (e.g. in the shared state cache) and then run ``bitbake-diffsigs`` | ||
4417 | over the matches to determine the stamps and delta where these two | ||
4418 | stamp trees diverge. | ||
4419 | |||
4420 | Building Software from an External Source | ||
4421 | ----------------------------------------- | ||
4422 | |||
4423 | By default, the OpenEmbedded build system uses the | ||
4424 | :term:`Build Directory` when building source | ||
4425 | code. The build process involves fetching the source files, unpacking | ||
4426 | them, and then patching them if necessary before the build takes place. | ||
4427 | |||
4428 | Situations exist where you might want to build software from source | ||
4429 | files that are external to and thus outside of the OpenEmbedded build | ||
4430 | system. For example, suppose you have a project that includes a new BSP | ||
4431 | with a heavily customized kernel. And, you want to minimize exposing the | ||
4432 | build system to the development team so that they can focus on their | ||
4433 | project and maintain everyone's workflow as much as possible. In this | ||
4434 | case, you want a kernel source directory on the development machine | ||
4435 | where the development occurs. You want the recipe's | ||
4436 | :term:`SRC_URI` variable to point to | ||
4437 | the external directory and use it as is, not copy it. | ||
4438 | |||
4439 | To build from software that comes from an external source, all you need | ||
4440 | to do is inherit the | ||
4441 | :ref:`externalsrc <ref-classes-externalsrc>` class | ||
4442 | and then set the | ||
4443 | :term:`EXTERNALSRC` variable to | ||
4444 | point to your external source code. Here are the statements to put in | ||
4445 | your ``local.conf`` file: | ||
4446 | :: | ||
4447 | |||
4448 | INHERIT += "externalsrc" | ||
4449 | EXTERNALSRC_pn-myrecipe = "path-to-your-source-tree" | ||
4450 | |||
4451 | This next example shows how to accomplish the same thing by setting | ||
4452 | ``EXTERNALSRC`` in the recipe itself or in the recipe's append file: | ||
4453 | :: | ||
4454 | |||
4455 | EXTERNALSRC = "path" | ||
4456 | EXTERNALSRC_BUILD = "path" | ||
4457 | |||
4458 | .. note:: | ||
4459 | |||
4460 | In order for these settings to take effect, you must globally or | ||
4461 | locally inherit the :ref:`externalsrc <ref-classes-externalsrc>` | ||
4462 | class. | ||
4463 | |||
4464 | By default, ``externalsrc.bbclass`` builds the source code in a | ||
4465 | directory separate from the external source directory as specified by | ||
4466 | :term:`EXTERNALSRC`. If you need | ||
4467 | to have the source built in the same directory in which it resides, or | ||
4468 | some other nominated directory, you can set | ||
4469 | :term:`EXTERNALSRC_BUILD` | ||
4470 | to point to that directory: | ||
4471 | :: | ||
4472 | |||
4473 | EXTERNALSRC_BUILD_pn-myrecipe = "path-to-your-source-tree" | ||
4474 | |||
4475 | Replicating a Build Offline | ||
4476 | --------------------------- | ||
4477 | |||
4478 | It can be useful to take a "snapshot" of upstream sources used in a | ||
4479 | build and then use that "snapshot" later to replicate the build offline. | ||
4480 | To do so, you need to first prepare and populate your downloads | ||
4481 | directory your "snapshot" of files. Once your downloads directory is | ||
4482 | ready, you can use it at any time and from any machine to replicate your | ||
4483 | build. | ||
4484 | |||
4485 | Follow these steps to populate your Downloads directory: | ||
4486 | |||
4487 | 1. *Create a Clean Downloads Directory:* Start with an empty downloads | ||
4488 | directory (:term:`DL_DIR`). You | ||
4489 | start with an empty downloads directory by either removing the files | ||
4490 | in the existing directory or by setting ``DL_DIR`` to point to either | ||
4491 | an empty location or one that does not yet exist. | ||
4492 | |||
4493 | 2. *Generate Tarballs of the Source Git Repositories:* Edit your | ||
4494 | ``local.conf`` configuration file as follows: | ||
4495 | :: | ||
4496 | |||
4497 | DL_DIR = "/home/your-download-dir/" | ||
4498 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
4499 | |||
4500 | During | ||
4501 | the fetch process in the next step, BitBake gathers the source files | ||
4502 | and creates tarballs in the directory pointed to by ``DL_DIR``. See | ||
4503 | the | ||
4504 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
4505 | variable for more information. | ||
4506 | |||
4507 | 3. *Populate Your Downloads Directory Without Building:* Use BitBake to | ||
4508 | fetch your sources but inhibit the build: | ||
4509 | :: | ||
4510 | |||
4511 | $ bitbake target --runonly=fetch | ||
4512 | |||
4513 | The downloads directory (i.e. ``${DL_DIR}``) now has | ||
4514 | a "snapshot" of the source files in the form of tarballs, which can | ||
4515 | be used for the build. | ||
4516 | |||
4517 | 4. *Optionally Remove Any Git or other SCM Subdirectories From the | ||
4518 | Downloads Directory:* If you want, you can clean up your downloads | ||
4519 | directory by removing any Git or other Source Control Management | ||
4520 | (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs | ||
4521 | already contain these subdirectories. | ||
4522 | |||
4523 | Once your downloads directory has everything it needs regarding source | ||
4524 | files, you can create your "own-mirror" and build your target. | ||
4525 | Understand that you can use the files to build the target offline from | ||
4526 | any machine and at any time. | ||
4527 | |||
4528 | Follow these steps to build your target using the files in the downloads | ||
4529 | directory: | ||
4530 | |||
4531 | 1. *Using Local Files Only:* Inside your ``local.conf`` file, add the | ||
4532 | :term:`SOURCE_MIRROR_URL` | ||
4533 | variable, inherit the | ||
4534 | :ref:`own-mirrors <ref-classes-own-mirrors>` | ||
4535 | class, and use the | ||
4536 | :term:`bitbake:BB_NO_NETWORK` | ||
4537 | variable to your ``local.conf``. | ||
4538 | :: | ||
4539 | |||
4540 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | ||
4541 | INHERIT += "own-mirrors" | ||
4542 | BB_NO_NETWORK = "1" | ||
4543 | |||
4544 | The ``SOURCE_MIRROR_URL`` and ``own-mirror`` | ||
4545 | class set up the system to use the downloads directory as your "own | ||
4546 | mirror". Using the ``BB_NO_NETWORK`` variable makes sure that | ||
4547 | BitBake's fetching process in step 3 stays local, which means files | ||
4548 | from your "own-mirror" are used. | ||
4549 | |||
4550 | 2. *Start With a Clean Build:* You can start with a clean build by | ||
4551 | removing the | ||
4552 | ``${``\ :term:`TMPDIR`\ ``}`` | ||
4553 | directory or using a new :term:`Build Directory`. | ||
4554 | |||
4555 | 3. *Build Your Target:* Use BitBake to build your target: | ||
4556 | :: | ||
4557 | |||
4558 | $ bitbake target | ||
4559 | |||
4560 | The build completes using the known local "snapshot" of source | ||
4561 | files from your mirror. The resulting tarballs for your "snapshot" of | ||
4562 | source files are in the downloads directory. | ||
4563 | |||
4564 | .. note:: | ||
4565 | |||
4566 | The offline build does not work if recipes attempt to find the | ||
4567 | latest version of software by setting | ||
4568 | :term:`SRCREV` to | ||
4569 | ``${``\ :term:`AUTOREV`\ ``}``: | ||
4570 | :: | ||
4571 | |||
4572 | SRCREV = "${AUTOREV}" | ||
4573 | |||
4574 | When a recipe sets ``SRCREV`` to | ||
4575 | ``${AUTOREV}``, the build system accesses the network in an | ||
4576 | attempt to determine the latest version of software from the SCM. | ||
4577 | Typically, recipes that use ``AUTOREV`` are custom or modified | ||
4578 | recipes. Recipes that reside in public repositories usually do not | ||
4579 | use ``AUTOREV``. | ||
4580 | |||
4581 | If you do have recipes that use ``AUTOREV``, you can take steps to | ||
4582 | still use the recipes in an offline build. Do the following: | ||
4583 | |||
4584 | 1. Use a configuration generated by enabling `build | ||
4585 | history <#maintaining-build-output-quality>`__. | ||
4586 | |||
4587 | 2. Use the ``buildhistory-collect-srcrevs`` command to collect the | ||
4588 | stored ``SRCREV`` values from the build's history. For more | ||
4589 | information on collecting these values, see the "`Build History | ||
4590 | Package Information <#build-history-package-information>`__" | ||
4591 | section. | ||
4592 | |||
4593 | 3. Once you have the correct source revisions, you can modify | ||
4594 | those recipes to to set ``SRCREV`` to specific versions of the | ||
4595 | software. | ||
4596 | |||
4597 | Speeding Up a Build | ||
4598 | =================== | ||
4599 | |||
4600 | Build time can be an issue. By default, the build system uses simple | ||
4601 | controls to try and maximize build efficiency. In general, the default | ||
4602 | settings for all the following variables result in the most efficient | ||
4603 | build times when dealing with single socket systems (i.e. a single CPU). | ||
4604 | If you have multiple CPUs, you might try increasing the default values | ||
4605 | to gain more speed. See the descriptions in the glossary for each | ||
4606 | variable for more information: | ||
4607 | |||
4608 | - :term:`BB_NUMBER_THREADS`: | ||
4609 | The maximum number of threads BitBake simultaneously executes. | ||
4610 | |||
4611 | - :term:`bitbake:BB_NUMBER_PARSE_THREADS`: | ||
4612 | The number of threads BitBake uses during parsing. | ||
4613 | |||
4614 | - :term:`PARALLEL_MAKE`: Extra | ||
4615 | options passed to the ``make`` command during the | ||
4616 | :ref:`ref-tasks-compile` task in | ||
4617 | order to specify parallel compilation on the local build host. | ||
4618 | |||
4619 | - :term:`PARALLEL_MAKEINST`: | ||
4620 | Extra options passed to the ``make`` command during the | ||
4621 | :ref:`ref-tasks-install` task in | ||
4622 | order to specify parallel installation on the local build host. | ||
4623 | |||
4624 | As mentioned, these variables all scale to the number of processor cores | ||
4625 | available on the build system. For single socket systems, this | ||
4626 | auto-scaling ensures that the build system fundamentally takes advantage | ||
4627 | of potential parallel operations during the build based on the build | ||
4628 | machine's capabilities. | ||
4629 | |||
4630 | Following are additional factors that can affect build speed: | ||
4631 | |||
4632 | - File system type: The file system type that the build is being | ||
4633 | performed on can also influence performance. Using ``ext4`` is | ||
4634 | recommended as compared to ``ext2`` and ``ext3`` due to ``ext4`` | ||
4635 | improved features such as extents. | ||
4636 | |||
4637 | - Disabling the updating of access time using ``noatime``: The | ||
4638 | ``noatime`` mount option prevents the build system from updating file | ||
4639 | and directory access times. | ||
4640 | |||
4641 | - Setting a longer commit: Using the "commit=" mount option increases | ||
4642 | the interval in seconds between disk cache writes. Changing this | ||
4643 | interval from the five second default to something longer increases | ||
4644 | the risk of data loss but decreases the need to write to the disk, | ||
4645 | thus increasing the build performance. | ||
4646 | |||
4647 | - Choosing the packaging backend: Of the available packaging backends, | ||
4648 | IPK is the fastest. Additionally, selecting a singular packaging | ||
4649 | backend also helps. | ||
4650 | |||
4651 | - Using ``tmpfs`` for :term:`TMPDIR` | ||
4652 | as a temporary file system: While this can help speed up the build, | ||
4653 | the benefits are limited due to the compiler using ``-pipe``. The | ||
4654 | build system goes to some lengths to avoid ``sync()`` calls into the | ||
4655 | file system on the principle that if there was a significant failure, | ||
4656 | the :term:`Build Directory` | ||
4657 | contents could easily be rebuilt. | ||
4658 | |||
4659 | - Inheriting the | ||
4660 | :ref:`rm_work <ref-classes-rm-work>` class: | ||
4661 | Inheriting this class has shown to speed up builds due to | ||
4662 | significantly lower amounts of data stored in the data cache as well | ||
4663 | as on disk. Inheriting this class also makes cleanup of | ||
4664 | :term:`TMPDIR` faster, at the | ||
4665 | expense of being easily able to dive into the source code. File | ||
4666 | system maintainers have recommended that the fastest way to clean up | ||
4667 | large numbers of files is to reformat partitions rather than delete | ||
4668 | files due to the linear nature of partitions. This, of course, | ||
4669 | assumes you structure the disk partitions and file systems in a way | ||
4670 | that this is practical. | ||
4671 | |||
4672 | Aside from the previous list, you should keep some trade offs in mind | ||
4673 | that can help you speed up the build: | ||
4674 | |||
4675 | - Remove items from | ||
4676 | :term:`DISTRO_FEATURES` | ||
4677 | that you might not need. | ||
4678 | |||
4679 | - Exclude debug symbols and other debug information: If you do not need | ||
4680 | these symbols and other debug information, disabling the ``*-dbg`` | ||
4681 | package generation can speed up the build. You can disable this | ||
4682 | generation by setting the | ||
4683 | :term:`INHIBIT_PACKAGE_DEBUG_SPLIT` | ||
4684 | variable to "1". | ||
4685 | |||
4686 | - Disable static library generation for recipes derived from | ||
4687 | ``autoconf`` or ``libtool``: Following is an example showing how to | ||
4688 | disable static libraries and still provide an override to handle | ||
4689 | exceptions: | ||
4690 | :: | ||
4691 | |||
4692 | STATICLIBCONF = "--disable-static" | ||
4693 | STATICLIBCONF_sqlite3-native = "" | ||
4694 | EXTRA_OECONF += "${STATICLIBCONF}" | ||
4695 | |||
4696 | .. note:: | ||
4697 | |||
4698 | - Some recipes need static libraries in order to work correctly | ||
4699 | (e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides, | ||
4700 | as in the previous example, account for these kinds of | ||
4701 | exceptions. | ||
4702 | |||
4703 | - Some packages have packaging code that assumes the presence of | ||
4704 | the static libraries. If so, you might need to exclude them as | ||
4705 | well. | ||
4706 | |||
4707 | Working With Libraries | ||
4708 | ====================== | ||
4709 | |||
4710 | Libraries are an integral part of your system. This section describes | ||
4711 | some common practices you might find helpful when working with libraries | ||
4712 | to build your system: | ||
4713 | |||
4714 | - `How to include static library | ||
4715 | files <#including-static-library-files>`__ | ||
4716 | |||
4717 | - `How to use the Multilib feature to combine multiple versions of | ||
4718 | library files into a single | ||
4719 | image <#combining-multiple-versions-library-files-into-one-image>`__ | ||
4720 | |||
4721 | - `How to install multiple versions of the same library in parallel on | ||
4722 | the same | ||
4723 | system <#installing-multiple-versions-of-the-same-library>`__ | ||
4724 | |||
4725 | Including Static Library Files | ||
4726 | ------------------------------ | ||
4727 | |||
4728 | If you are building a library and the library offers static linking, you | ||
4729 | can control which static library files (``*.a`` files) get included in | ||
4730 | the built library. | ||
4731 | |||
4732 | The :term:`PACKAGES` and | ||
4733 | :term:`FILES_* <FILES>` variables in the | ||
4734 | ``meta/conf/bitbake.conf`` configuration file define how files installed | ||
4735 | by the ``do_install`` task are packaged. By default, the ``PACKAGES`` | ||
4736 | variable includes ``${PN}-staticdev``, which represents all static | ||
4737 | library files. | ||
4738 | |||
4739 | .. note:: | ||
4740 | |||
4741 | Some previously released versions of the Yocto Project defined the | ||
4742 | static library files through ``${PN}-dev``. | ||
4743 | |||
4744 | Following is part of the BitBake configuration file, where you can see | ||
4745 | how the static library files are defined: | ||
4746 | :: | ||
4747 | |||
4748 | PACKAGE_BEFORE_PN ?= "" | ||
4749 | PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" | ||
4750 | PACKAGES_DYNAMIC = "^${PN}-locale-.*" | ||
4751 | FILES = "" | ||
4752 | |||
4753 | FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \ | ||
4754 | ${sysconfdir} ${sharedstatedir} ${localstatedir} \ | ||
4755 | ${base_bindir}/* ${base_sbindir}/* \ | ||
4756 | ${base_libdir}/*${SOLIBS} \ | ||
4757 | ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \ | ||
4758 | ${datadir}/${BPN} ${libdir}/${BPN}/* \ | ||
4759 | ${datadir}/pixmaps ${datadir}/applications \ | ||
4760 | ${datadir}/idl ${datadir}/omf ${datadir}/sounds \ | ||
4761 | ${libdir}/bonobo/servers" | ||
4762 | |||
4763 | FILES_${PN}-bin = "${bindir}/* ${sbindir}/*" | ||
4764 | |||
4765 | FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \ | ||
4766 | ${datadir}/gnome/help" | ||
4767 | SECTION_${PN}-doc = "doc" | ||
4768 | |||
4769 | FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}" | ||
4770 | FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \ | ||
4771 | ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \ | ||
4772 | ${datadir}/aclocal ${base_libdir}/*.o \ | ||
4773 | ${libdir}/${BPN}/*.la ${base_libdir}/*.la" | ||
4774 | SECTION_${PN}-dev = "devel" | ||
4775 | ALLOW_EMPTY_${PN}-dev = "1" | ||
4776 | RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" | ||
4777 | |||
4778 | FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" | ||
4779 | SECTION_${PN}-staticdev = "devel" | ||
4780 | RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})" | ||
4781 | |||
4782 | Combining Multiple Versions of Library Files into One Image | ||
4783 | ----------------------------------------------------------- | ||
4784 | |||
4785 | The build system offers the ability to build libraries with different | ||
4786 | target optimizations or architecture formats and combine these together | ||
4787 | into one system image. You can link different binaries in the image | ||
4788 | against the different libraries as needed for specific use cases. This | ||
4789 | feature is called "Multilib". | ||
4790 | |||
4791 | An example would be where you have most of a system compiled in 32-bit | ||
4792 | mode using 32-bit libraries, but you have something large, like a | ||
4793 | database engine, that needs to be a 64-bit application and uses 64-bit | ||
4794 | libraries. Multilib allows you to get the best of both 32-bit and 64-bit | ||
4795 | libraries. | ||
4796 | |||
4797 | While the Multilib feature is most commonly used for 32 and 64-bit | ||
4798 | differences, the approach the build system uses facilitates different | ||
4799 | target optimizations. You could compile some binaries to use one set of | ||
4800 | libraries and other binaries to use a different set of libraries. The | ||
4801 | libraries could differ in architecture, compiler options, or other | ||
4802 | optimizations. | ||
4803 | |||
4804 | Several examples exist in the ``meta-skeleton`` layer found in the | ||
4805 | :term:`Source Directory`: | ||
4806 | |||
4807 | - ``conf/multilib-example.conf`` configuration file | ||
4808 | |||
4809 | - ``conf/multilib-example2.conf`` configuration file | ||
4810 | |||
4811 | - ``recipes-multilib/images/core-image-multilib-example.bb`` recipe | ||
4812 | |||
4813 | Preparing to Use Multilib | ||
4814 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4815 | |||
4816 | User-specific requirements drive the Multilib feature. Consequently, | ||
4817 | there is no one "out-of-the-box" configuration that likely exists to | ||
4818 | meet your needs. | ||
4819 | |||
4820 | In order to enable Multilib, you first need to ensure your recipe is | ||
4821 | extended to support multiple libraries. Many standard recipes are | ||
4822 | already extended and support multiple libraries. You can check in the | ||
4823 | ``meta/conf/multilib.conf`` configuration file in the | ||
4824 | :term:`Source Directory` to see how this is | ||
4825 | done using the | ||
4826 | :term:`BBCLASSEXTEND` variable. | ||
4827 | Eventually, all recipes will be covered and this list will not be | ||
4828 | needed. | ||
4829 | |||
4830 | For the most part, the Multilib class extension works automatically to | ||
4831 | extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where | ||
4832 | ``MLPREFIX`` is the particular multilib (e.g. "lib32-" or "lib64-"). | ||
4833 | Standard variables such as | ||
4834 | :term:`DEPENDS`, | ||
4835 | :term:`RDEPENDS`, | ||
4836 | :term:`RPROVIDES`, | ||
4837 | :term:`RRECOMMENDS`, | ||
4838 | :term:`PACKAGES`, and | ||
4839 | :term:`PACKAGES_DYNAMIC` are | ||
4840 | automatically extended by the system. If you are extending any manual | ||
4841 | code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure | ||
4842 | those names are extended correctly. This automatic extension code | ||
4843 | resides in ``multilib.bbclass``. | ||
4844 | |||
4845 | Using Multilib | ||
4846 | ~~~~~~~~~~~~~~ | ||
4847 | |||
4848 | After you have set up the recipes, you need to define the actual | ||
4849 | combination of multiple libraries you want to build. You accomplish this | ||
4850 | through your ``local.conf`` configuration file in the | ||
4851 | :term:`Build Directory`. An example | ||
4852 | configuration would be as follows: | ||
4853 | :: | ||
4854 | |||
4855 | MACHINE = "qemux86-64" | ||
4856 | require conf/multilib.conf | ||
4857 | MULTILIBS = "multilib:lib32" | ||
4858 | DEFAULTTUNE_virtclass-multilib-lib32 = "x86" | ||
4859 | IMAGE_INSTALL_append = "lib32-glib-2.0" | ||
4860 | |||
4861 | This example enables an additional library named | ||
4862 | ``lib32`` alongside the normal target packages. When combining these | ||
4863 | "lib32" alternatives, the example uses "x86" for tuning. For information | ||
4864 | on this particular tuning, see | ||
4865 | ``meta/conf/machine/include/ia32/arch-ia32.inc``. | ||
4866 | |||
4867 | The example then includes ``lib32-glib-2.0`` in all the images, which | ||
4868 | illustrates one method of including a multiple library dependency. You | ||
4869 | can use a normal image build to include this dependency, for example: | ||
4870 | :: | ||
4871 | |||
4872 | $ bitbake core-image-sato | ||
4873 | |||
4874 | You can also build Multilib packages | ||
4875 | specifically with a command like this: | ||
4876 | :: | ||
4877 | |||
4878 | $ bitbake lib32-glib-2.0 | ||
4879 | |||
4880 | Additional Implementation Details | ||
4881 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4882 | |||
4883 | Generic implementation details as well as details that are specific to | ||
4884 | package management systems exist. Following are implementation details | ||
4885 | that exist regardless of the package management system: | ||
4886 | |||
4887 | - The typical convention used for the class extension code as used by | ||
4888 | Multilib assumes that all package names specified in | ||
4889 | :term:`PACKAGES` that contain | ||
4890 | ``${PN}`` have ``${PN}`` at the start of the name. When that | ||
4891 | convention is not followed and ``${PN}`` appears at the middle or the | ||
4892 | end of a name, problems occur. | ||
4893 | |||
4894 | - The :term:`TARGET_VENDOR` | ||
4895 | value under Multilib will be extended to "-vendormlmultilib" (e.g. | ||
4896 | "-pokymllib32" for a "lib32" Multilib with Poky). The reason for this | ||
4897 | slightly unwieldy contraction is that any "-" characters in the | ||
4898 | vendor string presently break Autoconf's ``config.sub``, and other | ||
4899 | separators are problematic for different reasons. | ||
4900 | |||
4901 | For the RPM Package Management System, the following implementation | ||
4902 | details exist: | ||
4903 | |||
4904 | - A unique architecture is defined for the Multilib packages, along | ||
4905 | with creating a unique deploy folder under ``tmp/deploy/rpm`` in the | ||
4906 | :term:`Build Directory`. For | ||
4907 | example, consider ``lib32`` in a ``qemux86-64`` image. The possible | ||
4908 | architectures in the system are "all", "qemux86_64", | ||
4909 | "lib32_qemux86_64", and "lib32_x86". | ||
4910 | |||
4911 | - The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM | ||
4912 | packaging. The naming for a normal RPM package and a Multilib RPM | ||
4913 | package in a ``qemux86-64`` system resolves to something similar to | ||
4914 | ``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``, | ||
4915 | respectively. | ||
4916 | |||
4917 | - When installing a Multilib image, the RPM backend first installs the | ||
4918 | base image and then installs the Multilib libraries. | ||
4919 | |||
4920 | - The build system relies on RPM to resolve the identical files in the | ||
4921 | two (or more) Multilib packages. | ||
4922 | |||
4923 | For the IPK Package Management System, the following implementation | ||
4924 | details exist: | ||
4925 | |||
4926 | - The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK | ||
4927 | packaging. The naming for a normal RPM package and a Multilib IPK | ||
4928 | package in a ``qemux86-64`` system resolves to something like | ||
4929 | ``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw_x86.ipk``, | ||
4930 | respectively. | ||
4931 | |||
4932 | - The IPK deploy folder is not modified with ``${MLPREFIX}`` because | ||
4933 | packages with and without the Multilib feature can exist in the same | ||
4934 | folder due to the ``${PN}`` differences. | ||
4935 | |||
4936 | - IPK defines a sanity check for Multilib installation using certain | ||
4937 | rules for file comparison, overridden, etc. | ||
4938 | |||
4939 | Installing Multiple Versions of the Same Library | ||
4940 | ------------------------------------------------ | ||
4941 | |||
4942 | Situations can exist where you need to install and use multiple versions | ||
4943 | of the same library on the same system at the same time. These | ||
4944 | situations almost always exist when a library API changes and you have | ||
4945 | multiple pieces of software that depend on the separate versions of the | ||
4946 | library. To accommodate these situations, you can install multiple | ||
4947 | versions of the same library in parallel on the same system. | ||
4948 | |||
4949 | The process is straightforward as long as the libraries use proper | ||
4950 | versioning. With properly versioned libraries, all you need to do to | ||
4951 | individually specify the libraries is create separate, appropriately | ||
4952 | named recipes where the :term:`PN` part of | ||
4953 | the name includes a portion that differentiates each library version | ||
4954 | (e.g. the major part of the version number). Thus, instead of having a | ||
4955 | single recipe that loads one version of a library (e.g. ``clutter``), | ||
4956 | you provide multiple recipes that result in different versions of the | ||
4957 | libraries you want. As an example, the following two recipes would allow | ||
4958 | the two separate versions of the ``clutter`` library to co-exist on the | ||
4959 | same system: | ||
4960 | |||
4961 | .. code-block:: none | ||
4962 | |||
4963 | clutter-1.6_1.6.20.bb | ||
4964 | clutter-1.8_1.8.4.bb | ||
4965 | |||
4966 | Additionally, if | ||
4967 | you have other recipes that depend on a given library, you need to use | ||
4968 | the :term:`DEPENDS` variable to | ||
4969 | create the dependency. Continuing with the same example, if you want to | ||
4970 | have a recipe depend on the 1.8 version of the ``clutter`` library, use | ||
4971 | the following in your recipe: | ||
4972 | :: | ||
4973 | |||
4974 | DEPENDS = "clutter-1.8" | ||
4975 | |||
4976 | Using x32 psABI | ||
4977 | =============== | ||
4978 | |||
4979 | x32 processor-specific Application Binary Interface (`x32 | ||
4980 | psABI <https://software.intel.com/en-us/node/628948>`__) is a native | ||
4981 | 32-bit processor-specific ABI for Intel 64 (x86-64) architectures. An | ||
4982 | ABI defines the calling conventions between functions in a processing | ||
4983 | environment. The interface determines what registers are used and what | ||
4984 | the sizes are for various C data types. | ||
4985 | |||
4986 | Some processing environments prefer using 32-bit applications even when | ||
4987 | running on Intel 64-bit platforms. Consider the i386 psABI, which is a | ||
4988 | very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not | ||
4989 | provide efficient use and access of the Intel 64-bit processor | ||
4990 | resources, leaving the system underutilized. Now consider the x86_64 | ||
4991 | psABI. This ABI is newer and uses 64-bits for data sizes and program | ||
4992 | pointers. The extra bits increase the footprint size of the programs, | ||
4993 | libraries, and also increases the memory and file system size | ||
4994 | requirements. Executing under the x32 psABI enables user programs to | ||
4995 | utilize CPU and system resources more efficiently while keeping the | ||
4996 | memory footprint of the applications low. Extra bits are used for | ||
4997 | registers but not for addressing mechanisms. | ||
4998 | |||
4999 | The Yocto Project supports the final specifications of x32 psABI as | ||
5000 | follows: | ||
5001 | |||
5002 | - You can create packages and images in x32 psABI format on x86_64 | ||
5003 | architecture targets. | ||
5004 | |||
5005 | - You can successfully build recipes with the x32 toolchain. | ||
5006 | |||
5007 | - You can create and boot ``core-image-minimal`` and | ||
5008 | ``core-image-sato`` images. | ||
5009 | |||
5010 | - RPM Package Manager (RPM) support exists for x32 binaries. | ||
5011 | |||
5012 | - Support for large images exists. | ||
5013 | |||
5014 | To use the x32 psABI, you need to edit your ``conf/local.conf`` | ||
5015 | configuration file as follows: | ||
5016 | :: | ||
5017 | |||
5018 | MACHINE = "qemux86-64" | ||
5019 | DEFAULTTUNE = "x86-64-x32" | ||
5020 | baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE') \ | ||
5021 | or 'INVALID')) or 'lib'}" | ||
5022 | |||
5023 | Once you have set | ||
5024 | up your configuration file, use BitBake to build an image that supports | ||
5025 | the x32 psABI. Here is an example: | ||
5026 | :: | ||
5027 | |||
5028 | $ bitbake core-image-sato | ||
5029 | |||
5030 | Enabling GObject Introspection Support | ||
5031 | ====================================== | ||
5032 | |||
5033 | `GObject | ||
5034 | introspection <https://wiki.gnome.org/Projects/GObjectIntrospection>`__ | ||
5035 | is the standard mechanism for accessing GObject-based software from | ||
5036 | runtime environments. GObject is a feature of the GLib library that | ||
5037 | provides an object framework for the GNOME desktop and related software. | ||
5038 | GObject Introspection adds information to GObject that allows objects | ||
5039 | created within it to be represented across different programming | ||
5040 | languages. If you want to construct GStreamer pipelines using Python, or | ||
5041 | control UPnP infrastructure using Javascript and GUPnP, GObject | ||
5042 | introspection is the only way to do it. | ||
5043 | |||
5044 | This section describes the Yocto Project support for generating and | ||
5045 | packaging GObject introspection data. GObject introspection data is a | ||
5046 | description of the API provided by libraries built on top of GLib | ||
5047 | framework, and, in particular, that framework's GObject mechanism. | ||
5048 | GObject Introspection Repository (GIR) files go to ``-dev`` packages, | ||
5049 | ``typelib`` files go to main packages as they are packaged together with | ||
5050 | libraries that are introspected. | ||
5051 | |||
5052 | The data is generated when building such a library, by linking the | ||
5053 | library with a small executable binary that asks the library to describe | ||
5054 | itself, and then executing the binary and processing its output. | ||
5055 | |||
5056 | Generating this data in a cross-compilation environment is difficult | ||
5057 | because the library is produced for the target architecture, but its | ||
5058 | code needs to be executed on the build host. This problem is solved with | ||
5059 | the OpenEmbedded build system by running the code through QEMU, which | ||
5060 | allows precisely that. Unfortunately, QEMU does not always work | ||
5061 | perfectly as mentioned in the "`Known Issues <#known-issues>`__" | ||
5062 | section. | ||
5063 | |||
5064 | Enabling the Generation of Introspection Data | ||
5065 | --------------------------------------------- | ||
5066 | |||
5067 | Enabling the generation of introspection data (GIR files) in your | ||
5068 | library package involves the following: | ||
5069 | |||
5070 | 1. Inherit the | ||
5071 | :ref:`gobject-introspection <ref-classes-gobject-introspection>` | ||
5072 | class. | ||
5073 | |||
5074 | 2. Make sure introspection is not disabled anywhere in the recipe or | ||
5075 | from anything the recipe includes. Also, make sure that | ||
5076 | "gobject-introspection-data" is not in | ||
5077 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED` | ||
5078 | and that "qemu-usermode" is not in | ||
5079 | :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`. | ||
5080 | If either of these conditions exist, nothing will happen. | ||
5081 | |||
5082 | 3. Try to build the recipe. If you encounter build errors that look like | ||
5083 | something is unable to find ``.so`` libraries, check where these | ||
5084 | libraries are located in the source tree and add the following to the | ||
5085 | recipe: | ||
5086 | :: | ||
5087 | |||
5088 | GIR_EXTRA_LIBS_PATH = "${B}/something/.libs" | ||
5089 | |||
5090 | .. note:: | ||
5091 | |||
5092 | See recipes in the ``oe-core`` repository that use that | ||
5093 | ``GIR_EXTRA_LIBS_PATH`` variable as an example. | ||
5094 | |||
5095 | 4. Look for any other errors, which probably mean that introspection | ||
5096 | support in a package is not entirely standard, and thus breaks down | ||
5097 | in a cross-compilation environment. For such cases, custom-made fixes | ||
5098 | are needed. A good place to ask and receive help in these cases is | ||
5099 | the :ref:`Yocto Project mailing | ||
5100 | lists <resources-mailinglist>`. | ||
5101 | |||
5102 | .. note:: | ||
5103 | |||
5104 | Using a library that no longer builds against the latest Yocto | ||
5105 | Project release and prints introspection related errors is a good | ||
5106 | candidate for the previous procedure. | ||
5107 | |||
5108 | Disabling the Generation of Introspection Data | ||
5109 | ---------------------------------------------- | ||
5110 | |||
5111 | You might find that you do not want to generate introspection data. Or, | ||
5112 | perhaps QEMU does not work on your build host and target architecture | ||
5113 | combination. If so, you can use either of the following methods to | ||
5114 | disable GIR file generations: | ||
5115 | |||
5116 | - Add the following to your distro configuration: | ||
5117 | :: | ||
5118 | |||
5119 | DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data" | ||
5120 | |||
5121 | Adding this statement disables generating introspection data using | ||
5122 | QEMU but will still enable building introspection tools and libraries | ||
5123 | (i.e. building them does not require the use of QEMU). | ||
5124 | |||
5125 | - Add the following to your machine configuration: | ||
5126 | :: | ||
5127 | |||
5128 | MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode" | ||
5129 | |||
5130 | Adding this statement disables the use of QEMU when building packages for your | ||
5131 | machine. Currently, this feature is used only by introspection | ||
5132 | recipes and has the same effect as the previously described option. | ||
5133 | |||
5134 | .. note:: | ||
5135 | |||
5136 | Future releases of the Yocto Project might have other features | ||
5137 | affected by this option. | ||
5138 | |||
5139 | If you disable introspection data, you can still obtain it through other | ||
5140 | means such as copying the data from a suitable sysroot, or by generating | ||
5141 | it on the target hardware. The OpenEmbedded build system does not | ||
5142 | currently provide specific support for these techniques. | ||
5143 | |||
5144 | Testing that Introspection Works in an Image | ||
5145 | -------------------------------------------- | ||
5146 | |||
5147 | Use the following procedure to test if generating introspection data is | ||
5148 | working in an image: | ||
5149 | |||
5150 | 1. Make sure that "gobject-introspection-data" is not in | ||
5151 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED` | ||
5152 | and that "qemu-usermode" is not in | ||
5153 | :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`. | ||
5154 | |||
5155 | 2. Build ``core-image-sato``. | ||
5156 | |||
5157 | 3. Launch a Terminal and then start Python in the terminal. | ||
5158 | |||
5159 | 4. Enter the following in the terminal: | ||
5160 | :: | ||
5161 | |||
5162 | >>> from gi.repository import GLib | ||
5163 | >>> GLib.get_host_name() | ||
5164 | |||
5165 | 5. For something a little more advanced, enter the following see: | ||
5166 | https://python-gtk-3-tutorial.readthedocs.io/en/latest/introduction.html | ||
5167 | |||
5168 | Known Issues | ||
5169 | ------------ | ||
5170 | |||
5171 | The following know issues exist for GObject Introspection Support: | ||
5172 | |||
5173 | - ``qemu-ppc64`` immediately crashes. Consequently, you cannot build | ||
5174 | introspection data on that architecture. | ||
5175 | |||
5176 | - x32 is not supported by QEMU. Consequently, introspection data is | ||
5177 | disabled. | ||
5178 | |||
5179 | - musl causes transient GLib binaries to crash on assertion failures. | ||
5180 | Consequently, generating introspection data is disabled. | ||
5181 | |||
5182 | - Because QEMU is not able to run the binaries correctly, introspection | ||
5183 | is disabled for some specific packages under specific architectures | ||
5184 | (e.g. ``gcr``, ``libsecret``, and ``webkit``). | ||
5185 | |||
5186 | - QEMU usermode might not work properly when running 64-bit binaries | ||
5187 | under 32-bit host machines. In particular, "qemumips64" is known to | ||
5188 | not work under i686. | ||
5189 | |||
5190 | Optionally Using an External Toolchain | ||
5191 | ====================================== | ||
5192 | |||
5193 | You might want to use an external toolchain as part of your development. | ||
5194 | If this is the case, the fundamental steps you need to accomplish are as | ||
5195 | follows: | ||
5196 | |||
5197 | - Understand where the installed toolchain resides. For cases where you | ||
5198 | need to build the external toolchain, you would need to take separate | ||
5199 | steps to build and install the toolchain. | ||
5200 | |||
5201 | - Make sure you add the layer that contains the toolchain to your | ||
5202 | ``bblayers.conf`` file through the | ||
5203 | :term:`BBLAYERS` variable. | ||
5204 | |||
5205 | - Set the ``EXTERNAL_TOOLCHAIN`` variable in your ``local.conf`` file | ||
5206 | to the location in which you installed the toolchain. | ||
5207 | |||
5208 | A good example of an external toolchain used with the Yocto Project is | ||
5209 | Mentor Graphics Sourcery G++ Toolchain. You can see information on how | ||
5210 | to use that particular layer in the ``README`` file at | ||
5211 | https://github.com/MentorEmbedded/meta-sourcery/. You can find | ||
5212 | further information by reading about the | ||
5213 | :term:`TCMODE` variable in the Yocto | ||
5214 | Project Reference Manual's variable glossary. | ||
5215 | |||
5216 | Creating Partitioned Images Using Wic | ||
5217 | ===================================== | ||
5218 | |||
5219 | Creating an image for a particular hardware target using the | ||
5220 | OpenEmbedded build system does not necessarily mean you can boot that | ||
5221 | image as is on your device. Physical devices accept and boot images in | ||
5222 | various ways depending on the specifics of the device. Usually, | ||
5223 | information about the hardware can tell you what image format the device | ||
5224 | requires. Should your device require multiple partitions on an SD card, | ||
5225 | flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to | ||
5226 | create the properly partitioned image. | ||
5227 | |||
5228 | The ``wic`` command generates partitioned images from existing | ||
5229 | OpenEmbedded build artifacts. Image generation is driven by partitioning | ||
5230 | commands contained in an Openembedded kickstart file (``.wks``) | ||
5231 | specified either directly on the command line or as one of a selection | ||
5232 | of canned kickstart files as shown with the ``wic list images`` command | ||
5233 | in the "`Using an Existing Kickstart | ||
5234 | File <#using-a-provided-kickstart-file>`__" section. When you apply the | ||
5235 | command to a given set of build artifacts, the result is an image or set | ||
5236 | of images that can be directly written onto media and used on a | ||
5237 | particular system. | ||
5238 | |||
5239 | .. note:: | ||
5240 | |||
5241 | For a kickstart file reference, see the | ||
5242 | ":ref:`ref-manual/kickstart:openembedded kickstart (\`\`.wks\`\`) reference`" | ||
5243 | Chapter in the Yocto Project Reference Manual. | ||
5244 | |||
5245 | The ``wic`` command and the infrastructure it is based on is by | ||
5246 | definition incomplete. The purpose of the command is to allow the | ||
5247 | generation of customized images, and as such, was designed to be | ||
5248 | completely extensible through a plugin interface. See the "`Using the | ||
5249 | Wic PlugIn Interface <#wic-using-the-wic-plugin-interface>`__" section | ||
5250 | for information on these plugins. | ||
5251 | |||
5252 | This section provides some background information on Wic, describes what | ||
5253 | you need to have in place to run the tool, provides instruction on how | ||
5254 | to use the Wic utility, provides information on using the Wic plugins | ||
5255 | interface, and provides several examples that show how to use Wic. | ||
5256 | |||
5257 | Background | ||
5258 | ---------- | ||
5259 | |||
5260 | This section provides some background on the Wic utility. While none of | ||
5261 | this information is required to use Wic, you might find it interesting. | ||
5262 | |||
5263 | - The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The | ||
5264 | "oe" diphthong in "oeic" was promoted to the letter "w", because | ||
5265 | "oeic" is both difficult to remember and to pronounce. | ||
5266 | |||
5267 | - Wic is loosely based on the Meego Image Creator (``mic``) framework. | ||
5268 | The Wic implementation has been heavily modified to make direct use | ||
5269 | of OpenEmbedded build artifacts instead of package installation and | ||
5270 | configuration, which are already incorporated within the OpenEmbedded | ||
5271 | artifacts. | ||
5272 | |||
5273 | - Wic is a completely independent standalone utility that initially | ||
5274 | provides easier-to-use and more flexible replacements for an existing | ||
5275 | functionality in OE-Core's | ||
5276 | :ref:`image-live <ref-classes-image-live>` | ||
5277 | class. The difference between Wic and those examples is that with Wic | ||
5278 | the functionality of those scripts is implemented by a | ||
5279 | general-purpose partitioning language, which is based on Redhat | ||
5280 | kickstart syntax. | ||
5281 | |||
5282 | Requirements | ||
5283 | ------------ | ||
5284 | |||
5285 | In order to use the Wic utility with the OpenEmbedded Build system, your | ||
5286 | system needs to meet the following requirements: | ||
5287 | |||
5288 | - The Linux distribution on your development host must support the | ||
5289 | Yocto Project. See the ":ref:`detailed-supported-distros`" | ||
5290 | section in the Yocto Project Reference Manual for the list of | ||
5291 | distributions that support the Yocto Project. | ||
5292 | |||
5293 | - The standard system utilities, such as ``cp``, must be installed on | ||
5294 | your development host system. | ||
5295 | |||
5296 | - You must have sourced the build environment setup script (i.e. | ||
5297 | :ref:`structure-core-script`) found in the | ||
5298 | :term:`Build Directory`. | ||
5299 | |||
5300 | - You need to have the build artifacts already available, which | ||
5301 | typically means that you must have already created an image using the | ||
5302 | Openembedded build system (e.g. ``core-image-minimal``). While it | ||
5303 | might seem redundant to generate an image in order to create an image | ||
5304 | using Wic, the current version of Wic requires the artifacts in the | ||
5305 | form generated by the OpenEmbedded build system. | ||
5306 | |||
5307 | - You must build several native tools, which are built to run on the | ||
5308 | build system: | ||
5309 | :: | ||
5310 | |||
5311 | $ bitbake parted-native dosfstools-native mtools-native | ||
5312 | |||
5313 | - Include "wic" as part of the | ||
5314 | :term:`IMAGE_FSTYPES` | ||
5315 | variable. | ||
5316 | |||
5317 | - Include the name of the :ref:`wic kickstart file <openembedded-kickstart-wks-reference>` | ||
5318 | as part of the :term:`WKS_FILE` variable | ||
5319 | |||
5320 | Getting Help | ||
5321 | ------------ | ||
5322 | |||
5323 | You can get general help for the ``wic`` command by entering the ``wic`` | ||
5324 | command by itself or by entering the command with a help argument as | ||
5325 | follows: | ||
5326 | :: | ||
5327 | |||
5328 | $ wic -h | ||
5329 | $ wic --help | ||
5330 | $ wic help | ||
5331 | |||
5332 | Currently, Wic supports seven commands: ``cp``, ``create``, ``help``, | ||
5333 | ``list``, ``ls``, ``rm``, and ``write``. You can get help for all these | ||
5334 | commands except "help" by using the following form: | ||
5335 | :: | ||
5336 | |||
5337 | $ wic help command | ||
5338 | |||
5339 | For example, the following command returns help for the ``write`` | ||
5340 | command: | ||
5341 | :: | ||
5342 | |||
5343 | $ wic help write | ||
5344 | |||
5345 | Wic supports help for three topics: ``overview``, ``plugins``, and | ||
5346 | ``kickstart``. You can get help for any topic using the following form: | ||
5347 | :: | ||
5348 | |||
5349 | $ wic help topic | ||
5350 | |||
5351 | For example, the following returns overview help for Wic: | ||
5352 | :: | ||
5353 | |||
5354 | $ wic help overview | ||
5355 | |||
5356 | One additional level of help exists for Wic. You can get help on | ||
5357 | individual images through the ``list`` command. You can use the ``list`` | ||
5358 | command to return the available Wic images as follows: | ||
5359 | :: | ||
5360 | |||
5361 | $ wic list images | ||
5362 | genericx86 Create an EFI disk image for genericx86* | ||
5363 | beaglebone-yocto Create SD card image for Beaglebone | ||
5364 | edgerouter Create SD card image for Edgerouter | ||
5365 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | ||
5366 | directdisk-gpt Create a 'pcbios' direct disk image | ||
5367 | mkefidisk Create an EFI disk image | ||
5368 | directdisk Create a 'pcbios' direct disk image | ||
5369 | systemd-bootdisk Create an EFI disk image with systemd-boot | ||
5370 | mkhybridiso Create a hybrid ISO image | ||
5371 | sdimage-bootpart Create SD card image with a boot partition | ||
5372 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | ||
5373 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | ||
5374 | |||
5375 | Once you know the list of available | ||
5376 | Wic images, you can use ``help`` with the command to get help on a | ||
5377 | particular image. For example, the following command returns help on the | ||
5378 | "beaglebone-yocto" image: | ||
5379 | :: | ||
5380 | |||
5381 | $ wic list beaglebone-yocto help | ||
5382 | |||
5383 | Creates a partitioned SD card image for Beaglebone. | ||
5384 | Boot files are located in the first vfat partition. | ||
5385 | |||
5386 | Operational Modes | ||
5387 | ----------------- | ||
5388 | |||
5389 | You can use Wic in two different modes, depending on how much control | ||
5390 | you need for specifying the Openembedded build artifacts that are used | ||
5391 | for creating the image: Raw and Cooked: | ||
5392 | |||
5393 | - *Raw Mode:* You explicitly specify build artifacts through Wic | ||
5394 | command-line arguments. | ||
5395 | |||
5396 | - *Cooked Mode:* The current | ||
5397 | :term:`MACHINE` setting and image | ||
5398 | name are used to automatically locate and provide the build | ||
5399 | artifacts. You just supply a kickstart file and the name of the image | ||
5400 | from which to use artifacts. | ||
5401 | |||
5402 | Regardless of the mode you use, you need to have the build artifacts | ||
5403 | ready and available. | ||
5404 | |||
5405 | Raw Mode | ||
5406 | ~~~~~~~~ | ||
5407 | |||
5408 | Running Wic in raw mode allows you to specify all the partitions through | ||
5409 | the ``wic`` command line. The primary use for raw mode is if you have | ||
5410 | built your kernel outside of the Yocto Project | ||
5411 | :term:`Build Directory`. In other words, you | ||
5412 | can point to arbitrary kernel, root filesystem locations, and so forth. | ||
5413 | Contrast this behavior with cooked mode where Wic looks in the Build | ||
5414 | Directory (e.g. ``tmp/deploy/images/``\ machine). | ||
5415 | |||
5416 | The general form of the ``wic`` command in raw mode is: | ||
5417 | :: | ||
5418 | |||
5419 | $ wic create wks_file options ... | ||
5420 | |||
5421 | Where: | ||
5422 | |||
5423 | wks_file: | ||
5424 | An OpenEmbedded kickstart file. You can provide | ||
5425 | your own custom file or use a file from a set of | ||
5426 | existing files as described by further options. | ||
5427 | |||
5428 | optional arguments: | ||
5429 | -h, --help show this help message and exit | ||
5430 | -o OUTDIR, --outdir OUTDIR | ||
5431 | name of directory to create image in | ||
5432 | -e IMAGE_NAME, --image-name IMAGE_NAME | ||
5433 | name of the image to use the artifacts from e.g. core- | ||
5434 | image-sato | ||
5435 | -r ROOTFS_DIR, --rootfs-dir ROOTFS_DIR | ||
5436 | path to the /rootfs dir to use as the .wks rootfs | ||
5437 | source | ||
5438 | -b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR | ||
5439 | path to the dir containing the boot artifacts (e.g. | ||
5440 | /EFI or /syslinux dirs) to use as the .wks bootimg | ||
5441 | source | ||
5442 | -k KERNEL_DIR, --kernel-dir KERNEL_DIR | ||
5443 | path to the dir containing the kernel to use in the | ||
5444 | .wks bootimg | ||
5445 | -n NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT | ||
5446 | path to the native sysroot containing the tools to use | ||
5447 | to build the image | ||
5448 | -s, --skip-build-check | ||
5449 | skip the build check | ||
5450 | -f, --build-rootfs build rootfs | ||
5451 | -c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz} | ||
5452 | compress image with specified compressor | ||
5453 | -m, --bmap generate .bmap | ||
5454 | --no-fstab-update Do not change fstab file. | ||
5455 | -v VARS_DIR, --vars VARS_DIR | ||
5456 | directory with <image>.env files that store bitbake | ||
5457 | variables | ||
5458 | -D, --debug output debug information | ||
5459 | |||
5460 | .. note:: | ||
5461 | |||
5462 | You do not need root privileges to run Wic. In fact, you should not | ||
5463 | run as root when using the utility. | ||
5464 | |||
5465 | Cooked Mode | ||
5466 | ~~~~~~~~~~~ | ||
5467 | |||
5468 | Running Wic in cooked mode leverages off artifacts in the Build | ||
5469 | Directory. In other words, you do not have to specify kernel or root | ||
5470 | filesystem locations as part of the command. All you need to provide is | ||
5471 | a kickstart file and the name of the image from which to use artifacts | ||
5472 | by using the "-e" option. Wic looks in the Build Directory (e.g. | ||
5473 | ``tmp/deploy/images/``\ machine) for artifacts. | ||
5474 | |||
5475 | The general form of the ``wic`` command using Cooked Mode is as follows: | ||
5476 | :: | ||
5477 | |||
5478 | $ wic create wks_file -e IMAGE_NAME | ||
5479 | |||
5480 | Where: | ||
5481 | |||
5482 | wks_file: | ||
5483 | An OpenEmbedded kickstart file. You can provide | ||
5484 | your own custom file or use a file from a set of | ||
5485 | existing files provided with the Yocto Project | ||
5486 | release. | ||
5487 | |||
5488 | required argument: | ||
5489 | -e IMAGE_NAME, --image-name IMAGE_NAME | ||
5490 | name of the image to use the artifacts from e.g. core- | ||
5491 | image-sato | ||
5492 | |||
5493 | Using an Existing Kickstart File | ||
5494 | -------------------------------- | ||
5495 | |||
5496 | If you do not want to create your own kickstart file, you can use an | ||
5497 | existing file provided by the Wic installation. As shipped, kickstart | ||
5498 | files can be found in the :ref:`overview-manual/development-environment:yocto project source repositories` in the | ||
5499 | following two locations: | ||
5500 | :: | ||
5501 | |||
5502 | poky/meta-yocto-bsp/wic | ||
5503 | poky/scripts/lib/wic/canned-wks | ||
5504 | |||
5505 | Use the following command to list the available kickstart files: | ||
5506 | :: | ||
5507 | |||
5508 | $ wic list images | ||
5509 | genericx86 Create an EFI disk image for genericx86* | ||
5510 | beaglebone-yocto Create SD card image for Beaglebone | ||
5511 | edgerouter Create SD card image for Edgerouter | ||
5512 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | ||
5513 | directdisk-gpt Create a 'pcbios' direct disk image | ||
5514 | mkefidisk Create an EFI disk image | ||
5515 | directdisk Create a 'pcbios' direct disk image | ||
5516 | systemd-bootdisk Create an EFI disk image with systemd-boot | ||
5517 | mkhybridiso Create a hybrid ISO image | ||
5518 | sdimage-bootpart Create SD card image with a boot partition | ||
5519 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | ||
5520 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | ||
5521 | |||
5522 | When you use an existing file, you | ||
5523 | do not have to use the ``.wks`` extension. Here is an example in Raw | ||
5524 | Mode that uses the ``directdisk`` file: | ||
5525 | :: | ||
5526 | |||
5527 | $ wic create directdisk -r rootfs_dir -b bootimg_dir \ | ||
5528 | -k kernel_dir -n native_sysroot | ||
5529 | |||
5530 | Here are the actual partition language commands used in the | ||
5531 | ``genericx86.wks`` file to generate an image: | ||
5532 | :: | ||
5533 | |||
5534 | # short-description: Create an EFI disk image for genericx86* | ||
5535 | # long-description: Creates a partitioned EFI disk image for genericx86* machines | ||
5536 | part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 | ||
5537 | part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid | ||
5538 | part swap --ondisk sda --size 44 --label swap1 --fstype=swap | ||
5539 | |||
5540 | bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0" | ||
5541 | |||
5542 | Using the Wic Plugin Interface | ||
5543 | ------------------------------ | ||
5544 | |||
5545 | You can extend and specialize Wic functionality by using Wic plugins. | ||
5546 | This section explains the Wic plugin interface. | ||
5547 | |||
5548 | .. note:: | ||
5549 | |||
5550 | Wic plugins consist of "source" and "imager" plugins. Imager plugins | ||
5551 | are beyond the scope of this section. | ||
5552 | |||
5553 | Source plugins provide a mechanism to customize partition content during | ||
5554 | the Wic image generation process. You can use source plugins to map | ||
5555 | values that you specify using ``--source`` commands in kickstart files | ||
5556 | (i.e. ``*.wks``) to a plugin implementation used to populate a given | ||
5557 | partition. | ||
5558 | |||
5559 | .. note:: | ||
5560 | |||
5561 | If you use plugins that have build-time dependencies (e.g. native | ||
5562 | tools, bootloaders, and so forth) when building a Wic image, you need | ||
5563 | to specify those dependencies using the :term:`WKS_FILE_DEPENDS` | ||
5564 | variable. | ||
5565 | |||
5566 | Source plugins are subclasses defined in plugin files. As shipped, the | ||
5567 | Yocto Project provides several plugin files. You can see the source | ||
5568 | plugin files that ship with the Yocto Project | ||
5569 | :yocto_git:`here </poky/tree/scripts/lib/wic/plugins/source>`. | ||
5570 | Each of these plugin files contains source plugins that are designed to | ||
5571 | populate a specific Wic image partition. | ||
5572 | |||
5573 | Source plugins are subclasses of the ``SourcePlugin`` class, which is | ||
5574 | defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example, | ||
5575 | the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py`` | ||
5576 | file is a subclass of the ``SourcePlugin`` class, which is found in the | ||
5577 | ``pluginbase.py`` file. | ||
5578 | |||
5579 | You can also implement source plugins in a layer outside of the Source | ||
5580 | Repositories (external layer). To do so, be sure that your plugin files | ||
5581 | are located in a directory whose path is | ||
5582 | ``scripts/lib/wic/plugins/source/`` within your external layer. When the | ||
5583 | plugin files are located there, the source plugins they contain are made | ||
5584 | available to Wic. | ||
5585 | |||
5586 | When the Wic implementation needs to invoke a partition-specific | ||
5587 | implementation, it looks for the plugin with the same name as the | ||
5588 | ``--source`` parameter used in the kickstart file given to that | ||
5589 | partition. For example, if the partition is set up using the following | ||
5590 | command in a kickstart file: | ||
5591 | :: | ||
5592 | |||
5593 | part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 | ||
5594 | |||
5595 | The methods defined as class | ||
5596 | members of the matching source plugin (i.e. ``bootimg-pcbios``) in the | ||
5597 | ``bootimg-pcbios.py`` plugin file are used. | ||
5598 | |||
5599 | To be more concrete, here is the corresponding plugin definition from | ||
5600 | the ``bootimg-pcbios.py`` file for the previous command along with an | ||
5601 | example method called by the Wic implementation when it needs to prepare | ||
5602 | a partition using an implementation-specific function: | ||
5603 | :: | ||
5604 | |||
5605 | . | ||
5606 | . | ||
5607 | . | ||
5608 | class BootimgPcbiosPlugin(SourcePlugin): | ||
5609 | """ | ||
5610 | Create MBR boot partition and install syslinux on it. | ||
5611 | """ | ||
5612 | |||
5613 | name = 'bootimg-pcbios' | ||
5614 | . | ||
5615 | . | ||
5616 | . | ||
5617 | @classmethod | ||
5618 | def do_prepare_partition(cls, part, source_params, creator, cr_workdir, | ||
5619 | oe_builddir, bootimg_dir, kernel_dir, | ||
5620 | rootfs_dir, native_sysroot): | ||
5621 | """ | ||
5622 | Called to do the actual content population for a partition i.e. it | ||
5623 | 'prepares' the partition to be incorporated into the image. | ||
5624 | In this case, prepare content for legacy bios boot partition. | ||
5625 | """ | ||
5626 | . | ||
5627 | . | ||
5628 | . | ||
5629 | |||
5630 | If a | ||
5631 | subclass (plugin) itself does not implement a particular function, Wic | ||
5632 | locates and uses the default version in the superclass. It is for this | ||
5633 | reason that all source plugins are derived from the ``SourcePlugin`` | ||
5634 | class. | ||
5635 | |||
5636 | The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines | ||
5637 | a set of methods that source plugins can implement or override. Any | ||
5638 | plugins (subclass of ``SourcePlugin``) that do not implement a | ||
5639 | particular method inherit the implementation of the method from the | ||
5640 | ``SourcePlugin`` class. For more information, see the ``SourcePlugin`` | ||
5641 | class in the ``pluginbase.py`` file for details: | ||
5642 | |||
5643 | The following list describes the methods implemented in the | ||
5644 | ``SourcePlugin`` class: | ||
5645 | |||
5646 | - ``do_prepare_partition()``: Called to populate a partition with | ||
5647 | actual content. In other words, the method prepares the final | ||
5648 | partition image that is incorporated into the disk image. | ||
5649 | |||
5650 | - ``do_configure_partition()``: Called before | ||
5651 | ``do_prepare_partition()`` to create custom configuration files for a | ||
5652 | partition (e.g. syslinux or grub configuration files). | ||
5653 | |||
5654 | - ``do_install_disk()``: Called after all partitions have been | ||
5655 | prepared and assembled into a disk image. This method provides a hook | ||
5656 | to allow finalization of a disk image (e.g. writing an MBR). | ||
5657 | |||
5658 | - ``do_stage_partition()``: Special content-staging hook called | ||
5659 | before ``do_prepare_partition()``. This method is normally empty. | ||
5660 | |||
5661 | Typically, a partition just uses the passed-in parameters (e.g. the | ||
5662 | unmodified value of ``bootimg_dir``). However, in some cases, things | ||
5663 | might need to be more tailored. As an example, certain files might | ||
5664 | additionally need to be taken from ``bootimg_dir + /boot``. This hook | ||
5665 | allows those files to be staged in a customized fashion. | ||
5666 | |||
5667 | .. note:: | ||
5668 | |||
5669 | ``get_bitbake_var()`` allows you to access non-standard variables that | ||
5670 | you might want to use for this behavior. | ||
5671 | |||
5672 | You can extend the source plugin mechanism. To add more hooks, create | ||
5673 | more source plugin methods within ``SourcePlugin`` and the corresponding | ||
5674 | derived subclasses. The code that calls the plugin methods uses the | ||
5675 | ``plugin.get_source_plugin_methods()`` function to find the method or | ||
5676 | methods needed by the call. Retrieval of those methods is accomplished | ||
5677 | by filling up a dict with keys that contain the method names of | ||
5678 | interest. On success, these will be filled in with the actual methods. | ||
5679 | See the Wic implementation for examples and details. | ||
5680 | |||
5681 | Wic Examples | ||
5682 | ------------ | ||
5683 | |||
5684 | This section provides several examples that show how to use the Wic | ||
5685 | utility. All the examples assume the list of requirements in the | ||
5686 | "`Requirements <#wic-requirements>`__" section have been met. The | ||
5687 | examples assume the previously generated image is | ||
5688 | ``core-image-minimal``. | ||
5689 | |||
5690 | Generate an Image using an Existing Kickstart File | ||
5691 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5692 | |||
5693 | This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart | ||
5694 | file: | ||
5695 | :: | ||
5696 | |||
5697 | $ wic create mkefidisk -e core-image-minimal | ||
5698 | INFO: Building wic-tools... | ||
5699 | . | ||
5700 | . | ||
5701 | . | ||
5702 | INFO: The new image(s) can be found here: | ||
5703 | ./mkefidisk-201804191017-sda.direct | ||
5704 | |||
5705 | The following build artifacts were used to create the image(s): | ||
5706 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
5707 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
5708 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
5709 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5710 | |||
5711 | INFO: The image(s) were created using OE kickstart file: | ||
5712 | /home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks | ||
5713 | |||
5714 | The previous example shows the easiest way to create an image by running | ||
5715 | in cooked mode and supplying a kickstart file and the "-e" option to | ||
5716 | point to the existing build artifacts. Your ``local.conf`` file needs to | ||
5717 | have the :term:`MACHINE` variable set | ||
5718 | to the machine you are using, which is "qemux86" in this example. | ||
5719 | |||
5720 | Once the image builds, the output provides image location, artifact use, | ||
5721 | and kickstart file information. | ||
5722 | |||
5723 | .. note:: | ||
5724 | |||
5725 | You should always verify the details provided in the output to make | ||
5726 | sure that the image was indeed created exactly as expected. | ||
5727 | |||
5728 | Continuing with the example, you can now write the image from the Build | ||
5729 | Directory onto a USB stick, or whatever media for which you built your | ||
5730 | image, and boot from the media. You can write the image by using | ||
5731 | ``bmaptool`` or ``dd``: | ||
5732 | :: | ||
5733 | |||
5734 | $ oe-run-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sdX | ||
5735 | |||
5736 | or :: | ||
5737 | |||
5738 | $ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sdX | ||
5739 | |||
5740 | .. note:: | ||
5741 | |||
5742 | For more information on how to use the ``bmaptool`` | ||
5743 | to flash a device with an image, see the | ||
5744 | ":ref:`dev-manual/common-tasks:flashing images using \`\`bmaptool\`\``" | ||
5745 | section. | ||
5746 | |||
5747 | Using a Modified Kickstart File | ||
5748 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5749 | |||
5750 | Because partitioned image creation is driven by the kickstart file, it | ||
5751 | is easy to affect image creation by changing the parameters in the file. | ||
5752 | This next example demonstrates that through modification of the | ||
5753 | ``directdisk-gpt`` kickstart file. | ||
5754 | |||
5755 | As mentioned earlier, you can use the command ``wic list images`` to | ||
5756 | show the list of existing kickstart files. The directory in which the | ||
5757 | ``directdisk-gpt.wks`` file resides is | ||
5758 | ``scripts/lib/image/canned-wks/``, which is located in the | ||
5759 | :term:`Source Directory` (e.g. ``poky``). | ||
5760 | Because available files reside in this directory, you can create and add | ||
5761 | your own custom files to the directory. Subsequent use of the | ||
5762 | ``wic list images`` command would then include your kickstart files. | ||
5763 | |||
5764 | In this example, the existing ``directdisk-gpt`` file already does most | ||
5765 | of what is needed. However, for the hardware in this example, the image | ||
5766 | will need to boot from ``sdb`` instead of ``sda``, which is what the | ||
5767 | ``directdisk-gpt`` kickstart file uses. | ||
5768 | |||
5769 | The example begins by making a copy of the ``directdisk-gpt.wks`` file | ||
5770 | in the ``scripts/lib/image/canned-wks`` directory and then by changing | ||
5771 | the lines that specify the target disk from which to boot. | ||
5772 | :: | ||
5773 | |||
5774 | $ cp /home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \ | ||
5775 | /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks | ||
5776 | |||
5777 | Next, the example modifies the ``directdisksdb-gpt.wks`` file and | ||
5778 | changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The | ||
5779 | example changes the following two lines and leaves the remaining lines | ||
5780 | untouched: | ||
5781 | :: | ||
5782 | |||
5783 | part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024 | ||
5784 | part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid | ||
5785 | |||
5786 | Once the lines are changed, the | ||
5787 | example generates the ``directdisksdb-gpt`` image. The command points | ||
5788 | the process at the ``core-image-minimal`` artifacts for the Next Unit of | ||
5789 | Computing (nuc) :term:`MACHINE` the | ||
5790 | ``local.conf``. | ||
5791 | :: | ||
5792 | |||
5793 | $ wic create directdisksdb-gpt -e core-image-minimal | ||
5794 | INFO: Building wic-tools... | ||
5795 | . | ||
5796 | . | ||
5797 | . | ||
5798 | Initialising tasks: 100% |#######################################| Time: 0:00:01 | ||
5799 | NOTE: Executing SetScene Tasks | ||
5800 | NOTE: Executing RunQueue Tasks | ||
5801 | NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded. | ||
5802 | INFO: Creating image(s)... | ||
5803 | |||
5804 | INFO: The new image(s) can be found here: | ||
5805 | ./directdisksdb-gpt-201710090938-sdb.direct | ||
5806 | |||
5807 | The following build artifacts were used to create the image(s): | ||
5808 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
5809 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
5810 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
5811 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5812 | |||
5813 | INFO: The image(s) were created using OE kickstart file: | ||
5814 | /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks | ||
5815 | |||
5816 | Continuing with the example, you can now directly ``dd`` the image to a | ||
5817 | USB stick, or whatever media for which you built your image, and boot | ||
5818 | the resulting media: | ||
5819 | :: | ||
5820 | |||
5821 | $ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb | ||
5822 | 140966+0 records in | ||
5823 | 140966+0 records out | ||
5824 | 72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s | ||
5825 | $ sudo eject /dev/sdb | ||
5826 | |||
5827 | Using a Modified Kickstart File and Running in Raw Mode | ||
5828 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5829 | |||
5830 | This next example manually specifies each build artifact (runs in Raw | ||
5831 | Mode) and uses a modified kickstart file. The example also uses the | ||
5832 | ``-o`` option to cause Wic to create the output somewhere other than the | ||
5833 | default output directory, which is the current directory: | ||
5834 | :: | ||
5835 | |||
5836 | $ wic create /home/stephano/my_yocto/test.wks -o /home/stephano/testwic \ | ||
5837 | --rootfs-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \ | ||
5838 | --bootimg-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \ | ||
5839 | --kernel-dir /home/stephano/build/master/build/tmp/deploy/images/qemux86 \ | ||
5840 | --native-sysroot /home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5841 | |||
5842 | INFO: Creating image(s)... | ||
5843 | |||
5844 | INFO: The new image(s) can be found here: | ||
5845 | /home/stephano/testwic/test-201710091445-sdb.direct | ||
5846 | |||
5847 | The following build artifacts were used to create the image(s): | ||
5848 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
5849 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
5850 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
5851 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5852 | |||
5853 | INFO: The image(s) were created using OE kickstart file: | ||
5854 | /home/stephano/my_yocto/test.wks | ||
5855 | |||
5856 | For this example, | ||
5857 | :term:`MACHINE` did not have to be | ||
5858 | specified in the ``local.conf`` file since the artifact is manually | ||
5859 | specified. | ||
5860 | |||
5861 | Using Wic to Manipulate an Image | ||
5862 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5863 | |||
5864 | Wic image manipulation allows you to shorten turnaround time during | ||
5865 | image development. For example, you can use Wic to delete the kernel | ||
5866 | partition of a Wic image and then insert a newly built kernel. This | ||
5867 | saves you time from having to rebuild the entire image each time you | ||
5868 | modify the kernel. | ||
5869 | |||
5870 | .. note:: | ||
5871 | |||
5872 | In order to use Wic to manipulate a Wic image as in this example, | ||
5873 | your development machine must have the ``mtools`` package installed. | ||
5874 | |||
5875 | The following example examines the contents of the Wic image, deletes | ||
5876 | the existing kernel, and then inserts a new kernel: | ||
5877 | |||
5878 | 1. *List the Partitions:* Use the ``wic ls`` command to list all the | ||
5879 | partitions in the Wic image: | ||
5880 | :: | ||
5881 | |||
5882 | $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic | ||
5883 | Num Start End Size Fstype | ||
5884 | 1 1048576 25041919 23993344 fat16 | ||
5885 | 2 25165824 72157183 46991360 ext4 | ||
5886 | |||
5887 | The previous output shows two partitions in the | ||
5888 | ``core-image-minimal-qemux86.wic`` image. | ||
5889 | |||
5890 | 2. *Examine a Particular Partition:* Use the ``wic ls`` command again | ||
5891 | but in a different form to examine a particular partition. | ||
5892 | |||
5893 | .. note:: | ||
5894 | |||
5895 | You can get command usage on any Wic command using the following | ||
5896 | form: | ||
5897 | :: | ||
5898 | |||
5899 | $ wic help command | ||
5900 | |||
5901 | |||
5902 | For example, the following command shows you the various ways to | ||
5903 | use the | ||
5904 | wic ls | ||
5905 | command: | ||
5906 | :: | ||
5907 | |||
5908 | $ wic help ls | ||
5909 | |||
5910 | |||
5911 | The following command shows what is in Partition one: | ||
5912 | :: | ||
5913 | |||
5914 | $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1 | ||
5915 | Volume in drive : is boot | ||
5916 | Volume Serial Number is E894-1809 | ||
5917 | Directory for ::/ | ||
5918 | |||
5919 | libcom32 c32 186500 2017-10-09 16:06 | ||
5920 | libutil c32 24148 2017-10-09 16:06 | ||
5921 | syslinux cfg 220 2017-10-09 16:06 | ||
5922 | vesamenu c32 27104 2017-10-09 16:06 | ||
5923 | vmlinuz 6904608 2017-10-09 16:06 | ||
5924 | 5 files 7 142 580 bytes | ||
5925 | 16 582 656 bytes free | ||
5926 | |||
5927 | The previous output shows five files, with the | ||
5928 | ``vmlinuz`` being the kernel. | ||
5929 | |||
5930 | .. note:: | ||
5931 | |||
5932 | If you see the following error, you need to update or create a | ||
5933 | ``~/.mtoolsrc`` file and be sure to have the line "mtools_skip_check=1" | ||
5934 | in the file. Then, run the Wic command again: | ||
5935 | :: | ||
5936 | |||
5937 | ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0 | ||
5938 | output: Total number of sectors (47824) not a multiple of sectors per track (32)! | ||
5939 | Add mtools_skip_check=1 to your .mtoolsrc file to skip this test | ||
5940 | |||
5941 | |||
5942 | 3. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the | ||
5943 | ``vmlinuz`` file (kernel): | ||
5944 | :: | ||
5945 | |||
5946 | $ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz | ||
5947 | |||
5948 | 4. *Add In the New Kernel:* Use the ``wic cp`` command to add the | ||
5949 | updated kernel to the Wic image. Depending on how you built your | ||
5950 | kernel, it could be in different places. If you used ``devtool`` and | ||
5951 | an SDK to build your kernel, it resides in the ``tmp/work`` directory | ||
5952 | of the extensible SDK. If you used ``make`` to build the kernel, the | ||
5953 | kernel will be in the ``workspace/sources`` area. | ||
5954 | |||
5955 | The following example assumes ``devtool`` was used to build the | ||
5956 | kernel: | ||
5957 | :: | ||
5958 | |||
5959 | cp ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \ | ||
5960 | ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz | ||
5961 | |||
5962 | Once the new kernel is added back into the image, you can use the | ||
5963 | ``dd`` command or :ref:`bmaptool | ||
5964 | <dev-manual/common-tasks:flashing images using \`\`bmaptool\`\`>` | ||
5965 | to flash your wic image onto an SD card or USB stick and test your | ||
5966 | target. | ||
5967 | |||
5968 | .. note:: | ||
5969 | |||
5970 | Using ``bmaptool`` is generally 10 to 20 times faster than using ``dd``. | ||
5971 | |||
5972 | Flashing Images Using ``bmaptool`` | ||
5973 | ================================== | ||
5974 | |||
5975 | A fast and easy way to flash an image to a bootable device is to use | ||
5976 | Bmaptool, which is integrated into the OpenEmbedded build system. | ||
5977 | Bmaptool is a generic tool that creates a file's block map (bmap) and | ||
5978 | then uses that map to copy the file. As compared to traditional tools | ||
5979 | such as dd or cp, Bmaptool can copy (or flash) large files like raw | ||
5980 | system image files much faster. | ||
5981 | |||
5982 | .. note:: | ||
5983 | |||
5984 | - If you are using Ubuntu or Debian distributions, you can install | ||
5985 | the ``bmap-tools`` package using the following command and then | ||
5986 | use the tool without specifying ``PATH`` even from the root | ||
5987 | account: | ||
5988 | :: | ||
5989 | |||
5990 | $ sudo apt-get install bmap-tools | ||
5991 | |||
5992 | - If you are unable to install the ``bmap-tools`` package, you will | ||
5993 | need to build Bmaptool before using it. Use the following command: | ||
5994 | :: | ||
5995 | |||
5996 | $ bitbake bmap-tools-native | ||
5997 | |||
5998 | Following, is an example that shows how to flash a Wic image. Realize | ||
5999 | that while this example uses a Wic image, you can use Bmaptool to flash | ||
6000 | any type of image. Use these steps to flash an image using Bmaptool: | ||
6001 | |||
6002 | 1. *Update your local.conf File:* You need to have the following set | ||
6003 | in your ``local.conf`` file before building your image: | ||
6004 | :: | ||
6005 | |||
6006 | IMAGE_FSTYPES += "wic wic.bmap" | ||
6007 | |||
6008 | 2. *Get Your Image:* Either have your image ready (pre-built with the | ||
6009 | :term:`IMAGE_FSTYPES` | ||
6010 | setting previously mentioned) or take the step to build the image: | ||
6011 | :: | ||
6012 | |||
6013 | $ bitbake image | ||
6014 | |||
6015 | 3. *Flash the Device:* Flash the device with the image by using Bmaptool | ||
6016 | depending on your particular setup. The following commands assume the | ||
6017 | image resides in the Build Directory's ``deploy/images/`` area: | ||
6018 | |||
6019 | - If you have write access to the media, use this command form: | ||
6020 | :: | ||
6021 | |||
6022 | $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | ||
6023 | |||
6024 | - If you do not have write access to the media, set your permissions | ||
6025 | first and then use the same command form: | ||
6026 | :: | ||
6027 | |||
6028 | $ sudo chmod 666 /dev/sdX | ||
6029 | $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | ||
6030 | |||
6031 | For help on the ``bmaptool`` command, use the following command: | ||
6032 | :: | ||
6033 | |||
6034 | $ bmaptool --help | ||
6035 | |||
6036 | Making Images More Secure | ||
6037 | ========================= | ||
6038 | |||
6039 | Security is of increasing concern for embedded devices. Consider the | ||
6040 | issues and problems discussed in just this sampling of work found across | ||
6041 | the Internet: | ||
6042 | |||
6043 | - *"*\ `Security Risks of Embedded | ||
6044 | Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"* | ||
6045 | by Bruce Schneier | ||
6046 | |||
6047 | - *"*\ `Internet Census | ||
6048 | 2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna | ||
6049 | Botnet | ||
6050 | |||
6051 | - *"*\ `Security Issues for Embedded | ||
6052 | Devices <https://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"* | ||
6053 | by Jake Edge | ||
6054 | |||
6055 | When securing your image is of concern, there are steps, tools, and | ||
6056 | variables that you can consider to help you reach the security goals you | ||
6057 | need for your particular device. Not all situations are identical when | ||
6058 | it comes to making an image secure. Consequently, this section provides | ||
6059 | some guidance and suggestions for consideration when you want to make | ||
6060 | your image more secure. | ||
6061 | |||
6062 | .. note:: | ||
6063 | |||
6064 | Because the security requirements and risks are different for every | ||
6065 | type of device, this section cannot provide a complete reference on | ||
6066 | securing your custom OS. It is strongly recommended that you also | ||
6067 | consult other sources of information on embedded Linux system | ||
6068 | hardening and on security. | ||
6069 | |||
6070 | General Considerations | ||
6071 | ---------------------- | ||
6072 | |||
6073 | General considerations exist that help you create more secure images. | ||
6074 | You should consider the following suggestions to help make your device | ||
6075 | more secure: | ||
6076 | |||
6077 | - Scan additional code you are adding to the system (e.g. application | ||
6078 | code) by using static analysis tools. Look for buffer overflows and | ||
6079 | other potential security problems. | ||
6080 | |||
6081 | - Pay particular attention to the security for any web-based | ||
6082 | administration interface. | ||
6083 | |||
6084 | Web interfaces typically need to perform administrative functions and | ||
6085 | tend to need to run with elevated privileges. Thus, the consequences | ||
6086 | resulting from the interface's security becoming compromised can be | ||
6087 | serious. Look for common web vulnerabilities such as | ||
6088 | cross-site-scripting (XSS), unvalidated inputs, and so forth. | ||
6089 | |||
6090 | As with system passwords, the default credentials for accessing a | ||
6091 | web-based interface should not be the same across all devices. This | ||
6092 | is particularly true if the interface is enabled by default as it can | ||
6093 | be assumed that many end-users will not change the credentials. | ||
6094 | |||
6095 | - Ensure you can update the software on the device to mitigate | ||
6096 | vulnerabilities discovered in the future. This consideration | ||
6097 | especially applies when your device is network-enabled. | ||
6098 | |||
6099 | - Ensure you remove or disable debugging functionality before producing | ||
6100 | the final image. For information on how to do this, see the | ||
6101 | "`Considerations Specific to the OpenEmbedded Build | ||
6102 | System <#considerations-specific-to-the-openembedded-build-system>`__" | ||
6103 | section. | ||
6104 | |||
6105 | - Ensure you have no network services listening that are not needed. | ||
6106 | |||
6107 | - Remove any software from the image that is not needed. | ||
6108 | |||
6109 | - Enable hardware support for secure boot functionality when your | ||
6110 | device supports this functionality. | ||
6111 | |||
6112 | Security Flags | ||
6113 | -------------- | ||
6114 | |||
6115 | The Yocto Project has security flags that you can enable that help make | ||
6116 | your build output more secure. The security flags are in the | ||
6117 | ``meta/conf/distro/include/security_flags.inc`` file in your | ||
6118 | :term:`Source Directory` (e.g. ``poky``). | ||
6119 | |||
6120 | .. note:: | ||
6121 | |||
6122 | Depending on the recipe, certain security flags are enabled and | ||
6123 | disabled by default. | ||
6124 | |||
6125 | Use the following line in your ``local.conf`` file or in your custom | ||
6126 | distribution configuration file to enable the security compiler and | ||
6127 | linker flags for your build: | ||
6128 | :: | ||
6129 | |||
6130 | require conf/distro/include/security_flags.inc | ||
6131 | |||
6132 | Considerations Specific to the OpenEmbedded Build System | ||
6133 | -------------------------------------------------------- | ||
6134 | |||
6135 | You can take some steps that are specific to the OpenEmbedded build | ||
6136 | system to make your images more secure: | ||
6137 | |||
6138 | - Ensure "debug-tweaks" is not one of your selected | ||
6139 | :term:`IMAGE_FEATURES`. | ||
6140 | When creating a new project, the default is to provide you with an | ||
6141 | initial ``local.conf`` file that enables this feature using the | ||
6142 | :term:`EXTRA_IMAGE_FEATURES` | ||
6143 | variable with the line: | ||
6144 | :: | ||
6145 | |||
6146 | EXTRA_IMAGE_FEATURES = "debug-tweaks" | ||
6147 | |||
6148 | To disable that feature, simply comment out that line in your | ||
6149 | ``local.conf`` file, or make sure ``IMAGE_FEATURES`` does not contain | ||
6150 | "debug-tweaks" before producing your final image. Among other things, | ||
6151 | leaving this in place sets the root password as blank, which makes | ||
6152 | logging in for debugging or inspection easy during development but | ||
6153 | also means anyone can easily log in during production. | ||
6154 | |||
6155 | - It is possible to set a root password for the image and also to set | ||
6156 | passwords for any extra users you might add (e.g. administrative or | ||
6157 | service type users). When you set up passwords for multiple images or | ||
6158 | users, you should not duplicate passwords. | ||
6159 | |||
6160 | To set up passwords, use the | ||
6161 | :ref:`extrausers <ref-classes-extrausers>` | ||
6162 | class, which is the preferred method. For an example on how to set up | ||
6163 | both root and user passwords, see the | ||
6164 | ":ref:`extrausers.bbclass <ref-classes-extrausers>`" | ||
6165 | section. | ||
6166 | |||
6167 | .. note:: | ||
6168 | |||
6169 | When adding extra user accounts or setting a root password, be | ||
6170 | cautious about setting the same password on every device. If you | ||
6171 | do this, and the password you have set is exposed, then every | ||
6172 | device is now potentially compromised. If you need this access but | ||
6173 | want to ensure security, consider setting a different, random | ||
6174 | password for each device. Typically, you do this as a separate | ||
6175 | step after you deploy the image onto the device. | ||
6176 | |||
6177 | - Consider enabling a Mandatory Access Control (MAC) framework such as | ||
6178 | SMACK or SELinux and tuning it appropriately for your device's usage. | ||
6179 | You can find more information in the | ||
6180 | :yocto_git:`meta-selinux </meta-selinux/>` layer. | ||
6181 | |||
6182 | Tools for Hardening Your Image | ||
6183 | ------------------------------ | ||
6184 | |||
6185 | The Yocto Project provides tools for making your image more secure. You | ||
6186 | can find these tools in the ``meta-security`` layer of the | ||
6187 | :yocto_git:`Yocto Project Source Repositories <>`. | ||
6188 | |||
6189 | Creating Your Own Distribution | ||
6190 | ============================== | ||
6191 | |||
6192 | When you build an image using the Yocto Project and do not alter any | ||
6193 | distribution :term:`Metadata`, you are | ||
6194 | creating a Poky distribution. If you wish to gain more control over | ||
6195 | package alternative selections, compile-time options, and other | ||
6196 | low-level configurations, you can create your own distribution. | ||
6197 | |||
6198 | To create your own distribution, the basic steps consist of creating | ||
6199 | your own distribution layer, creating your own distribution | ||
6200 | configuration file, and then adding any needed code and Metadata to the | ||
6201 | layer. The following steps provide some more detail: | ||
6202 | |||
6203 | - *Create a layer for your new distro:* Create your distribution layer | ||
6204 | so that you can keep your Metadata and code for the distribution | ||
6205 | separate. It is strongly recommended that you create and use your own | ||
6206 | layer for configuration and code. Using your own layer as compared to | ||
6207 | just placing configurations in a ``local.conf`` configuration file | ||
6208 | makes it easier to reproduce the same build configuration when using | ||
6209 | multiple build machines. See the | ||
6210 | ":ref:`dev-manual/common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`" | ||
6211 | section for information on how to quickly set up a layer. | ||
6212 | |||
6213 | - *Create the distribution configuration file:* The distribution | ||
6214 | configuration file needs to be created in the ``conf/distro`` | ||
6215 | directory of your layer. You need to name it using your distribution | ||
6216 | name (e.g. ``mydistro.conf``). | ||
6217 | |||
6218 | .. note:: | ||
6219 | |||
6220 | The :term:`DISTRO` variable in your ``local.conf`` file determines the | ||
6221 | name of your distribution. | ||
6222 | |||
6223 | You can split out parts of your configuration file into include files | ||
6224 | and then "require" them from within your distribution configuration | ||
6225 | file. Be sure to place the include files in the | ||
6226 | ``conf/distro/include`` directory of your layer. A common example | ||
6227 | usage of include files would be to separate out the selection of | ||
6228 | desired version and revisions for individual recipes. | ||
6229 | |||
6230 | Your configuration file needs to set the following required | ||
6231 | variables: | ||
6232 | |||
6233 | - :term:`DISTRO_NAME` | ||
6234 | |||
6235 | - :term:`DISTRO_VERSION` | ||
6236 | |||
6237 | These following variables are optional and you typically set them | ||
6238 | from the distribution configuration file: | ||
6239 | |||
6240 | - :term:`DISTRO_FEATURES` | ||
6241 | |||
6242 | - :term:`DISTRO_EXTRA_RDEPENDS` | ||
6243 | |||
6244 | - :term:`DISTRO_EXTRA_RRECOMMENDS` | ||
6245 | |||
6246 | - :term:`TCLIBC` | ||
6247 | |||
6248 | .. tip:: | ||
6249 | |||
6250 | If you want to base your distribution configuration file on the | ||
6251 | very basic configuration from OE-Core, you can use | ||
6252 | ``conf/distro/defaultsetup.conf`` as a reference and just include | ||
6253 | variables that differ as compared to ``defaultsetup.conf``. | ||
6254 | Alternatively, you can create a distribution configuration file | ||
6255 | from scratch using the ``defaultsetup.conf`` file or configuration files | ||
6256 | from other distributions such as Poky or Angstrom as references. | ||
6257 | |||
6258 | - *Provide miscellaneous variables:* Be sure to define any other | ||
6259 | variables for which you want to create a default or enforce as part | ||
6260 | of the distribution configuration. You can include nearly any | ||
6261 | variable from the ``local.conf`` file. The variables you use are not | ||
6262 | limited to the list in the previous bulleted item. | ||
6263 | |||
6264 | - *Point to Your distribution configuration file:* In your | ||
6265 | ``local.conf`` file in the :term:`Build Directory`, | ||
6266 | set your | ||
6267 | :term:`DISTRO` variable to point to | ||
6268 | your distribution's configuration file. For example, if your | ||
6269 | distribution's configuration file is named ``mydistro.conf``, then | ||
6270 | you point to it as follows: | ||
6271 | :: | ||
6272 | |||
6273 | DISTRO = "mydistro" | ||
6274 | |||
6275 | - *Add more to the layer if necessary:* Use your layer to hold other | ||
6276 | information needed for the distribution: | ||
6277 | |||
6278 | - Add recipes for installing distro-specific configuration files | ||
6279 | that are not already installed by another recipe. If you have | ||
6280 | distro-specific configuration files that are included by an | ||
6281 | existing recipe, you should add an append file (``.bbappend``) for | ||
6282 | those. For general information and recommendations on how to add | ||
6283 | recipes to your layer, see the "`Creating Your Own | ||
6284 | Layer <#creating-your-own-layer>`__" and "`Following Best | ||
6285 | Practices When Creating | ||
6286 | Layers <#best-practices-to-follow-when-creating-layers>`__" | ||
6287 | sections. | ||
6288 | |||
6289 | - Add any image recipes that are specific to your distribution. | ||
6290 | |||
6291 | - Add a ``psplash`` append file for a branded splash screen. For | ||
6292 | information on append files, see the "`Using .bbappend Files in | ||
6293 | Your Layer <#using-bbappend-files>`__" section. | ||
6294 | |||
6295 | - Add any other append files to make custom changes that are | ||
6296 | specific to individual recipes. | ||
6297 | |||
6298 | Creating a Custom Template Configuration Directory | ||
6299 | ================================================== | ||
6300 | |||
6301 | If you are producing your own customized version of the build system for | ||
6302 | use by other users, you might want to customize the message shown by the | ||
6303 | setup script or you might want to change the template configuration | ||
6304 | files (i.e. ``local.conf`` and ``bblayers.conf``) that are created in a | ||
6305 | new build directory. | ||
6306 | |||
6307 | The OpenEmbedded build system uses the environment variable | ||
6308 | ``TEMPLATECONF`` to locate the directory from which it gathers | ||
6309 | configuration information that ultimately ends up in the | ||
6310 | :term:`Build Directory` ``conf`` directory. | ||
6311 | By default, ``TEMPLATECONF`` is set as follows in the ``poky`` | ||
6312 | repository: | ||
6313 | :: | ||
6314 | |||
6315 | TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} | ||
6316 | |||
6317 | This is the | ||
6318 | directory used by the build system to find templates from which to build | ||
6319 | some key configuration files. If you look at this directory, you will | ||
6320 | see the ``bblayers.conf.sample``, ``local.conf.sample``, and | ||
6321 | ``conf-notes.txt`` files. The build system uses these files to form the | ||
6322 | respective ``bblayers.conf`` file, ``local.conf`` file, and display the | ||
6323 | list of BitBake targets when running the setup script. | ||
6324 | |||
6325 | To override these default configuration files with configurations you | ||
6326 | want used within every new Build Directory, simply set the | ||
6327 | ``TEMPLATECONF`` variable to your directory. The ``TEMPLATECONF`` | ||
6328 | variable is set in the ``.templateconf`` file, which is in the top-level | ||
6329 | :term:`Source Directory` folder | ||
6330 | (e.g. ``poky``). Edit the ``.templateconf`` so that it can locate your | ||
6331 | directory. | ||
6332 | |||
6333 | Best practices dictate that you should keep your template configuration | ||
6334 | directory in your custom distribution layer. For example, suppose you | ||
6335 | have a layer named ``meta-mylayer`` located in your home directory and | ||
6336 | you want your template configuration directory named ``myconf``. | ||
6337 | Changing the ``.templateconf`` as follows causes the OpenEmbedded build | ||
6338 | system to look in your directory and base its configuration files on the | ||
6339 | ``*.sample`` configuration files it finds. The final configuration files | ||
6340 | (i.e. ``local.conf`` and ``bblayers.conf`` ultimately still end up in | ||
6341 | your Build Directory, but they are based on your ``*.sample`` files. | ||
6342 | :: | ||
6343 | |||
6344 | TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf} | ||
6345 | |||
6346 | Aside from the ``*.sample`` configuration files, the ``conf-notes.txt`` | ||
6347 | also resides in the default ``meta-poky/conf`` directory. The script | ||
6348 | that sets up the build environment (i.e. | ||
6349 | :ref:`structure-core-script`) uses this file to | ||
6350 | display BitBake targets as part of the script output. Customizing this | ||
6351 | ``conf-notes.txt`` file is a good way to make sure your list of custom | ||
6352 | targets appears as part of the script's output. | ||
6353 | |||
6354 | Here is the default list of targets displayed as a result of running | ||
6355 | either of the setup scripts: | ||
6356 | :: | ||
6357 | |||
6358 | You can now run 'bitbake <target>' | ||
6359 | |||
6360 | Common targets are: | ||
6361 | core-image-minimal | ||
6362 | core-image-sato | ||
6363 | meta-toolchain | ||
6364 | meta-ide-support | ||
6365 | |||
6366 | Changing the listed common targets is as easy as editing your version of | ||
6367 | ``conf-notes.txt`` in your custom template configuration directory and | ||
6368 | making sure you have ``TEMPLATECONF`` set to your directory. | ||
6369 | |||
6370 | Conserving Disk Space During Builds | ||
6371 | =================================== | ||
6372 | |||
6373 | To help conserve disk space during builds, you can add the following | ||
6374 | statement to your project's ``local.conf`` configuration file found in | ||
6375 | the :term:`Build Directory`: | ||
6376 | :: | ||
6377 | |||
6378 | INHERIT += "rm_work" | ||
6379 | |||
6380 | Adding this statement deletes the work directory used for | ||
6381 | building a recipe once the recipe is built. For more information on | ||
6382 | "rm_work", see the | ||
6383 | :ref:`rm_work <ref-classes-rm-work>` class in the | ||
6384 | Yocto Project Reference Manual. | ||
6385 | |||
6386 | Working with Packages | ||
6387 | ===================== | ||
6388 | |||
6389 | This section describes a few tasks that involve packages: | ||
6390 | |||
6391 | - `Excluding packages from an | ||
6392 | image <#excluding-packages-from-an-image>`__ | ||
6393 | |||
6394 | - `Incrementing a binary package | ||
6395 | version <#incrementing-a-binary-package-version>`__ | ||
6396 | |||
6397 | - `Handling optional module | ||
6398 | packaging <#handling-optional-module-packaging>`__ | ||
6399 | |||
6400 | - `Using runtime package | ||
6401 | management <#using-runtime-package-management>`__ | ||
6402 | |||
6403 | - `Generating and using signed | ||
6404 | packages <#generating-and-using-signed-packages>`__ | ||
6405 | |||
6406 | - `Setting up and running package test | ||
6407 | (ptest) <#testing-packages-with-ptest>`__ | ||
6408 | |||
6409 | - `Creating node package manager (NPM) | ||
6410 | packages <#creating-node-package-manager-npm-packages>`__ | ||
6411 | |||
6412 | - `Adding custom metadata to | ||
6413 | packages <#adding-custom-metadata-to-packages>`__ | ||
6414 | |||
6415 | Excluding Packages from an Image | ||
6416 | -------------------------------- | ||
6417 | |||
6418 | You might find it necessary to prevent specific packages from being | ||
6419 | installed into an image. If so, you can use several variables to direct | ||
6420 | the build system to essentially ignore installing recommended packages | ||
6421 | or to not install a package at all. | ||
6422 | |||
6423 | The following list introduces variables you can use to prevent packages | ||
6424 | from being installed into your image. Each of these variables only works | ||
6425 | with IPK and RPM package types. Support for Debian packages does not | ||
6426 | exist. Also, you can use these variables from your ``local.conf`` file | ||
6427 | or attach them to a specific image recipe by using a recipe name | ||
6428 | override. For more detail on the variables, see the descriptions in the | ||
6429 | Yocto Project Reference Manual's glossary chapter. | ||
6430 | |||
6431 | - :term:`BAD_RECOMMENDATIONS`: | ||
6432 | Use this variable to specify "recommended-only" packages that you do | ||
6433 | not want installed. | ||
6434 | |||
6435 | - :term:`NO_RECOMMENDATIONS`: | ||
6436 | Use this variable to prevent all "recommended-only" packages from | ||
6437 | being installed. | ||
6438 | |||
6439 | - :term:`PACKAGE_EXCLUDE`: | ||
6440 | Use this variable to prevent specific packages from being installed | ||
6441 | regardless of whether they are "recommended-only" or not. You need to | ||
6442 | realize that the build process could fail with an error when you | ||
6443 | prevent the installation of a package whose presence is required by | ||
6444 | an installed package. | ||
6445 | |||
6446 | Incrementing a Package Version | ||
6447 | ------------------------------ | ||
6448 | |||
6449 | This section provides some background on how binary package versioning | ||
6450 | is accomplished and presents some of the services, variables, and | ||
6451 | terminology involved. | ||
6452 | |||
6453 | In order to understand binary package versioning, you need to consider | ||
6454 | the following: | ||
6455 | |||
6456 | - Binary Package: The binary package that is eventually built and | ||
6457 | installed into an image. | ||
6458 | |||
6459 | - Binary Package Version: The binary package version is composed of two | ||
6460 | components - a version and a revision. | ||
6461 | |||
6462 | .. note:: | ||
6463 | |||
6464 | Technically, a third component, the "epoch" (i.e. :term:`PE`) is involved | ||
6465 | but this discussion for the most part ignores ``PE``. | ||
6466 | |||
6467 | The version and revision are taken from the | ||
6468 | :term:`PV` and | ||
6469 | :term:`PR` variables, respectively. | ||
6470 | |||
6471 | - ``PV``: The recipe version. ``PV`` represents the version of the | ||
6472 | software being packaged. Do not confuse ``PV`` with the binary | ||
6473 | package version. | ||
6474 | |||
6475 | - ``PR``: The recipe revision. | ||
6476 | |||
6477 | - :term:`SRCPV`: The OpenEmbedded | ||
6478 | build system uses this string to help define the value of ``PV`` when | ||
6479 | the source code revision needs to be included in it. | ||
6480 | |||
6481 | - :yocto_wiki:`PR Service </PR_Service>`: A | ||
6482 | network-based service that helps automate keeping package feeds | ||
6483 | compatible with existing package manager applications such as RPM, | ||
6484 | APT, and OPKG. | ||
6485 | |||
6486 | Whenever the binary package content changes, the binary package version | ||
6487 | must change. Changing the binary package version is accomplished by | ||
6488 | changing or "bumping" the ``PR`` and/or ``PV`` values. Increasing these | ||
6489 | values occurs one of two ways: | ||
6490 | |||
6491 | - Automatically using a Package Revision Service (PR Service). | ||
6492 | |||
6493 | - Manually incrementing the ``PR`` and/or ``PV`` variables. | ||
6494 | |||
6495 | Given a primary challenge of any build system and its users is how to | ||
6496 | maintain a package feed that is compatible with existing package manager | ||
6497 | applications such as RPM, APT, and OPKG, using an automated system is | ||
6498 | much preferred over a manual system. In either system, the main | ||
6499 | requirement is that binary package version numbering increases in a | ||
6500 | linear fashion and that a number of version components exist that | ||
6501 | support that linear progression. For information on how to ensure | ||
6502 | package revisioning remains linear, see the "`Automatically Incrementing | ||
6503 | a Binary Package Revision | ||
6504 | Number <#automatically-incrementing-a-binary-package-revision-number>`__" | ||
6505 | section. | ||
6506 | |||
6507 | The following three sections provide related information on the PR | ||
6508 | Service, the manual method for "bumping" ``PR`` and/or ``PV``, and on | ||
6509 | how to ensure binary package revisioning remains linear. | ||
6510 | |||
6511 | Working With a PR Service | ||
6512 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6513 | |||
6514 | As mentioned, attempting to maintain revision numbers in the | ||
6515 | :term:`Metadata` is error prone, inaccurate, | ||
6516 | and causes problems for people submitting recipes. Conversely, the PR | ||
6517 | Service automatically generates increasing numbers, particularly the | ||
6518 | revision field, which removes the human element. | ||
6519 | |||
6520 | .. note:: | ||
6521 | |||
6522 | For additional information on using a PR Service, you can see the | ||
6523 | :yocto_wiki:`PR Service </PR_Service>` wiki page. | ||
6524 | |||
6525 | The Yocto Project uses variables in order of decreasing priority to | ||
6526 | facilitate revision numbering (i.e. | ||
6527 | :term:`PE`, | ||
6528 | :term:`PV`, and | ||
6529 | :term:`PR` for epoch, version, and | ||
6530 | revision, respectively). The values are highly dependent on the policies | ||
6531 | and procedures of a given distribution and package feed. | ||
6532 | |||
6533 | Because the OpenEmbedded build system uses | ||
6534 | ":ref:`signatures <overview-manual/concepts:checksums (signatures)>`", which are | ||
6535 | unique to a given build, the build system knows when to rebuild | ||
6536 | packages. All the inputs into a given task are represented by a | ||
6537 | signature, which can trigger a rebuild when different. Thus, the build | ||
6538 | system itself does not rely on the ``PR``, ``PV``, and ``PE`` numbers to | ||
6539 | trigger a rebuild. The signatures, however, can be used to generate | ||
6540 | these values. | ||
6541 | |||
6542 | The PR Service works with both ``OEBasic`` and ``OEBasicHash`` | ||
6543 | generators. The value of ``PR`` bumps when the checksum changes and the | ||
6544 | different generator mechanisms change signatures under different | ||
6545 | circumstances. | ||
6546 | |||
6547 | As implemented, the build system includes values from the PR Service | ||
6548 | into the ``PR`` field as an addition using the form "``.x``" so ``r0`` | ||
6549 | becomes ``r0.1``, ``r0.2`` and so forth. This scheme allows existing | ||
6550 | ``PR`` values to be used for whatever reasons, which include manual | ||
6551 | ``PR`` bumps, should it be necessary. | ||
6552 | |||
6553 | By default, the PR Service is not enabled or running. Thus, the packages | ||
6554 | generated are just "self consistent". The build system adds and removes | ||
6555 | packages and there are no guarantees about upgrade paths but images will | ||
6556 | be consistent and correct with the latest changes. | ||
6557 | |||
6558 | The simplest form for a PR Service is for it to exist for a single host | ||
6559 | development system that builds the package feed (building system). For | ||
6560 | this scenario, you can enable a local PR Service by setting | ||
6561 | :term:`PRSERV_HOST` in your | ||
6562 | ``local.conf`` file in the :term:`Build Directory`: | ||
6563 | :: | ||
6564 | |||
6565 | PRSERV_HOST = "localhost:0" | ||
6566 | |||
6567 | Once the service is started, packages will automatically | ||
6568 | get increasing ``PR`` values and BitBake takes care of starting and | ||
6569 | stopping the server. | ||
6570 | |||
6571 | If you have a more complex setup where multiple host development systems | ||
6572 | work against a common, shared package feed, you have a single PR Service | ||
6573 | running and it is connected to each building system. For this scenario, | ||
6574 | you need to start the PR Service using the ``bitbake-prserv`` command: | ||
6575 | :: | ||
6576 | |||
6577 | bitbake-prserv --host ip --port port --start | ||
6578 | |||
6579 | In addition to | ||
6580 | hand-starting the service, you need to update the ``local.conf`` file of | ||
6581 | each building system as described earlier so each system points to the | ||
6582 | server and port. | ||
6583 | |||
6584 | It is also recommended you use build history, which adds some sanity | ||
6585 | checks to binary package versions, in conjunction with the server that | ||
6586 | is running the PR Service. To enable build history, add the following to | ||
6587 | each building system's ``local.conf`` file: | ||
6588 | :: | ||
6589 | |||
6590 | # It is recommended to activate "buildhistory" for testing the PR service | ||
6591 | INHERIT += "buildhistory" | ||
6592 | BUILDHISTORY_COMMIT = "1" | ||
6593 | |||
6594 | For information on build | ||
6595 | history, see the "`Maintaining Build Output | ||
6596 | Quality <#maintaining-build-output-quality>`__" section. | ||
6597 | |||
6598 | .. note:: | ||
6599 | |||
6600 | The OpenEmbedded build system does not maintain ``PR`` information as | ||
6601 | part of the shared state (sstate) packages. If you maintain an sstate | ||
6602 | feed, its expected that either all your building systems that | ||
6603 | contribute to the sstate feed use a shared PR Service, or you do not | ||
6604 | run a PR Service on any of your building systems. Having some systems | ||
6605 | use a PR Service while others do not leads to obvious problems. | ||
6606 | |||
6607 | For more information on shared state, see the | ||
6608 | ":ref:`overview-manual/concepts:shared state cache`" | ||
6609 | section in the Yocto Project Overview and Concepts Manual. | ||
6610 | |||
6611 | Manually Bumping PR | ||
6612 | ~~~~~~~~~~~~~~~~~~~ | ||
6613 | |||
6614 | The alternative to setting up a PR Service is to manually "bump" the | ||
6615 | :term:`PR` variable. | ||
6616 | |||
6617 | If a committed change results in changing the package output, then the | ||
6618 | value of the PR variable needs to be increased (or "bumped") as part of | ||
6619 | that commit. For new recipes you should add the ``PR`` variable and set | ||
6620 | its initial value equal to "r0", which is the default. Even though the | ||
6621 | default value is "r0", the practice of adding it to a new recipe makes | ||
6622 | it harder to forget to bump the variable when you make changes to the | ||
6623 | recipe in future. | ||
6624 | |||
6625 | If you are sharing a common ``.inc`` file with multiple recipes, you can | ||
6626 | also use the ``INC_PR`` variable to ensure that the recipes sharing the | ||
6627 | ``.inc`` file are rebuilt when the ``.inc`` file itself is changed. The | ||
6628 | ``.inc`` file must set ``INC_PR`` (initially to "r0"), and all recipes | ||
6629 | referring to it should set ``PR`` to "${INC_PR}.0" initially, | ||
6630 | incrementing the last number when the recipe is changed. If the ``.inc`` | ||
6631 | file is changed then its ``INC_PR`` should be incremented. | ||
6632 | |||
6633 | When upgrading the version of a binary package, assuming the ``PV`` | ||
6634 | changes, the ``PR`` variable should be reset to "r0" (or "${INC_PR}.0" | ||
6635 | if you are using ``INC_PR``). | ||
6636 | |||
6637 | Usually, version increases occur only to binary packages. However, if | ||
6638 | for some reason ``PV`` changes but does not increase, you can increase | ||
6639 | the ``PE`` variable (Package Epoch). The ``PE`` variable defaults to | ||
6640 | "0". | ||
6641 | |||
6642 | Binary package version numbering strives to follow the `Debian Version | ||
6643 | Field Policy | ||
6644 | Guidelines <https://www.debian.org/doc/debian-policy/ch-controlfields.html>`__. | ||
6645 | These guidelines define how versions are compared and what "increasing" | ||
6646 | a version means. | ||
6647 | |||
6648 | Automatically Incrementing a Package Version Number | ||
6649 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6650 | |||
6651 | When fetching a repository, BitBake uses the | ||
6652 | :term:`SRCREV` variable to determine | ||
6653 | the specific source code revision from which to build. You set the | ||
6654 | ``SRCREV`` variable to | ||
6655 | :term:`AUTOREV` to cause the | ||
6656 | OpenEmbedded build system to automatically use the latest revision of | ||
6657 | the software: | ||
6658 | :: | ||
6659 | |||
6660 | SRCREV = "${AUTOREV}" | ||
6661 | |||
6662 | Furthermore, you need to reference ``SRCPV`` in ``PV`` in order to | ||
6663 | automatically update the version whenever the revision of the source | ||
6664 | code changes. Here is an example: | ||
6665 | :: | ||
6666 | |||
6667 | PV = "1.0+git${SRCPV}" | ||
6668 | |||
6669 | The OpenEmbedded build system substitutes ``SRCPV`` with the following: | ||
6670 | |||
6671 | .. code-block:: none | ||
6672 | |||
6673 | AUTOINC+source_code_revision | ||
6674 | |||
6675 | The build system replaces the ``AUTOINC`` | ||
6676 | with a number. The number used depends on the state of the PR Service: | ||
6677 | |||
6678 | - If PR Service is enabled, the build system increments the number, | ||
6679 | which is similar to the behavior of | ||
6680 | :term:`PR`. This behavior results in | ||
6681 | linearly increasing package versions, which is desirable. Here is an | ||
6682 | example: | ||
6683 | |||
6684 | .. code-block:: none | ||
6685 | |||
6686 | hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk | ||
6687 | hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk | ||
6688 | |||
6689 | - If PR Service is not enabled, the build system replaces the | ||
6690 | ``AUTOINC`` placeholder with zero (i.e. "0"). This results in | ||
6691 | changing the package version since the source revision is included. | ||
6692 | However, package versions are not increased linearly. Here is an | ||
6693 | example: | ||
6694 | |||
6695 | .. code-block:: none | ||
6696 | |||
6697 | hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk | ||
6698 | hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk | ||
6699 | |||
6700 | In summary, the OpenEmbedded build system does not track the history of | ||
6701 | binary package versions for this purpose. ``AUTOINC``, in this case, is | ||
6702 | comparable to ``PR``. If PR server is not enabled, ``AUTOINC`` in the | ||
6703 | package version is simply replaced by "0". If PR server is enabled, the | ||
6704 | build system keeps track of the package versions and bumps the number | ||
6705 | when the package revision changes. | ||
6706 | |||
6707 | Handling Optional Module Packaging | ||
6708 | ---------------------------------- | ||
6709 | |||
6710 | Many pieces of software split functionality into optional modules (or | ||
6711 | plugins) and the plugins that are built might depend on configuration | ||
6712 | options. To avoid having to duplicate the logic that determines what | ||
6713 | modules are available in your recipe or to avoid having to package each | ||
6714 | module by hand, the OpenEmbedded build system provides functionality to | ||
6715 | handle module packaging dynamically. | ||
6716 | |||
6717 | To handle optional module packaging, you need to do two things: | ||
6718 | |||
6719 | - Ensure the module packaging is actually done. | ||
6720 | |||
6721 | - Ensure that any dependencies on optional modules from other recipes | ||
6722 | are satisfied by your recipe. | ||
6723 | |||
6724 | Making Sure the Packaging is Done | ||
6725 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6726 | |||
6727 | To ensure the module packaging actually gets done, you use the | ||
6728 | ``do_split_packages`` function within the ``populate_packages`` Python | ||
6729 | function in your recipe. The ``do_split_packages`` function searches for | ||
6730 | a pattern of files or directories under a specified path and creates a | ||
6731 | package for each one it finds by appending to the | ||
6732 | :term:`PACKAGES` variable and | ||
6733 | setting the appropriate values for ``FILES_packagename``, | ||
6734 | ``RDEPENDS_packagename``, ``DESCRIPTION_packagename``, and so forth. | ||
6735 | Here is an example from the ``lighttpd`` recipe: | ||
6736 | :: | ||
6737 | |||
6738 | python populate_packages_prepend () { | ||
6739 | lighttpd_libdir = d.expand('${libdir}') | ||
6740 | do_split_packages(d, lighttpd_libdir, '^mod_(.*).so$', | ||
6741 | 'lighttpd-module-%s', 'Lighttpd module for %s', | ||
6742 | extra_depends='') | ||
6743 | } | ||
6744 | |||
6745 | The previous example specifies a number of things in the call to | ||
6746 | ``do_split_packages``. | ||
6747 | |||
6748 | - A directory within the files installed by your recipe through | ||
6749 | ``do_install`` in which to search. | ||
6750 | |||
6751 | - A regular expression used to match module files in that directory. In | ||
6752 | the example, note the parentheses () that mark the part of the | ||
6753 | expression from which the module name should be derived. | ||
6754 | |||
6755 | - A pattern to use for the package names. | ||
6756 | |||
6757 | - A description for each package. | ||
6758 | |||
6759 | - An empty string for ``extra_depends``, which disables the default | ||
6760 | dependency on the main ``lighttpd`` package. Thus, if a file in | ||
6761 | ``${libdir}`` called ``mod_alias.so`` is found, a package called | ||
6762 | ``lighttpd-module-alias`` is created for it and the | ||
6763 | :term:`DESCRIPTION` is set to | ||
6764 | "Lighttpd module for alias". | ||
6765 | |||
6766 | Often, packaging modules is as simple as the previous example. However, | ||
6767 | more advanced options exist that you can use within | ||
6768 | ``do_split_packages`` to modify its behavior. And, if you need to, you | ||
6769 | can add more logic by specifying a hook function that is called for each | ||
6770 | package. It is also perfectly acceptable to call ``do_split_packages`` | ||
6771 | multiple times if you have more than one set of modules to package. | ||
6772 | |||
6773 | For more examples that show how to use ``do_split_packages``, see the | ||
6774 | ``connman.inc`` file in the ``meta/recipes-connectivity/connman/`` | ||
6775 | directory of the ``poky`` :ref:`source repository <overview-manual/development-environment:yocto project source repositories>`. You can | ||
6776 | also find examples in ``meta/classes/kernel.bbclass``. | ||
6777 | |||
6778 | Following is a reference that shows ``do_split_packages`` mandatory and | ||
6779 | optional arguments: | ||
6780 | :: | ||
6781 | |||
6782 | Mandatory arguments | ||
6783 | |||
6784 | root | ||
6785 | The path in which to search | ||
6786 | file_regex | ||
6787 | Regular expression to match searched files. | ||
6788 | Use parentheses () to mark the part of this | ||
6789 | expression that should be used to derive the | ||
6790 | module name (to be substituted where %s is | ||
6791 | used in other function arguments as noted below) | ||
6792 | output_pattern | ||
6793 | Pattern to use for the package names. Must | ||
6794 | include %s. | ||
6795 | description | ||
6796 | Description to set for each package. Must | ||
6797 | include %s. | ||
6798 | |||
6799 | Optional arguments | ||
6800 | |||
6801 | postinst | ||
6802 | Postinstall script to use for all packages | ||
6803 | (as a string) | ||
6804 | recursive | ||
6805 | True to perform a recursive search - default | ||
6806 | False | ||
6807 | hook | ||
6808 | A hook function to be called for every match. | ||
6809 | The function will be called with the following | ||
6810 | arguments (in the order listed): | ||
6811 | |||
6812 | f | ||
6813 | Full path to the file/directory match | ||
6814 | pkg | ||
6815 | The package name | ||
6816 | file_regex | ||
6817 | As above | ||
6818 | output_pattern | ||
6819 | As above | ||
6820 | modulename | ||
6821 | The module name derived using file_regex | ||
6822 | extra_depends | ||
6823 | Extra runtime dependencies (RDEPENDS) to be | ||
6824 | set for all packages. The default value of None | ||
6825 | causes a dependency on the main package | ||
6826 | (${PN}) - if you do not want this, pass empty | ||
6827 | string '' for this parameter. | ||
6828 | aux_files_pattern | ||
6829 | Extra item(s) to be added to FILES for each | ||
6830 | package. Can be a single string item or a list | ||
6831 | of strings for multiple items. Must include %s. | ||
6832 | postrm | ||
6833 | postrm script to use for all packages (as a | ||
6834 | string) | ||
6835 | allow_dirs | ||
6836 | True to allow directories to be matched - | ||
6837 | default False | ||
6838 | prepend | ||
6839 | If True, prepend created packages to PACKAGES | ||
6840 | instead of the default False which appends them | ||
6841 | match_path | ||
6842 | match file_regex on the whole relative path to | ||
6843 | the root rather than just the file name | ||
6844 | aux_files_pattern_verbatim | ||
6845 | Extra item(s) to be added to FILES for each | ||
6846 | package, using the actual derived module name | ||
6847 | rather than converting it to something legal | ||
6848 | for a package name. Can be a single string item | ||
6849 | or a list of strings for multiple items. Must | ||
6850 | include %s. | ||
6851 | allow_links | ||
6852 | True to allow symlinks to be matched - default | ||
6853 | False | ||
6854 | summary | ||
6855 | Summary to set for each package. Must include %s; | ||
6856 | defaults to description if not set. | ||
6857 | |||
6858 | |||
6859 | |||
6860 | Satisfying Dependencies | ||
6861 | ~~~~~~~~~~~~~~~~~~~~~~~ | ||
6862 | |||
6863 | The second part for handling optional module packaging is to ensure that | ||
6864 | any dependencies on optional modules from other recipes are satisfied by | ||
6865 | your recipe. You can be sure these dependencies are satisfied by using | ||
6866 | the :term:`PACKAGES_DYNAMIC` | ||
6867 | variable. Here is an example that continues with the ``lighttpd`` recipe | ||
6868 | shown earlier: | ||
6869 | :: | ||
6870 | |||
6871 | PACKAGES_DYNAMIC = "lighttpd-module-.*" | ||
6872 | |||
6873 | The name | ||
6874 | specified in the regular expression can of course be anything. In this | ||
6875 | example, it is ``lighttpd-module-`` and is specified as the prefix to | ||
6876 | ensure that any :term:`RDEPENDS` and | ||
6877 | :term:`RRECOMMENDS` on a package | ||
6878 | name starting with the prefix are satisfied during build time. If you | ||
6879 | are using ``do_split_packages`` as described in the previous section, | ||
6880 | the value you put in ``PACKAGES_DYNAMIC`` should correspond to the name | ||
6881 | pattern specified in the call to ``do_split_packages``. | ||
6882 | |||
6883 | Using Runtime Package Management | ||
6884 | -------------------------------- | ||
6885 | |||
6886 | During a build, BitBake always transforms a recipe into one or more | ||
6887 | packages. For example, BitBake takes the ``bash`` recipe and produces a | ||
6888 | number of packages (e.g. ``bash``, ``bash-bashbug``, | ||
6889 | ``bash-completion``, ``bash-completion-dbg``, ``bash-completion-dev``, | ||
6890 | ``bash-completion-extra``, ``bash-dbg``, and so forth). Not all | ||
6891 | generated packages are included in an image. | ||
6892 | |||
6893 | In several situations, you might need to update, add, remove, or query | ||
6894 | the packages on a target device at runtime (i.e. without having to | ||
6895 | generate a new image). Examples of such situations include: | ||
6896 | |||
6897 | - You want to provide in-the-field updates to deployed devices (e.g. | ||
6898 | security updates). | ||
6899 | |||
6900 | - You want to have a fast turn-around development cycle for one or more | ||
6901 | applications that run on your device. | ||
6902 | |||
6903 | - You want to temporarily install the "debug" packages of various | ||
6904 | applications on your device so that debugging can be greatly improved | ||
6905 | by allowing access to symbols and source debugging. | ||
6906 | |||
6907 | - You want to deploy a more minimal package selection of your device | ||
6908 | but allow in-the-field updates to add a larger selection for | ||
6909 | customization. | ||
6910 | |||
6911 | In all these situations, you have something similar to a more | ||
6912 | traditional Linux distribution in that in-field devices are able to | ||
6913 | receive pre-compiled packages from a server for installation or update. | ||
6914 | Being able to install these packages on a running, in-field device is | ||
6915 | what is termed "runtime package management". | ||
6916 | |||
6917 | In order to use runtime package management, you need a host or server | ||
6918 | machine that serves up the pre-compiled packages plus the required | ||
6919 | metadata. You also need package manipulation tools on the target. The | ||
6920 | build machine is a likely candidate to act as the server. However, that | ||
6921 | machine does not necessarily have to be the package server. The build | ||
6922 | machine could push its artifacts to another machine that acts as the | ||
6923 | server (e.g. Internet-facing). In fact, doing so is advantageous for a | ||
6924 | production environment as getting the packages away from the development | ||
6925 | system's build directory prevents accidental overwrites. | ||
6926 | |||
6927 | A simple build that targets just one device produces more than one | ||
6928 | package database. In other words, the packages produced by a build are | ||
6929 | separated out into a couple of different package groupings based on | ||
6930 | criteria such as the target's CPU architecture, the target board, or the | ||
6931 | C library used on the target. For example, a build targeting the | ||
6932 | ``qemux86`` device produces the following three package databases: | ||
6933 | ``noarch``, ``i586``, and ``qemux86``. If you wanted your ``qemux86`` | ||
6934 | device to be aware of all the packages that were available to it, you | ||
6935 | would need to point it to each of these databases individually. In a | ||
6936 | similar way, a traditional Linux distribution usually is configured to | ||
6937 | be aware of a number of software repositories from which it retrieves | ||
6938 | packages. | ||
6939 | |||
6940 | Using runtime package management is completely optional and not required | ||
6941 | for a successful build or deployment in any way. But if you want to make | ||
6942 | use of runtime package management, you need to do a couple things above | ||
6943 | and beyond the basics. The remainder of this section describes what you | ||
6944 | need to do. | ||
6945 | |||
6946 | Build Considerations | ||
6947 | ~~~~~~~~~~~~~~~~~~~~ | ||
6948 | |||
6949 | This section describes build considerations of which you need to be | ||
6950 | aware in order to provide support for runtime package management. | ||
6951 | |||
6952 | When BitBake generates packages, it needs to know what format or formats | ||
6953 | to use. In your configuration, you use the | ||
6954 | :term:`PACKAGE_CLASSES` | ||
6955 | variable to specify the format: | ||
6956 | |||
6957 | 1. Open the ``local.conf`` file inside your | ||
6958 | :term:`Build Directory` (e.g. | ||
6959 | ``~/poky/build/conf/local.conf``). | ||
6960 | |||
6961 | 2. Select the desired package format as follows: | ||
6962 | :: | ||
6963 | |||
6964 | PACKAGE_CLASSES ?= "package_packageformat" | ||
6965 | |||
6966 | where packageformat can be "ipk", "rpm", | ||
6967 | "deb", or "tar" which are the supported package formats. | ||
6968 | |||
6969 | .. note:: | ||
6970 | |||
6971 | Because the Yocto Project supports four different package formats, | ||
6972 | you can set the variable with more than one argument. However, the | ||
6973 | OpenEmbedded build system only uses the first argument when | ||
6974 | creating an image or Software Development Kit (SDK). | ||
6975 | |||
6976 | If you would like your image to start off with a basic package database | ||
6977 | containing the packages in your current build as well as to have the | ||
6978 | relevant tools available on the target for runtime package management, | ||
6979 | you can include "package-management" in the | ||
6980 | :term:`IMAGE_FEATURES` | ||
6981 | variable. Including "package-management" in this configuration variable | ||
6982 | ensures that when the image is assembled for your target, the image | ||
6983 | includes the currently-known package databases as well as the | ||
6984 | target-specific tools required for runtime package management to be | ||
6985 | performed on the target. However, this is not strictly necessary. You | ||
6986 | could start your image off without any databases but only include the | ||
6987 | required on-target package tool(s). As an example, you could include | ||
6988 | "opkg" in your | ||
6989 | :term:`IMAGE_INSTALL` variable | ||
6990 | if you are using the IPK package format. You can then initialize your | ||
6991 | target's package database(s) later once your image is up and running. | ||
6992 | |||
6993 | Whenever you perform any sort of build step that can potentially | ||
6994 | generate a package or modify existing package, it is always a good idea | ||
6995 | to re-generate the package index after the build by using the following | ||
6996 | command: | ||
6997 | :: | ||
6998 | |||
6999 | $ bitbake package-index | ||
7000 | |||
7001 | It might be tempting to build the | ||
7002 | package and the package index at the same time with a command such as | ||
7003 | the following: | ||
7004 | :: | ||
7005 | |||
7006 | $ bitbake some-package package-index | ||
7007 | |||
7008 | Do not do this as | ||
7009 | BitBake does not schedule the package index for after the completion of | ||
7010 | the package you are building. Consequently, you cannot be sure of the | ||
7011 | package index including information for the package you just built. | ||
7012 | Thus, be sure to run the package update step separately after building | ||
7013 | any packages. | ||
7014 | |||
7015 | You can use the | ||
7016 | :term:`PACKAGE_FEED_ARCHS`, | ||
7017 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7018 | and | ||
7019 | :term:`PACKAGE_FEED_URIS` | ||
7020 | variables to pre-configure target images to use a package feed. If you | ||
7021 | do not define these variables, then manual steps as described in the | ||
7022 | subsequent sections are necessary to configure the target. You should | ||
7023 | set these variables before building the image in order to produce a | ||
7024 | correctly configured image. | ||
7025 | |||
7026 | When your build is complete, your packages reside in the | ||
7027 | ``${TMPDIR}/deploy/packageformat`` directory. For example, if | ||
7028 | ``${``\ :term:`TMPDIR`\ ``}`` is | ||
7029 | ``tmp`` and your selected package type is RPM, then your RPM packages | ||
7030 | are available in ``tmp/deploy/rpm``. | ||
7031 | |||
7032 | Host or Server Machine Setup | ||
7033 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7034 | |||
7035 | Although other protocols are possible, a server using HTTP typically | ||
7036 | serves packages. If you want to use HTTP, then set up and configure a | ||
7037 | web server such as Apache 2, lighttpd, or Python web server on the | ||
7038 | machine serving the packages. | ||
7039 | |||
7040 | To keep things simple, this section describes how to set up a | ||
7041 | Python web server to share package feeds from the developer's | ||
7042 | machine. Although this server might not be the best for a production | ||
7043 | environment, the setup is simple and straight forward. Should you want | ||
7044 | to use a different server more suited for production (e.g. Apache 2, | ||
7045 | Lighttpd, or Nginx), take the appropriate steps to do so. | ||
7046 | |||
7047 | From within the build directory where you have built an image based on | ||
7048 | your packaging choice (i.e. the | ||
7049 | :term:`PACKAGE_CLASSES` | ||
7050 | setting), simply start the server. The following example assumes a build | ||
7051 | directory of ``~/poky/build/tmp/deploy/rpm`` and a ``PACKAGE_CLASSES`` | ||
7052 | setting of "package_rpm": | ||
7053 | :: | ||
7054 | |||
7055 | $ cd ~/poky/build/tmp/deploy/rpm | ||
7056 | $ python3 -m http.server | ||
7057 | |||
7058 | Target Setup | ||
7059 | ~~~~~~~~~~~~ | ||
7060 | |||
7061 | Setting up the target differs depending on the package management | ||
7062 | system. This section provides information for RPM, IPK, and DEB. | ||
7063 | |||
7064 | Using RPM | ||
7065 | ^^^^^^^^^ | ||
7066 | |||
7067 | The `Dandified Packaging | ||
7068 | Tool <https://en.wikipedia.org/wiki/DNF_(software)>`__ (DNF) performs | ||
7069 | runtime package management of RPM packages. In order to use DNF for | ||
7070 | runtime package management, you must perform an initial setup on the | ||
7071 | target machine for cases where the ``PACKAGE_FEED_*`` variables were not | ||
7072 | set as part of the image that is running on the target. This means if | ||
7073 | you built your image and did not not use these variables as part of the | ||
7074 | build and your image is now running on the target, you need to perform | ||
7075 | the steps in this section if you want to use runtime package management. | ||
7076 | |||
7077 | .. note:: | ||
7078 | |||
7079 | For information on the ``PACKAGE_FEED_*`` variables, see | ||
7080 | :term:`PACKAGE_FEED_ARCHS`, :term:`PACKAGE_FEED_BASE_PATHS`, and | ||
7081 | :term:`PACKAGE_FEED_URIS` in the Yocto Project Reference Manual variables | ||
7082 | glossary. | ||
7083 | |||
7084 | On the target, you must inform DNF that package databases are available. | ||
7085 | You do this by creating a file named | ||
7086 | ``/etc/yum.repos.d/oe-packages.repo`` and defining the ``oe-packages``. | ||
7087 | |||
7088 | As an example, assume the target is able to use the following package | ||
7089 | databases: ``all``, ``i586``, and ``qemux86`` from a server named | ||
7090 | ``my.server``. The specifics for setting up the web server are up to | ||
7091 | you. The critical requirement is that the URIs in the target repository | ||
7092 | configuration point to the correct remote location for the feeds. | ||
7093 | |||
7094 | .. note:: | ||
7095 | |||
7096 | For development purposes, you can point the web server to the build | ||
7097 | system's ``deploy`` directory. However, for production use, it is better to | ||
7098 | copy the package directories to a location outside of the build area and use | ||
7099 | that location. Doing so avoids situations where the build system | ||
7100 | overwrites or changes the ``deploy`` directory. | ||
7101 | |||
7102 | When telling DNF where to look for the package databases, you must | ||
7103 | declare individual locations per architecture or a single location used | ||
7104 | for all architectures. You cannot do both: | ||
7105 | |||
7106 | - *Create an Explicit List of Architectures:* Define individual base | ||
7107 | URLs to identify where each package database is located: | ||
7108 | |||
7109 | .. code-block:: none | ||
7110 | |||
7111 | [oe-packages] | ||
7112 | baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all | ||
7113 | |||
7114 | This example | ||
7115 | informs DNF about individual package databases for all three | ||
7116 | architectures. | ||
7117 | |||
7118 | - *Create a Single (Full) Package Index:* Define a single base URL that | ||
7119 | identifies where a full package database is located: | ||
7120 | :: | ||
7121 | |||
7122 | [oe-packages] | ||
7123 | baseurl=http://my.server/rpm | ||
7124 | |||
7125 | This example informs DNF about a single | ||
7126 | package database that contains all the package index information for | ||
7127 | all supported architectures. | ||
7128 | |||
7129 | Once you have informed DNF where to find the package databases, you need | ||
7130 | to fetch them: | ||
7131 | |||
7132 | .. code-block:: none | ||
7133 | |||
7134 | # dnf makecache | ||
7135 | |||
7136 | DNF is now able to find, install, and | ||
7137 | upgrade packages from the specified repository or repositories. | ||
7138 | |||
7139 | .. note:: | ||
7140 | |||
7141 | See the `DNF documentation <https://dnf.readthedocs.io/en/latest/>`__ for | ||
7142 | additional information. | ||
7143 | |||
7144 | Using IPK | ||
7145 | ^^^^^^^^^ | ||
7146 | |||
7147 | The ``opkg`` application performs runtime package management of IPK | ||
7148 | packages. You must perform an initial setup for ``opkg`` on the target | ||
7149 | machine if the | ||
7150 | :term:`PACKAGE_FEED_ARCHS`, | ||
7151 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7152 | and | ||
7153 | :term:`PACKAGE_FEED_URIS` | ||
7154 | variables have not been set or the target image was built before the | ||
7155 | variables were set. | ||
7156 | |||
7157 | The ``opkg`` application uses configuration files to find available | ||
7158 | package databases. Thus, you need to create a configuration file inside | ||
7159 | the ``/etc/opkg/`` direction, which informs ``opkg`` of any repository | ||
7160 | you want to use. | ||
7161 | |||
7162 | As an example, suppose you are serving packages from a ``ipk/`` | ||
7163 | directory containing the ``i586``, ``all``, and ``qemux86`` databases | ||
7164 | through an HTTP server named ``my.server``. On the target, create a | ||
7165 | configuration file (e.g. ``my_repo.conf``) inside the ``/etc/opkg/`` | ||
7166 | directory containing the following: | ||
7167 | |||
7168 | .. code-block:: none | ||
7169 | |||
7170 | src/gz all http://my.server/ipk/all | ||
7171 | src/gz i586 http://my.server/ipk/i586 | ||
7172 | src/gz qemux86 http://my.server/ipk/qemux86 | ||
7173 | |||
7174 | Next, instruct ``opkg`` to fetch the | ||
7175 | repository information: | ||
7176 | |||
7177 | .. code-block:: none | ||
7178 | |||
7179 | # opkg update | ||
7180 | |||
7181 | The ``opkg`` application is now able to find, install, and upgrade packages | ||
7182 | from the specified repository. | ||
7183 | |||
7184 | Using DEB | ||
7185 | ^^^^^^^^^ | ||
7186 | |||
7187 | The ``apt`` application performs runtime package management of DEB | ||
7188 | packages. This application uses a source list file to find available | ||
7189 | package databases. You must perform an initial setup for ``apt`` on the | ||
7190 | target machine if the | ||
7191 | :term:`PACKAGE_FEED_ARCHS`, | ||
7192 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7193 | and | ||
7194 | :term:`PACKAGE_FEED_URIS` | ||
7195 | variables have not been set or the target image was built before the | ||
7196 | variables were set. | ||
7197 | |||
7198 | To inform ``apt`` of the repository you want to use, you might create a | ||
7199 | list file (e.g. ``my_repo.list``) inside the | ||
7200 | ``/etc/apt/sources.list.d/`` directory. As an example, suppose you are | ||
7201 | serving packages from a ``deb/`` directory containing the ``i586``, | ||
7202 | ``all``, and ``qemux86`` databases through an HTTP server named | ||
7203 | ``my.server``. The list file should contain: | ||
7204 | |||
7205 | .. code-block:: none | ||
7206 | |||
7207 | deb http://my.server/deb/all ./ | ||
7208 | deb http://my.server/deb/i586 ./ | ||
7209 | deb http://my.server/deb/qemux86 ./ | ||
7210 | |||
7211 | Next, instruct the ``apt`` application | ||
7212 | to fetch the repository information: | ||
7213 | |||
7214 | .. code-block:: none | ||
7215 | |||
7216 | # apt-get update | ||
7217 | |||
7218 | After this step, | ||
7219 | ``apt`` is able to find, install, and upgrade packages from the | ||
7220 | specified repository. | ||
7221 | |||
7222 | Generating and Using Signed Packages | ||
7223 | ------------------------------------ | ||
7224 | |||
7225 | In order to add security to RPM packages used during a build, you can | ||
7226 | take steps to securely sign them. Once a signature is verified, the | ||
7227 | OpenEmbedded build system can use the package in the build. If security | ||
7228 | fails for a signed package, the build system aborts the build. | ||
7229 | |||
7230 | This section describes how to sign RPM packages during a build and how | ||
7231 | to use signed package feeds (repositories) when doing a build. | ||
7232 | |||
7233 | Signing RPM Packages | ||
7234 | ~~~~~~~~~~~~~~~~~~~~ | ||
7235 | |||
7236 | To enable signing RPM packages, you must set up the following | ||
7237 | configurations in either your ``local.config`` or ``distro.config`` | ||
7238 | file: | ||
7239 | :: | ||
7240 | |||
7241 | # Inherit sign_rpm.bbclass to enable signing functionality | ||
7242 | INHERIT += " sign_rpm" | ||
7243 | # Define the GPG key that will be used for signing. | ||
7244 | RPM_GPG_NAME = "key_name" | ||
7245 | # Provide passphrase for the key | ||
7246 | RPM_GPG_PASSPHRASE = "passphrase" | ||
7247 | |||
7248 | .. note:: | ||
7249 | |||
7250 | Be sure to supply appropriate values for both `key_name` and | ||
7251 | `passphrase`. | ||
7252 | |||
7253 | Aside from the ``RPM_GPG_NAME`` and ``RPM_GPG_PASSPHRASE`` variables in | ||
7254 | the previous example, two optional variables related to signing exist: | ||
7255 | |||
7256 | - *GPG_BIN:* Specifies a ``gpg`` binary/wrapper that is executed | ||
7257 | when the package is signed. | ||
7258 | |||
7259 | - *GPG_PATH:* Specifies the ``gpg`` home directory used when the | ||
7260 | package is signed. | ||
7261 | |||
7262 | Processing Package Feeds | ||
7263 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7264 | |||
7265 | In addition to being able to sign RPM packages, you can also enable | ||
7266 | signed package feeds for IPK and RPM packages. | ||
7267 | |||
7268 | The steps you need to take to enable signed package feed use are similar | ||
7269 | to the steps used to sign RPM packages. You must define the following in | ||
7270 | your ``local.config`` or ``distro.config`` file: | ||
7271 | :: | ||
7272 | |||
7273 | INHERIT += "sign_package_feed" | ||
7274 | PACKAGE_FEED_GPG_NAME = "key_name" | ||
7275 | PACKAGE_FEED_GPG_PASSPHRASE_FILE = "path_to_file_containing_passphrase" | ||
7276 | |||
7277 | For signed package feeds, the passphrase must exist in a separate file, | ||
7278 | which is pointed to by the ``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` | ||
7279 | variable. Regarding security, keeping a plain text passphrase out of the | ||
7280 | configuration is more secure. | ||
7281 | |||
7282 | Aside from the ``PACKAGE_FEED_GPG_NAME`` and | ||
7283 | ``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` variables, three optional variables | ||
7284 | related to signed package feeds exist: | ||
7285 | |||
7286 | - *GPG_BIN* Specifies a ``gpg`` binary/wrapper that is executed | ||
7287 | when the package is signed. | ||
7288 | |||
7289 | - *GPG_PATH:* Specifies the ``gpg`` home directory used when the | ||
7290 | package is signed. | ||
7291 | |||
7292 | - *PACKAGE_FEED_GPG_SIGNATURE_TYPE:* Specifies the type of ``gpg`` | ||
7293 | signature. This variable applies only to RPM and IPK package feeds. | ||
7294 | Allowable values for the ``PACKAGE_FEED_GPG_SIGNATURE_TYPE`` are | ||
7295 | "ASC", which is the default and specifies ascii armored, and "BIN", | ||
7296 | which specifies binary. | ||
7297 | |||
7298 | Testing Packages With ptest | ||
7299 | --------------------------- | ||
7300 | |||
7301 | A Package Test (ptest) runs tests against packages built by the | ||
7302 | OpenEmbedded build system on the target machine. A ptest contains at | ||
7303 | least two items: the actual test, and a shell script (``run-ptest``) | ||
7304 | that starts the test. The shell script that starts the test must not | ||
7305 | contain the actual test - the script only starts the test. On the other | ||
7306 | hand, the test can be anything from a simple shell script that runs a | ||
7307 | binary and checks the output to an elaborate system of test binaries and | ||
7308 | data files. | ||
7309 | |||
7310 | The test generates output in the format used by Automake: | ||
7311 | :: | ||
7312 | |||
7313 | result: testname | ||
7314 | |||
7315 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
7316 | the testname can be any identifying string. | ||
7317 | |||
7318 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
7319 | see the :yocto_wiki:`Ptest </Ptest>` wiki page. | ||
7320 | |||
7321 | .. note:: | ||
7322 | |||
7323 | A recipe is "ptest-enabled" if it inherits the | ||
7324 | :ref:`ptest <ref-classes-ptest>` class. | ||
7325 | |||
7326 | Adding ptest to Your Build | ||
7327 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7328 | |||
7329 | To add package testing to your build, add the | ||
7330 | :term:`DISTRO_FEATURES` and | ||
7331 | :term:`EXTRA_IMAGE_FEATURES` | ||
7332 | variables to your ``local.conf`` file, which is found in the | ||
7333 | :term:`Build Directory`: | ||
7334 | :: | ||
7335 | |||
7336 | DISTRO_FEATURES_append = " ptest" | ||
7337 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
7338 | |||
7339 | Once your build is complete, the ptest files are installed into the | ||
7340 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
7341 | is the name of the package. | ||
7342 | |||
7343 | Running ptest | ||
7344 | ~~~~~~~~~~~~~ | ||
7345 | |||
7346 | The ``ptest-runner`` package installs a shell script that loops through | ||
7347 | all installed ptest test suites and runs them in sequence. Consequently, | ||
7348 | you might want to add this package to your image. | ||
7349 | |||
7350 | Getting Your Package Ready | ||
7351 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7352 | |||
7353 | In order to enable a recipe to run installed ptests on target hardware, | ||
7354 | you need to prepare the recipes that build the packages you want to | ||
7355 | test. Here is what you have to do for each recipe: | ||
7356 | |||
7357 | - *Be sure the recipe inherits | ||
7358 | the* :ref:`ptest <ref-classes-ptest>` *class:* | ||
7359 | Include the following line in each recipe: | ||
7360 | :: | ||
7361 | |||
7362 | inherit ptest | ||
7363 | |||
7364 | - *Create run-ptest:* This script starts your test. Locate the | ||
7365 | script where you will refer to it using | ||
7366 | :term:`SRC_URI`. Here is an | ||
7367 | example that starts a test for ``dbus``: | ||
7368 | :: | ||
7369 | |||
7370 | #!/bin/sh | ||
7371 | cd test | ||
7372 | make -k runtest-TESTS | ||
7373 | |||
7374 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
7375 | dependencies that normally do not exist for the package (such as | ||
7376 | requiring "make" to run the test suite), use the | ||
7377 | :term:`DEPENDS` and | ||
7378 | :term:`RDEPENDS` variables in | ||
7379 | your recipe in order for the package to meet the dependencies. Here | ||
7380 | is an example where the package has a runtime dependency on "make": | ||
7381 | :: | ||
7382 | |||
7383 | RDEPENDS_${PN}-ptest += "make" | ||
7384 | |||
7385 | - *Add a function to build the test suite:* Not many packages support | ||
7386 | cross-compilation of their test suites. Consequently, you usually | ||
7387 | need to add a cross-compilation function to the package. | ||
7388 | |||
7389 | Many packages based on Automake compile and run the test suite by | ||
7390 | using a single command such as ``make check``. However, the host | ||
7391 | ``make check`` builds and runs on the same computer, while | ||
7392 | cross-compiling requires that the package is built on the host but | ||
7393 | executed for the target architecture (though often, as in the case | ||
7394 | for ptest, the execution occurs on the host). The built version of | ||
7395 | Automake that ships with the Yocto Project includes a patch that | ||
7396 | separates building and execution. Consequently, packages that use the | ||
7397 | unaltered, patched version of ``make check`` automatically | ||
7398 | cross-compiles. | ||
7399 | |||
7400 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
7401 | build the test suite. Add a function similar to the following to your | ||
7402 | recipe: | ||
7403 | :: | ||
7404 | |||
7405 | do_compile_ptest() { | ||
7406 | oe_runmake buildtest-TESTS | ||
7407 | } | ||
7408 | |||
7409 | - *Ensure special configurations are set:* If the package requires | ||
7410 | special configurations prior to compiling the test code, you must | ||
7411 | insert a ``do_configure_ptest`` function into the recipe. | ||
7412 | |||
7413 | - *Install the test suite:* The ``ptest`` class automatically copies | ||
7414 | the file ``run-ptest`` to the target and then runs make | ||
7415 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
7416 | to create a ``do_install_ptest`` function and make sure it gets | ||
7417 | called after the "make install-ptest" completes. | ||
7418 | |||
7419 | Creating Node Package Manager (NPM) Packages | ||
7420 | -------------------------------------------- | ||
7421 | |||
7422 | `NPM <https://en.wikipedia.org/wiki/Npm_(software)>`__ is a package | ||
7423 | manager for the JavaScript programming language. The Yocto Project | ||
7424 | supports the NPM :ref:`fetcher <bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`. You can | ||
7425 | use this fetcher in combination with | ||
7426 | :doc:`devtool </ref-manual/devtool-reference>` to create | ||
7427 | recipes that produce NPM packages. | ||
7428 | |||
7429 | Two workflows exist that allow you to create NPM packages using | ||
7430 | ``devtool``: the NPM registry modules method and the NPM project code | ||
7431 | method. | ||
7432 | |||
7433 | .. note:: | ||
7434 | |||
7435 | While it is possible to create NPM recipes manually, using | ||
7436 | ``devtool`` is far simpler. | ||
7437 | |||
7438 | Additionally, some requirements and caveats exist. | ||
7439 | |||
7440 | Requirements and Caveats | ||
7441 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7442 | |||
7443 | You need to be aware of the following before using ``devtool`` to create | ||
7444 | NPM packages: | ||
7445 | |||
7446 | - Of the two methods that you can use ``devtool`` to create NPM | ||
7447 | packages, the registry approach is slightly simpler. However, you | ||
7448 | might consider the project approach because you do not have to | ||
7449 | publish your module in the NPM registry | ||
7450 | (`npm-registry <https://docs.npmjs.com/misc/registry>`_), which | ||
7451 | is NPM's public registry. | ||
7452 | |||
7453 | - Be familiar with | ||
7454 | :doc:`devtool </ref-manual/devtool-reference>`. | ||
7455 | |||
7456 | - The NPM host tools need the native ``nodejs-npm`` package, which is | ||
7457 | part of the OpenEmbedded environment. You need to get the package by | ||
7458 | cloning the https://github.com/openembedded/meta-openembedded | ||
7459 | repository out of GitHub. Be sure to add the path to your local copy | ||
7460 | to your ``bblayers.conf`` file. | ||
7461 | |||
7462 | - ``devtool`` cannot detect native libraries in module dependencies. | ||
7463 | Consequently, you must manually add packages to your recipe. | ||
7464 | |||
7465 | - While deploying NPM packages, ``devtool`` cannot determine which | ||
7466 | dependent packages are missing on the target (e.g. the node runtime | ||
7467 | ``nodejs``). Consequently, you need to find out what files are | ||
7468 | missing and be sure they are on the target. | ||
7469 | |||
7470 | - Although you might not need NPM to run your node package, it is | ||
7471 | useful to have NPM on your target. The NPM package name is | ||
7472 | ``nodejs-npm``. | ||
7473 | |||
7474 | Using the Registry Modules Method | ||
7475 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7476 | |||
7477 | This section presents an example that uses the ``cute-files`` module, | ||
7478 | which is a file browser web application. | ||
7479 | |||
7480 | .. note:: | ||
7481 | |||
7482 | You must know the ``cute-files`` module version. | ||
7483 | |||
7484 | The first thing you need to do is use ``devtool`` and the NPM fetcher to | ||
7485 | create the recipe: | ||
7486 | :: | ||
7487 | |||
7488 | $ devtool add "npm://registry.npmjs.org;package=cute-files;version=1.0.2" | ||
7489 | |||
7490 | The | ||
7491 | ``devtool add`` command runs ``recipetool create`` and uses the same | ||
7492 | fetch URI to download each dependency and capture license details where | ||
7493 | possible. The result is a generated recipe. | ||
7494 | |||
7495 | The recipe file is fairly simple and contains every license that | ||
7496 | ``recipetool`` finds and includes the licenses in the recipe's | ||
7497 | :term:`LIC_FILES_CHKSUM` | ||
7498 | variables. You need to examine the variables and look for those with | ||
7499 | "unknown" in the :term:`LICENSE` | ||
7500 | field. You need to track down the license information for "unknown" | ||
7501 | modules and manually add the information to the recipe. | ||
7502 | |||
7503 | ``recipetool`` creates a "shrinkwrap" file for your recipe. Shrinkwrap | ||
7504 | files capture the version of all dependent modules. Many packages do not | ||
7505 | provide shrinkwrap files. ``recipetool`` create a shrinkwrap file as it | ||
7506 | runs. | ||
7507 | |||
7508 | .. note:: | ||
7509 | |||
7510 | A package is created for each sub-module. This policy is the only | ||
7511 | practical way to have the licenses for all of the dependencies | ||
7512 | represented in the license manifest of the image. | ||
7513 | |||
7514 | The ``devtool edit-recipe`` command lets you take a look at the recipe: | ||
7515 | :: | ||
7516 | |||
7517 | $ devtool edit-recipe cute-files | ||
7518 | SUMMARY = "Turn any folder on your computer into a cute file browser, available on the local network." | ||
7519 | LICENSE = "MIT & ISC & Unknown" | ||
7520 | LIC_FILES_CHKSUM = "file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \ | ||
7521 | file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502 \ | ||
7522 | file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99 \ | ||
7523 | ... | ||
7524 | SRC_URI = " \ | ||
7525 | npm://registry.npmjs.org/;package=cute-files;version=${PV} \ | ||
7526 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ | ||
7527 | " | ||
7528 | S = "${WORKDIR}/npm" | ||
7529 | inherit npm LICENSE_${PN} = "MIT" | ||
7530 | LICENSE_${PN}-accepts = "MIT" | ||
7531 | LICENSE_${PN}-array-flatten = "MIT" | ||
7532 | ... | ||
7533 | LICENSE_${PN}-vary = "MIT" | ||
7534 | |||
7535 | Three key points exist in the previous example: | ||
7536 | |||
7537 | - :term:`SRC_URI` uses the NPM | ||
7538 | scheme so that the NPM fetcher is used. | ||
7539 | |||
7540 | - ``recipetool`` collects all the license information. If a | ||
7541 | sub-module's license is unavailable, the sub-module's name appears in | ||
7542 | the comments. | ||
7543 | |||
7544 | - The ``inherit npm`` statement causes the | ||
7545 | :ref:`npm <ref-classes-npm>` class to package | ||
7546 | up all the modules. | ||
7547 | |||
7548 | You can run the following command to build the ``cute-files`` package: | ||
7549 | :: | ||
7550 | |||
7551 | $ devtool build cute-files | ||
7552 | |||
7553 | Remember that ``nodejs`` must be installed on | ||
7554 | the target before your package. | ||
7555 | |||
7556 | Assuming 192.168.7.2 for the target's IP address, use the following | ||
7557 | command to deploy your package: | ||
7558 | :: | ||
7559 | |||
7560 | $ devtool deploy-target -s cute-files root@192.168.7.2 | ||
7561 | |||
7562 | Once the package is installed on the target, you can | ||
7563 | test the application: | ||
7564 | |||
7565 | .. note:: | ||
7566 | |||
7567 | Because of a known issue, you cannot simply run ``cute-files`` as you would | ||
7568 | if you had run ``npm install``. | ||
7569 | |||
7570 | :: | ||
7571 | |||
7572 | $ cd /usr/lib/node_modules/cute-files | ||
7573 | $ node cute-files.js | ||
7574 | |||
7575 | On a browser, | ||
7576 | go to ``http://192.168.7.2:3000`` and you see the following: | ||
7577 | |||
7578 | .. image:: figures/cute-files-npm-example.png | ||
7579 | :align: center | ||
7580 | |||
7581 | You can find the recipe in ``workspace/recipes/cute-files``. You can use | ||
7582 | the recipe in any layer you choose. | ||
7583 | |||
7584 | Using the NPM Projects Code Method | ||
7585 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7586 | |||
7587 | Although it is useful to package modules already in the NPM registry, | ||
7588 | adding ``node.js`` projects under development is a more common developer | ||
7589 | use case. | ||
7590 | |||
7591 | This section covers the NPM projects code method, which is very similar | ||
7592 | to the "registry" approach described in the previous section. In the NPM | ||
7593 | projects method, you provide ``devtool`` with an URL that points to the | ||
7594 | source files. | ||
7595 | |||
7596 | Replicating the same example, (i.e. ``cute-files``) use the following | ||
7597 | command: | ||
7598 | :: | ||
7599 | |||
7600 | $ devtool add https://github.com/martinaglv/cute-files.git | ||
7601 | |||
7602 | The | ||
7603 | recipe this command generates is very similar to the recipe created in | ||
7604 | the previous section. However, the ``SRC_URI`` looks like the following: | ||
7605 | :: | ||
7606 | |||
7607 | SRC_URI = " \ | ||
7608 | git://github.com/martinaglv/cute-files.git;protocol=https \ | ||
7609 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ | ||
7610 | " | ||
7611 | |||
7612 | In this example, | ||
7613 | the main module is taken from the Git repository and dependencies are | ||
7614 | taken from the NPM registry. Other than those differences, the recipe is | ||
7615 | basically the same between the two methods. You can build and deploy the | ||
7616 | package exactly as described in the previous section that uses the | ||
7617 | registry modules method. | ||
7618 | |||
7619 | Adding custom metadata to packages | ||
7620 | ---------------------------------- | ||
7621 | |||
7622 | The variable | ||
7623 | :term:`PACKAGE_ADD_METADATA` | ||
7624 | can be used to add additional metadata to packages. This is reflected in | ||
7625 | the package control/spec file. To take the ipk format for example, the | ||
7626 | CONTROL file stored inside would contain the additional metadata as | ||
7627 | additional lines. | ||
7628 | |||
7629 | The variable can be used in multiple ways, including using suffixes to | ||
7630 | set it for a specific package type and/or package. Note that the order | ||
7631 | of precedence is the same as this list: | ||
7632 | |||
7633 | - ``PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>`` | ||
7634 | |||
7635 | - ``PACKAGE_ADD_METADATA_<PKGTYPE>`` | ||
7636 | |||
7637 | - ``PACKAGE_ADD_METADATA_<PN>`` | ||
7638 | |||
7639 | - ``PACKAGE_ADD_METADATA`` | ||
7640 | |||
7641 | `<PKGTYPE>` is a parameter and expected to be a distinct name of specific | ||
7642 | package type: | ||
7643 | |||
7644 | - IPK for .ipk packages | ||
7645 | |||
7646 | - DEB for .deb packages | ||
7647 | |||
7648 | - RPM for .rpm packages | ||
7649 | |||
7650 | `<PN>` is a parameter and expected to be a package name. | ||
7651 | |||
7652 | The variable can contain multiple [one-line] metadata fields separated | ||
7653 | by the literal sequence '\\n'. The separator can be redefined using the | ||
7654 | variable flag ``separator``. | ||
7655 | |||
7656 | The following is an example that adds two custom fields for ipk | ||
7657 | packages: | ||
7658 | :: | ||
7659 | |||
7660 | PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup:Applications/Spreadsheets" | ||
7661 | |||
7662 | Efficiently Fetching Source Files During a Build | ||
7663 | ================================================ | ||
7664 | |||
7665 | The OpenEmbedded build system works with source files located through | ||
7666 | the :term:`SRC_URI` variable. When | ||
7667 | you build something using BitBake, a big part of the operation is | ||
7668 | locating and downloading all the source tarballs. For images, | ||
7669 | downloading all the source for various packages can take a significant | ||
7670 | amount of time. | ||
7671 | |||
7672 | This section shows you how you can use mirrors to speed up fetching | ||
7673 | source files and how you can pre-fetch files all of which leads to more | ||
7674 | efficient use of resources and time. | ||
7675 | |||
7676 | Setting up Effective Mirrors | ||
7677 | ---------------------------- | ||
7678 | |||
7679 | A good deal that goes into a Yocto Project build is simply downloading | ||
7680 | all of the source tarballs. Maybe you have been working with another | ||
7681 | build system (OpenEmbedded or Angstrom) for which you have built up a | ||
7682 | sizable directory of source tarballs. Or, perhaps someone else has such | ||
7683 | a directory for which you have read access. If so, you can save time by | ||
7684 | adding statements to your configuration file so that the build process | ||
7685 | checks local directories first for existing tarballs before checking the | ||
7686 | Internet. | ||
7687 | |||
7688 | Here is an efficient way to set it up in your ``local.conf`` file: | ||
7689 | :: | ||
7690 | |||
7691 | SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" | ||
7692 | INHERIT += "own-mirrors" | ||
7693 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
7694 | # BB_NO_NETWORK = "1" | ||
7695 | |||
7696 | In the previous example, the | ||
7697 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
7698 | variable causes the OpenEmbedded build system to generate tarballs of | ||
7699 | the Git repositories and store them in the | ||
7700 | :term:`DL_DIR` directory. Due to | ||
7701 | performance reasons, generating and storing these tarballs is not the | ||
7702 | build system's default behavior. | ||
7703 | |||
7704 | You can also use the | ||
7705 | :term:`PREMIRRORS` variable. For | ||
7706 | an example, see the variable's glossary entry in the Yocto Project | ||
7707 | Reference Manual. | ||
7708 | |||
7709 | Getting Source Files and Suppressing the Build | ||
7710 | ---------------------------------------------- | ||
7711 | |||
7712 | Another technique you can use to ready yourself for a successive string | ||
7713 | of build operations, is to pre-fetch all the source files without | ||
7714 | actually starting a build. This technique lets you work through any | ||
7715 | download issues and ultimately gathers all the source files into your | ||
7716 | download directory :ref:`structure-build-downloads`, | ||
7717 | which is located with :term:`DL_DIR`. | ||
7718 | |||
7719 | Use the following BitBake command form to fetch all the necessary | ||
7720 | sources without starting the build: | ||
7721 | :: | ||
7722 | |||
7723 | $ bitbake target --runall=fetch | ||
7724 | |||
7725 | This | ||
7726 | variation of the BitBake command guarantees that you have all the | ||
7727 | sources for that BitBake target should you disconnect from the Internet | ||
7728 | and want to do the build later offline. | ||
7729 | |||
7730 | Selecting an Initialization Manager | ||
7731 | =================================== | ||
7732 | |||
7733 | By default, the Yocto Project uses SysVinit as the initialization | ||
7734 | manager. However, support also exists for systemd, which is a full | ||
7735 | replacement for init with parallel starting of services, reduced shell | ||
7736 | overhead and other features that are used by many distributions. | ||
7737 | |||
7738 | Within the system, SysVinit treats system components as services. These | ||
7739 | services are maintained as shell scripts stored in the ``/etc/init.d/`` | ||
7740 | directory. Services organize into different run levels. This | ||
7741 | organization is maintained by putting links to the services in the | ||
7742 | ``/etc/rcN.d/`` directories, where `N/` is one of the following options: | ||
7743 | "S", "0", "1", "2", "3", "4", "5", or "6". | ||
7744 | |||
7745 | .. note:: | ||
7746 | |||
7747 | Each runlevel has a dependency on the previous runlevel. This | ||
7748 | dependency allows the services to work properly. | ||
7749 | |||
7750 | In comparison, systemd treats components as units. Using units is a | ||
7751 | broader concept as compared to using a service. A unit includes several | ||
7752 | different types of entities. Service is one of the types of entities. | ||
7753 | The runlevel concept in SysVinit corresponds to the concept of a target | ||
7754 | in systemd, where target is also a type of supported unit. | ||
7755 | |||
7756 | In a SysVinit-based system, services load sequentially (i.e. one by one) | ||
7757 | during init and parallelization is not supported. With systemd, services | ||
7758 | start in parallel. Needless to say, the method can have an impact on | ||
7759 | system startup performance. | ||
7760 | |||
7761 | If you want to use SysVinit, you do not have to do anything. But, if you | ||
7762 | want to use systemd, you must take some steps as described in the | ||
7763 | following sections. | ||
7764 | |||
7765 | Using systemd Exclusively | ||
7766 | ------------------------- | ||
7767 | |||
7768 | Set these variables in your distribution configuration file as follows: | ||
7769 | :: | ||
7770 | |||
7771 | DISTRO_FEATURES_append = " systemd" | ||
7772 | VIRTUAL-RUNTIME_init_manager = "systemd" | ||
7773 | |||
7774 | You can also prevent the SysVinit distribution feature from | ||
7775 | being automatically enabled as follows: | ||
7776 | :: | ||
7777 | |||
7778 | DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" | ||
7779 | |||
7780 | Doing so removes any | ||
7781 | redundant SysVinit scripts. | ||
7782 | |||
7783 | To remove initscripts from your image altogether, set this variable | ||
7784 | also: | ||
7785 | :: | ||
7786 | |||
7787 | VIRTUAL-RUNTIME_initscripts = "" | ||
7788 | |||
7789 | For information on the backfill variable, see | ||
7790 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`. | ||
7791 | |||
7792 | Using systemd for the Main Image and Using SysVinit for the Rescue Image | ||
7793 | ------------------------------------------------------------------------ | ||
7794 | |||
7795 | Set these variables in your distribution configuration file as follows: | ||
7796 | :: | ||
7797 | |||
7798 | DISTRO_FEATURES_append = " systemd" | ||
7799 | VIRTUAL-RUNTIME_init_manager = "systemd" | ||
7800 | |||
7801 | Doing so causes your main image to use the | ||
7802 | ``packagegroup-core-boot.bb`` recipe and systemd. The rescue/minimal | ||
7803 | image cannot use this package group. However, it can install SysVinit | ||
7804 | and the appropriate packages will have support for both systemd and | ||
7805 | SysVinit. | ||
7806 | |||
7807 | Selecting a Device Manager | ||
7808 | ========================== | ||
7809 | |||
7810 | The Yocto Project provides multiple ways to manage the device manager | ||
7811 | (``/dev``): | ||
7812 | |||
7813 | - Persistent and Pre-Populated\ ``/dev``: For this case, the ``/dev`` | ||
7814 | directory is persistent and the required device nodes are created | ||
7815 | during the build. | ||
7816 | |||
7817 | - Use ``devtmpfs`` with a Device Manager: For this case, the ``/dev`` | ||
7818 | directory is provided by the kernel as an in-memory file system and | ||
7819 | is automatically populated by the kernel at runtime. Additional | ||
7820 | configuration of device nodes is done in user space by a device | ||
7821 | manager like ``udev`` or ``busybox-mdev``. | ||
7822 | |||
7823 | Using Persistent and Pre-Populated\ ``/dev`` | ||
7824 | -------------------------------------------- | ||
7825 | |||
7826 | To use the static method for device population, you need to set the | ||
7827 | :term:`USE_DEVFS` variable to "0" | ||
7828 | as follows: | ||
7829 | :: | ||
7830 | |||
7831 | USE_DEVFS = "0" | ||
7832 | |||
7833 | The content of the resulting ``/dev`` directory is defined in a Device | ||
7834 | Table file. The | ||
7835 | :term:`IMAGE_DEVICE_TABLES` | ||
7836 | variable defines the Device Table to use and should be set in the | ||
7837 | machine or distro configuration file. Alternatively, you can set this | ||
7838 | variable in your ``local.conf`` configuration file. | ||
7839 | |||
7840 | If you do not define the ``IMAGE_DEVICE_TABLES`` variable, the default | ||
7841 | ``device_table-minimal.txt`` is used: | ||
7842 | :: | ||
7843 | |||
7844 | IMAGE_DEVICE_TABLES = "device_table-mymachine.txt" | ||
7845 | |||
7846 | The population is handled by the ``makedevs`` utility during image | ||
7847 | creation: | ||
7848 | |||
7849 | Using ``devtmpfs`` and a Device Manager | ||
7850 | --------------------------------------- | ||
7851 | |||
7852 | To use the dynamic method for device population, you need to use (or be | ||
7853 | sure to set) the :term:`USE_DEVFS` | ||
7854 | variable to "1", which is the default: | ||
7855 | :: | ||
7856 | |||
7857 | USE_DEVFS = "1" | ||
7858 | |||
7859 | With this | ||
7860 | setting, the resulting ``/dev`` directory is populated by the kernel | ||
7861 | using ``devtmpfs``. Make sure the corresponding kernel configuration | ||
7862 | variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux | ||
7863 | kernel. | ||
7864 | |||
7865 | All devices created by ``devtmpfs`` will be owned by ``root`` and have | ||
7866 | permissions ``0600``. | ||
7867 | |||
7868 | To have more control over the device nodes, you can use a device manager | ||
7869 | like ``udev`` or ``busybox-mdev``. You choose the device manager by | ||
7870 | defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or | ||
7871 | distro configuration file. Alternatively, you can set this variable in | ||
7872 | your ``local.conf`` configuration file: | ||
7873 | :: | ||
7874 | |||
7875 | VIRTUAL-RUNTIME_dev_manager = "udev" | ||
7876 | |||
7877 | # Some alternative values | ||
7878 | # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev" | ||
7879 | # VIRTUAL-RUNTIME_dev_manager = "systemd" | ||
7880 | |||
7881 | Using an External SCM | ||
7882 | ===================== | ||
7883 | |||
7884 | If you're working on a recipe that pulls from an external Source Code | ||
7885 | Manager (SCM), it is possible to have the OpenEmbedded build system | ||
7886 | notice new recipe changes added to the SCM and then build the resulting | ||
7887 | packages that depend on the new recipes by using the latest versions. | ||
7888 | This only works for SCMs from which it is possible to get a sensible | ||
7889 | revision number for changes. Currently, you can do this with Apache | ||
7890 | Subversion (SVN), Git, and Bazaar (BZR) repositories. | ||
7891 | |||
7892 | To enable this behavior, the :term:`PV` of | ||
7893 | the recipe needs to reference | ||
7894 | :term:`SRCPV`. Here is an example: | ||
7895 | :: | ||
7896 | |||
7897 | PV = "1.2.3+git${SRCPV}" | ||
7898 | |||
7899 | Then, you can add the following to your | ||
7900 | ``local.conf``: | ||
7901 | :: | ||
7902 | |||
7903 | SRCREV_pn-PN = "${AUTOREV}" | ||
7904 | |||
7905 | :term:`PN` is the name of the recipe for | ||
7906 | which you want to enable automatic source revision updating. | ||
7907 | |||
7908 | If you do not want to update your local configuration file, you can add | ||
7909 | the following directly to the recipe to finish enabling the feature: | ||
7910 | :: | ||
7911 | |||
7912 | SRCREV = "${AUTOREV}" | ||
7913 | |||
7914 | The Yocto Project provides a distribution named ``poky-bleeding``, whose | ||
7915 | configuration file contains the line: | ||
7916 | :: | ||
7917 | |||
7918 | require conf/distro/include/poky-floating-revisions.inc | ||
7919 | |||
7920 | This line pulls in the | ||
7921 | listed include file that contains numerous lines of exactly that form: | ||
7922 | :: | ||
7923 | |||
7924 | #SRCREV_pn-opkg-native ?= "${AUTOREV}" | ||
7925 | #SRCREV_pn-opkg-sdk ?= "${AUTOREV}" | ||
7926 | #SRCREV_pn-opkg ?= "${AUTOREV}" | ||
7927 | #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" | ||
7928 | #SRCREV_pn-opkg-utils ?= "${AUTOREV}" | ||
7929 | SRCREV_pn-gconf-dbus ?= "${AUTOREV}" | ||
7930 | SRCREV_pn-matchbox-common ?= "${AUTOREV}" | ||
7931 | SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}" | ||
7932 | SRCREV_pn-matchbox-desktop ?= "${AUTOREV}" | ||
7933 | SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" | ||
7934 | SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}" | ||
7935 | SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}" | ||
7936 | SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" | ||
7937 | SRCREV_pn-matchbox-wm ?= "${AUTOREV}" | ||
7938 | SRCREV_pn-settings-daemon ?= "${AUTOREV}" | ||
7939 | SRCREV_pn-screenshot ?= "${AUTOREV}" | ||
7940 | . . . | ||
7941 | |||
7942 | These lines allow you to | ||
7943 | experiment with building a distribution that tracks the latest | ||
7944 | development source for numerous packages. | ||
7945 | |||
7946 | .. note:: | ||
7947 | |||
7948 | The ``poky-bleeding`` distribution is not tested on a regular basis. Keep | ||
7949 | this in mind if you use it. | ||
7950 | |||
7951 | Creating a Read-Only Root Filesystem | ||
7952 | ==================================== | ||
7953 | |||
7954 | Suppose, for security reasons, you need to disable your target device's | ||
7955 | root filesystem's write permissions (i.e. you need a read-only root | ||
7956 | filesystem). Or, perhaps you are running the device's operating system | ||
7957 | from a read-only storage device. For either case, you can customize your | ||
7958 | image for that behavior. | ||
7959 | |||
7960 | .. note:: | ||
7961 | |||
7962 | Supporting a read-only root filesystem requires that the system and | ||
7963 | applications do not try to write to the root filesystem. You must | ||
7964 | configure all parts of the target system to write elsewhere, or to | ||
7965 | gracefully fail in the event of attempting to write to the root | ||
7966 | filesystem. | ||
7967 | |||
7968 | Creating the Root Filesystem | ||
7969 | ---------------------------- | ||
7970 | |||
7971 | To create the read-only root filesystem, simply add the | ||
7972 | "read-only-rootfs" feature to your image, normally in one of two ways. | ||
7973 | The first way is to add the "read-only-rootfs" image feature in the | ||
7974 | image's recipe file via the ``IMAGE_FEATURES`` variable: | ||
7975 | :: | ||
7976 | |||
7977 | IMAGE_FEATURES += "read-only-rootfs" | ||
7978 | |||
7979 | As an alternative, you can add the same feature | ||
7980 | from within your build directory's ``local.conf`` file with the | ||
7981 | associated ``EXTRA_IMAGE_FEATURES`` variable, as in: | ||
7982 | :: | ||
7983 | |||
7984 | EXTRA_IMAGE_FEATURES = "read-only-rootfs" | ||
7985 | |||
7986 | For more information on how to use these variables, see the | ||
7987 | ":ref:`dev-manual/common-tasks:Customizing Images Using Custom \`\`IMAGE_FEATURES\`\` and \`\`EXTRA_IMAGE_FEATURES\`\``" | ||
7988 | section. For information on the variables, see | ||
7989 | :term:`IMAGE_FEATURES` and | ||
7990 | :term:`EXTRA_IMAGE_FEATURES`. | ||
7991 | |||
7992 | Post-Installation Scripts and Read-Only Root Filesystem | ||
7993 | ------------------------------------------------------- | ||
7994 | |||
7995 | It is very important that you make sure all post-Installation | ||
7996 | (``pkg_postinst``) scripts for packages that are installed into the | ||
7997 | image can be run at the time when the root filesystem is created during | ||
7998 | the build on the host system. These scripts cannot attempt to run during | ||
7999 | first-boot on the target device. With the "read-only-rootfs" feature | ||
8000 | enabled, the build system checks during root filesystem creation to make | ||
8001 | sure all post-installation scripts succeed. If any of these scripts | ||
8002 | still need to be run after the root filesystem is created, the build | ||
8003 | immediately fails. These build-time checks ensure that the build fails | ||
8004 | rather than the target device fails later during its initial boot | ||
8005 | operation. | ||
8006 | |||
8007 | Most of the common post-installation scripts generated by the build | ||
8008 | system for the out-of-the-box Yocto Project are engineered so that they | ||
8009 | can run during root filesystem creation (e.g. post-installation scripts | ||
8010 | for caching fonts). However, if you create and add custom scripts, you | ||
8011 | need to be sure they can be run during this file system creation. | ||
8012 | |||
8013 | Here are some common problems that prevent post-installation scripts | ||
8014 | from running during root filesystem creation: | ||
8015 | |||
8016 | - *Not using $D in front of absolute paths:* The build system defines | ||
8017 | ``$``\ :term:`D` when the root | ||
8018 | filesystem is created. Furthermore, ``$D`` is blank when the script | ||
8019 | is run on the target device. This implies two purposes for ``$D``: | ||
8020 | ensuring paths are valid in both the host and target environments, | ||
8021 | and checking to determine which environment is being used as a method | ||
8022 | for taking appropriate actions. | ||
8023 | |||
8024 | - *Attempting to run processes that are specific to or dependent on the | ||
8025 | target architecture:* You can work around these attempts by using | ||
8026 | native tools, which run on the host system, to accomplish the same | ||
8027 | tasks, or by alternatively running the processes under QEMU, which | ||
8028 | has the ``qemu_run_binary`` function. For more information, see the | ||
8029 | :ref:`qemu <ref-classes-qemu>` class. | ||
8030 | |||
8031 | Areas With Write Access | ||
8032 | ----------------------- | ||
8033 | |||
8034 | With the "read-only-rootfs" feature enabled, any attempt by the target | ||
8035 | to write to the root filesystem at runtime fails. Consequently, you must | ||
8036 | make sure that you configure processes and applications that attempt | ||
8037 | these types of writes do so to directories with write access (e.g. | ||
8038 | ``/tmp`` or ``/var/run``). | ||
8039 | |||
8040 | Maintaining Build Output Quality | ||
8041 | ================================ | ||
8042 | |||
8043 | Many factors can influence the quality of a build. For example, if you | ||
8044 | upgrade a recipe to use a new version of an upstream software package or | ||
8045 | you experiment with some new configuration options, subtle changes can | ||
8046 | occur that you might not detect until later. Consider the case where | ||
8047 | your recipe is using a newer version of an upstream package. In this | ||
8048 | case, a new version of a piece of software might introduce an optional | ||
8049 | dependency on another library, which is auto-detected. If that library | ||
8050 | has already been built when the software is building, the software will | ||
8051 | link to the built library and that library will be pulled into your | ||
8052 | image along with the new software even if you did not want the library. | ||
8053 | |||
8054 | The :ref:`buildhistory <ref-classes-buildhistory>` | ||
8055 | class exists to help you maintain the quality of your build output. You | ||
8056 | can use the class to highlight unexpected and possibly unwanted changes | ||
8057 | in the build output. When you enable build history, it records | ||
8058 | information about the contents of each package and image and then | ||
8059 | commits that information to a local Git repository where you can examine | ||
8060 | the information. | ||
8061 | |||
8062 | The remainder of this section describes the following: | ||
8063 | |||
8064 | - :ref:`How you can enable and disable build history <dev-manual/common-tasks:enabling and disabling build history>` | ||
8065 | |||
8066 | - :ref:`How to understand what the build history contains <dev-manual/common-tasks:understanding what the build history contains>` | ||
8067 | |||
8068 | - :ref:`How to limit the information used for build history <dev-manual/common-tasks:using build history to gather image information only>` | ||
8069 | |||
8070 | - :ref:`How to examine the build history from both a command-line and web interface <dev-manual/common-tasks:examining build history information>` | ||
8071 | |||
8072 | Enabling and Disabling Build History | ||
8073 | ------------------------------------ | ||
8074 | |||
8075 | Build history is disabled by default. To enable it, add the following | ||
8076 | ``INHERIT`` statement and set the | ||
8077 | :term:`BUILDHISTORY_COMMIT` | ||
8078 | variable to "1" at the end of your ``conf/local.conf`` file found in the | ||
8079 | :term:`Build Directory`: | ||
8080 | :: | ||
8081 | |||
8082 | INHERIT += "buildhistory" | ||
8083 | BUILDHISTORY_COMMIT = "1" | ||
8084 | |||
8085 | Enabling build history as | ||
8086 | previously described causes the OpenEmbedded build system to collect | ||
8087 | build output information and commit it as a single commit to a local | ||
8088 | :ref:`overview-manual/development-environment:git` repository. | ||
8089 | |||
8090 | .. note:: | ||
8091 | |||
8092 | Enabling build history increases your build times slightly, | ||
8093 | particularly for images, and increases the amount of disk space used | ||
8094 | during the build. | ||
8095 | |||
8096 | You can disable build history by removing the previous statements from | ||
8097 | your ``conf/local.conf`` file. | ||
8098 | |||
8099 | Understanding What the Build History Contains | ||
8100 | --------------------------------------------- | ||
8101 | |||
8102 | Build history information is kept in | ||
8103 | ``${``\ :term:`TOPDIR`\ ``}/buildhistory`` | ||
8104 | in the Build Directory as defined by the | ||
8105 | :term:`BUILDHISTORY_DIR` | ||
8106 | variable. The following is an example abbreviated listing: | ||
8107 | |||
8108 | .. image:: figures/buildhistory.png | ||
8109 | :align: center | ||
8110 | |||
8111 | At the top level, a ``metadata-revs`` file exists that lists the | ||
8112 | revisions of the repositories for the enabled layers when the build was | ||
8113 | produced. The rest of the data splits into separate ``packages``, | ||
8114 | ``images`` and ``sdk`` directories, the contents of which are described | ||
8115 | as follows. | ||
8116 | |||
8117 | Build History Package Information | ||
8118 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8119 | |||
8120 | The history for each package contains a text file that has name-value | ||
8121 | pairs with information about the package. For example, | ||
8122 | ``buildhistory/packages/i586-poky-linux/busybox/busybox/latest`` | ||
8123 | contains the following: | ||
8124 | |||
8125 | .. code-block:: none | ||
8126 | |||
8127 | PV = 1.22.1 | ||
8128 | PR = r32 | ||
8129 | RPROVIDES = | ||
8130 | RDEPENDS = glibc (>= 2.20) update-alternatives-opkg | ||
8131 | RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d | ||
8132 | PKGSIZE = 540168 | ||
8133 | FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \ | ||
8134 | /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \ | ||
8135 | /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \ | ||
8136 | /usr/share/pixmaps /usr/share/applications /usr/share/idl \ | ||
8137 | /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers | ||
8138 | FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \ | ||
8139 | /etc/busybox.links.nosuid /etc/busybox.links.suid | ||
8140 | |||
8141 | Most of these | ||
8142 | name-value pairs correspond to variables used to produce the package. | ||
8143 | The exceptions are ``FILELIST``, which is the actual list of files in | ||
8144 | the package, and ``PKGSIZE``, which is the total size of files in the | ||
8145 | package in bytes. | ||
8146 | |||
8147 | A file also exists that corresponds to the recipe from which the package | ||
8148 | came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``): | ||
8149 | |||
8150 | .. code-block:: none | ||
8151 | |||
8152 | PV = 1.22.1 | ||
8153 | PR = r32 | ||
8154 | DEPENDS = initscripts kern-tools-native update-rc.d-native \ | ||
8155 | virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \ | ||
8156 | virtual/libc virtual/update-alternatives | ||
8157 | PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \ | ||
8158 | busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \ | ||
8159 | busybox-staticdev busybox-dev busybox-doc busybox-locale busybox | ||
8160 | |||
8161 | Finally, for those recipes fetched from a version control system (e.g., | ||
8162 | Git), a file exists that lists source revisions that are specified in | ||
8163 | the recipe and lists the actual revisions used during the build. Listed | ||
8164 | and actual revisions might differ when | ||
8165 | :term:`SRCREV` is set to | ||
8166 | ${:term:`AUTOREV`}. Here is an | ||
8167 | example assuming | ||
8168 | ``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``): | ||
8169 | :: | ||
8170 | |||
8171 | # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8172 | SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8173 | # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8174 | SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8175 | |||
8176 | You can use the | ||
8177 | ``buildhistory-collect-srcrevs`` command with the ``-a`` option to | ||
8178 | collect the stored ``SRCREV`` values from build history and report them | ||
8179 | in a format suitable for use in global configuration (e.g., | ||
8180 | ``local.conf`` or a distro include file) to override floating | ||
8181 | ``AUTOREV`` values to a fixed set of revisions. Here is some example | ||
8182 | output from this command: | ||
8183 | :: | ||
8184 | |||
8185 | $ buildhistory-collect-srcrevs -a | ||
8186 | # i586-poky-linux | ||
8187 | SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8188 | SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8189 | SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a" | ||
8190 | SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" | ||
8191 | # x86_64-linux | ||
8192 | SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa" | ||
8193 | SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" | ||
8194 | SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11" | ||
8195 | SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8196 | SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3" | ||
8197 | SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca" | ||
8198 | SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a" | ||
8199 | SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" | ||
8200 | SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" | ||
8201 | # qemux86-poky-linux | ||
8202 | SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8203 | SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8204 | # all-poky-linux | ||
8205 | SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11" | ||
8206 | |||
8207 | .. note:: | ||
8208 | |||
8209 | Here are some notes on using the ``buildhistory-collect-srcrevs`` command: | ||
8210 | |||
8211 | - By default, only values where the ``SRCREV`` was not hardcoded | ||
8212 | (usually when ``AUTOREV`` is used) are reported. Use the ``-a`` | ||
8213 | option to see all ``SRCREV`` values. | ||
8214 | |||
8215 | - The output statements might not have any effect if overrides are | ||
8216 | applied elsewhere in the build system configuration. Use the | ||
8217 | ``-f`` option to add the ``forcevariable`` override to each output | ||
8218 | line if you need to work around this restriction. | ||
8219 | |||
8220 | - The script does apply special handling when building for multiple | ||
8221 | machines. However, the script does place a comment before each set | ||
8222 | of values that specifies which triplet to which they belong as | ||
8223 | previously shown (e.g., ``i586-poky-linux``). | ||
8224 | |||
8225 | Build History Image Information | ||
8226 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8227 | |||
8228 | The files produced for each image are as follows: | ||
8229 | |||
8230 | - ``image-files:`` A directory containing selected files from the root | ||
8231 | filesystem. The files are defined by | ||
8232 | :term:`BUILDHISTORY_IMAGE_FILES`. | ||
8233 | |||
8234 | - ``build-id.txt:`` Human-readable information about the build | ||
8235 | configuration and metadata source revisions. This file contains the | ||
8236 | full build header as printed by BitBake. | ||
8237 | |||
8238 | - ``*.dot:`` Dependency graphs for the image that are compatible with | ||
8239 | ``graphviz``. | ||
8240 | |||
8241 | - ``files-in-image.txt:`` A list of files in the image with | ||
8242 | permissions, owner, group, size, and symlink information. | ||
8243 | |||
8244 | - ``image-info.txt:`` A text file containing name-value pairs with | ||
8245 | information about the image. See the following listing example for | ||
8246 | more information. | ||
8247 | |||
8248 | - ``installed-package-names.txt:`` A list of installed packages by name | ||
8249 | only. | ||
8250 | |||
8251 | - ``installed-package-sizes.txt:`` A list of installed packages ordered | ||
8252 | by size. | ||
8253 | |||
8254 | - ``installed-packages.txt:`` A list of installed packages with full | ||
8255 | package filenames. | ||
8256 | |||
8257 | .. note:: | ||
8258 | |||
8259 | Installed package information is able to be gathered and produced | ||
8260 | even if package management is disabled for the final image. | ||
8261 | |||
8262 | Here is an example of ``image-info.txt``: | ||
8263 | |||
8264 | .. code-block:: none | ||
8265 | |||
8266 | DISTRO = poky | ||
8267 | DISTRO_VERSION = 1.7 | ||
8268 | USER_CLASSES = buildstats image-mklibs image-prelink | ||
8269 | IMAGE_CLASSES = image_types | ||
8270 | IMAGE_FEATURES = debug-tweaks | ||
8271 | IMAGE_LINGUAS = | ||
8272 | IMAGE_INSTALL = packagegroup-core-boot run-postinsts | ||
8273 | BAD_RECOMMENDATIONS = | ||
8274 | NO_RECOMMENDATIONS = | ||
8275 | PACKAGE_EXCLUDE = | ||
8276 | ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \ | ||
8277 | write_image_manifest ; buildhistory_list_installed_image ; \ | ||
8278 | buildhistory_get_image_installed ; ssh_allow_empty_password; \ | ||
8279 | postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ; | ||
8280 | IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ; | ||
8281 | IMAGESIZE = 6900 | ||
8282 | |||
8283 | Other than ``IMAGESIZE``, | ||
8284 | which is the total size of the files in the image in Kbytes, the | ||
8285 | name-value pairs are variables that may have influenced the content of | ||
8286 | the image. This information is often useful when you are trying to | ||
8287 | determine why a change in the package or file listings has occurred. | ||
8288 | |||
8289 | Using Build History to Gather Image Information Only | ||
8290 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8291 | |||
8292 | As you can see, build history produces image information, including | ||
8293 | dependency graphs, so you can see why something was pulled into the | ||
8294 | image. If you are just interested in this information and not interested | ||
8295 | in collecting specific package or SDK information, you can enable | ||
8296 | writing only image information without any history by adding the | ||
8297 | following to your ``conf/local.conf`` file found in the | ||
8298 | :term:`Build Directory`: | ||
8299 | :: | ||
8300 | |||
8301 | INHERIT += "buildhistory" | ||
8302 | BUILDHISTORY_COMMIT = "0" | ||
8303 | BUILDHISTORY_FEATURES = "image" | ||
8304 | |||
8305 | Here, you set the | ||
8306 | :term:`BUILDHISTORY_FEATURES` | ||
8307 | variable to use the image feature only. | ||
8308 | |||
8309 | Build History SDK Information | ||
8310 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8311 | |||
8312 | Build history collects similar information on the contents of SDKs (e.g. | ||
8313 | ``bitbake -c populate_sdk imagename``) as compared to information it | ||
8314 | collects for images. Furthermore, this information differs depending on | ||
8315 | whether an extensible or standard SDK is being produced. | ||
8316 | |||
8317 | The following list shows the files produced for SDKs: | ||
8318 | |||
8319 | - ``files-in-sdk.txt:`` A list of files in the SDK with permissions, | ||
8320 | owner, group, size, and symlink information. This list includes both | ||
8321 | the host and target parts of the SDK. | ||
8322 | |||
8323 | - ``sdk-info.txt:`` A text file containing name-value pairs with | ||
8324 | information about the SDK. See the following listing example for more | ||
8325 | information. | ||
8326 | |||
8327 | - ``sstate-task-sizes.txt:`` A text file containing name-value pairs | ||
8328 | with information about task group sizes (e.g. ``do_populate_sysroot`` | ||
8329 | tasks have a total size). The ``sstate-task-sizes.txt`` file exists | ||
8330 | only when an extensible SDK is created. | ||
8331 | |||
8332 | - ``sstate-package-sizes.txt:`` A text file containing name-value pairs | ||
8333 | with information for the shared-state packages and sizes in the SDK. | ||
8334 | The ``sstate-package-sizes.txt`` file exists only when an extensible | ||
8335 | SDK is created. | ||
8336 | |||
8337 | - ``sdk-files:`` A folder that contains copies of the files mentioned | ||
8338 | in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output. | ||
8339 | Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is | ||
8340 | specific to the extensible SDK although you can set it differently if | ||
8341 | you would like to pull in specific files from the standard SDK. | ||
8342 | |||
8343 | The default files are ``conf/local.conf``, ``conf/bblayers.conf``, | ||
8344 | ``conf/auto.conf``, ``conf/locked-sigs.inc``, and | ||
8345 | ``conf/devtool.conf``. Thus, for an extensible SDK, these files get | ||
8346 | copied into the ``sdk-files`` directory. | ||
8347 | |||
8348 | - The following information appears under each of the ``host`` and | ||
8349 | ``target`` directories for the portions of the SDK that run on the | ||
8350 | host and on the target, respectively: | ||
8351 | |||
8352 | .. note:: | ||
8353 | |||
8354 | The following files for the most part are empty when producing an | ||
8355 | extensible SDK because this type of SDK is not constructed from | ||
8356 | packages as is the standard SDK. | ||
8357 | |||
8358 | - ``depends.dot:`` Dependency graph for the SDK that is compatible | ||
8359 | with ``graphviz``. | ||
8360 | |||
8361 | - ``installed-package-names.txt:`` A list of installed packages by | ||
8362 | name only. | ||
8363 | |||
8364 | - ``installed-package-sizes.txt:`` A list of installed packages | ||
8365 | ordered by size. | ||
8366 | |||
8367 | - ``installed-packages.txt:`` A list of installed packages with full | ||
8368 | package filenames. | ||
8369 | |||
8370 | Here is an example of ``sdk-info.txt``: | ||
8371 | |||
8372 | .. code-block:: none | ||
8373 | |||
8374 | DISTRO = poky | ||
8375 | DISTRO_VERSION = 1.3+snapshot-20130327 | ||
8376 | SDK_NAME = poky-glibc-i686-arm | ||
8377 | SDK_VERSION = 1.3+snapshot | ||
8378 | SDKMACHINE = | ||
8379 | SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs | ||
8380 | BAD_RECOMMENDATIONS = | ||
8381 | SDKSIZE = 352712 | ||
8382 | |||
8383 | Other than ``SDKSIZE``, which is | ||
8384 | the total size of the files in the SDK in Kbytes, the name-value pairs | ||
8385 | are variables that might have influenced the content of the SDK. This | ||
8386 | information is often useful when you are trying to determine why a | ||
8387 | change in the package or file listings has occurred. | ||
8388 | |||
8389 | Examining Build History Information | ||
8390 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8391 | |||
8392 | You can examine build history output from the command line or from a web | ||
8393 | interface. | ||
8394 | |||
8395 | To see any changes that have occurred (assuming you have | ||
8396 | :term:`BUILDHISTORY_COMMIT` = "1"), | ||
8397 | you can simply use any Git command that allows you to view the history | ||
8398 | of a repository. Here is one method: | ||
8399 | :: | ||
8400 | |||
8401 | $ git log -p | ||
8402 | |||
8403 | You need to realize, | ||
8404 | however, that this method does show changes that are not significant | ||
8405 | (e.g. a package's size changing by a few bytes). | ||
8406 | |||
8407 | A command-line tool called ``buildhistory-diff`` does exist, though, | ||
8408 | that queries the Git repository and prints just the differences that | ||
8409 | might be significant in human-readable form. Here is an example: | ||
8410 | :: | ||
8411 | |||
8412 | $ ~/poky/poky/scripts/buildhistory-diff . HEAD^ | ||
8413 | Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt): | ||
8414 | /etc/anotherpkg.conf was added | ||
8415 | /sbin/anotherpkg was added | ||
8416 | * (installed-package-names.txt): | ||
8417 | * anotherpkg was added | ||
8418 | Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt): | ||
8419 | anotherpkg was added | ||
8420 | packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" | ||
8421 | * PR changed from "r0" to "r1" | ||
8422 | * PV changed from "0.1.10" to "0.1.12" | ||
8423 | packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%) | ||
8424 | * PR changed from "r0" to "r1" | ||
8425 | * PV changed from "0.1.10" to "0.1.12" | ||
8426 | |||
8427 | .. note:: | ||
8428 | |||
8429 | The ``buildhistory-diff`` tool requires the ``GitPython`` | ||
8430 | package. Be sure to install it using Pip3 as follows: | ||
8431 | :: | ||
8432 | |||
8433 | $ pip3 install GitPython --user | ||
8434 | |||
8435 | |||
8436 | Alternatively, you can install ``python3-git`` using the appropriate | ||
8437 | distribution package manager (e.g. ``apt-get``, ``dnf``, or ``zipper``). | ||
8438 | |||
8439 | To see changes to the build history using a web interface, follow the | ||
8440 | instruction in the ``README`` file | ||
8441 | :yocto_git:`here </buildhistory-web/>`. | ||
8442 | |||
8443 | Here is a sample screenshot of the interface: | ||
8444 | |||
8445 | .. image:: figures/buildhistory-web.png | ||
8446 | :align: center | ||
8447 | |||
8448 | Performing Automated Runtime Testing | ||
8449 | ==================================== | ||
8450 | |||
8451 | The OpenEmbedded build system makes available a series of automated | ||
8452 | tests for images to verify runtime functionality. You can run these | ||
8453 | tests on either QEMU or actual target hardware. Tests are written in | ||
8454 | Python making use of the ``unittest`` module, and the majority of them | ||
8455 | run commands on the target system over SSH. This section describes how | ||
8456 | you set up the environment to use these tests, run available tests, and | ||
8457 | write and add your own tests. | ||
8458 | |||
8459 | For information on the test and QA infrastructure available within the | ||
8460 | Yocto Project, see the ":ref:`ref-manual/release-process:testing and quality assurance`" | ||
8461 | section in the Yocto Project Reference Manual. | ||
8462 | |||
8463 | Enabling Tests | ||
8464 | -------------- | ||
8465 | |||
8466 | Depending on whether you are planning to run tests using QEMU or on the | ||
8467 | hardware, you have to take different steps to enable the tests. See the | ||
8468 | following subsections for information on how to enable both types of | ||
8469 | tests. | ||
8470 | |||
8471 | Enabling Runtime Tests on QEMU | ||
8472 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8473 | |||
8474 | In order to run tests, you need to do the following: | ||
8475 | |||
8476 | - *Set up to avoid interaction with sudo for networking:* To | ||
8477 | accomplish this, you must do one of the following: | ||
8478 | |||
8479 | - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all | ||
8480 | commands or just for ``runqemu-ifup``. You must provide the full | ||
8481 | path as that can change if you are using multiple clones of the | ||
8482 | source repository. | ||
8483 | |||
8484 | .. note:: | ||
8485 | |||
8486 | On some distributions, you also need to comment out "Defaults | ||
8487 | requiretty" in ``/etc/sudoers``. | ||
8488 | |||
8489 | - Manually configure a tap interface for your system. | ||
8490 | |||
8491 | - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which | ||
8492 | should generate a list of tap devices. This is the option | ||
8493 | typically chosen for Autobuilder-type environments. | ||
8494 | |||
8495 | .. note:: | ||
8496 | |||
8497 | - Be sure to use an absolute path when calling this script | ||
8498 | with sudo. | ||
8499 | |||
8500 | - The package recipe ``qemu-helper-native`` is required to run | ||
8501 | this script. Build the package using the following command: | ||
8502 | :: | ||
8503 | |||
8504 | $ bitbake qemu-helper-native | ||
8505 | |||
8506 | - *Set the DISPLAY variable:* You need to set this variable so that | ||
8507 | you have an X server available (e.g. start ``vncserver`` for a | ||
8508 | headless machine). | ||
8509 | |||
8510 | - *Be sure your host's firewall accepts incoming connections from | ||
8511 | 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an | ||
8512 | HTTP server on a random high number port, which is used to serve | ||
8513 | files to the target. The DNF module serves | ||
8514 | ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands. | ||
8515 | That means your host's firewall must accept incoming connections from | ||
8516 | 192.168.7.0/24, which is the default IP range used for tap devices by | ||
8517 | ``runqemu``. | ||
8518 | |||
8519 | - *Be sure your host has the correct packages installed:* Depending | ||
8520 | your host's distribution, you need to have the following packages | ||
8521 | installed: | ||
8522 | |||
8523 | - Ubuntu and Debian: ``sysstat`` and ``iproute2`` | ||
8524 | |||
8525 | - OpenSUSE: ``sysstat`` and ``iproute2`` | ||
8526 | |||
8527 | - Fedora: ``sysstat`` and ``iproute`` | ||
8528 | |||
8529 | - CentOS: ``sysstat`` and ``iproute`` | ||
8530 | |||
8531 | Once you start running the tests, the following happens: | ||
8532 | |||
8533 | 1. A copy of the root filesystem is written to ``${WORKDIR}/testimage``. | ||
8534 | |||
8535 | 2. The image is booted under QEMU using the standard ``runqemu`` script. | ||
8536 | |||
8537 | 3. A default timeout of 500 seconds occurs to allow for the boot process | ||
8538 | to reach the login prompt. You can change the timeout period by | ||
8539 | setting | ||
8540 | :term:`TEST_QEMUBOOT_TIMEOUT` | ||
8541 | in the ``local.conf`` file. | ||
8542 | |||
8543 | 4. Once the boot process is reached and the login prompt appears, the | ||
8544 | tests run. The full boot log is written to | ||
8545 | ``${WORKDIR}/testimage/qemu_boot_log``. | ||
8546 | |||
8547 | 5. Each test module loads in the order found in ``TEST_SUITES``. You can | ||
8548 | find the full output of the commands run over SSH in | ||
8549 | ``${WORKDIR}/testimgage/ssh_target_log``. | ||
8550 | |||
8551 | 6. If no failures occur, the task running the tests ends successfully. | ||
8552 | You can find the output from the ``unittest`` in the task log at | ||
8553 | ``${WORKDIR}/temp/log.do_testimage``. | ||
8554 | |||
8555 | Enabling Runtime Tests on Hardware | ||
8556 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8557 | |||
8558 | The OpenEmbedded build system can run tests on real hardware, and for | ||
8559 | certain devices it can also deploy the image to be tested onto the | ||
8560 | device beforehand. | ||
8561 | |||
8562 | For automated deployment, a "master image" is installed onto the | ||
8563 | hardware once as part of setup. Then, each time tests are to be run, the | ||
8564 | following occurs: | ||
8565 | |||
8566 | 1. The master image is booted into and used to write the image to be | ||
8567 | tested to a second partition. | ||
8568 | |||
8569 | 2. The device is then rebooted using an external script that you need to | ||
8570 | provide. | ||
8571 | |||
8572 | 3. The device boots into the image to be tested. | ||
8573 | |||
8574 | When running tests (independent of whether the image has been deployed | ||
8575 | automatically or not), the device is expected to be connected to a | ||
8576 | network on a pre-determined IP address. You can either use static IP | ||
8577 | addresses written into the image, or set the image to use DHCP and have | ||
8578 | your DHCP server on the test network assign a known IP address based on | ||
8579 | the MAC address of the device. | ||
8580 | |||
8581 | In order to run tests on hardware, you need to set ``TEST_TARGET`` to an | ||
8582 | appropriate value. For QEMU, you do not have to change anything, the | ||
8583 | default value is "qemu". For running tests on hardware, the following | ||
8584 | options exist: | ||
8585 | |||
8586 | - *"simpleremote":* Choose "simpleremote" if you are going to run tests | ||
8587 | on a target system that is already running the image to be tested and | ||
8588 | is available on the network. You can use "simpleremote" in | ||
8589 | conjunction with either real hardware or an image running within a | ||
8590 | separately started QEMU or any other virtual machine manager. | ||
8591 | |||
8592 | - *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is | ||
8593 | an EFI-based machine with ``systemd-boot`` as bootloader and | ||
8594 | ``core-image-testmaster`` (or something similar) is installed. Also, | ||
8595 | your hardware under test must be in a DHCP-enabled network that gives | ||
8596 | it the same IP address for each reboot. | ||
8597 | |||
8598 | If you choose "SystemdbootTarget", there are additional requirements | ||
8599 | and considerations. See the "`Selecting | ||
8600 | SystemdbootTarget <#selecting-systemdboottarget>`__" section, which | ||
8601 | follows, for more information. | ||
8602 | |||
8603 | - *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying | ||
8604 | images and running tests on the BeagleBone "Black" or original | ||
8605 | "White" hardware. For information on how to use these tests, see the | ||
8606 | comments at the top of the BeagleBoneTarget | ||
8607 | ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file. | ||
8608 | |||
8609 | - *"EdgeRouterTarget":* Choose "EdgeRouterTarget" if you are deploying | ||
8610 | images and running tests on the Ubiquiti Networks EdgeRouter Lite. | ||
8611 | For information on how to use these tests, see the comments at the | ||
8612 | top of the EdgeRouterTarget | ||
8613 | ``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file. | ||
8614 | |||
8615 | - *"GrubTarget":* Choose "GrubTarget" if you are deploying images and running | ||
8616 | tests on any generic PC that boots using GRUB. For information on how | ||
8617 | to use these tests, see the comments at the top of the GrubTarget | ||
8618 | ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file. | ||
8619 | |||
8620 | - *"your-target":* Create your own custom target if you want to run | ||
8621 | tests when you are deploying images and running tests on a custom | ||
8622 | machine within your BSP layer. To do this, you need to add a Python | ||
8623 | unit that defines the target class under ``lib/oeqa/controllers/`` | ||
8624 | within your layer. You must also provide an empty ``__init__.py``. | ||
8625 | For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``. | ||
8626 | |||
8627 | Selecting SystemdbootTarget | ||
8628 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8629 | |||
8630 | If you did not set ``TEST_TARGET`` to "SystemdbootTarget", then you do | ||
8631 | not need any information in this section. You can skip down to the | ||
8632 | "`Running Tests <#qemu-image-running-tests>`__" section. | ||
8633 | |||
8634 | If you did set ``TEST_TARGET`` to "SystemdbootTarget", you also need to | ||
8635 | perform a one-time setup of your master image by doing the following: | ||
8636 | |||
8637 | 1. *Set EFI_PROVIDER:* Be sure that ``EFI_PROVIDER`` is as follows: | ||
8638 | :: | ||
8639 | |||
8640 | EFI_PROVIDER = "systemd-boot" | ||
8641 | |||
8642 | 2. *Build the master image:* Build the ``core-image-testmaster`` image. | ||
8643 | The ``core-image-testmaster`` recipe is provided as an example for a | ||
8644 | "master" image and you can customize the image recipe as you would | ||
8645 | any other recipe. | ||
8646 | |||
8647 | Here are the image recipe requirements: | ||
8648 | |||
8649 | - Inherits ``core-image`` so that kernel modules are installed. | ||
8650 | |||
8651 | - Installs normal linux utilities not busybox ones (e.g. ``bash``, | ||
8652 | ``coreutils``, ``tar``, ``gzip``, and ``kmod``). | ||
8653 | |||
8654 | - Uses a custom Initial RAM Disk (initramfs) image with a custom | ||
8655 | installer. A normal image that you can install usually creates a | ||
8656 | single rootfs partition. This image uses another installer that | ||
8657 | creates a specific partition layout. Not all Board Support | ||
8658 | Packages (BSPs) can use an installer. For such cases, you need to | ||
8659 | manually create the following partition layout on the target: | ||
8660 | |||
8661 | - First partition mounted under ``/boot``, labeled "boot". | ||
8662 | |||
8663 | - The main rootfs partition where this image gets installed, | ||
8664 | which is mounted under ``/``. | ||
8665 | |||
8666 | - Another partition labeled "testrootfs" where test images get | ||
8667 | deployed. | ||
8668 | |||
8669 | 3. *Install image:* Install the image that you just built on the target | ||
8670 | system. | ||
8671 | |||
8672 | The final thing you need to do when setting ``TEST_TARGET`` to | ||
8673 | "SystemdbootTarget" is to set up the test image: | ||
8674 | |||
8675 | 1. *Set up your local.conf file:* Make sure you have the following | ||
8676 | statements in your ``local.conf`` file: | ||
8677 | :: | ||
8678 | |||
8679 | IMAGE_FSTYPES += "tar.gz" | ||
8680 | INHERIT += "testimage" | ||
8681 | TEST_TARGET = "SystemdbootTarget" | ||
8682 | TEST_TARGET_IP = "192.168.2.3" | ||
8683 | |||
8684 | 2. *Build your test image:* Use BitBake to build the image: | ||
8685 | :: | ||
8686 | |||
8687 | $ bitbake core-image-sato | ||
8688 | |||
8689 | Power Control | ||
8690 | ~~~~~~~~~~~~~ | ||
8691 | |||
8692 | For most hardware targets other than "simpleremote", you can control | ||
8693 | power: | ||
8694 | |||
8695 | - You can use ``TEST_POWERCONTROL_CMD`` together with | ||
8696 | ``TEST_POWERCONTROL_EXTRA_ARGS`` as a command that runs on the host | ||
8697 | and does power cycling. The test code passes one argument to that | ||
8698 | command: off, on or cycle (off then on). Here is an example that | ||
8699 | could appear in your ``local.conf`` file: | ||
8700 | :: | ||
8701 | |||
8702 | TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" | ||
8703 | |||
8704 | In this example, the expect | ||
8705 | script does the following: | ||
8706 | |||
8707 | .. code-block:: shell | ||
8708 | |||
8709 | ssh test@10.11.12.1 "pyctl nuc1 arg" | ||
8710 | |||
8711 | It then runs a Python script that controls power for a label called | ||
8712 | ``nuc1``. | ||
8713 | |||
8714 | .. note:: | ||
8715 | |||
8716 | You need to customize ``TEST_POWERCONTROL_CMD`` and | ||
8717 | ``TEST_POWERCONTROL_EXTRA_ARGS`` for your own setup. The one requirement | ||
8718 | is that it accepts "on", "off", and "cycle" as the last argument. | ||
8719 | |||
8720 | - When no command is defined, it connects to the device over SSH and | ||
8721 | uses the classic reboot command to reboot the device. Classic reboot | ||
8722 | is fine as long as the machine actually reboots (i.e. the SSH test | ||
8723 | has not failed). It is useful for scenarios where you have a simple | ||
8724 | setup, typically with a single board, and where some manual | ||
8725 | interaction is okay from time to time. | ||
8726 | |||
8727 | If you have no hardware to automatically perform power control but still | ||
8728 | wish to experiment with automated hardware testing, you can use the | ||
8729 | ``dialog-power-control`` script that shows a dialog prompting you to perform | ||
8730 | the required power action. This script requires either KDialog or Zenity | ||
8731 | to be installed. To use this script, set the | ||
8732 | :term:`TEST_POWERCONTROL_CMD` | ||
8733 | variable as follows: | ||
8734 | :: | ||
8735 | |||
8736 | TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" | ||
8737 | |||
8738 | Serial Console Connection | ||
8739 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8740 | |||
8741 | For test target classes requiring a serial console to interact with the | ||
8742 | bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget), | ||
8743 | you need to specify a command to use to connect to the serial console of | ||
8744 | the target machine by using the | ||
8745 | :term:`TEST_SERIALCONTROL_CMD` | ||
8746 | variable and optionally the | ||
8747 | :term:`TEST_SERIALCONTROL_EXTRA_ARGS` | ||
8748 | variable. | ||
8749 | |||
8750 | These cases could be a serial terminal program if the machine is | ||
8751 | connected to a local serial port, or a ``telnet`` or ``ssh`` command | ||
8752 | connecting to a remote console server. Regardless of the case, the | ||
8753 | command simply needs to connect to the serial console and forward that | ||
8754 | connection to standard input and output as any normal terminal program | ||
8755 | does. For example, to use the picocom terminal program on serial device | ||
8756 | ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows: | ||
8757 | :: | ||
8758 | |||
8759 | TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" | ||
8760 | |||
8761 | For local | ||
8762 | devices where the serial port device disappears when the device reboots, | ||
8763 | an additional "serdevtry" wrapper script is provided. To use this | ||
8764 | wrapper, simply prefix the terminal command with | ||
8765 | ``${COREBASE}/scripts/contrib/serdevtry``: | ||
8766 | :: | ||
8767 | |||
8768 | TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0" | ||
8769 | |||
8770 | Running Tests | ||
8771 | ------------- | ||
8772 | |||
8773 | You can start the tests automatically or manually: | ||
8774 | |||
8775 | - *Automatically running tests:* To run the tests automatically after | ||
8776 | the OpenEmbedded build system successfully creates an image, first | ||
8777 | set the | ||
8778 | :term:`TESTIMAGE_AUTO` | ||
8779 | variable to "1" in your ``local.conf`` file in the | ||
8780 | :term:`Build Directory`: | ||
8781 | :: | ||
8782 | |||
8783 | TESTIMAGE_AUTO = "1" | ||
8784 | |||
8785 | Next, build your image. If the image successfully builds, the | ||
8786 | tests run: | ||
8787 | :: | ||
8788 | |||
8789 | bitbake core-image-sato | ||
8790 | |||
8791 | - *Manually running tests:* To manually run the tests, first globally | ||
8792 | inherit the | ||
8793 | :ref:`testimage <ref-classes-testimage*>` class | ||
8794 | by editing your ``local.conf`` file: | ||
8795 | :: | ||
8796 | |||
8797 | INHERIT += "testimage" | ||
8798 | |||
8799 | Next, use BitBake to run the tests: | ||
8800 | :: | ||
8801 | |||
8802 | bitbake -c testimage image | ||
8803 | |||
8804 | All test files reside in ``meta/lib/oeqa/runtime`` in the | ||
8805 | :term:`Source Directory`. A test name maps | ||
8806 | directly to a Python module. Each test module may contain a number of | ||
8807 | individual tests. Tests are usually grouped together by the area tested | ||
8808 | (e.g tests for systemd reside in ``meta/lib/oeqa/runtime/systemd.py``). | ||
8809 | |||
8810 | You can add tests to any layer provided you place them in the proper | ||
8811 | area and you extend :term:`BBPATH` in | ||
8812 | the ``local.conf`` file as normal. Be sure that tests reside in | ||
8813 | ``layer/lib/oeqa/runtime``. | ||
8814 | |||
8815 | .. note:: | ||
8816 | |||
8817 | Be sure that module names do not collide with module names used in | ||
8818 | the default set of test modules in ``meta/lib/oeqa/runtime``. | ||
8819 | |||
8820 | You can change the set of tests run by appending or overriding | ||
8821 | :term:`TEST_SUITES` variable in | ||
8822 | ``local.conf``. Each name in ``TEST_SUITES`` represents a required test | ||
8823 | for the image. Test modules named within ``TEST_SUITES`` cannot be | ||
8824 | skipped even if a test is not suitable for an image (e.g. running the | ||
8825 | RPM tests on an image without ``rpm``). Appending "auto" to | ||
8826 | ``TEST_SUITES`` causes the build system to try to run all tests that are | ||
8827 | suitable for the image (i.e. each test module may elect to skip itself). | ||
8828 | |||
8829 | The order you list tests in ``TEST_SUITES`` is important and influences | ||
8830 | test dependencies. Consequently, tests that depend on other tests should | ||
8831 | be added after the test on which they depend. For example, since the | ||
8832 | ``ssh`` test depends on the ``ping`` test, "ssh" needs to come after | ||
8833 | "ping" in the list. The test class provides no re-ordering or dependency | ||
8834 | handling. | ||
8835 | |||
8836 | .. note:: | ||
8837 | |||
8838 | Each module can have multiple classes with multiple test methods. | ||
8839 | And, Python ``unittest`` rules apply. | ||
8840 | |||
8841 | Here are some things to keep in mind when running tests: | ||
8842 | |||
8843 | - The default tests for the image are defined as: | ||
8844 | :: | ||
8845 | |||
8846 | DEFAULT_TEST_SUITES_pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg" | ||
8847 | |||
8848 | - Add your own test to the list of the by using the following: | ||
8849 | :: | ||
8850 | |||
8851 | TEST_SUITES_append = " mytest" | ||
8852 | |||
8853 | - Run a specific list of tests as follows: | ||
8854 | :: | ||
8855 | |||
8856 | TEST_SUITES = "test1 test2 test3" | ||
8857 | |||
8858 | Remember, order is important. Be sure to place a test that is | ||
8859 | dependent on another test later in the order. | ||
8860 | |||
8861 | Exporting Tests | ||
8862 | --------------- | ||
8863 | |||
8864 | You can export tests so that they can run independently of the build | ||
8865 | system. Exporting tests is required if you want to be able to hand the | ||
8866 | test execution off to a scheduler. You can only export tests that are | ||
8867 | defined in :term:`TEST_SUITES`. | ||
8868 | |||
8869 | If your image is already built, make sure the following are set in your | ||
8870 | ``local.conf`` file: | ||
8871 | :: | ||
8872 | |||
8873 | INHERIT += "testexport" | ||
8874 | TEST_TARGET_IP = "IP-address-for-the-test-target" | ||
8875 | TEST_SERVER_IP = "IP-address-for-the-test-server" | ||
8876 | |||
8877 | You can then export the tests with the | ||
8878 | following BitBake command form: | ||
8879 | :: | ||
8880 | |||
8881 | $ bitbake image -c testexport | ||
8882 | |||
8883 | Exporting the tests places them in the | ||
8884 | :term:`Build Directory` in | ||
8885 | ``tmp/testexport/``\ image, which is controlled by the | ||
8886 | ``TEST_EXPORT_DIR`` variable. | ||
8887 | |||
8888 | You can now run the tests outside of the build environment: | ||
8889 | :: | ||
8890 | |||
8891 | $ cd tmp/testexport/image | ||
8892 | $ ./runexported.py testdata.json | ||
8893 | |||
8894 | Here is a complete example that shows IP addresses and uses the | ||
8895 | ``core-image-sato`` image: | ||
8896 | :: | ||
8897 | |||
8898 | INHERIT += "testexport" | ||
8899 | TEST_TARGET_IP = "192.168.7.2" | ||
8900 | TEST_SERVER_IP = "192.168.7.1" | ||
8901 | |||
8902 | Use BitBake to export the tests: | ||
8903 | :: | ||
8904 | |||
8905 | $ bitbake core-image-sato -c testexport | ||
8906 | |||
8907 | Run the tests outside of | ||
8908 | the build environment using the following: | ||
8909 | :: | ||
8910 | |||
8911 | $ cd tmp/testexport/core-image-sato | ||
8912 | $ ./runexported.py testdata.json | ||
8913 | |||
8914 | Writing New Tests | ||
8915 | ----------------- | ||
8916 | |||
8917 | As mentioned previously, all new test files need to be in the proper | ||
8918 | place for the build system to find them. New tests for additional | ||
8919 | functionality outside of the core should be added to the layer that adds | ||
8920 | the functionality, in ``layer/lib/oeqa/runtime`` (as long as | ||
8921 | :term:`BBPATH` is extended in the | ||
8922 | layer's ``layer.conf`` file as normal). Just remember the following: | ||
8923 | |||
8924 | - Filenames need to map directly to test (module) names. | ||
8925 | |||
8926 | - Do not use module names that collide with existing core tests. | ||
8927 | |||
8928 | - Minimally, an empty ``__init__.py`` file must exist in the runtime | ||
8929 | directory. | ||
8930 | |||
8931 | To create a new test, start by copying an existing module (e.g. | ||
8932 | ``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use | ||
8933 | code from ``meta/lib/oeqa/utils``, which are helper classes. | ||
8934 | |||
8935 | .. note:: | ||
8936 | |||
8937 | Structure shell commands such that you rely on them and they return a | ||
8938 | single code for success. Be aware that sometimes you will need to | ||
8939 | parse the output. See the ``df.py`` and ``date.py`` modules for examples. | ||
8940 | |||
8941 | You will notice that all test classes inherit ``oeRuntimeTest``, which | ||
8942 | is found in ``meta/lib/oetest.py``. This base class offers some helper | ||
8943 | attributes, which are described in the following sections: | ||
8944 | |||
8945 | Class Methods | ||
8946 | ~~~~~~~~~~~~~ | ||
8947 | |||
8948 | Class methods are as follows: | ||
8949 | |||
8950 | - *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed | ||
8951 | package list of the image, which is based on the manifest file that | ||
8952 | is generated during the ``do_rootfs`` task. | ||
8953 | |||
8954 | - *hasFeature(feature):* Returns "True" if the feature is in | ||
8955 | :term:`IMAGE_FEATURES` or | ||
8956 | :term:`DISTRO_FEATURES`. | ||
8957 | |||
8958 | Class Attributes | ||
8959 | ~~~~~~~~~~~~~~~~ | ||
8960 | |||
8961 | Class attributes are as follows: | ||
8962 | |||
8963 | - *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image. | ||
8964 | Otherwise, ``pscmd`` equals "ps" (busybox). | ||
8965 | |||
8966 | - *tc:* The called test context, which gives access to the | ||
8967 | following attributes: | ||
8968 | |||
8969 | - *d:* The BitBake datastore, which allows you to use stuff such | ||
8970 | as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``. | ||
8971 | |||
8972 | - *testslist and testsrequired:* Used internally. The tests | ||
8973 | do not need these. | ||
8974 | |||
8975 | - *filesdir:* The absolute path to | ||
8976 | ``meta/lib/oeqa/runtime/files``, which contains helper files for | ||
8977 | tests meant for copying on the target such as small files written | ||
8978 | in C for compilation. | ||
8979 | |||
8980 | - *target:* The target controller object used to deploy and | ||
8981 | start an image on a particular target (e.g. Qemu, SimpleRemote, | ||
8982 | and SystemdbootTarget). Tests usually use the following: | ||
8983 | |||
8984 | - *ip:* The target's IP address. | ||
8985 | |||
8986 | - *server_ip:* The host's IP address, which is usually used | ||
8987 | by the DNF test suite. | ||
8988 | |||
8989 | - *run(cmd, timeout=None):* The single, most used method. | ||
8990 | This command is a wrapper for: ``ssh root@host "cmd"``. The | ||
8991 | command returns a tuple: (status, output), which are what their | ||
8992 | names imply - the return code of "cmd" and whatever output it | ||
8993 | produces. The optional timeout argument represents the number | ||
8994 | of seconds the test should wait for "cmd" to return. If the | ||
8995 | argument is "None", the test uses the default instance's | ||
8996 | timeout period, which is 300 seconds. If the argument is "0", | ||
8997 | the test runs until the command returns. | ||
8998 | |||
8999 | - *copy_to(localpath, remotepath):* | ||
9000 | ``scp localpath root@ip:remotepath``. | ||
9001 | |||
9002 | - *copy_from(remotepath, localpath):* | ||
9003 | ``scp root@host:remotepath localpath``. | ||
9004 | |||
9005 | Instance Attributes | ||
9006 | ~~~~~~~~~~~~~~~~~~~ | ||
9007 | |||
9008 | A single instance attribute exists, which is ``target``. The ``target`` | ||
9009 | instance attribute is identical to the class attribute of the same name, | ||
9010 | which is described in the previous section. This attribute exists as | ||
9011 | both an instance and class attribute so tests can use | ||
9012 | ``self.target.run(cmd)`` in instance methods instead of | ||
9013 | ``oeRuntimeTest.tc.target.run(cmd)``. | ||
9014 | |||
9015 | Installing Packages in the DUT Without the Package Manager | ||
9016 | ---------------------------------------------------------- | ||
9017 | |||
9018 | When a test requires a package built by BitBake, it is possible to | ||
9019 | install that package. Installing the package does not require a package | ||
9020 | manager be installed in the device under test (DUT). It does, however, | ||
9021 | require an SSH connection and the target must be using the | ||
9022 | ``sshcontrol`` class. | ||
9023 | |||
9024 | .. note:: | ||
9025 | |||
9026 | This method uses ``scp`` to copy files from the host to the target, which | ||
9027 | causes permissions and special attributes to be lost. | ||
9028 | |||
9029 | A JSON file is used to define the packages needed by a test. This file | ||
9030 | must be in the same path as the file used to define the tests. | ||
9031 | Furthermore, the filename must map directly to the test module name with | ||
9032 | a ``.json`` extension. | ||
9033 | |||
9034 | The JSON file must include an object with the test name as keys of an | ||
9035 | object or an array. This object (or array of objects) uses the following | ||
9036 | data: | ||
9037 | |||
9038 | - "pkg" - A mandatory string that is the name of the package to be | ||
9039 | installed. | ||
9040 | |||
9041 | - "rm" - An optional boolean, which defaults to "false", that specifies | ||
9042 | to remove the package after the test. | ||
9043 | |||
9044 | - "extract" - An optional boolean, which defaults to "false", that | ||
9045 | specifies if the package must be extracted from the package format. | ||
9046 | When set to "true", the package is not automatically installed into | ||
9047 | the DUT. | ||
9048 | |||
9049 | Following is an example JSON file that handles test "foo" installing | ||
9050 | package "bar" and test "foobar" installing packages "foo" and "bar". | ||
9051 | Once the test is complete, the packages are removed from the DUT. | ||
9052 | :: | ||
9053 | |||
9054 | { | ||
9055 | "foo": { | ||
9056 | "pkg": "bar" | ||
9057 | }, | ||
9058 | "foobar": [ | ||
9059 | { | ||
9060 | "pkg": "foo", | ||
9061 | "rm": true | ||
9062 | }, | ||
9063 | { | ||
9064 | "pkg": "bar", | ||
9065 | "rm": true | ||
9066 | } | ||
9067 | ] | ||
9068 | } | ||
9069 | |||
9070 | Debugging Tools and Techniques | ||
9071 | ============================== | ||
9072 | |||
9073 | The exact method for debugging build failures depends on the nature of | ||
9074 | the problem and on the system's area from which the bug originates. | ||
9075 | Standard debugging practices such as comparison against the last known | ||
9076 | working version with examination of the changes and the re-application | ||
9077 | of steps to identify the one causing the problem are valid for the Yocto | ||
9078 | Project just as they are for any other system. Even though it is | ||
9079 | impossible to detail every possible potential failure, this section | ||
9080 | provides some general tips to aid in debugging given a variety of | ||
9081 | situations. | ||
9082 | |||
9083 | .. note:: | ||
9084 | |||
9085 | A useful feature for debugging is the error reporting tool. | ||
9086 | Configuring the Yocto Project to use this tool causes the | ||
9087 | OpenEmbedded build system to produce error reporting commands as part | ||
9088 | of the console output. You can enter the commands after the build | ||
9089 | completes to log error information into a common database, that can | ||
9090 | help you figure out what might be going wrong. For information on how | ||
9091 | to enable and use this feature, see the | ||
9092 | ":ref:`dev-manual/common-tasks:using the error reporting tool`" | ||
9093 | section. | ||
9094 | |||
9095 | The following list shows the debugging topics in the remainder of this | ||
9096 | section: | ||
9097 | |||
9098 | - "`Viewing Logs from Failed | ||
9099 | Tasks <#dev-debugging-viewing-logs-from-failed-tasks>`__" describes | ||
9100 | how to find and view logs from tasks that failed during the build | ||
9101 | process. | ||
9102 | |||
9103 | - "`Viewing Variable | ||
9104 | Values <#dev-debugging-viewing-variable-values>`__" describes how to | ||
9105 | use the BitBake ``-e`` option to examine variable values after a | ||
9106 | recipe has been parsed. | ||
9107 | |||
9108 | - ":ref:`dev-manual/common-tasks:viewing package information with \`\`oe-pkgdata-util\`\``" | ||
9109 | describes how to use the ``oe-pkgdata-util`` utility to query | ||
9110 | :term:`PKGDATA_DIR` and | ||
9111 | display package-related information for built packages. | ||
9112 | |||
9113 | - "`Viewing Dependencies Between Recipes and | ||
9114 | Tasks <#dev-viewing-dependencies-between-recipes-and-tasks>`__" | ||
9115 | describes how to use the BitBake ``-g`` option to display recipe | ||
9116 | dependency information used during the build. | ||
9117 | |||
9118 | - "`Viewing Task Variable | ||
9119 | Dependencies <#dev-viewing-task-variable-dependencies>`__" describes | ||
9120 | how to use the ``bitbake-dumpsig`` command in conjunction with key | ||
9121 | subdirectories in the | ||
9122 | :term:`Build Directory` to determine | ||
9123 | variable dependencies. | ||
9124 | |||
9125 | - "`Running Specific Tasks <#dev-debugging-taskrunning>`__" describes | ||
9126 | how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``) | ||
9127 | to run specific tasks in the build chain. It can be useful to run | ||
9128 | tasks "out-of-order" when trying isolate build issues. | ||
9129 | |||
9130 | - "`General BitBake Problems <#dev-debugging-bitbake>`__" describes how | ||
9131 | to use BitBake's ``-D`` debug output option to reveal more about what | ||
9132 | BitBake is doing during the build. | ||
9133 | |||
9134 | - "`Building with No Dependencies <#dev-debugging-buildfile>`__" | ||
9135 | describes how to use the BitBake ``-b`` option to build a recipe | ||
9136 | while ignoring dependencies. | ||
9137 | |||
9138 | - "`Recipe Logging Mechanisms <#recipe-logging-mechanisms>`__" | ||
9139 | describes how to use the many recipe logging functions to produce | ||
9140 | debugging output and report errors and warnings. | ||
9141 | |||
9142 | - "`Debugging Parallel Make Races <#debugging-parallel-make-races>`__" | ||
9143 | describes how to debug situations where the build consists of several | ||
9144 | parts that are run simultaneously and when the output or result of | ||
9145 | one part is not ready for use with a different part of the build that | ||
9146 | depends on that output. | ||
9147 | |||
9148 | - "`Debugging With the GNU Project Debugger (GDB) | ||
9149 | Remotely <#platdev-gdb-remotedebug>`__" describes how to use GDB to | ||
9150 | allow you to examine running programs, which can help you fix | ||
9151 | problems. | ||
9152 | |||
9153 | - "`Debugging with the GNU Project Debugger (GDB) on the | ||
9154 | Target <#debugging-with-the-gnu-project-debugger-gdb-on-the-target>`__" | ||
9155 | describes how to use GDB directly on target hardware for debugging. | ||
9156 | |||
9157 | - "`Other Debugging Tips <#dev-other-debugging-others>`__" describes | ||
9158 | miscellaneous debugging tips that can be useful. | ||
9159 | |||
9160 | Viewing Logs from Failed Tasks | ||
9161 | ------------------------------ | ||
9162 | |||
9163 | You can find the log for a task in the file | ||
9164 | ``${``\ :term:`WORKDIR`\ ``}/temp/log.do_``\ `taskname`. | ||
9165 | For example, the log for the | ||
9166 | :ref:`ref-tasks-compile` task of the | ||
9167 | QEMU minimal image for the x86 machine (``qemux86``) might be in | ||
9168 | ``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``. | ||
9169 | To see the commands :term:`BitBake` ran | ||
9170 | to generate a log, look at the corresponding ``run.do_``\ `taskname` file | ||
9171 | in the same directory. | ||
9172 | |||
9173 | ``log.do_``\ `taskname` and ``run.do_``\ `taskname` are actually symbolic | ||
9174 | links to ``log.do_``\ `taskname`\ ``.``\ `pid` and | ||
9175 | ``log.run_``\ `taskname`\ ``.``\ `pid`, where `pid` is the PID the task had | ||
9176 | when it ran. The symlinks always point to the files corresponding to the | ||
9177 | most recent run. | ||
9178 | |||
9179 | Viewing Variable Values | ||
9180 | ----------------------- | ||
9181 | |||
9182 | Sometimes you need to know the value of a variable as a result of | ||
9183 | BitBake's parsing step. This could be because some unexpected behavior | ||
9184 | occurred in your project. Perhaps an attempt to :ref:`modify a variable | ||
9185 | <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:modifying existing | ||
9186 | variables>` did not work out as expected. | ||
9187 | |||
9188 | BitBake's ``-e`` option is used to display variable values after | ||
9189 | parsing. The following command displays the variable values after the | ||
9190 | configuration files (i.e. ``local.conf``, ``bblayers.conf``, | ||
9191 | ``bitbake.conf`` and so forth) have been parsed: | ||
9192 | :: | ||
9193 | |||
9194 | $ bitbake -e | ||
9195 | |||
9196 | The following command displays variable values after a specific recipe has | ||
9197 | been parsed. The variables include those from the configuration as well: | ||
9198 | :: | ||
9199 | |||
9200 | $ bitbake -e recipename | ||
9201 | |||
9202 | .. note:: | ||
9203 | |||
9204 | Each recipe has its own private set of variables (datastore). | ||
9205 | Internally, after parsing the configuration, a copy of the resulting | ||
9206 | datastore is made prior to parsing each recipe. This copying implies | ||
9207 | that variables set in one recipe will not be visible to other | ||
9208 | recipes. | ||
9209 | |||
9210 | Likewise, each task within a recipe gets a private datastore based on | ||
9211 | the recipe datastore, which means that variables set within one task | ||
9212 | will not be visible to other tasks. | ||
9213 | |||
9214 | In the output of ``bitbake -e``, each variable is preceded by a | ||
9215 | description of how the variable got its value, including temporary | ||
9216 | values that were later overridden. This description also includes | ||
9217 | variable flags (varflags) set on the variable. The output can be very | ||
9218 | helpful during debugging. | ||
9219 | |||
9220 | Variables that are exported to the environment are preceded by | ||
9221 | ``export`` in the output of ``bitbake -e``. See the following example: | ||
9222 | :: | ||
9223 | |||
9224 | export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" | ||
9225 | |||
9226 | In addition to variable values, the output of the ``bitbake -e`` and | ||
9227 | ``bitbake -e`` recipe commands includes the following information: | ||
9228 | |||
9229 | - The output starts with a tree listing all configuration files and | ||
9230 | classes included globally, recursively listing the files they include | ||
9231 | or inherit in turn. Much of the behavior of the OpenEmbedded build | ||
9232 | system (including the behavior of the :ref:`ref-manual/tasks:normal recipe build tasks`) is | ||
9233 | implemented in the | ||
9234 | :ref:`base <ref-classes-base>` class and the | ||
9235 | classes it inherits, rather than being built into BitBake itself. | ||
9236 | |||
9237 | - After the variable values, all functions appear in the output. For | ||
9238 | shell functions, variables referenced within the function body are | ||
9239 | expanded. If a function has been modified using overrides or using | ||
9240 | override-style operators like ``_append`` and ``_prepend``, then the | ||
9241 | final assembled function body appears in the output. | ||
9242 | |||
9243 | Viewing Package Information with ``oe-pkgdata-util`` | ||
9244 | ---------------------------------------------------- | ||
9245 | |||
9246 | You can use the ``oe-pkgdata-util`` command-line utility to query | ||
9247 | :term:`PKGDATA_DIR` and display | ||
9248 | various package-related information. When you use the utility, you must | ||
9249 | use it to view information on packages that have already been built. | ||
9250 | |||
9251 | Following are a few of the available ``oe-pkgdata-util`` subcommands. | ||
9252 | |||
9253 | .. note:: | ||
9254 | |||
9255 | You can use the standard \* and ? globbing wildcards as part of | ||
9256 | package names and paths. | ||
9257 | |||
9258 | - ``oe-pkgdata-util list-pkgs [pattern]``: Lists all packages | ||
9259 | that have been built, optionally limiting the match to packages that | ||
9260 | match pattern. | ||
9261 | |||
9262 | - ``oe-pkgdata-util list-pkg-files package ...``: Lists the | ||
9263 | files and directories contained in the given packages. | ||
9264 | |||
9265 | .. note:: | ||
9266 | |||
9267 | A different way to view the contents of a package is to look at | ||
9268 | the | ||
9269 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
9270 | directory of the recipe that generates the package. This directory | ||
9271 | is created by the | ||
9272 | :ref:`ref-tasks-package` task | ||
9273 | and has one subdirectory for each package the recipe generates, | ||
9274 | which contains the files stored in that package. | ||
9275 | |||
9276 | If you want to inspect the ``${WORKDIR}/packages-split`` | ||
9277 | directory, make sure that | ||
9278 | :ref:`rm_work <ref-classes-rm-work>` is not | ||
9279 | enabled when you build the recipe. | ||
9280 | |||
9281 | - ``oe-pkgdata-util find-path path ...``: Lists the names of | ||
9282 | the packages that contain the given paths. For example, the following | ||
9283 | tells us that ``/usr/share/man/man1/make.1`` is contained in the | ||
9284 | ``make-doc`` package: | ||
9285 | :: | ||
9286 | |||
9287 | $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 | ||
9288 | make-doc: /usr/share/man/man1/make.1 | ||
9289 | |||
9290 | - ``oe-pkgdata-util lookup-recipe package ...``: Lists the name | ||
9291 | of the recipes that produce the given packages. | ||
9292 | |||
9293 | For more information on the ``oe-pkgdata-util`` command, use the help | ||
9294 | facility: | ||
9295 | :: | ||
9296 | |||
9297 | $ oe-pkgdata-util --help | ||
9298 | $ oe-pkgdata-util subcommand --help | ||
9299 | |||
9300 | Viewing Dependencies Between Recipes and Tasks | ||
9301 | ---------------------------------------------- | ||
9302 | |||
9303 | Sometimes it can be hard to see why BitBake wants to build other recipes | ||
9304 | before the one you have specified. Dependency information can help you | ||
9305 | understand why a recipe is built. | ||
9306 | |||
9307 | To generate dependency information for a recipe, run the following | ||
9308 | command: | ||
9309 | :: | ||
9310 | |||
9311 | $ bitbake -g recipename | ||
9312 | |||
9313 | This command writes the following files in the current directory: | ||
9314 | |||
9315 | - ``pn-buildlist``: A list of recipes/targets involved in building | ||
9316 | `recipename`. "Involved" here means that at least one task from the | ||
9317 | recipe needs to run when building `recipename` from scratch. Targets | ||
9318 | that are in | ||
9319 | :term:`ASSUME_PROVIDED` | ||
9320 | are not listed. | ||
9321 | |||
9322 | - ``task-depends.dot``: A graph showing dependencies between tasks. | ||
9323 | |||
9324 | The graphs are in | ||
9325 | `DOT <https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29>`__ | ||
9326 | format and can be converted to images (e.g. using the ``dot`` tool from | ||
9327 | `Graphviz <https://www.graphviz.org/>`__). | ||
9328 | |||
9329 | .. note:: | ||
9330 | |||
9331 | - DOT files use a plain text format. The graphs generated using the | ||
9332 | ``bitbake -g`` command are often so large as to be difficult to | ||
9333 | read without special pruning (e.g. with Bitbake's ``-I`` option) | ||
9334 | and processing. Despite the form and size of the graphs, the | ||
9335 | corresponding ``.dot`` files can still be possible to read and | ||
9336 | provide useful information. | ||
9337 | |||
9338 | As an example, the ``task-depends.dot`` file contains lines such | ||
9339 | as the following: | ||
9340 | :: | ||
9341 | |||
9342 | "libxslt.do_configure" -> "libxml2.do_populate_sysroot" | ||
9343 | |||
9344 | The above example line reveals that the | ||
9345 | :ref:`ref-tasks-configure` | ||
9346 | task in ``libxslt`` depends on the | ||
9347 | :ref:`ref-tasks-populate_sysroot` | ||
9348 | task in ``libxml2``, which is a normal | ||
9349 | :term:`DEPENDS` dependency | ||
9350 | between the two recipes. | ||
9351 | |||
9352 | - For an example of how ``.dot`` files can be processed, see the | ||
9353 | ``scripts/contrib/graph-tool`` Python script, which finds and | ||
9354 | displays paths between graph nodes. | ||
9355 | |||
9356 | You can use a different method to view dependency information by using | ||
9357 | the following command: | ||
9358 | :: | ||
9359 | |||
9360 | $ bitbake -g -u taskexp recipename | ||
9361 | |||
9362 | This command | ||
9363 | displays a GUI window from which you can view build-time and runtime | ||
9364 | dependencies for the recipes involved in building recipename. | ||
9365 | |||
9366 | Viewing Task Variable Dependencies | ||
9367 | ---------------------------------- | ||
9368 | |||
9369 | As mentioned in the | ||
9370 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section of the BitBake | ||
9371 | User Manual, BitBake tries to automatically determine what variables a | ||
9372 | task depends on so that it can rerun the task if any values of the | ||
9373 | variables change. This determination is usually reliable. However, if | ||
9374 | you do things like construct variable names at runtime, then you might | ||
9375 | have to manually declare dependencies on those variables using | ||
9376 | ``vardeps`` as described in the | ||
9377 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section of the BitBake | ||
9378 | User Manual. | ||
9379 | |||
9380 | If you are unsure whether a variable dependency is being picked up | ||
9381 | automatically for a given task, you can list the variable dependencies | ||
9382 | BitBake has determined by doing the following: | ||
9383 | |||
9384 | 1. Build the recipe containing the task: | ||
9385 | :: | ||
9386 | |||
9387 | $ bitbake recipename | ||
9388 | |||
9389 | 2. Inside the :term:`STAMPS_DIR` | ||
9390 | directory, find the signature data (``sigdata``) file that | ||
9391 | corresponds to the task. The ``sigdata`` files contain a pickled | ||
9392 | Python database of all the metadata that went into creating the input | ||
9393 | checksum for the task. As an example, for the | ||
9394 | :ref:`ref-tasks-fetch` task of the | ||
9395 | ``db`` recipe, the ``sigdata`` file might be found in the following | ||
9396 | location: | ||
9397 | :: | ||
9398 | |||
9399 | ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
9400 | |||
9401 | For tasks that are accelerated through the shared state | ||
9402 | (:ref:`sstate <overview-manual/concepts:shared state cache>`) cache, an | ||
9403 | additional ``siginfo`` file is written into | ||
9404 | :term:`SSTATE_DIR` along with | ||
9405 | the cached task output. The ``siginfo`` files contain exactly the | ||
9406 | same information as ``sigdata`` files. | ||
9407 | |||
9408 | 3. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here | ||
9409 | is an example: | ||
9410 | :: | ||
9411 | |||
9412 | $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
9413 | |||
9414 | In the output of the above command, you will find a line like the | ||
9415 | following, which lists all the (inferred) variable dependencies for | ||
9416 | the task. This list also includes indirect dependencies from | ||
9417 | variables depending on other variables, recursively. | ||
9418 | :: | ||
9419 | |||
9420 | Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] | ||
9421 | |||
9422 | .. note:: | ||
9423 | |||
9424 | Functions (e.g. ``base_do_fetch``) also count as variable dependencies. | ||
9425 | These functions in turn depend on the variables they reference. | ||
9426 | |||
9427 | The output of ``bitbake-dumpsig`` also includes the value each | ||
9428 | variable had, a list of dependencies for each variable, and | ||
9429 | :term:`bitbake:BB_HASHBASE_WHITELIST` | ||
9430 | information. | ||
9431 | |||
9432 | There is also a ``bitbake-diffsigs`` command for comparing two | ||
9433 | ``siginfo`` or ``sigdata`` files. This command can be helpful when | ||
9434 | trying to figure out what changed between two versions of a task. If you | ||
9435 | call ``bitbake-diffsigs`` with just one file, the command behaves like | ||
9436 | ``bitbake-dumpsig``. | ||
9437 | |||
9438 | You can also use BitBake to dump out the signature construction | ||
9439 | information without executing tasks by using either of the following | ||
9440 | BitBake command-line options: | ||
9441 | :: | ||
9442 | |||
9443 | ‐‐dump-signatures=SIGNATURE_HANDLER | ||
9444 | -S SIGNATURE_HANDLER | ||
9445 | |||
9446 | |||
9447 | .. note:: | ||
9448 | |||
9449 | Two common values for `SIGNATURE_HANDLER` are "none" and "printdiff", which | ||
9450 | dump only the signature or compare the dumped signature with the cached one, | ||
9451 | respectively. | ||
9452 | |||
9453 | Using BitBake with either of these options causes BitBake to dump out | ||
9454 | ``sigdata`` files in the ``stamps`` directory for every task it would | ||
9455 | have executed instead of building the specified target package. | ||
9456 | |||
9457 | Viewing Metadata Used to Create the Input Signature of a Shared State Task | ||
9458 | -------------------------------------------------------------------------- | ||
9459 | |||
9460 | Seeing what metadata went into creating the input signature of a shared | ||
9461 | state (sstate) task can be a useful debugging aid. This information is | ||
9462 | available in signature information (``siginfo``) files in | ||
9463 | :term:`SSTATE_DIR`. For | ||
9464 | information on how to view and interpret information in ``siginfo`` | ||
9465 | files, see the "`Viewing Task Variable | ||
9466 | Dependencies <#dev-viewing-task-variable-dependencies>`__" section. | ||
9467 | |||
9468 | For conceptual information on shared state, see the | ||
9469 | ":ref:`overview-manual/concepts:shared state`" | ||
9470 | section in the Yocto Project Overview and Concepts Manual. | ||
9471 | |||
9472 | Invalidating Shared State to Force a Task to Run | ||
9473 | ------------------------------------------------ | ||
9474 | |||
9475 | The OpenEmbedded build system uses | ||
9476 | :ref:`checksums <overview-manual/concepts:checksums (signatures)>` and | ||
9477 | :ref:`overview-manual/concepts:shared state` cache to avoid unnecessarily | ||
9478 | rebuilding tasks. Collectively, this scheme is known as "shared state | ||
9479 | code". | ||
9480 | |||
9481 | As with all schemes, this one has some drawbacks. It is possible that | ||
9482 | you could make implicit changes to your code that the checksum | ||
9483 | calculations do not take into account. These implicit changes affect a | ||
9484 | task's output but do not trigger the shared state code into rebuilding a | ||
9485 | recipe. Consider an example during which a tool changes its output. | ||
9486 | Assume that the output of ``rpmdeps`` changes. The result of the change | ||
9487 | should be that all the ``package`` and ``package_write_rpm`` shared | ||
9488 | state cache items become invalid. However, because the change to the | ||
9489 | output is external to the code and therefore implicit, the associated | ||
9490 | shared state cache items do not become invalidated. In this case, the | ||
9491 | build process uses the cached items rather than running the task again. | ||
9492 | Obviously, these types of implicit changes can cause problems. | ||
9493 | |||
9494 | To avoid these problems during the build, you need to understand the | ||
9495 | effects of any changes you make. Realize that changes you make directly | ||
9496 | to a function are automatically factored into the checksum calculation. | ||
9497 | Thus, these explicit changes invalidate the associated area of shared | ||
9498 | state cache. However, you need to be aware of any implicit changes that | ||
9499 | are not obvious changes to the code and could affect the output of a | ||
9500 | given task. | ||
9501 | |||
9502 | When you identify an implicit change, you can easily take steps to | ||
9503 | invalidate the cache and force the tasks to run. The steps you can take | ||
9504 | are as simple as changing a function's comments in the source code. For | ||
9505 | example, to invalidate package shared state files, change the comment | ||
9506 | statements of | ||
9507 | :ref:`ref-tasks-package` or the | ||
9508 | comments of one of the functions it calls. Even though the change is | ||
9509 | purely cosmetic, it causes the checksum to be recalculated and forces | ||
9510 | the build system to run the task again. | ||
9511 | |||
9512 | .. note:: | ||
9513 | |||
9514 | For an example of a commit that makes a cosmetic change to invalidate | ||
9515 | shared state, see this | ||
9516 | :yocto_git:`commit </poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54>`. | ||
9517 | |||
9518 | Running Specific Tasks | ||
9519 | ---------------------- | ||
9520 | |||
9521 | Any given recipe consists of a set of tasks. The standard BitBake | ||
9522 | behavior in most cases is: ``do_fetch``, ``do_unpack``, ``do_patch``, | ||
9523 | ``do_configure``, ``do_compile``, ``do_install``, ``do_package``, | ||
9524 | ``do_package_write_*``, and ``do_build``. The default task is | ||
9525 | ``do_build`` and any tasks on which it depends build first. Some tasks, | ||
9526 | such as ``do_devshell``, are not part of the default build chain. If you | ||
9527 | wish to run a task that is not part of the default build chain, you can | ||
9528 | use the ``-c`` option in BitBake. Here is an example: | ||
9529 | :: | ||
9530 | |||
9531 | $ bitbake matchbox-desktop -c devshell | ||
9532 | |||
9533 | The ``-c`` option respects task dependencies, which means that all other | ||
9534 | tasks (including tasks from other recipes) that the specified task | ||
9535 | depends on will be run before the task. Even when you manually specify a | ||
9536 | task to run with ``-c``, BitBake will only run the task if it considers | ||
9537 | it "out of date". See the | ||
9538 | ":ref:`overview-manual/concepts:stamp files and the rerunning of tasks`" | ||
9539 | section in the Yocto Project Overview and Concepts Manual for how | ||
9540 | BitBake determines whether a task is "out of date". | ||
9541 | |||
9542 | If you want to force an up-to-date task to be rerun (e.g. because you | ||
9543 | made manual modifications to the recipe's | ||
9544 | :term:`WORKDIR` that you want to try | ||
9545 | out), then you can use the ``-f`` option. | ||
9546 | |||
9547 | .. note:: | ||
9548 | |||
9549 | The reason ``-f`` is never required when running the | ||
9550 | :ref:`ref-tasks-devshell` task is because the | ||
9551 | [\ :ref:`nostamp <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ] | ||
9552 | variable flag is already set for the task. | ||
9553 | |||
9554 | The following example shows one way you can use the ``-f`` option: | ||
9555 | :: | ||
9556 | |||
9557 | $ bitbake matchbox-desktop | ||
9558 | . | ||
9559 | . | ||
9560 | make some changes to the source code in the work directory | ||
9561 | . | ||
9562 | . | ||
9563 | $ bitbake matchbox-desktop -c compile -f | ||
9564 | $ bitbake matchbox-desktop | ||
9565 | |||
9566 | This sequence first builds and then recompiles ``matchbox-desktop``. The | ||
9567 | last command reruns all tasks (basically the packaging tasks) after the | ||
9568 | compile. BitBake recognizes that the ``do_compile`` task was rerun and | ||
9569 | therefore understands that the other tasks also need to be run again. | ||
9570 | |||
9571 | Another, shorter way to rerun a task and all | ||
9572 | :ref:`ref-manual/tasks:normal recipe build tasks` | ||
9573 | that depend on it is to use the ``-C`` option. | ||
9574 | |||
9575 | .. note:: | ||
9576 | |||
9577 | This option is upper-cased and is separate from the ``-c`` | ||
9578 | option, which is lower-cased. | ||
9579 | |||
9580 | Using this option invalidates the given task and then runs the | ||
9581 | :ref:`ref-tasks-build` task, which is | ||
9582 | the default task if no task is given, and the tasks on which it depends. | ||
9583 | You could replace the final two commands in the previous example with | ||
9584 | the following single command: | ||
9585 | :: | ||
9586 | |||
9587 | $ bitbake matchbox-desktop -C compile | ||
9588 | |||
9589 | Internally, the ``-f`` and ``-C`` options work by tainting (modifying) | ||
9590 | the input checksum of the specified task. This tainting indirectly | ||
9591 | causes the task and its dependent tasks to be rerun through the normal | ||
9592 | task dependency mechanisms. | ||
9593 | |||
9594 | .. note:: | ||
9595 | |||
9596 | BitBake explicitly keeps track of which tasks have been tainted in | ||
9597 | this fashion, and will print warnings such as the following for | ||
9598 | builds involving such tasks: | ||
9599 | |||
9600 | .. code-block:: none | ||
9601 | |||
9602 | WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run | ||
9603 | |||
9604 | |||
9605 | The purpose of the warning is to let you know that the work directory | ||
9606 | and build output might not be in the clean state they would be in for | ||
9607 | a "normal" build, depending on what actions you took. To get rid of | ||
9608 | such warnings, you can remove the work directory and rebuild the | ||
9609 | recipe, as follows: | ||
9610 | :: | ||
9611 | |||
9612 | $ bitbake matchbox-desktop -c clean | ||
9613 | $ bitbake matchbox-desktop | ||
9614 | |||
9615 | |||
9616 | You can view a list of tasks in a given package by running the | ||
9617 | ``do_listtasks`` task as follows: | ||
9618 | :: | ||
9619 | |||
9620 | $ bitbake matchbox-desktop -c listtasks | ||
9621 | |||
9622 | The results appear as output to the console and are also in | ||
9623 | the file ``${WORKDIR}/temp/log.do_listtasks``. | ||
9624 | |||
9625 | General BitBake Problems | ||
9626 | ------------------------ | ||
9627 | |||
9628 | You can see debug output from BitBake by using the ``-D`` option. The | ||
9629 | debug output gives more information about what BitBake is doing and the | ||
9630 | reason behind it. Each ``-D`` option you use increases the logging | ||
9631 | level. The most common usage is ``-DDD``. | ||
9632 | |||
9633 | The output from ``bitbake -DDD -v targetname`` can reveal why BitBake | ||
9634 | chose a certain version of a package or why BitBake picked a certain | ||
9635 | provider. This command could also help you in a situation where you | ||
9636 | think BitBake did something unexpected. | ||
9637 | |||
9638 | Building with No Dependencies | ||
9639 | ----------------------------- | ||
9640 | |||
9641 | To build a specific recipe (``.bb`` file), you can use the following | ||
9642 | command form: | ||
9643 | :: | ||
9644 | |||
9645 | $ bitbake -b somepath/somerecipe.bb | ||
9646 | |||
9647 | This command form does | ||
9648 | not check for dependencies. Consequently, you should use it only when | ||
9649 | you know existing dependencies have been met. | ||
9650 | |||
9651 | .. note:: | ||
9652 | |||
9653 | You can also specify fragments of the filename. In this case, BitBake | ||
9654 | checks for a unique match. | ||
9655 | |||
9656 | Recipe Logging Mechanisms | ||
9657 | ------------------------- | ||
9658 | |||
9659 | The Yocto Project provides several logging functions for producing | ||
9660 | debugging output and reporting errors and warnings. For Python | ||
9661 | functions, the following logging functions exist. All of these functions | ||
9662 | log to ``${T}/log.do_``\ `task`, and can also log to standard output | ||
9663 | (stdout) with the right settings: | ||
9664 | |||
9665 | - ``bb.plain(msg)``: Writes msg as is to the log while also | ||
9666 | logging to stdout. | ||
9667 | |||
9668 | - ``bb.note(msg)``: Writes "NOTE: msg" to the log. Also logs to | ||
9669 | stdout if BitBake is called with "-v". | ||
9670 | |||
9671 | - ``bb.debug(level, msg)``: Writes "DEBUG: msg" to the | ||
9672 | log. Also logs to stdout if the log level is greater than or equal to | ||
9673 | level. See the ":ref:`-D <bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax>`" option | ||
9674 | in the BitBake User Manual for more information. | ||
9675 | |||
9676 | - ``bb.warn(msg)``: Writes "WARNING: msg" to the log while also | ||
9677 | logging to stdout. | ||
9678 | |||
9679 | - ``bb.error(msg)``: Writes "ERROR: msg" to the log while also | ||
9680 | logging to standard out (stdout). | ||
9681 | |||
9682 | .. note:: | ||
9683 | |||
9684 | Calling this function does not cause the task to fail. | ||
9685 | |||
9686 | - ``bb.fatal(``\ msg\ ``)``: This logging function is similar to | ||
9687 | ``bb.error(``\ msg\ ``)`` but also causes the calling task to fail. | ||
9688 | |||
9689 | .. note:: | ||
9690 | |||
9691 | ``bb.fatal()`` raises an exception, which means you do not need to put a | ||
9692 | "return" statement after the function. | ||
9693 | |||
9694 | The same logging functions are also available in shell functions, under | ||
9695 | the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``, | ||
9696 | and ``bbfatal``. The | ||
9697 | :ref:`logging <ref-classes-logging>` class | ||
9698 | implements these functions. See that class in the ``meta/classes`` | ||
9699 | folder of the :term:`Source Directory` for information. | ||
9700 | |||
9701 | Logging With Python | ||
9702 | ~~~~~~~~~~~~~~~~~~~ | ||
9703 | |||
9704 | When creating recipes using Python and inserting code that handles build | ||
9705 | logs, keep in mind the goal is to have informative logs while keeping | ||
9706 | the console as "silent" as possible. Also, if you want status messages | ||
9707 | in the log, use the "debug" loglevel. | ||
9708 | |||
9709 | Following is an example written in Python. The code handles logging for | ||
9710 | a function that determines the number of tasks needed to be run. See the | ||
9711 | ":ref:`ref-tasks-listtasks`" | ||
9712 | section for additional information: | ||
9713 | :: | ||
9714 | |||
9715 | python do_listtasks() { | ||
9716 | bb.debug(2, "Starting to figure out the task list") | ||
9717 | if noteworthy_condition: | ||
9718 | bb.note("There are 47 tasks to run") | ||
9719 | bb.debug(2, "Got to point xyz") | ||
9720 | if warning_trigger: | ||
9721 | bb.warn("Detected warning_trigger, this might be a problem later.") | ||
9722 | if recoverable_error: | ||
9723 | bb.error("Hit recoverable_error, you really need to fix this!") | ||
9724 | if fatal_error: | ||
9725 | bb.fatal("fatal_error detected, unable to print the task list") | ||
9726 | bb.plain("The tasks present are abc") | ||
9727 | bb.debug(2, "Finished figuring out the tasklist") | ||
9728 | } | ||
9729 | |||
9730 | Logging With Bash | ||
9731 | ~~~~~~~~~~~~~~~~~ | ||
9732 | |||
9733 | When creating recipes using Bash and inserting code that handles build | ||
9734 | logs, you have the same goals - informative with minimal console output. | ||
9735 | The syntax you use for recipes written in Bash is similar to that of | ||
9736 | recipes written in Python described in the previous section. | ||
9737 | |||
9738 | Following is an example written in Bash. The code logs the progress of | ||
9739 | the ``do_my_function`` function. | ||
9740 | :: | ||
9741 | |||
9742 | do_my_function() { | ||
9743 | bbdebug 2 "Running do_my_function" | ||
9744 | if [ exceptional_condition ]; then | ||
9745 | bbnote "Hit exceptional_condition" | ||
9746 | fi | ||
9747 | bbdebug 2 "Got to point xyz" | ||
9748 | if [ warning_trigger ]; then | ||
9749 | bbwarn "Detected warning_trigger, this might cause a problem later." | ||
9750 | fi | ||
9751 | if [ recoverable_error ]; then | ||
9752 | bberror "Hit recoverable_error, correcting" | ||
9753 | fi | ||
9754 | if [ fatal_error ]; then | ||
9755 | bbfatal "fatal_error detected" | ||
9756 | fi | ||
9757 | bbdebug 2 "Completed do_my_function" | ||
9758 | } | ||
9759 | |||
9760 | |||
9761 | Debugging Parallel Make Races | ||
9762 | ----------------------------- | ||
9763 | |||
9764 | A parallel ``make`` race occurs when the build consists of several parts | ||
9765 | that are run simultaneously and a situation occurs when the output or | ||
9766 | result of one part is not ready for use with a different part of the | ||
9767 | build that depends on that output. Parallel make races are annoying and | ||
9768 | can sometimes be difficult to reproduce and fix. However, some simple | ||
9769 | tips and tricks exist that can help you debug and fix them. This section | ||
9770 | presents a real-world example of an error encountered on the Yocto | ||
9771 | Project autobuilder and the process used to fix it. | ||
9772 | |||
9773 | .. note:: | ||
9774 | |||
9775 | If you cannot properly fix a ``make`` race condition, you can work around it | ||
9776 | by clearing either the :term:`PARALLEL_MAKE` or :term:`PARALLEL_MAKEINST` | ||
9777 | variables. | ||
9778 | |||
9779 | The Failure | ||
9780 | ~~~~~~~~~~~ | ||
9781 | |||
9782 | For this example, assume that you are building an image that depends on | ||
9783 | the "neard" package. And, during the build, BitBake runs into problems | ||
9784 | and creates the following output. | ||
9785 | |||
9786 | .. note:: | ||
9787 | |||
9788 | This example log file has longer lines artificially broken to make | ||
9789 | the listing easier to read. | ||
9790 | |||
9791 | If you examine the output or the log file, you see the failure during | ||
9792 | ``make``: | ||
9793 | |||
9794 | .. code-block:: none | ||
9795 | |||
9796 | | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] | ||
9797 | | DEBUG: Executing shell function do_compile | ||
9798 | | NOTE: make -j 16 | ||
9799 | | make --no-print-directory all-am | ||
9800 | | /bin/mkdir -p include/near | ||
9801 | | /bin/mkdir -p include/near | ||
9802 | | /bin/mkdir -p include/near | ||
9803 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9804 | 0.14-r0/neard-0.14/include/types.h include/near/types.h | ||
9805 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9806 | 0.14-r0/neard-0.14/include/log.h include/near/log.h | ||
9807 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9808 | 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h | ||
9809 | | /bin/mkdir -p include/near | ||
9810 | | /bin/mkdir -p include/near | ||
9811 | | /bin/mkdir -p include/near | ||
9812 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9813 | 0.14-r0/neard-0.14/include/tag.h include/near/tag.h | ||
9814 | | /bin/mkdir -p include/near | ||
9815 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9816 | 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h | ||
9817 | | /bin/mkdir -p include/near | ||
9818 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9819 | 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h | ||
9820 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9821 | 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h | ||
9822 | | /bin/mkdir -p include/near | ||
9823 | | /bin/mkdir -p include/near | ||
9824 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9825 | 0.14-r0/neard-0.14/include/setting.h include/near/setting.h | ||
9826 | | /bin/mkdir -p include/near | ||
9827 | | /bin/mkdir -p include/near | ||
9828 | | /bin/mkdir -p include/near | ||
9829 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9830 | 0.14-r0/neard-0.14/include/device.h include/near/device.h | ||
9831 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9832 | 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h | ||
9833 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9834 | 0.14-r0/neard-0.14/include/snep.h include/near/snep.h | ||
9835 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9836 | 0.14-r0/neard-0.14/include/version.h include/near/version.h | ||
9837 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
9838 | 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h | ||
9839 | | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h | ||
9840 | | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/ | ||
9841 | build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ | ||
9842 | yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 | ||
9843 | -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ | ||
9844 | lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/ | ||
9845 | tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/ | ||
9846 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ | ||
9847 | yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 | ||
9848 | -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" | ||
9849 | -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c | ||
9850 | -o tools/snep-send.o tools/snep-send.c | ||
9851 | | In file included from tools/snep-send.c:16:0: | ||
9852 | | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
9853 | | #include <near/dbus.h> | ||
9854 | | ^ | ||
9855 | | compilation terminated. | ||
9856 | | make[1]: *** [tools/snep-send.o] Error 1 | ||
9857 | | make[1]: *** Waiting for unfinished jobs.... | ||
9858 | | make: *** [all] Error 2 | ||
9859 | | ERROR: oe_runmake failed | ||
9860 | |||
9861 | Reproducing the Error | ||
9862 | ~~~~~~~~~~~~~~~~~~~~~ | ||
9863 | |||
9864 | Because race conditions are intermittent, they do not manifest | ||
9865 | themselves every time you do the build. In fact, most times the build | ||
9866 | will complete without problems even though the potential race condition | ||
9867 | exists. Thus, once the error surfaces, you need a way to reproduce it. | ||
9868 | |||
9869 | In this example, compiling the "neard" package is causing the problem. | ||
9870 | So the first thing to do is build "neard" locally. Before you start the | ||
9871 | build, set the | ||
9872 | :term:`PARALLEL_MAKE` variable | ||
9873 | in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a | ||
9874 | high value for ``PARALLEL_MAKE`` increases the chances of the race | ||
9875 | condition showing up: | ||
9876 | :: | ||
9877 | |||
9878 | $ bitbake neard | ||
9879 | |||
9880 | Once the local build for "neard" completes, start a ``devshell`` build: | ||
9881 | :: | ||
9882 | |||
9883 | $ bitbake neard -c devshell | ||
9884 | |||
9885 | For information on how to use a | ||
9886 | ``devshell``, see the "`Using a Development | ||
9887 | Shell <#platdev-appdev-devshell>`__" section. | ||
9888 | |||
9889 | In the ``devshell``, do the following: | ||
9890 | :: | ||
9891 | |||
9892 | $ make clean | ||
9893 | $ make tools/snep-send.o | ||
9894 | |||
9895 | The ``devshell`` commands cause the failure to clearly | ||
9896 | be visible. In this case, a missing dependency exists for the "neard" | ||
9897 | Makefile target. Here is some abbreviated, sample output with the | ||
9898 | missing dependency clearly visible at the end: | ||
9899 | :: | ||
9900 | |||
9901 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... | ||
9902 | . | ||
9903 | . | ||
9904 | . | ||
9905 | tools/snep-send.c | ||
9906 | In file included from tools/snep-send.c:16:0: | ||
9907 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
9908 | #include <near/dbus.h> | ||
9909 | ^ | ||
9910 | compilation terminated. | ||
9911 | make: *** [tools/snep-send.o] Error 1 | ||
9912 | $ | ||
9913 | |||
9914 | |||
9915 | Creating a Patch for the Fix | ||
9916 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
9917 | |||
9918 | Because there is a missing dependency for the Makefile target, you need | ||
9919 | to patch the ``Makefile.am`` file, which is generated from | ||
9920 | ``Makefile.in``. You can use Quilt to create the patch: | ||
9921 | :: | ||
9922 | |||
9923 | $ quilt new parallelmake.patch | ||
9924 | Patch patches/parallelmake.patch is now on top | ||
9925 | $ quilt add Makefile.am | ||
9926 | File Makefile.am added to patch patches/parallelmake.patch | ||
9927 | |||
9928 | For more information on using Quilt, see the | ||
9929 | "`Using Quilt in Your Workflow <#using-a-quilt-workflow>`__" section. | ||
9930 | |||
9931 | At this point you need to make the edits to ``Makefile.am`` to add the | ||
9932 | missing dependency. For our example, you have to add the following line | ||
9933 | to the file: | ||
9934 | :: | ||
9935 | |||
9936 | tools/snep-send.$(OBJEXT): include/near/dbus.h | ||
9937 | |||
9938 | Once you have edited the file, use the ``refresh`` command to create the | ||
9939 | patch: | ||
9940 | :: | ||
9941 | |||
9942 | $ quilt refresh | ||
9943 | Refreshed patch patches/parallelmake.patch | ||
9944 | |||
9945 | Once | ||
9946 | the patch file exists, you need to add it back to the originating recipe | ||
9947 | folder. Here is an example assuming a top-level | ||
9948 | :term:`Source Directory` named ``poky``: | ||
9949 | :: | ||
9950 | |||
9951 | $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard | ||
9952 | |||
9953 | The final thing you need to do to implement the fix in the build is to | ||
9954 | update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the | ||
9955 | :term:`SRC_URI` statement includes | ||
9956 | the patch file. The recipe file is in the folder above the patch. Here | ||
9957 | is what the edited ``SRC_URI`` statement would look like: | ||
9958 | :: | ||
9959 | |||
9960 | SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ | ||
9961 | file://neard.in \ | ||
9962 | file://neard.service.in \ | ||
9963 | file://parallelmake.patch \ | ||
9964 | " | ||
9965 | |||
9966 | With the patch complete and moved to the correct folder and the | ||
9967 | ``SRC_URI`` statement updated, you can exit the ``devshell``: | ||
9968 | :: | ||
9969 | |||
9970 | $ exit | ||
9971 | |||
9972 | Testing the Build | ||
9973 | ~~~~~~~~~~~~~~~~~ | ||
9974 | |||
9975 | With everything in place, you can get back to trying the build again | ||
9976 | locally: | ||
9977 | :: | ||
9978 | |||
9979 | $ bitbake neard | ||
9980 | |||
9981 | This build should succeed. | ||
9982 | |||
9983 | Now you can open up a ``devshell`` again and repeat the clean and make | ||
9984 | operations as follows: | ||
9985 | :: | ||
9986 | |||
9987 | $ bitbake neard -c devshell | ||
9988 | $ make clean | ||
9989 | $ make tools/snep-send.o | ||
9990 | |||
9991 | The build should work without issue. | ||
9992 | |||
9993 | As with all solved problems, if they originated upstream, you need to | ||
9994 | submit the fix for the recipe in OE-Core and upstream so that the | ||
9995 | problem is taken care of at its source. See the "`Submitting a Change to | ||
9996 | the Yocto Project <#how-to-submit-a-change>`__" section for more | ||
9997 | information. | ||
9998 | |||
9999 | Debugging With the GNU Project Debugger (GDB) Remotely | ||
10000 | ------------------------------------------------------ | ||
10001 | |||
10002 | GDB allows you to examine running programs, which in turn helps you to | ||
10003 | understand and fix problems. It also allows you to perform post-mortem | ||
10004 | style analysis of program crashes. GDB is available as a package within | ||
10005 | the Yocto Project and is installed in SDK images by default. See the | ||
10006 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
10007 | Project Reference Manual for a description of these images. You can find | ||
10008 | information on GDB at https://sourceware.org/gdb/. | ||
10009 | |||
10010 | .. note:: | ||
10011 | |||
10012 | For best results, install debug (``-dbg``) packages for the applications you | ||
10013 | are going to debug. Doing so makes extra debug symbols available that give | ||
10014 | you more meaningful output. | ||
10015 | |||
10016 | Sometimes, due to memory or disk space constraints, it is not possible | ||
10017 | to use GDB directly on the remote target to debug applications. These | ||
10018 | constraints arise because GDB needs to load the debugging information | ||
10019 | and the binaries of the process being debugged. Additionally, GDB needs | ||
10020 | to perform many computations to locate information such as function | ||
10021 | names, variable names and values, stack traces and so forth - even | ||
10022 | before starting the debugging process. These extra computations place | ||
10023 | more load on the target system and can alter the characteristics of the | ||
10024 | program being debugged. | ||
10025 | |||
10026 | To help get past the previously mentioned constraints, you can use | ||
10027 | gdbserver, which runs on the remote target and does not load any | ||
10028 | debugging information from the debugged process. Instead, a GDB instance | ||
10029 | processes the debugging information that is run on a remote computer - | ||
10030 | the host GDB. The host GDB then sends control commands to gdbserver to | ||
10031 | make it stop or start the debugged program, as well as read or write | ||
10032 | memory regions of that debugged program. All the debugging information | ||
10033 | loaded and processed as well as all the heavy debugging is done by the | ||
10034 | host GDB. Offloading these processes gives the gdbserver running on the | ||
10035 | target a chance to remain small and fast. | ||
10036 | |||
10037 | Because the host GDB is responsible for loading the debugging | ||
10038 | information and for doing the necessary processing to make actual | ||
10039 | debugging happen, you have to make sure the host can access the | ||
10040 | unstripped binaries complete with their debugging information and also | ||
10041 | be sure the target is compiled with no optimizations. The host GDB must | ||
10042 | also have local access to all the libraries used by the debugged | ||
10043 | program. Because gdbserver does not need any local debugging | ||
10044 | information, the binaries on the remote target can remain stripped. | ||
10045 | However, the binaries must also be compiled without optimization so they | ||
10046 | match the host's binaries. | ||
10047 | |||
10048 | To remain consistent with GDB documentation and terminology, the binary | ||
10049 | being debugged on the remote target machine is referred to as the | ||
10050 | "inferior" binary. For documentation on GDB see the `GDB | ||
10051 | site <https://sourceware.org/gdb/documentation/>`__. | ||
10052 | |||
10053 | The following steps show you how to debug using the GNU project | ||
10054 | debugger. | ||
10055 | |||
10056 | 1. *Configure your build system to construct the companion debug | ||
10057 | filesystem:* | ||
10058 | |||
10059 | In your ``local.conf`` file, set the following: | ||
10060 | :: | ||
10061 | |||
10062 | IMAGE_GEN_DEBUGFS = "1" | ||
10063 | IMAGE_FSTYPES_DEBUGFS = "tar.bz2" | ||
10064 | |||
10065 | These options cause the | ||
10066 | OpenEmbedded build system to generate a special companion filesystem | ||
10067 | fragment, which contains the matching source and debug symbols to | ||
10068 | your deployable filesystem. The build system does this by looking at | ||
10069 | what is in the deployed filesystem, and pulling the corresponding | ||
10070 | ``-dbg`` packages. | ||
10071 | |||
10072 | The companion debug filesystem is not a complete filesystem, but only | ||
10073 | contains the debug fragments. This filesystem must be combined with | ||
10074 | the full filesystem for debugging. Subsequent steps in this procedure | ||
10075 | show how to combine the partial filesystem with the full filesystem. | ||
10076 | |||
10077 | 2. *Configure the system to include gdbserver in the target filesystem:* | ||
10078 | |||
10079 | Make the following addition in either your ``local.conf`` file or in | ||
10080 | an image recipe: | ||
10081 | :: | ||
10082 | |||
10083 | IMAGE_INSTALL_append = " gdbserver" | ||
10084 | |||
10085 | The change makes | ||
10086 | sure the ``gdbserver`` package is included. | ||
10087 | |||
10088 | 3. *Build the environment:* | ||
10089 | |||
10090 | Use the following command to construct the image and the companion | ||
10091 | Debug Filesystem: | ||
10092 | :: | ||
10093 | |||
10094 | $ bitbake image | ||
10095 | |||
10096 | Build the cross GDB component and | ||
10097 | make it available for debugging. Build the SDK that matches the | ||
10098 | image. Building the SDK is best for a production build that can be | ||
10099 | used later for debugging, especially during long term maintenance: | ||
10100 | :: | ||
10101 | |||
10102 | $ bitbake -c populate_sdk image | ||
10103 | |||
10104 | Alternatively, you can build the minimal toolchain components that | ||
10105 | match the target. Doing so creates a smaller than typical SDK and | ||
10106 | only contains a minimal set of components with which to build simple | ||
10107 | test applications, as well as run the debugger: | ||
10108 | :: | ||
10109 | |||
10110 | $ bitbake meta-toolchain | ||
10111 | |||
10112 | A final method is to build Gdb itself within the build system: | ||
10113 | :: | ||
10114 | |||
10115 | $ bitbake gdb-cross-<architecture> | ||
10116 | |||
10117 | Doing so produces a temporary copy of | ||
10118 | ``cross-gdb`` you can use for debugging during development. While | ||
10119 | this is the quickest approach, the two previous methods in this step | ||
10120 | are better when considering long-term maintenance strategies. | ||
10121 | |||
10122 | .. note:: | ||
10123 | |||
10124 | If you run ``bitbake gdb-cross``, the OpenEmbedded build system suggests | ||
10125 | the actual image (e.g. ``gdb-cross-i586``). The suggestion is usually the | ||
10126 | actual name you want to use. | ||
10127 | |||
10128 | 4. *Set up the* ``debugfs``\ *:* | ||
10129 | |||
10130 | Run the following commands to set up the ``debugfs``: | ||
10131 | :: | ||
10132 | |||
10133 | $ mkdir debugfs | ||
10134 | $ cd debugfs | ||
10135 | $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2 | ||
10136 | $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2 | ||
10137 | |||
10138 | 5. *Set up GDB:* | ||
10139 | |||
10140 | Install the SDK (if you built one) and then source the correct | ||
10141 | environment file. Sourcing the environment file puts the SDK in your | ||
10142 | ``PATH`` environment variable. | ||
10143 | |||
10144 | If you are using the build system, Gdb is located in | ||
10145 | `build-dir`\ ``/tmp/sysroots/``\ `host`\ ``/usr/bin/``\ `architecture`\ ``/``\ `architecture`\ ``-gdb`` | ||
10146 | |||
10147 | 6. *Boot the target:* | ||
10148 | |||
10149 | For information on how to run QEMU, see the `QEMU | ||
10150 | Documentation <https://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__. | ||
10151 | |||
10152 | .. note:: | ||
10153 | |||
10154 | Be sure to verify that your host can access the target via TCP. | ||
10155 | |||
10156 | 7. *Debug a program:* | ||
10157 | |||
10158 | Debugging a program involves running gdbserver on the target and then | ||
10159 | running Gdb on the host. The example in this step debugs ``gzip``: | ||
10160 | |||
10161 | .. code-block:: shell | ||
10162 | |||
10163 | root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help | ||
10164 | |||
10165 | For | ||
10166 | additional gdbserver options, see the `GDB Server | ||
10167 | Documentation <https://www.gnu.org/software/gdb/documentation/>`__. | ||
10168 | |||
10169 | After running gdbserver on the target, you need to run Gdb on the | ||
10170 | host and configure it and connect to the target. Use these commands: | ||
10171 | :: | ||
10172 | |||
10173 | $ cd directory-holding-the-debugfs-directory | ||
10174 | $ arch-gdb | ||
10175 | (gdb) set sysroot debugfs | ||
10176 | (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug | ||
10177 | (gdb) target remote IP-of-target:1234 | ||
10178 | |||
10179 | At this | ||
10180 | point, everything should automatically load (i.e. matching binaries, | ||
10181 | symbols and headers). | ||
10182 | |||
10183 | .. note:: | ||
10184 | |||
10185 | The Gdb ``set`` commands in the previous example can be placed into the | ||
10186 | users ``~/.gdbinit`` file. Upon starting, Gdb automatically runs whatever | ||
10187 | commands are in that file. | ||
10188 | |||
10189 | 8. *Deploying without a full image rebuild:* | ||
10190 | |||
10191 | In many cases, during development you want a quick method to deploy a | ||
10192 | new binary to the target and debug it, without waiting for a full | ||
10193 | image build. | ||
10194 | |||
10195 | One approach to solving this situation is to just build the component | ||
10196 | you want to debug. Once you have built the component, copy the | ||
10197 | executable directly to both the target and the host ``debugfs``. | ||
10198 | |||
10199 | If the binary is processed through the debug splitting in | ||
10200 | OpenEmbedded, you should also copy the debug items (i.e. ``.debug`` | ||
10201 | contents and corresponding ``/usr/src/debug`` files) from the work | ||
10202 | directory. Here is an example: | ||
10203 | :: | ||
10204 | |||
10205 | $ bitbake bash | ||
10206 | $ bitbake -c devshell bash | ||
10207 | $ cd .. | ||
10208 | $ scp packages-split/bash/bin/bash target:/bin/bash | ||
10209 | $ cp -a packages-split/bash-dbg/\* path/debugfs | ||
10210 | |||
10211 | Debugging with the GNU Project Debugger (GDB) on the Target | ||
10212 | ----------------------------------------------------------- | ||
10213 | |||
10214 | The previous section addressed using GDB remotely for debugging | ||
10215 | purposes, which is the most usual case due to the inherent hardware | ||
10216 | limitations on many embedded devices. However, debugging in the target | ||
10217 | hardware itself is also possible with more powerful devices. This | ||
10218 | section describes what you need to do in order to support using GDB to | ||
10219 | debug on the target hardware. | ||
10220 | |||
10221 | To support this kind of debugging, you need do the following: | ||
10222 | |||
10223 | - Ensure that GDB is on the target. You can do this by adding "gdb" to | ||
10224 | :term:`IMAGE_INSTALL`: | ||
10225 | :: | ||
10226 | |||
10227 | IMAGE_INSTALL_append = " gdb" | ||
10228 | |||
10229 | Alternatively, you can add "tools-debug" to :term:`IMAGE_FEATURES`: | ||
10230 | :: | ||
10231 | |||
10232 | IMAGE_FEATURES_append = " tools-debug" | ||
10233 | |||
10234 | - Ensure that debug symbols are present. You can make sure these | ||
10235 | symbols are present by installing ``-dbg``: | ||
10236 | :: | ||
10237 | |||
10238 | IMAGE_INSTALL_append = "packagename-dbg" | ||
10239 | |||
10240 | Alternatively, you can do the following to include | ||
10241 | all the debug symbols: | ||
10242 | :: | ||
10243 | |||
10244 | IMAGE_FEATURES_append = " dbg-pkgs" | ||
10245 | |||
10246 | .. note:: | ||
10247 | |||
10248 | To improve the debug information accuracy, you can reduce the level | ||
10249 | of optimization used by the compiler. For example, when adding the | ||
10250 | following line to your ``local.conf`` file, you will reduce optimization | ||
10251 | from :term:`FULL_OPTIMIZATION` of "-O2" to :term:`DEBUG_OPTIMIZATION` | ||
10252 | of "-O -fno-omit-frame-pointer": | ||
10253 | :: | ||
10254 | |||
10255 | DEBUG_BUILD = "1" | ||
10256 | |||
10257 | Consider that this will reduce the application's performance and is | ||
10258 | recommended only for debugging purposes. | ||
10259 | |||
10260 | Other Debugging Tips | ||
10261 | -------------------- | ||
10262 | |||
10263 | Here are some other tips that you might find useful: | ||
10264 | |||
10265 | - When adding new packages, it is worth watching for undesirable items | ||
10266 | making their way into compiler command lines. For example, you do not | ||
10267 | want references to local system files like ``/usr/lib/`` or | ||
10268 | ``/usr/include/``. | ||
10269 | |||
10270 | - If you want to remove the ``psplash`` boot splashscreen, add | ||
10271 | ``psplash=false`` to the kernel command line. Doing so prevents | ||
10272 | ``psplash`` from loading and thus allows you to see the console. It | ||
10273 | is also possible to switch out of the splashscreen by switching the | ||
10274 | virtual console (e.g. Fn+Left or Fn+Right on a Zaurus). | ||
10275 | |||
10276 | - Removing :term:`TMPDIR` (usually | ||
10277 | ``tmp/``, within the | ||
10278 | :term:`Build Directory`) can often fix | ||
10279 | temporary build issues. Removing ``TMPDIR`` is usually a relatively | ||
10280 | cheap operation, because task output will be cached in | ||
10281 | :term:`SSTATE_DIR` (usually | ||
10282 | ``sstate-cache/``, which is also in the Build Directory). | ||
10283 | |||
10284 | .. note:: | ||
10285 | |||
10286 | Removing ``TMPDIR`` might be a workaround rather than a fix. | ||
10287 | Consequently, trying to determine the underlying cause of an issue before | ||
10288 | removing the directory is a good idea. | ||
10289 | |||
10290 | - Understanding how a feature is used in practice within existing | ||
10291 | recipes can be very helpful. It is recommended that you configure | ||
10292 | some method that allows you to quickly search through files. | ||
10293 | |||
10294 | Using GNU Grep, you can use the following shell function to | ||
10295 | recursively search through common recipe-related files, skipping | ||
10296 | binary files, ``.git`` directories, and the Build Directory (assuming | ||
10297 | its name starts with "build"): | ||
10298 | :: | ||
10299 | |||
10300 | g() { | ||
10301 | grep -Ir \ | ||
10302 | --exclude-dir=.git \ | ||
10303 | --exclude-dir='build*' \ | ||
10304 | --include='*.bb*' \ | ||
10305 | --include='*.inc*' \ | ||
10306 | --include='*.conf*' \ | ||
10307 | --include='*.py*' \ | ||
10308 | "$@" | ||
10309 | } | ||
10310 | |||
10311 | Following are some usage examples: | ||
10312 | :: | ||
10313 | |||
10314 | $ g FOO # Search recursively for "FOO" | ||
10315 | $ g -i foo # Search recursively for "foo", ignoring case | ||
10316 | $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" | ||
10317 | |||
10318 | If figuring | ||
10319 | out how some feature works requires a lot of searching, it might | ||
10320 | indicate that the documentation should be extended or improved. In | ||
10321 | such cases, consider filing a documentation bug using the Yocto | ||
10322 | Project implementation of | ||
10323 | :yocto_bugs:`Bugzilla <>`. For information on | ||
10324 | how to submit a bug against the Yocto Project, see the Yocto Project | ||
10325 | Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` | ||
10326 | and the "`Submitting a Defect Against the Yocto | ||
10327 | Project <#submitting-a-defect-against-the-yocto-project>`__" section. | ||
10328 | |||
10329 | .. note:: | ||
10330 | |||
10331 | The manuals might not be the right place to document variables | ||
10332 | that are purely internal and have a limited scope (e.g. internal | ||
10333 | variables used to implement a single ``.bbclass`` file). | ||
10334 | |||
10335 | Making Changes to the Yocto Project | ||
10336 | =================================== | ||
10337 | |||
10338 | Because the Yocto Project is an open-source, community-based project, | ||
10339 | you can effect changes to the project. This section presents procedures | ||
10340 | that show you how to submit a defect against the project and how to | ||
10341 | submit a change. | ||
10342 | |||
10343 | Submitting a Defect Against the Yocto Project | ||
10344 | --------------------------------------------- | ||
10345 | |||
10346 | Use the Yocto Project implementation of | ||
10347 | `Bugzilla <https://www.bugzilla.org/about/>`__ to submit a defect (bug) | ||
10348 | against the Yocto Project. For additional information on this | ||
10349 | implementation of Bugzilla see the ":ref:`Yocto Project | ||
10350 | Bugzilla <resources-bugtracker>`" section in the | ||
10351 | Yocto Project Reference Manual. For more detail on any of the following | ||
10352 | steps, see the Yocto Project | ||
10353 | :yocto_wiki:`Bugzilla wiki page </Bugzilla_Configuration_and_Bug_Tracking>`. | ||
10354 | |||
10355 | Use the following general steps to submit a bug: | ||
10356 | |||
10357 | 1. Open the Yocto Project implementation of :yocto_bugs:`Bugzilla <>`. | ||
10358 | |||
10359 | 2. Click "File a Bug" to enter a new bug. | ||
10360 | |||
10361 | 3. Choose the appropriate "Classification", "Product", and "Component" | ||
10362 | for which the bug was found. Bugs for the Yocto Project fall into | ||
10363 | one of several classifications, which in turn break down into | ||
10364 | several products and components. For example, for a bug against the | ||
10365 | ``meta-intel`` layer, you would choose "Build System, Metadata & | ||
10366 | Runtime", "BSPs", and "bsps-meta-intel", respectively. | ||
10367 | |||
10368 | 4. Choose the "Version" of the Yocto Project for which you found the | ||
10369 | bug (e.g. &DISTRO;). | ||
10370 | |||
10371 | 5. Determine and select the "Severity" of the bug. The severity | ||
10372 | indicates how the bug impacted your work. | ||
10373 | |||
10374 | 6. Choose the "Hardware" that the bug impacts. | ||
10375 | |||
10376 | 7. Choose the "Architecture" that the bug impacts. | ||
10377 | |||
10378 | 8. Choose a "Documentation change" item for the bug. Fixing a bug might | ||
10379 | or might not affect the Yocto Project documentation. If you are | ||
10380 | unsure of the impact to the documentation, select "Don't Know". | ||
10381 | |||
10382 | 9. Provide a brief "Summary" of the bug. Try to limit your summary to | ||
10383 | just a line or two and be sure to capture the essence of the bug. | ||
10384 | |||
10385 | 10. Provide a detailed "Description" of the bug. You should provide as | ||
10386 | much detail as you can about the context, behavior, output, and so | ||
10387 | forth that surrounds the bug. You can even attach supporting files | ||
10388 | for output from logs by using the "Add an attachment" button. | ||
10389 | |||
10390 | 11. Click the "Submit Bug" button submit the bug. A new Bugzilla number | ||
10391 | is assigned to the bug and the defect is logged in the bug tracking | ||
10392 | system. | ||
10393 | |||
10394 | Once you file a bug, the bug is processed by the Yocto Project Bug | ||
10395 | Triage Team and further details concerning the bug are assigned (e.g. | ||
10396 | priority and owner). You are the "Submitter" of the bug and any further | ||
10397 | categorization, progress, or comments on the bug result in Bugzilla | ||
10398 | sending you an automated email concerning the particular change or | ||
10399 | progress to the bug. | ||
10400 | |||
10401 | Submitting a Change to the Yocto Project | ||
10402 | ---------------------------------------- | ||
10403 | |||
10404 | Contributions to the Yocto Project and OpenEmbedded are very welcome. | ||
10405 | Because the system is extremely configurable and flexible, we recognize | ||
10406 | that developers will want to extend, configure or optimize it for their | ||
10407 | specific uses. | ||
10408 | |||
10409 | The Yocto Project uses a mailing list and a patch-based workflow that is | ||
10410 | similar to the Linux kernel but contains important differences. In | ||
10411 | general, a mailing list exists through which you can submit patches. You | ||
10412 | should send patches to the appropriate mailing list so that they can be | ||
10413 | reviewed and merged by the appropriate maintainer. The specific mailing | ||
10414 | list you need to use depends on the location of the code you are | ||
10415 | changing. Each component (e.g. layer) should have a ``README`` file that | ||
10416 | indicates where to send the changes and which process to follow. | ||
10417 | |||
10418 | You can send the patch to the mailing list using whichever approach you | ||
10419 | feel comfortable with to generate the patch. Once sent, the patch is | ||
10420 | usually reviewed by the community at large. If somebody has concerns | ||
10421 | with the patch, they will usually voice their concern over the mailing | ||
10422 | list. If a patch does not receive any negative reviews, the maintainer | ||
10423 | of the affected layer typically takes the patch, tests it, and then | ||
10424 | based on successful testing, merges the patch. | ||
10425 | |||
10426 | The "poky" repository, which is the Yocto Project's reference build | ||
10427 | environment, is a hybrid repository that contains several individual | ||
10428 | pieces (e.g. BitBake, Metadata, documentation, and so forth) built using | ||
10429 | the combo-layer tool. The upstream location used for submitting changes | ||
10430 | varies by component: | ||
10431 | |||
10432 | - *Core Metadata:* Send your patch to the | ||
10433 | :oe_lists:`openembedded-core </g/openembedded-core>` | ||
10434 | mailing list. For example, a change to anything under the ``meta`` or | ||
10435 | ``scripts`` directories should be sent to this mailing list. | ||
10436 | |||
10437 | - *BitBake:* For changes to BitBake (i.e. anything under the | ||
10438 | ``bitbake`` directory), send your patch to the | ||
10439 | :oe_lists:`bitbake-devel </g/bitbake-devel>` | ||
10440 | mailing list. | ||
10441 | |||
10442 | - *"meta-\*" trees:* These trees contain Metadata. Use the | ||
10443 | :yocto_lists:`poky </g/poky>` mailing list. | ||
10444 | |||
10445 | - *Documentation*: For changes to the Yocto Project documentation, use the | ||
10446 | :yocto_lists:`docs </g/docs>` mailing list. | ||
10447 | |||
10448 | For changes to other layers hosted in the Yocto Project source | ||
10449 | repositories (i.e. ``yoctoproject.org``) and tools use the | ||
10450 | :yocto_lists:`Yocto Project </g/yocto/>` general mailing list. | ||
10451 | |||
10452 | .. note:: | ||
10453 | |||
10454 | Sometimes a layer's documentation specifies to use a particular | ||
10455 | mailing list. If so, use that list. | ||
10456 | |||
10457 | For additional recipes that do not fit into the core Metadata, you | ||
10458 | should determine which layer the recipe should go into and submit the | ||
10459 | change in the manner recommended by the documentation (e.g. the | ||
10460 | ``README`` file) supplied with the layer. If in doubt, please ask on the | ||
10461 | Yocto general mailing list or on the openembedded-devel mailing list. | ||
10462 | |||
10463 | You can also push a change upstream and request a maintainer to pull the | ||
10464 | change into the component's upstream repository. You do this by pushing | ||
10465 | to a contribution repository that is upstream. See the | ||
10466 | ":ref:`overview-manual/development-environment:git workflows and the yocto project`" | ||
10467 | section in the Yocto Project Overview and Concepts Manual for additional | ||
10468 | concepts on working in the Yocto Project development environment. | ||
10469 | |||
10470 | Maintainers commonly use ``-next`` branches to test submissions prior to | ||
10471 | merging patches. Thus, you can get an idea of the status of a patch based on | ||
10472 | whether the patch has been merged into one of these branches. The commonly | ||
10473 | used testing branches for OpenEmbedded-Core are as follows: | ||
10474 | |||
10475 | - *openembedded-core "master-next" branch:* This branch is part of the | ||
10476 | :oe_git:`openembedded-core </openembedded-core/>` repository and contains | ||
10477 | proposed changes to the core metadata. | ||
10478 | |||
10479 | - *poky "master-next" branch:* This branch is part of the | ||
10480 | :yocto_git:`poky </poky/>` repository and combines proposed | ||
10481 | changes to bitbake, the core metadata and the poky distro. | ||
10482 | |||
10483 | Similarly, stable branches maintained by the project may have corresponding | ||
10484 | ``-next`` branches which collect proposed changes. For example, | ||
10485 | ``&DISTRO_NAME_NO_CAP;-next`` and ``&DISTRO_NAME_NO_CAP_MINUS_ONE;-next`` | ||
10486 | branches in both the "openembdedded-core" and "poky" repositories. | ||
10487 | |||
10488 | Other layers may have similar testing branches but there is no formal | ||
10489 | requirement or standard for these so please check the documentation for the | ||
10490 | layers you are contributing to. | ||
10491 | |||
10492 | The following sections provide procedures for submitting a change. | ||
10493 | |||
10494 | Preparing Changes for Submission | ||
10495 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10496 | |||
10497 | 1. *Make Your Changes Locally:* Make your changes in your local Git | ||
10498 | repository. You should make small, controlled, isolated changes. | ||
10499 | Keeping changes small and isolated aids review, makes | ||
10500 | merging/rebasing easier and keeps the change history clean should | ||
10501 | anyone need to refer to it in future. | ||
10502 | |||
10503 | 2. *Stage Your Changes:* Stage your changes by using the ``git add`` | ||
10504 | command on each file you changed. | ||
10505 | |||
10506 | 3. *Commit Your Changes:* Commit the change by using the ``git commit`` | ||
10507 | command. Make sure your commit information follows standards by | ||
10508 | following these accepted conventions: | ||
10509 | |||
10510 | - Be sure to include a "Signed-off-by:" line in the same style as | ||
10511 | required by the Linux kernel. Adding this line signifies that you, | ||
10512 | the submitter, have agreed to the Developer's Certificate of | ||
10513 | Origin 1.1 as follows: | ||
10514 | |||
10515 | .. code-block:: none | ||
10516 | |||
10517 | Developer's Certificate of Origin 1.1 | ||
10518 | |||
10519 | By making a contribution to this project, I certify that: | ||
10520 | |||
10521 | (a) The contribution was created in whole or in part by me and I | ||
10522 | have the right to submit it under the open source license | ||
10523 | indicated in the file; or | ||
10524 | |||
10525 | (b) The contribution is based upon previous work that, to the best | ||
10526 | of my knowledge, is covered under an appropriate open source | ||
10527 | license and I have the right under that license to submit that | ||
10528 | work with modifications, whether created in whole or in part | ||
10529 | by me, under the same open source license (unless I am | ||
10530 | permitted to submit under a different license), as indicated | ||
10531 | in the file; or | ||
10532 | |||
10533 | (c) The contribution was provided directly to me by some other | ||
10534 | person who certified (a), (b) or (c) and I have not modified | ||
10535 | it. | ||
10536 | |||
10537 | (d) I understand and agree that this project and the contribution | ||
10538 | are public and that a record of the contribution (including all | ||
10539 | personal information I submit with it, including my sign-off) is | ||
10540 | maintained indefinitely and may be redistributed consistent with | ||
10541 | this project or the open source license(s) involved. | ||
10542 | |||
10543 | - Provide a single-line summary of the change and, if more | ||
10544 | explanation is needed, provide more detail in the body of the | ||
10545 | commit. This summary is typically viewable in the "shortlist" of | ||
10546 | changes. Thus, providing something short and descriptive that | ||
10547 | gives the reader a summary of the change is useful when viewing a | ||
10548 | list of many commits. You should prefix this short description | ||
10549 | with the recipe name (if changing a recipe), or else with the | ||
10550 | short form path to the file being changed. | ||
10551 | |||
10552 | - For the body of the commit message, provide detailed information | ||
10553 | that describes what you changed, why you made the change, and the | ||
10554 | approach you used. It might also be helpful if you mention how you | ||
10555 | tested the change. Provide as much detail as you can in the body | ||
10556 | of the commit message. | ||
10557 | |||
10558 | .. note:: | ||
10559 | |||
10560 | You do not need to provide a more detailed explanation of a | ||
10561 | change if the change is minor to the point of the single line | ||
10562 | summary providing all the information. | ||
10563 | |||
10564 | - If the change addresses a specific bug or issue that is associated | ||
10565 | with a bug-tracking ID, include a reference to that ID in your | ||
10566 | detailed description. For example, the Yocto Project uses a | ||
10567 | specific convention for bug references - any commit that addresses | ||
10568 | a specific bug should use the following form for the detailed | ||
10569 | description. Be sure to use the actual bug-tracking ID from | ||
10570 | Bugzilla for bug-id: | ||
10571 | :: | ||
10572 | |||
10573 | Fixes [YOCTO #bug-id] | ||
10574 | |||
10575 | detailed description of change | ||
10576 | |||
10577 | Using Email to Submit a Patch | ||
10578 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10579 | |||
10580 | Depending on the components changed, you need to submit the email to a | ||
10581 | specific mailing list. For some guidance on which mailing list to use, | ||
10582 | see the `list <#figuring-out-the-mailing-list-to-use>`__ at the | ||
10583 | beginning of this section. For a description of all the available | ||
10584 | mailing lists, see the ":ref:`Mailing Lists <resources-mailinglist>`" section in the | ||
10585 | Yocto Project Reference Manual. | ||
10586 | |||
10587 | Here is the general procedure on how to submit a patch through email | ||
10588 | without using the scripts once the steps in | ||
10589 | :ref:`dev-manual/common-tasks:preparing changes for submission` have been followed: | ||
10590 | |||
10591 | 1. *Format the Commit:* Format the commit into an email message. To | ||
10592 | format commits, use the ``git format-patch`` command. When you | ||
10593 | provide the command, you must include a revision list or a number of | ||
10594 | patches as part of the command. For example, either of these two | ||
10595 | commands takes your most recent single commit and formats it as an | ||
10596 | email message in the current directory: | ||
10597 | :: | ||
10598 | |||
10599 | $ git format-patch -1 | ||
10600 | |||
10601 | or :: | ||
10602 | |||
10603 | $ git format-patch HEAD~ | ||
10604 | |||
10605 | After the command is run, the current directory contains a numbered | ||
10606 | ``.patch`` file for the commit. | ||
10607 | |||
10608 | If you provide several commits as part of the command, the | ||
10609 | ``git format-patch`` command produces a series of numbered files in | ||
10610 | the current directory – one for each commit. If you have more than | ||
10611 | one patch, you should also use the ``--cover`` option with the | ||
10612 | command, which generates a cover letter as the first "patch" in the | ||
10613 | series. You can then edit the cover letter to provide a description | ||
10614 | for the series of patches. For information on the | ||
10615 | ``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed | ||
10616 | using the ``man git-format-patch`` command. | ||
10617 | |||
10618 | .. note:: | ||
10619 | |||
10620 | If you are or will be a frequent contributor to the Yocto Project | ||
10621 | or to OpenEmbedded, you might consider requesting a contrib area | ||
10622 | and the necessary associated rights. | ||
10623 | |||
10624 | 2. *Send the patches via email:* Send the patches to the recipients and | ||
10625 | relevant mailing lists by using the ``git send-email`` command. | ||
10626 | |||
10627 | .. note:: | ||
10628 | |||
10629 | In order to use ``git send-email``, you must have the proper Git packages | ||
10630 | installed on your host. | ||
10631 | For Ubuntu, Debian, and Fedora the package is ``git-email``. | ||
10632 | |||
10633 | The ``git send-email`` command sends email by using a local or remote | ||
10634 | Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or | ||
10635 | through a direct ``smtp`` configuration in your Git ``~/.gitconfig`` | ||
10636 | file. If you are submitting patches through email only, it is very | ||
10637 | important that you submit them without any whitespace or HTML | ||
10638 | formatting that either you or your mailer introduces. The maintainer | ||
10639 | that receives your patches needs to be able to save and apply them | ||
10640 | directly from your emails. A good way to verify that what you are | ||
10641 | sending will be applicable by the maintainer is to do a dry run and | ||
10642 | send them to yourself and then save and apply them as the maintainer | ||
10643 | would. | ||
10644 | |||
10645 | The ``git send-email`` command is the preferred method for sending | ||
10646 | your patches using email since there is no risk of compromising | ||
10647 | whitespace in the body of the message, which can occur when you use | ||
10648 | your own mail client. The command also has several options that let | ||
10649 | you specify recipients and perform further editing of the email | ||
10650 | message. For information on how to use the ``git send-email`` | ||
10651 | command, see ``GIT-SEND-EMAIL(1)`` displayed using the | ||
10652 | ``man git-send-email`` command. | ||
10653 | |||
10654 | The Yocto Project uses a `Patchwork instance <https://patchwork.openembedded.org/>`__ | ||
10655 | to track the status of patches submitted to the various mailing lists and to | ||
10656 | support automated patch testing. Each submitted patch is checked for common | ||
10657 | mistakes and deviations from the expected patch format and submitters are | ||
10658 | notified by patchtest if such mistakes are found. This process helps to | ||
10659 | reduce the burden of patch review on maintainers. | ||
10660 | |||
10661 | .. note:: | ||
10662 | |||
10663 | This system is imperfect and changes can sometimes get lost in the flow. | ||
10664 | Asking about the status of a patch or change is reasonable if the change | ||
10665 | has been idle for a while with no feedback. | ||
10666 | |||
10667 | Using Scripts to Push a Change Upstream and Request a Pull | ||
10668 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10669 | |||
10670 | For larger patch series it is preferable to send a pull request which not | ||
10671 | only includes the patch but also a pointer to a branch that can be pulled | ||
10672 | from. This involves making a local branch for your changes, pushing this | ||
10673 | branch to an accessible repository and then using the ``create-pull-request`` | ||
10674 | and ``send-pull-request`` scripts from openembedded-core to create and send a | ||
10675 | patch series with a link to the branch for review. | ||
10676 | |||
10677 | Follow this procedure to push a change to an upstream "contrib" Git | ||
10678 | repository once the steps in :ref:`dev-manual/common-tasks:preparing changes for submission` have | ||
10679 | been followed: | ||
10680 | |||
10681 | .. note:: | ||
10682 | |||
10683 | You can find general Git information on how to push a change upstream | ||
10684 | in the | ||
10685 | `Git Community Book <https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows>`__. | ||
10686 | |||
10687 | 1. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for | ||
10688 | permissions to push to an upstream contrib repository, push the | ||
10689 | change to that repository: | ||
10690 | :: | ||
10691 | |||
10692 | $ git push upstream_remote_repo local_branch_name | ||
10693 | |||
10694 | For example, suppose you have permissions to push | ||
10695 | into the upstream ``meta-intel-contrib`` repository and you are | ||
10696 | working in a local branch named `your_name`\ ``/README``. The following | ||
10697 | command pushes your local commits to the ``meta-intel-contrib`` | ||
10698 | upstream repository and puts the commit in a branch named | ||
10699 | `your_name`\ ``/README``: | ||
10700 | :: | ||
10701 | |||
10702 | $ git push meta-intel-contrib your_name/README | ||
10703 | |||
10704 | 2. *Determine Who to Notify:* Determine the maintainer or the mailing | ||
10705 | list that you need to notify for the change. | ||
10706 | |||
10707 | Before submitting any change, you need to be sure who the maintainer | ||
10708 | is or what mailing list that you need to notify. Use either these | ||
10709 | methods to find out: | ||
10710 | |||
10711 | - *Maintenance File:* Examine the ``maintainers.inc`` file, which is | ||
10712 | located in the :term:`Source Directory` at | ||
10713 | ``meta/conf/distro/include``, to see who is responsible for code. | ||
10714 | |||
10715 | - *Search by File:* Using :ref:`overview-manual/development-environment:git`, you can | ||
10716 | enter the following command to bring up a short list of all | ||
10717 | commits against a specific file: | ||
10718 | :: | ||
10719 | |||
10720 | git shortlog -- filename | ||
10721 | |||
10722 | Just provide the name of the file for which you are interested. The | ||
10723 | information returned is not ordered by history but does include a | ||
10724 | list of everyone who has committed grouped by name. From the list, | ||
10725 | you can see who is responsible for the bulk of the changes against | ||
10726 | the file. | ||
10727 | |||
10728 | - *Examine the List of Mailing Lists:* For a list of the Yocto | ||
10729 | Project and related mailing lists, see the ":ref:`Mailing | ||
10730 | lists <resources-mailinglist>`" section in | ||
10731 | the Yocto Project Reference Manual. | ||
10732 | |||
10733 | 3. *Make a Pull Request:* Notify the maintainer or the mailing list that | ||
10734 | you have pushed a change by making a pull request. | ||
10735 | |||
10736 | The Yocto Project provides two scripts that conveniently let you | ||
10737 | generate and send pull requests to the Yocto Project. These scripts | ||
10738 | are ``create-pull-request`` and ``send-pull-request``. You can find | ||
10739 | these scripts in the ``scripts`` directory within the | ||
10740 | :term:`Source Directory` (e.g. | ||
10741 | ``~/poky/scripts``). | ||
10742 | |||
10743 | Using these scripts correctly formats the requests without | ||
10744 | introducing any whitespace or HTML formatting. The maintainer that | ||
10745 | receives your patches either directly or through the mailing list | ||
10746 | needs to be able to save and apply them directly from your emails. | ||
10747 | Using these scripts is the preferred method for sending patches. | ||
10748 | |||
10749 | First, create the pull request. For example, the following command | ||
10750 | runs the script, specifies the upstream repository in the contrib | ||
10751 | directory into which you pushed the change, and provides a subject | ||
10752 | line in the created patch files: | ||
10753 | :: | ||
10754 | |||
10755 | $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README" | ||
10756 | |||
10757 | Running this script forms ``*.patch`` files in a folder named | ||
10758 | ``pull-``\ `PID` in the current directory. One of the patch files is a | ||
10759 | cover letter. | ||
10760 | |||
10761 | Before running the ``send-pull-request`` script, you must edit the | ||
10762 | cover letter patch to insert information about your change. After | ||
10763 | editing the cover letter, send the pull request. For example, the | ||
10764 | following command runs the script and specifies the patch directory | ||
10765 | and email address. In this example, the email address is a mailing | ||
10766 | list: | ||
10767 | :: | ||
10768 | |||
10769 | $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org | ||
10770 | |||
10771 | You need to follow the prompts as the script is interactive. | ||
10772 | |||
10773 | .. note:: | ||
10774 | |||
10775 | For help on using these scripts, simply provide the ``-h`` | ||
10776 | argument as follows: | ||
10777 | :: | ||
10778 | |||
10779 | $ poky/scripts/create-pull-request -h | ||
10780 | $ poky/scripts/send-pull-request -h | ||
10781 | |||
10782 | Responding to Patch Review | ||
10783 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10784 | |||
10785 | You may get feedback on your submitted patches from other community members | ||
10786 | or from the automated patchtest service. If issues are identified in your | ||
10787 | patch then it is usually necessary to address these before the patch will be | ||
10788 | accepted into the project. In this case you should amend the patch according | ||
10789 | to the feedback and submit an updated version to the relevant mailing list, | ||
10790 | copying in the reviewers who provided feedback to the previous version of the | ||
10791 | patch. | ||
10792 | |||
10793 | The patch should be amended using ``git commit --amend`` or perhaps ``git | ||
10794 | rebase`` for more expert git users. You should also modify the ``[PATCH]`` | ||
10795 | tag in the email subject line when sending the revised patch to mark the new | ||
10796 | iteration as ``[PATCH v2]``, ``[PATCH v3]``, etc as appropriate. This can be | ||
10797 | done by passing the ``-v`` argument to ``git format-patch`` with a version | ||
10798 | number. | ||
10799 | |||
10800 | Lastly please ensure that you also test your revised changes. In particular | ||
10801 | please don't just edit the patch file written out by ``git format-patch`` and | ||
10802 | resend it. | ||
10803 | |||
10804 | Submitting Changes to Stable Release Branches | ||
10805 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10806 | |||
10807 | The process for proposing changes to a Yocto Project stable branch differs | ||
10808 | from the steps described above. Changes to a stable branch must address | ||
10809 | identified bugs or CVEs and should be made carefully in order to avoid the | ||
10810 | risk of introducing new bugs or breaking backwards compatibility. Typically | ||
10811 | bug fixes must already be accepted into the master branch before they can be | ||
10812 | backported to a stable branch unless the bug in question does not affect the | ||
10813 | master branch or the fix on the master branch is unsuitable for backporting. | ||
10814 | |||
10815 | The list of stable branches along with the status and maintainer for each | ||
10816 | branch can be obtained from the | ||
10817 | :yocto_wiki:`Releases wiki page </Releases>`. | ||
10818 | |||
10819 | .. note:: | ||
10820 | |||
10821 | Changes will not typically be accepted for branches which are marked as | ||
10822 | End-Of-Life (EOL). | ||
10823 | |||
10824 | With this in mind, the steps to submit a change for a stable branch are as | ||
10825 | follows: | ||
10826 | |||
10827 | 1. *Identify the bug or CVE to be fixed:* This information should be | ||
10828 | collected so that it can be included in your submission. | ||
10829 | |||
10830 | 2. *Check if the fix is already present in the master branch:* This will | ||
10831 | result in the most straightforward path into the stable branch for the | ||
10832 | fix. | ||
10833 | |||
10834 | a. *If the fix is present in the master branch - Submit a backport request | ||
10835 | by email:* You should send an email to the relevant stable branch | ||
10836 | maintainer and the mailing list with details of the bug or CVE to be | ||
10837 | fixed, the commit hash on the master branch that fixes the issue and | ||
10838 | the stable branches which you would like this fix to be backported to. | ||
10839 | |||
10840 | b. *If the fix is not present in the master branch - Submit the fix to the | ||
10841 | master branch first:* This will ensure that the fix passes through the | ||
10842 | project's usual patch review and test processes before being accepted. | ||
10843 | It will also ensure that bugs are not left unresolved in the master | ||
10844 | branch itself. Once the fix is accepted in the master branch a backport | ||
10845 | request can be submitted as above. | ||
10846 | |||
10847 | c. *If the fix is unsuitable for the master branch - Submit a patch | ||
10848 | directly for the stable branch:* This method should be considered as a | ||
10849 | last resort. It is typically necessary when the master branch is using | ||
10850 | a newer version of the software which includes an upstream fix for the | ||
10851 | issue or when the issue has been fixed on the master branch in a way | ||
10852 | that introduces backwards incompatible changes. In this case follow the | ||
10853 | steps in :ref:`dev-manual/common-tasks:preparing changes for submission` and | ||
10854 | :ref:`dev-manual/common-tasks:using email to submit a patch` but modify the subject header of your patch | ||
10855 | email to include the name of the stable branch which you are | ||
10856 | targetting. This can be done using the ``--subject-prefix`` argument to | ||
10857 | ``git format-patch``, for example to submit a patch to the dunfell | ||
10858 | branch use | ||
10859 | ``git format-patch --subject-prefix='&DISTRO_NAME_NO_CAP_MINUS_ONE;][PATCH' ...``. | ||
10860 | |||
10861 | Working With Licenses | ||
10862 | ===================== | ||
10863 | |||
10864 | As mentioned in the ":ref:`overview-manual/development-environment:licensing`" | ||
10865 | section in the Yocto Project Overview and Concepts Manual, open source | ||
10866 | projects are open to the public and they consequently have different | ||
10867 | licensing structures in place. This section describes the mechanism by | ||
10868 | which the :term:`OpenEmbedded Build System` | ||
10869 | tracks changes to | ||
10870 | licensing text and covers how to maintain open source license compliance | ||
10871 | during your project's lifecycle. The section also describes how to | ||
10872 | enable commercially licensed recipes, which by default are disabled. | ||
10873 | |||
10874 | Tracking License Changes | ||
10875 | ------------------------ | ||
10876 | |||
10877 | The license of an upstream project might change in the future. In order | ||
10878 | to prevent these changes going unnoticed, the | ||
10879 | :term:`LIC_FILES_CHKSUM` | ||
10880 | variable tracks changes to the license text. The checksums are validated | ||
10881 | at the end of the configure step, and if the checksums do not match, the | ||
10882 | build will fail. | ||
10883 | |||
10884 | Specifying the ``LIC_FILES_CHKSUM`` Variable | ||
10885 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10886 | |||
10887 | The ``LIC_FILES_CHKSUM`` variable contains checksums of the license text | ||
10888 | in the source code for the recipe. Following is an example of how to | ||
10889 | specify ``LIC_FILES_CHKSUM``: | ||
10890 | :: | ||
10891 | |||
10892 | LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ | ||
10893 | file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ | ||
10894 | file://licfile2.txt;endline=50;md5=zzzz \ | ||
10895 | ..." | ||
10896 | |||
10897 | .. note:: | ||
10898 | |||
10899 | - When using "beginline" and "endline", realize that line numbering | ||
10900 | begins with one and not zero. Also, the included lines are | ||
10901 | inclusive (i.e. lines five through and including 29 in the | ||
10902 | previous example for ``licfile1.txt``). | ||
10903 | |||
10904 | - When a license check fails, the selected license text is included | ||
10905 | as part of the QA message. Using this output, you can determine | ||
10906 | the exact start and finish for the needed license text. | ||
10907 | |||
10908 | The build system uses the :term:`S` | ||
10909 | variable as the default directory when searching files listed in | ||
10910 | ``LIC_FILES_CHKSUM``. The previous example employs the default | ||
10911 | directory. | ||
10912 | |||
10913 | Consider this next example: | ||
10914 | :: | ||
10915 | |||
10916 | LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ | ||
10917 | md5=bb14ed3c4cda583abc85401304b5cd4e" | ||
10918 | LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" | ||
10919 | |||
10920 | The first line locates a file in ``${S}/src/ls.c`` and isolates lines | ||
10921 | five through 16 as license text. The second line refers to a file in | ||
10922 | :term:`WORKDIR`. | ||
10923 | |||
10924 | Note that ``LIC_FILES_CHKSUM`` variable is mandatory for all recipes, | ||
10925 | unless the ``LICENSE`` variable is set to "CLOSED". | ||
10926 | |||
10927 | Explanation of Syntax | ||
10928 | ~~~~~~~~~~~~~~~~~~~~~ | ||
10929 | |||
10930 | As mentioned in the previous section, the ``LIC_FILES_CHKSUM`` variable | ||
10931 | lists all the important files that contain the license text for the | ||
10932 | source code. It is possible to specify a checksum for an entire file, or | ||
10933 | a specific section of a file (specified by beginning and ending line | ||
10934 | numbers with the "beginline" and "endline" parameters, respectively). | ||
10935 | The latter is useful for source files with a license notice header, | ||
10936 | README documents, and so forth. If you do not use the "beginline" | ||
10937 | parameter, then it is assumed that the text begins on the first line of | ||
10938 | the file. Similarly, if you do not use the "endline" parameter, it is | ||
10939 | assumed that the license text ends with the last line of the file. | ||
10940 | |||
10941 | The "md5" parameter stores the md5 checksum of the license text. If the | ||
10942 | license text changes in any way as compared to this parameter then a | ||
10943 | mismatch occurs. This mismatch triggers a build failure and notifies the | ||
10944 | developer. Notification allows the developer to review and address the | ||
10945 | license text changes. Also note that if a mismatch occurs during the | ||
10946 | build, the correct md5 checksum is placed in the build log and can be | ||
10947 | easily copied to the recipe. | ||
10948 | |||
10949 | There is no limit to how many files you can specify using the | ||
10950 | ``LIC_FILES_CHKSUM`` variable. Generally, however, every project | ||
10951 | requires a few specifications for license tracking. Many projects have a | ||
10952 | "COPYING" file that stores the license information for all the source | ||
10953 | code files. This practice allows you to just track the "COPYING" file as | ||
10954 | long as it is kept up to date. | ||
10955 | |||
10956 | .. note:: | ||
10957 | |||
10958 | - If you specify an empty or invalid "md5" parameter, | ||
10959 | :term:`BitBake` returns an md5 | ||
10960 | mis-match error and displays the correct "md5" parameter value | ||
10961 | during the build. The correct parameter is also captured in the | ||
10962 | build log. | ||
10963 | |||
10964 | - If the whole file contains only license text, you do not need to | ||
10965 | use the "beginline" and "endline" parameters. | ||
10966 | |||
10967 | Enabling Commercially Licensed Recipes | ||
10968 | -------------------------------------- | ||
10969 | |||
10970 | By default, the OpenEmbedded build system disables components that have | ||
10971 | commercial or other special licensing requirements. Such requirements | ||
10972 | are defined on a recipe-by-recipe basis through the | ||
10973 | :term:`LICENSE_FLAGS` variable | ||
10974 | definition in the affected recipe. For instance, the | ||
10975 | ``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe | ||
10976 | contains the following statement: | ||
10977 | :: | ||
10978 | |||
10979 | LICENSE_FLAGS = "commercial" | ||
10980 | |||
10981 | Here is a | ||
10982 | slightly more complicated example that contains both an explicit recipe | ||
10983 | name and version (after variable expansion): | ||
10984 | :: | ||
10985 | |||
10986 | LICENSE_FLAGS = "license_${PN}_${PV}" | ||
10987 | |||
10988 | In order for a component restricted by a | ||
10989 | ``LICENSE_FLAGS`` definition to be enabled and included in an image, it | ||
10990 | needs to have a matching entry in the global | ||
10991 | :term:`LICENSE_FLAGS_WHITELIST` | ||
10992 | variable, which is a variable typically defined in your ``local.conf`` | ||
10993 | file. For example, to enable the | ||
10994 | ``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you | ||
10995 | could add either the string "commercial_gst-plugins-ugly" or the more | ||
10996 | general string "commercial" to ``LICENSE_FLAGS_WHITELIST``. See the | ||
10997 | "`License Flag Matching <#license-flag-matching>`__" section for a full | ||
10998 | explanation of how ``LICENSE_FLAGS`` matching works. Here is the | ||
10999 | example: | ||
11000 | :: | ||
11001 | |||
11002 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly" | ||
11003 | |||
11004 | Likewise, to additionally enable the package built from the recipe | ||
11005 | containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that | ||
11006 | the actual recipe name was ``emgd_1.10.bb``, the following string would | ||
11007 | enable that package as well as the original ``gst-plugins-ugly`` | ||
11008 | package: | ||
11009 | :: | ||
11010 | |||
11011 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10" | ||
11012 | |||
11013 | As a convenience, you do not need to specify the | ||
11014 | complete license string in the whitelist for every package. You can use | ||
11015 | an abbreviated form, which consists of just the first portion or | ||
11016 | portions of the license string before the initial underscore character | ||
11017 | or characters. A partial string will match any license that contains the | ||
11018 | given string as the first portion of its license. For example, the | ||
11019 | following whitelist string will also match both of the packages | ||
11020 | previously mentioned as well as any other packages that have licenses | ||
11021 | starting with "commercial" or "license". | ||
11022 | :: | ||
11023 | |||
11024 | LICENSE_FLAGS_WHITELIST = "commercial license" | ||
11025 | |||
11026 | License Flag Matching | ||
11027 | ~~~~~~~~~~~~~~~~~~~~~ | ||
11028 | |||
11029 | License flag matching allows you to control what recipes the | ||
11030 | OpenEmbedded build system includes in the build. Fundamentally, the | ||
11031 | build system attempts to match ``LICENSE_FLAGS`` strings found in | ||
11032 | recipes against ``LICENSE_FLAGS_WHITELIST`` strings found in the | ||
11033 | whitelist. A match causes the build system to include a recipe in the | ||
11034 | build, while failure to find a match causes the build system to exclude | ||
11035 | a recipe. | ||
11036 | |||
11037 | In general, license flag matching is simple. However, understanding some | ||
11038 | concepts will help you correctly and effectively use matching. | ||
11039 | |||
11040 | Before a flag defined by a particular recipe is tested against the | ||
11041 | contents of the whitelist, the expanded string ``_${PN}`` is appended to | ||
11042 | the flag. This expansion makes each ``LICENSE_FLAGS`` value | ||
11043 | recipe-specific. After expansion, the string is then matched against the | ||
11044 | whitelist. Thus, specifying ``LICENSE_FLAGS = "commercial"`` in recipe | ||
11045 | "foo", for example, results in the string ``"commercial_foo"``. And, to | ||
11046 | create a match, that string must appear in the whitelist. | ||
11047 | |||
11048 | Judicious use of the ``LICENSE_FLAGS`` strings and the contents of the | ||
11049 | ``LICENSE_FLAGS_WHITELIST`` variable allows you a lot of flexibility for | ||
11050 | including or excluding recipes based on licensing. For example, you can | ||
11051 | broaden the matching capabilities by using license flags string subsets | ||
11052 | in the whitelist. | ||
11053 | |||
11054 | .. note:: | ||
11055 | |||
11056 | When using a string subset, be sure to use the part of the expanded | ||
11057 | string that precedes the appended underscore character (e.g. | ||
11058 | ``usethispart_1.3``, ``usethispart_1.4``, and so forth). | ||
11059 | |||
11060 | For example, simply specifying the string "commercial" in the whitelist | ||
11061 | matches any expanded ``LICENSE_FLAGS`` definition that starts with the | ||
11062 | string "commercial" such as "commercial_foo" and "commercial_bar", which | ||
11063 | are the strings the build system automatically generates for | ||
11064 | hypothetical recipes named "foo" and "bar" assuming those recipes simply | ||
11065 | specify the following: | ||
11066 | :: | ||
11067 | |||
11068 | LICENSE_FLAGS = "commercial" | ||
11069 | |||
11070 | Thus, you can choose | ||
11071 | to exhaustively enumerate each license flag in the whitelist and allow | ||
11072 | only specific recipes into the image, or you can use a string subset | ||
11073 | that causes a broader range of matches to allow a range of recipes into | ||
11074 | the image. | ||
11075 | |||
11076 | This scheme works even if the ``LICENSE_FLAGS`` string already has | ||
11077 | ``_${PN}`` appended. For example, the build system turns the license | ||
11078 | flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match | ||
11079 | both the general "commercial" and the specific "commercial_1.2_foo" | ||
11080 | strings found in the whitelist, as expected. | ||
11081 | |||
11082 | Here are some other scenarios: | ||
11083 | |||
11084 | - You can specify a versioned string in the recipe such as | ||
11085 | "commercial_foo_1.2" in a "foo" recipe. The build system expands this | ||
11086 | string to "commercial_foo_1.2_foo". Combine this license flag with a | ||
11087 | whitelist that has the string "commercial" and you match the flag | ||
11088 | along with any other flag that starts with the string "commercial". | ||
11089 | |||
11090 | - Under the same circumstances, you can use "commercial_foo" in the | ||
11091 | whitelist and the build system not only matches "commercial_foo_1.2" | ||
11092 | but also matches any license flag with the string "commercial_foo", | ||
11093 | regardless of the version. | ||
11094 | |||
11095 | - You can be very specific and use both the package and version parts | ||
11096 | in the whitelist (e.g. "commercial_foo_1.2") to specifically match a | ||
11097 | versioned recipe. | ||
11098 | |||
11099 | Other Variables Related to Commercial Licenses | ||
11100 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11101 | |||
11102 | Other helpful variables related to commercial license handling exist and | ||
11103 | are defined in the | ||
11104 | ``poky/meta/conf/distro/include/default-distrovars.inc`` file: | ||
11105 | :: | ||
11106 | |||
11107 | COMMERCIAL_AUDIO_PLUGINS ?= "" | ||
11108 | COMMERCIAL_VIDEO_PLUGINS ?= "" | ||
11109 | |||
11110 | If you | ||
11111 | want to enable these components, you can do so by making sure you have | ||
11112 | statements similar to the following in your ``local.conf`` configuration | ||
11113 | file: | ||
11114 | :: | ||
11115 | |||
11116 | COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \ | ||
11117 | gst-plugins-ugly-mpegaudioparse" | ||
11118 | COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \ | ||
11119 | gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse" | ||
11120 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp" | ||
11121 | |||
11122 | |||
11123 | Of course, you could also create a matching whitelist for those | ||
11124 | components using the more general "commercial" in the whitelist, but | ||
11125 | that would also enable all the other packages with ``LICENSE_FLAGS`` | ||
11126 | containing "commercial", which you may or may not want: | ||
11127 | :: | ||
11128 | |||
11129 | LICENSE_FLAGS_WHITELIST = "commercial" | ||
11130 | |||
11131 | Specifying audio and video plugins as part of the | ||
11132 | ``COMMERCIAL_AUDIO_PLUGINS`` and ``COMMERCIAL_VIDEO_PLUGINS`` statements | ||
11133 | (along with the enabling ``LICENSE_FLAGS_WHITELIST``) includes the | ||
11134 | plugins or components into built images, thus adding support for media | ||
11135 | formats or components. | ||
11136 | |||
11137 | Maintaining Open Source License Compliance During Your Product's Lifecycle | ||
11138 | -------------------------------------------------------------------------- | ||
11139 | |||
11140 | One of the concerns for a development organization using open source | ||
11141 | software is how to maintain compliance with various open source | ||
11142 | licensing during the lifecycle of the product. While this section does | ||
11143 | not provide legal advice or comprehensively cover all scenarios, it does | ||
11144 | present methods that you can use to assist you in meeting the compliance | ||
11145 | requirements during a software release. | ||
11146 | |||
11147 | With hundreds of different open source licenses that the Yocto Project | ||
11148 | tracks, it is difficult to know the requirements of each and every | ||
11149 | license. However, the requirements of the major FLOSS licenses can begin | ||
11150 | to be covered by assuming that three main areas of concern exist: | ||
11151 | |||
11152 | - Source code must be provided. | ||
11153 | |||
11154 | - License text for the software must be provided. | ||
11155 | |||
11156 | - Compilation scripts and modifications to the source code must be | ||
11157 | provided. | ||
11158 | |||
11159 | - spdx files can be provided. | ||
11160 | |||
11161 | There are other requirements beyond the scope of these three and the | ||
11162 | methods described in this section (e.g. the mechanism through which | ||
11163 | source code is distributed). | ||
11164 | |||
11165 | As different organizations have different methods of complying with open | ||
11166 | source licensing, this section is not meant to imply that there is only | ||
11167 | one single way to meet your compliance obligations, but rather to | ||
11168 | describe one method of achieving compliance. The remainder of this | ||
11169 | section describes methods supported to meet the previously mentioned | ||
11170 | three requirements. Once you take steps to meet these requirements, and | ||
11171 | prior to releasing images, sources, and the build system, you should | ||
11172 | audit all artifacts to ensure completeness. | ||
11173 | |||
11174 | .. note:: | ||
11175 | |||
11176 | The Yocto Project generates a license manifest during image creation | ||
11177 | that is located in ``${DEPLOY_DIR}/licenses/``\ `image_name`\ ``-``\ `datestamp` | ||
11178 | to assist with any audits. | ||
11179 | |||
11180 | Providing the Source Code | ||
11181 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11182 | |||
11183 | Compliance activities should begin before you generate the final image. | ||
11184 | The first thing you should look at is the requirement that tops the list | ||
11185 | for most compliance groups - providing the source. The Yocto Project has | ||
11186 | a few ways of meeting this requirement. | ||
11187 | |||
11188 | One of the easiest ways to meet this requirement is to provide the | ||
11189 | entire :term:`DL_DIR` used by the | ||
11190 | build. This method, however, has a few issues. The most obvious is the | ||
11191 | size of the directory since it includes all sources used in the build | ||
11192 | and not just the source used in the released image. It will include | ||
11193 | toolchain source, and other artifacts, which you would not generally | ||
11194 | release. However, the more serious issue for most companies is | ||
11195 | accidental release of proprietary software. The Yocto Project provides | ||
11196 | an :ref:`archiver <ref-classes-archiver>` class to | ||
11197 | help avoid some of these concerns. | ||
11198 | |||
11199 | Before you employ ``DL_DIR`` or the ``archiver`` class, you need to | ||
11200 | decide how you choose to provide source. The source ``archiver`` class | ||
11201 | can generate tarballs and SRPMs and can create them with various levels | ||
11202 | of compliance in mind. | ||
11203 | |||
11204 | One way of doing this (but certainly not the only way) is to release | ||
11205 | just the source as a tarball. You can do this by adding the following to | ||
11206 | the ``local.conf`` file found in the | ||
11207 | :term:`Build Directory`: | ||
11208 | :: | ||
11209 | |||
11210 | INHERIT += "archiver" | ||
11211 | ARCHIVER_MODE[src] = "original" | ||
11212 | |||
11213 | During the creation of your | ||
11214 | image, the source from all recipes that deploy packages to the image is | ||
11215 | placed within subdirectories of ``DEPLOY_DIR/sources`` based on the | ||
11216 | :term:`LICENSE` for each recipe. | ||
11217 | Releasing the entire directory enables you to comply with requirements | ||
11218 | concerning providing the unmodified source. It is important to note that | ||
11219 | the size of the directory can get large. | ||
11220 | |||
11221 | A way to help mitigate the size issue is to only release tarballs for | ||
11222 | licenses that require the release of source. Let us assume you are only | ||
11223 | concerned with GPL code as identified by running the following script: | ||
11224 | |||
11225 | .. code-block:: shell | ||
11226 | |||
11227 | # Script to archive a subset of packages matching specific license(s) | ||
11228 | # Source and license files are copied into sub folders of package folder | ||
11229 | # Must be run from build folder | ||
11230 | #!/bin/bash | ||
11231 | src_release_dir="source-release" | ||
11232 | mkdir -p $src_release_dir | ||
11233 | for a in tmp/deploy/sources/*; do | ||
11234 | for d in $a/*; do | ||
11235 | # Get package name from path | ||
11236 | p=`basename $d` | ||
11237 | p=${p%-*} | ||
11238 | p=${p%-*} | ||
11239 | # Only archive GPL packages (update *GPL* regex for your license check) | ||
11240 | numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l` | ||
11241 | if [ $numfiles -gt 1 ]; then | ||
11242 | echo Archiving $p | ||
11243 | mkdir -p $src_release_dir/$p/source | ||
11244 | cp $d/* $src_release_dir/$p/source 2> /dev/null | ||
11245 | mkdir -p $src_release_dir/$p/license | ||
11246 | cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null | ||
11247 | fi | ||
11248 | done | ||
11249 | done | ||
11250 | |||
11251 | At this point, you | ||
11252 | could create a tarball from the ``gpl_source_release`` directory and | ||
11253 | provide that to the end user. This method would be a step toward | ||
11254 | achieving compliance with section 3a of GPLv2 and with section 6 of | ||
11255 | GPLv3. | ||
11256 | |||
11257 | Providing License Text | ||
11258 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
11259 | |||
11260 | One requirement that is often overlooked is inclusion of license text. | ||
11261 | This requirement also needs to be dealt with prior to generating the | ||
11262 | final image. Some licenses require the license text to accompany the | ||
11263 | binary. You can achieve this by adding the following to your | ||
11264 | ``local.conf`` file: | ||
11265 | :: | ||
11266 | |||
11267 | COPY_LIC_MANIFEST = "1" | ||
11268 | COPY_LIC_DIRS = "1" | ||
11269 | LICENSE_CREATE_PACKAGE = "1" | ||
11270 | |||
11271 | Adding these statements to the | ||
11272 | configuration file ensures that the licenses collected during package | ||
11273 | generation are included on your image. | ||
11274 | |||
11275 | .. note:: | ||
11276 | |||
11277 | Setting all three variables to "1" results in the image having two | ||
11278 | copies of the same license file. One copy resides in | ||
11279 | ``/usr/share/common-licenses`` and the other resides in | ||
11280 | ``/usr/share/license``. | ||
11281 | |||
11282 | The reason for this behavior is because | ||
11283 | :term:`COPY_LIC_DIRS` and | ||
11284 | :term:`COPY_LIC_MANIFEST` | ||
11285 | add a copy of the license when the image is built but do not offer a | ||
11286 | path for adding licenses for newly installed packages to an image. | ||
11287 | :term:`LICENSE_CREATE_PACKAGE` | ||
11288 | adds a separate package and an upgrade path for adding licenses to an | ||
11289 | image. | ||
11290 | |||
11291 | As the source ``archiver`` class has already archived the original | ||
11292 | unmodified source that contains the license files, you would have | ||
11293 | already met the requirements for inclusion of the license information | ||
11294 | with source as defined by the GPL and other open source licenses. | ||
11295 | |||
11296 | Providing Compilation Scripts and Source Code Modifications | ||
11297 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11298 | |||
11299 | At this point, we have addressed all we need to prior to generating the | ||
11300 | image. The next two requirements are addressed during the final | ||
11301 | packaging of the release. | ||
11302 | |||
11303 | By releasing the version of the OpenEmbedded build system and the layers | ||
11304 | used during the build, you will be providing both compilation scripts | ||
11305 | and the source code modifications in one step. | ||
11306 | |||
11307 | If the deployment team has a :ref:`overview-manual/concepts:bsp layer` | ||
11308 | and a distro layer, and those | ||
11309 | those layers are used to patch, compile, package, or modify (in any way) | ||
11310 | any open source software included in your released images, you might be | ||
11311 | required to release those layers under section 3 of GPLv2 or section 1 | ||
11312 | of GPLv3. One way of doing that is with a clean checkout of the version | ||
11313 | of the Yocto Project and layers used during your build. Here is an | ||
11314 | example: | ||
11315 | |||
11316 | .. code-block:: shell | ||
11317 | |||
11318 | # We built using the dunfell branch of the poky repo | ||
11319 | $ git clone -b dunfell git://git.yoctoproject.org/poky | ||
11320 | $ cd poky | ||
11321 | # We built using the release_branch for our layers | ||
11322 | $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer | ||
11323 | $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer | ||
11324 | # clean up the .git repos | ||
11325 | $ find . -name ".git" -type d -exec rm -rf {} \; | ||
11326 | |||
11327 | One | ||
11328 | thing a development organization might want to consider for end-user | ||
11329 | convenience is to modify ``meta-poky/conf/bblayers.conf.sample`` to | ||
11330 | ensure that when the end user utilizes the released build system to | ||
11331 | build an image, the development organization's layers are included in | ||
11332 | the ``bblayers.conf`` file automatically: | ||
11333 | :: | ||
11334 | |||
11335 | # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf | ||
11336 | # changes incompatibly | ||
11337 | POKY_BBLAYERS_CONF_VERSION = "2" | ||
11338 | |||
11339 | BBPATH = "${TOPDIR}" | ||
11340 | BBFILES ?= "" | ||
11341 | |||
11342 | BBLAYERS ?= " \ | ||
11343 | ##OEROOT##/meta \ | ||
11344 | ##OEROOT##/meta-poky \ | ||
11345 | ##OEROOT##/meta-yocto-bsp \ | ||
11346 | ##OEROOT##/meta-mylayer \ | ||
11347 | " | ||
11348 | |||
11349 | Creating and | ||
11350 | providing an archive of the :term:`Metadata` | ||
11351 | layers (recipes, configuration files, and so forth) enables you to meet | ||
11352 | your requirements to include the scripts to control compilation as well | ||
11353 | as any modifications to the original source. | ||
11354 | |||
11355 | Providing spdx files | ||
11356 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11357 | |||
11358 | The spdx module has been integrated to a layer named meta-spdxscanner. | ||
11359 | meta-spdxscanner provides several kinds of scanner. If you want to enable | ||
11360 | this function, you have to follow the following steps: | ||
11361 | |||
11362 | 1. Add meta-spdxscanner layer into ``bblayers.conf``. | ||
11363 | |||
11364 | 2. Refer to the README in meta-spdxscanner to setup the environment (e.g, | ||
11365 | setup a fossology server) needed for the scanner. | ||
11366 | |||
11367 | 3. Meta-spdxscanner provides several methods within the bbclass to create spdx files. | ||
11368 | Please choose one that you want to use and enable the spdx task. You have to | ||
11369 | add some config options in ``local.conf`` file in your :term:`Build | ||
11370 | Directory`. The following is an example showing how to generate spdx files | ||
11371 | during bitbake using the fossology-python.bbclass:: | ||
11372 | |||
11373 | # Select fossology-python.bbclass. | ||
11374 | INHERIT += "fossology-python" | ||
11375 | # For fossology-python.bbclass, TOKEN is necessary, so, after setup a | ||
11376 | # Fossology server, you have to create a token. | ||
11377 | TOKEN = "eyJ0eXAiO..." | ||
11378 | # The fossology server is necessary for fossology-python.bbclass. | ||
11379 | FOSSOLOGY_SERVER = "http://xx.xx.xx.xx:8081/repo" | ||
11380 | # If you want to upload the source code to a special folder: | ||
11381 | FOLDER_NAME = "xxxx" //Optional | ||
11382 | # If you don't want to put spdx files in tmp/deploy/spdx, you can enable: | ||
11383 | SPDX_DEPLOY_DIR = "${DEPLOY_DIR}" //Optional | ||
11384 | |||
11385 | For more usage information refer to :yocto_git:`the meta-spdxscanner repository | ||
11386 | </meta-spdxscanner/>`. | ||
11387 | |||
11388 | |||
11389 | Copying Licenses that Do Not Exist | ||
11390 | ---------------------------------- | ||
11391 | |||
11392 | Some packages, such as the linux-firmware package, have many licenses | ||
11393 | that are not in any way common. You can avoid adding a lot of these | ||
11394 | types of common license files, which are only applicable to a specific | ||
11395 | package, by using the | ||
11396 | :term:`NO_GENERIC_LICENSE` | ||
11397 | variable. Using this variable also avoids QA errors when you use a | ||
11398 | non-common, non-CLOSED license in a recipe. | ||
11399 | |||
11400 | The following is an example that uses the ``LICENSE.Abilis.txt`` file as | ||
11401 | the license from the fetched source: | ||
11402 | :: | ||
11403 | |||
11404 | NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt" | ||
11405 | |||
11406 | Using the Error Reporting Tool | ||
11407 | ============================== | ||
11408 | |||
11409 | The error reporting tool allows you to submit errors encountered during | ||
11410 | builds to a central database. Outside of the build environment, you can | ||
11411 | use a web interface to browse errors, view statistics, and query for | ||
11412 | errors. The tool works using a client-server system where the client | ||
11413 | portion is integrated with the installed Yocto Project | ||
11414 | :term:`Source Directory` (e.g. ``poky``). | ||
11415 | The server receives the information collected and saves it in a | ||
11416 | database. | ||
11417 | |||
11418 | A live instance of the error reporting server exists at | ||
11419 | https://errors.yoctoproject.org. This server exists so that when | ||
11420 | you want to get help with build failures, you can submit all of the | ||
11421 | information on the failure easily and then point to the URL in your bug | ||
11422 | report or send an email to the mailing list. | ||
11423 | |||
11424 | .. note:: | ||
11425 | |||
11426 | If you send error reports to this server, the reports become publicly | ||
11427 | visible. | ||
11428 | |||
11429 | Enabling and Using the Tool | ||
11430 | --------------------------- | ||
11431 | |||
11432 | By default, the error reporting tool is disabled. You can enable it by | ||
11433 | inheriting the | ||
11434 | :ref:`report-error <ref-classes-report-error>` | ||
11435 | class by adding the following statement to the end of your | ||
11436 | ``local.conf`` file in your | ||
11437 | :term:`Build Directory`. | ||
11438 | :: | ||
11439 | |||
11440 | INHERIT += "report-error" | ||
11441 | |||
11442 | By default, the error reporting feature stores information in | ||
11443 | ``${``\ :term:`LOG_DIR`\ ``}/error-report``. | ||
11444 | However, you can specify a directory to use by adding the following to | ||
11445 | your ``local.conf`` file: | ||
11446 | :: | ||
11447 | |||
11448 | ERR_REPORT_DIR = "path" | ||
11449 | |||
11450 | Enabling error | ||
11451 | reporting causes the build process to collect the errors and store them | ||
11452 | in a file as previously described. When the build system encounters an | ||
11453 | error, it includes a command as part of the console output. You can run | ||
11454 | the command to send the error file to the server. For example, the | ||
11455 | following command sends the errors to an upstream server: | ||
11456 | :: | ||
11457 | |||
11458 | $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt | ||
11459 | |||
11460 | In the previous example, the errors are sent to a public database | ||
11461 | available at https://errors.yoctoproject.org, which is used by the | ||
11462 | entire community. If you specify a particular server, you can send the | ||
11463 | errors to a different database. Use the following command for more | ||
11464 | information on available options: | ||
11465 | :: | ||
11466 | |||
11467 | $ send-error-report --help | ||
11468 | |||
11469 | When sending the error file, you are prompted to review the data being | ||
11470 | sent as well as to provide a name and optional email address. Once you | ||
11471 | satisfy these prompts, the command returns a link from the server that | ||
11472 | corresponds to your entry in the database. For example, here is a | ||
11473 | typical link: https://errors.yoctoproject.org/Errors/Details/9522/ | ||
11474 | |||
11475 | Following the link takes you to a web interface where you can browse, | ||
11476 | query the errors, and view statistics. | ||
11477 | |||
11478 | Disabling the Tool | ||
11479 | ------------------ | ||
11480 | |||
11481 | To disable the error reporting feature, simply remove or comment out the | ||
11482 | following statement from the end of your ``local.conf`` file in your | ||
11483 | :term:`Build Directory`. | ||
11484 | :: | ||
11485 | |||
11486 | INHERIT += "report-error" | ||
11487 | |||
11488 | Setting Up Your Own Error Reporting Server | ||
11489 | ------------------------------------------ | ||
11490 | |||
11491 | If you want to set up your own error reporting server, you can obtain | ||
11492 | the code from the Git repository at :yocto_git:`/error-report-web/`. | ||
11493 | Instructions on how to set it up are in the README document. | ||
11494 | |||
11495 | Using Wayland and Weston | ||
11496 | ======================== | ||
11497 | |||
11498 | `Wayland <https://en.wikipedia.org/wiki/Wayland_(display_server_protocol)>`__ | ||
11499 | is a computer display server protocol that provides a method for | ||
11500 | compositing window managers to communicate directly with applications | ||
11501 | and video hardware and expects them to communicate with input hardware | ||
11502 | using other libraries. Using Wayland with supporting targets can result | ||
11503 | in better control over graphics frame rendering than an application | ||
11504 | might otherwise achieve. | ||
11505 | |||
11506 | The Yocto Project provides the Wayland protocol libraries and the | ||
11507 | reference | ||
11508 | `Weston <https://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston>`__ | ||
11509 | compositor as part of its release. You can find the integrated packages | ||
11510 | in the ``meta`` layer of the :term:`Source Directory`. | ||
11511 | Specifically, you | ||
11512 | can find the recipes that build both Wayland and Weston at | ||
11513 | ``meta/recipes-graphics/wayland``. | ||
11514 | |||
11515 | You can build both the Wayland and Weston packages for use only with | ||
11516 | targets that accept the `Mesa 3D and Direct Rendering | ||
11517 | Infrastructure <https://en.wikipedia.org/wiki/Mesa_(computer_graphics)>`__, | ||
11518 | which is also known as Mesa DRI. This implies that you cannot build and | ||
11519 | use the packages if your target uses, for example, the Intel Embedded | ||
11520 | Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI. | ||
11521 | |||
11522 | .. note:: | ||
11523 | |||
11524 | Due to lack of EGL support, Weston 1.0.3 will not run directly on the | ||
11525 | emulated QEMU hardware. However, this version of Weston will run | ||
11526 | under X emulation without issues. | ||
11527 | |||
11528 | This section describes what you need to do to implement Wayland and use | ||
11529 | the Weston compositor when building an image for a supporting target. | ||
11530 | |||
11531 | Enabling Wayland in an Image | ||
11532 | ---------------------------- | ||
11533 | |||
11534 | To enable Wayland, you need to enable it to be built and enable it to be | ||
11535 | included (installed) in the image. | ||
11536 | |||
11537 | Building Wayland | ||
11538 | ~~~~~~~~~~~~~~~~ | ||
11539 | |||
11540 | To cause Mesa to build the ``wayland-egl`` platform and Weston to build | ||
11541 | Wayland with Kernel Mode Setting | ||
11542 | (`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__) | ||
11543 | support, include the "wayland" flag in the | ||
11544 | :term:`DISTRO_FEATURES` | ||
11545 | statement in your ``local.conf`` file: | ||
11546 | :: | ||
11547 | |||
11548 | DISTRO_FEATURES_append = " wayland" | ||
11549 | |||
11550 | .. note:: | ||
11551 | |||
11552 | If X11 has been enabled elsewhere, Weston will build Wayland with X11 | ||
11553 | support | ||
11554 | |||
11555 | Installing Wayland and Weston | ||
11556 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11557 | |||
11558 | To install the Wayland feature into an image, you must include the | ||
11559 | following | ||
11560 | :term:`CORE_IMAGE_EXTRA_INSTALL` | ||
11561 | statement in your ``local.conf`` file: | ||
11562 | :: | ||
11563 | |||
11564 | CORE_IMAGE_EXTRA_INSTALL += "wayland weston" | ||
11565 | |||
11566 | Running Weston | ||
11567 | -------------- | ||
11568 | |||
11569 | To run Weston inside X11, enabling it as described earlier and building | ||
11570 | a Sato image is sufficient. If you are running your image under Sato, a | ||
11571 | Weston Launcher appears in the "Utility" category. | ||
11572 | |||
11573 | Alternatively, you can run Weston through the command-line interpretor | ||
11574 | (CLI), which is better suited for development work. To run Weston under | ||
11575 | the CLI, you need to do the following after your image is built: | ||
11576 | |||
11577 | 1. Run these commands to export ``XDG_RUNTIME_DIR``: | ||
11578 | :: | ||
11579 | |||
11580 | mkdir -p /tmp/$USER-weston | ||
11581 | chmod 0700 /tmp/$USER-weston | ||
11582 | export XDG_RUNTIME_DIR=/tmp/$USER-weston | ||
11583 | |||
11584 | 2. Launch Weston in the shell: | ||
11585 | :: | ||
11586 | |||
11587 | weston | ||