diff options
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.rst')
-rw-r--r-- | documentation/dev-manual/dev-manual-common-tasks.rst | 11803 |
1 files changed, 11803 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.rst b/documentation/dev-manual/dev-manual-common-tasks.rst new file mode 100644 index 0000000000..179979c763 --- /dev/null +++ b/documentation/dev-manual/dev-manual-common-tasks.rst | |||
@@ -0,0 +1,11803 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-2.0-UK | ||
2 | |||
3 | ************ | ||
4 | Common Tasks | ||
5 | ************ | ||
6 | |||
7 | This chapter describes fundamental procedures such as creating layers, | ||
8 | adding new software packages, extending or customizing images, porting | ||
9 | work to new hardware (adding a new machine), and so forth. You will find | ||
10 | that the procedures documented here occur often in the development cycle | ||
11 | using the Yocto Project. | ||
12 | |||
13 | Understanding and Creating Layers | ||
14 | ================================= | ||
15 | |||
16 | The OpenEmbedded build system supports organizing | ||
17 | :term:`Metadata` into multiple layers. | ||
18 | Layers allow you to isolate different types of customizations from each | ||
19 | other. For introductory information on the Yocto Project Layer Model, | ||
20 | see the | ||
21 | ":ref:`overview-manual/overview-manual-yp-intro:the yocto project layer model`" | ||
22 | section in the Yocto Project Overview and Concepts Manual. | ||
23 | |||
24 | Creating Your Own Layer | ||
25 | ----------------------- | ||
26 | |||
27 | It is very easy to create your own layers to use with the OpenEmbedded | ||
28 | build system. The Yocto Project ships with tools that speed up creating | ||
29 | layers. This section describes the steps you perform by hand to create | ||
30 | layers so that you can better understand them. For information about the | ||
31 | layer-creation tools, see the | ||
32 | ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`" | ||
33 | section in the Yocto Project Board Support Package (BSP) Developer's | ||
34 | Guide and the ":ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`" | ||
35 | section further down in this manual. | ||
36 | |||
37 | Follow these general steps to create your layer without using tools: | ||
38 | |||
39 | 1. *Check Existing Layers:* Before creating a new layer, you should be | ||
40 | sure someone has not already created a layer containing the Metadata | ||
41 | you need. You can see the `OpenEmbedded Metadata | ||
42 | Index <http://layers.openembedded.org/layerindex/layers/>`__ for a | ||
43 | list of layers from the OpenEmbedded community that can be used in | ||
44 | the Yocto Project. You could find a layer that is identical or close | ||
45 | to what you need. | ||
46 | |||
47 | 2. *Create a Directory:* Create the directory for your layer. When you | ||
48 | create the layer, be sure to create the directory in an area not | ||
49 | associated with the Yocto Project :term:`Source Directory` | ||
50 | (e.g. the cloned ``poky`` repository). | ||
51 | |||
52 | While not strictly required, prepend the name of the directory with | ||
53 | the string "meta-". For example: | ||
54 | :: | ||
55 | |||
56 | meta-mylayer | ||
57 | meta-GUI_xyz | ||
58 | meta-mymachine | ||
59 | |||
60 | With rare exceptions, a layer's name follows this form: | ||
61 | :: | ||
62 | |||
63 | meta-root_name | ||
64 | |||
65 | Following this layer naming convention can save | ||
66 | you trouble later when tools, components, or variables "assume" your | ||
67 | layer name begins with "meta-". A notable example is in configuration | ||
68 | files as shown in the following step where layer names without the | ||
69 | "meta-" string are appended to several variables used in the | ||
70 | configuration. | ||
71 | |||
72 | 3. *Create a Layer Configuration File:* Inside your new layer folder, | ||
73 | you need to create a ``conf/layer.conf`` file. It is easiest to take | ||
74 | an existing layer configuration file and copy that to your layer's | ||
75 | ``conf`` directory and then modify the file as needed. | ||
76 | |||
77 | The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project | ||
78 | :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf>` | ||
79 | demonstrates the required syntax. For your layer, you need to replace | ||
80 | "yoctobsp" with a unique identifier for your layer (e.g. "machinexyz" | ||
81 | for a layer named "meta-machinexyz"): | ||
82 | :: | ||
83 | |||
84 | # We have a conf and classes directory, add to BBPATH | ||
85 | BBPATH .= ":${LAYERDIR}" | ||
86 | |||
87 | # We have recipes-\* directories, add to BBFILES | ||
88 | BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ | ||
89 | ${LAYERDIR}/recipes-*/*/*.bbappend" | ||
90 | |||
91 | BBFILE_COLLECTIONS += "yoctobsp" | ||
92 | BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" | ||
93 | BBFILE_PRIORITY_yoctobsp = "5" | ||
94 | LAYERVERSION_yoctobsp = "4" | ||
95 | LAYERSERIES_COMPAT_yoctobsp = "dunfell" | ||
96 | |||
97 | Following is an explanation of the layer configuration file: | ||
98 | |||
99 | - :term:`BBPATH`: Adds the layer's | ||
100 | root directory to BitBake's search path. Through the use of the | ||
101 | ``BBPATH`` variable, BitBake locates class files (``.bbclass``), | ||
102 | configuration files, and files that are included with ``include`` | ||
103 | and ``require`` statements. For these cases, BitBake uses the | ||
104 | first file that matches the name found in ``BBPATH``. This is | ||
105 | similar to the way the ``PATH`` variable is used for binaries. It | ||
106 | is recommended, therefore, that you use unique class and | ||
107 | configuration filenames in your custom layer. | ||
108 | |||
109 | - :term:`BBFILES`: Defines the | ||
110 | location for all recipes in the layer. | ||
111 | |||
112 | - :term:`BBFILE_COLLECTIONS`: | ||
113 | Establishes the current layer through a unique identifier that is | ||
114 | used throughout the OpenEmbedded build system to refer to the | ||
115 | layer. In this example, the identifier "yoctobsp" is the | ||
116 | representation for the container layer named "meta-yocto-bsp". | ||
117 | |||
118 | - :term:`BBFILE_PATTERN`: | ||
119 | Expands immediately during parsing to provide the directory of the | ||
120 | layer. | ||
121 | |||
122 | - :term:`BBFILE_PRIORITY`: | ||
123 | Establishes a priority to use for recipes in the layer when the | ||
124 | OpenEmbedded build finds recipes of the same name in different | ||
125 | layers. | ||
126 | |||
127 | - :term:`LAYERVERSION`: | ||
128 | Establishes a version number for the layer. You can use this | ||
129 | version number to specify this exact version of the layer as a | ||
130 | dependency when using the | ||
131 | :term:`LAYERDEPENDS` | ||
132 | variable. | ||
133 | |||
134 | - :term:`LAYERDEPENDS`: | ||
135 | Lists all layers on which this layer depends (if any). | ||
136 | |||
137 | - :term:`LAYERSERIES_COMPAT`: | ||
138 | Lists the :yocto_wiki:`Yocto Project </wiki/Releases>` | ||
139 | releases for which the current version is compatible. This | ||
140 | variable is a good way to indicate if your particular layer is | ||
141 | current. | ||
142 | |||
143 | 4. *Add Content:* Depending on the type of layer, add the content. If | ||
144 | the layer adds support for a machine, add the machine configuration | ||
145 | in a ``conf/machine/`` file within the layer. If the layer adds | ||
146 | distro policy, add the distro configuration in a ``conf/distro/`` | ||
147 | file within the layer. If the layer introduces new recipes, put the | ||
148 | recipes you need in ``recipes-*`` subdirectories within the layer. | ||
149 | |||
150 | .. note:: | ||
151 | |||
152 | For an explanation of layer hierarchy that is compliant with the | ||
153 | Yocto Project, see the " | ||
154 | Example Filesystem Layout | ||
155 | " section in the Yocto Project Board Support Package (BSP) | ||
156 | Developer's Guide. | ||
157 | |||
158 | 5. *Optionally Test for Compatibility:* If you want permission to use | ||
159 | the Yocto Project Compatibility logo with your layer or application | ||
160 | that uses your layer, perform the steps to apply for compatibility. | ||
161 | See the "`Making Sure Your Layer is Compatible With Yocto | ||
162 | Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__" | ||
163 | section for more information. | ||
164 | |||
165 | .. _best-practices-to-follow-when-creating-layers: | ||
166 | |||
167 | Following Best Practices When Creating Layers | ||
168 | --------------------------------------------- | ||
169 | |||
170 | To create layers that are easier to maintain and that will not impact | ||
171 | builds for other machines, you should consider the information in the | ||
172 | following list: | ||
173 | |||
174 | - *Avoid "Overlaying" Entire Recipes from Other Layers in Your | ||
175 | Configuration:* In other words, do not copy an entire recipe into | ||
176 | your layer and then modify it. Rather, use an append file | ||
177 | (``.bbappend``) to override only those parts of the original recipe | ||
178 | you need to modify. | ||
179 | |||
180 | - *Avoid Duplicating Include Files:* Use append files (``.bbappend``) | ||
181 | for each recipe that uses an include file. Or, if you are introducing | ||
182 | a new recipe that requires the included file, use the path relative | ||
183 | to the original layer directory to refer to the file. For example, | ||
184 | use ``require recipes-core/``\ package\ ``/``\ file\ ``.inc`` instead | ||
185 | of ``require``\ file\ ``.inc``. If you're finding you have to overlay | ||
186 | the include file, it could indicate a deficiency in the include file | ||
187 | in the layer to which it originally belongs. If this is the case, you | ||
188 | should try to address that deficiency instead of overlaying the | ||
189 | include file. For example, you could address this by getting the | ||
190 | maintainer of the include file to add a variable or variables to make | ||
191 | it easy to override the parts needing to be overridden. | ||
192 | |||
193 | - *Structure Your Layers:* Proper use of overrides within append files | ||
194 | and placement of machine-specific files within your layer can ensure | ||
195 | that a build is not using the wrong Metadata and negatively impacting | ||
196 | a build for a different machine. Following are some examples: | ||
197 | |||
198 | - *Modify Variables to Support a Different Machine:* Suppose you | ||
199 | have a layer named ``meta-one`` that adds support for building | ||
200 | machine "one". To do so, you use an append file named | ||
201 | ``base-files.bbappend`` and create a dependency on "foo" by | ||
202 | altering the :term:`DEPENDS` | ||
203 | variable: | ||
204 | :: | ||
205 | |||
206 | DEPENDS = "foo" | ||
207 | |||
208 | The dependency is created during any | ||
209 | build that includes the layer ``meta-one``. However, you might not | ||
210 | want this dependency for all machines. For example, suppose you | ||
211 | are building for machine "two" but your ``bblayers.conf`` file has | ||
212 | the ``meta-one`` layer included. During the build, the | ||
213 | ``base-files`` for machine "two" will also have the dependency on | ||
214 | ``foo``. | ||
215 | |||
216 | To make sure your changes apply only when building machine "one", | ||
217 | use a machine override with the ``DEPENDS`` statement: DEPENDS_one | ||
218 | = "foo" You should follow the same strategy when using ``_append`` | ||
219 | and ``_prepend`` operations: | ||
220 | :: | ||
221 | |||
222 | DEPENDS_append_one = " foo" | ||
223 | DEPENDS_prepend_one = "foo " | ||
224 | |||
225 | As an actual example, here's a | ||
226 | snippet from the generic kernel include file ``linux-yocto.inc``, | ||
227 | wherein the kernel compile and link options are adjusted in the | ||
228 | case of a subset of the supported architectures: | ||
229 | :: | ||
230 | |||
231 | DEPENDS_append_aarch64 = " libgcc" | ||
232 | KERNEL_CC_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" | ||
233 | KERNEL_LD_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" | ||
234 | |||
235 | DEPENDS_append_nios2 = " libgcc" | ||
236 | KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}" | ||
237 | KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}" | ||
238 | |||
239 | DEPENDS_append_arc = " libgcc" | ||
240 | KERNEL_CC_append_arc = " ${TOOLCHAIN_OPTIONS}" | ||
241 | KERNEL_LD_append_arc = " ${TOOLCHAIN_OPTIONS}" | ||
242 | |||
243 | KERNEL_FEATURES_append_qemuall=" features/debug/printk.scc" | ||
244 | |||
245 | .. note:: | ||
246 | |||
247 | Avoiding "+=" and "=+" and using machine-specific | ||
248 | \_append | ||
249 | and | ||
250 | \_prepend | ||
251 | operations is recommended as well. | ||
252 | |||
253 | - *Place Machine-Specific Files in Machine-Specific Locations:* When | ||
254 | you have a base recipe, such as ``base-files.bb``, that contains a | ||
255 | :term:`SRC_URI` statement to a | ||
256 | file, you can use an append file to cause the build to use your | ||
257 | own version of the file. For example, an append file in your layer | ||
258 | at ``meta-one/recipes-core/base-files/base-files.bbappend`` could | ||
259 | extend :term:`FILESPATH` | ||
260 | using | ||
261 | :term:`FILESEXTRAPATHS` | ||
262 | as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" The | ||
263 | build for machine "one" will pick up your machine-specific file as | ||
264 | long as you have the file in | ||
265 | ``meta-one/recipes-core/base-files/base-files/``. However, if you | ||
266 | are building for a different machine and the ``bblayers.conf`` | ||
267 | file includes the ``meta-one`` layer and the location of your | ||
268 | machine-specific file is the first location where that file is | ||
269 | found according to ``FILESPATH``, builds for all machines will | ||
270 | also use that machine-specific file. | ||
271 | |||
272 | You can make sure that a machine-specific file is used for a | ||
273 | particular machine by putting the file in a subdirectory specific | ||
274 | to the machine. For example, rather than placing the file in | ||
275 | ``meta-one/recipes-core/base-files/base-files/`` as shown above, | ||
276 | put it in ``meta-one/recipes-core/base-files/base-files/one/``. | ||
277 | Not only does this make sure the file is used only when building | ||
278 | for machine "one", but the build process locates the file more | ||
279 | quickly. | ||
280 | |||
281 | In summary, you need to place all files referenced from | ||
282 | ``SRC_URI`` in a machine-specific subdirectory within the layer in | ||
283 | order to restrict those files to machine-specific builds. | ||
284 | |||
285 | - *Perform Steps to Apply for Yocto Project Compatibility:* If you want | ||
286 | permission to use the Yocto Project Compatibility logo with your | ||
287 | layer or application that uses your layer, perform the steps to apply | ||
288 | for compatibility. See the "`Making Sure Your Layer is Compatible | ||
289 | With Yocto | ||
290 | Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__" | ||
291 | section for more information. | ||
292 | |||
293 | - *Follow the Layer Naming Convention:* Store custom layers in a Git | ||
294 | repository that use the ``meta-layer_name`` format. | ||
295 | |||
296 | - *Group Your Layers Locally:* Clone your repository alongside other | ||
297 | cloned ``meta`` directories from the :term:`Source Directory`. | ||
298 | |||
299 | Making Sure Your Layer is Compatible With Yocto Project | ||
300 | ------------------------------------------------------- | ||
301 | |||
302 | When you create a layer used with the Yocto Project, it is advantageous | ||
303 | to make sure that the layer interacts well with existing Yocto Project | ||
304 | layers (i.e. the layer is compatible with the Yocto Project). Ensuring | ||
305 | compatibility makes the layer easy to be consumed by others in the Yocto | ||
306 | Project community and could allow you permission to use the Yocto | ||
307 | Project Compatible Logo. | ||
308 | |||
309 | .. note:: | ||
310 | |||
311 | Only Yocto Project member organizations are permitted to use the | ||
312 | Yocto Project Compatible Logo. The logo is not available for general | ||
313 | use. For information on how to become a Yocto Project member | ||
314 | organization, see the | ||
315 | Yocto Project Website | ||
316 | . | ||
317 | |||
318 | The Yocto Project Compatibility Program consists of a layer application | ||
319 | process that requests permission to use the Yocto Project Compatibility | ||
320 | Logo for your layer and application. The process consists of two parts: | ||
321 | |||
322 | 1. Successfully passing a script (``yocto-check-layer``) that when run | ||
323 | against your layer, tests it against constraints based on experiences | ||
324 | of how layers have worked in the real world and where pitfalls have | ||
325 | been found. Getting a "PASS" result from the script is required for | ||
326 | successful compatibility registration. | ||
327 | |||
328 | 2. Completion of an application acceptance form, which you can find at | ||
329 | https://www.yoctoproject.org/webform/yocto-project-compatible-registration. | ||
330 | |||
331 | To be granted permission to use the logo, you need to satisfy the | ||
332 | following: | ||
333 | |||
334 | - Be able to check the box indicating that you got a "PASS" when | ||
335 | running the script against your layer. | ||
336 | |||
337 | - Answer "Yes" to the questions on the form or have an acceptable | ||
338 | explanation for any questions answered "No". | ||
339 | |||
340 | - Be a Yocto Project Member Organization. | ||
341 | |||
342 | The remainder of this section presents information on the registration | ||
343 | form and on the ``yocto-check-layer`` script. | ||
344 | |||
345 | Yocto Project Compatible Program Application | ||
346 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
347 | |||
348 | Use the form to apply for your layer's approval. Upon successful | ||
349 | application, you can use the Yocto Project Compatibility Logo with your | ||
350 | layer and the application that uses your layer. | ||
351 | |||
352 | To access the form, use this link: | ||
353 | https://www.yoctoproject.org/webform/yocto-project-compatible-registration. | ||
354 | Follow the instructions on the form to complete your application. | ||
355 | |||
356 | The application consists of the following sections: | ||
357 | |||
358 | - *Contact Information:* Provide your contact information as the fields | ||
359 | require. Along with your information, provide the released versions | ||
360 | of the Yocto Project for which your layer is compatible. | ||
361 | |||
362 | - *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the | ||
363 | items in the checklist. Space exists at the bottom of the form for | ||
364 | any explanations for items for which you answered "No". | ||
365 | |||
366 | - *Recommendations:* Provide answers for the questions regarding Linux | ||
367 | kernel use and build success. | ||
368 | |||
369 | ``yocto-check-layer`` Script | ||
370 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
371 | |||
372 | The ``yocto-check-layer`` script provides you a way to assess how | ||
373 | compatible your layer is with the Yocto Project. You should run this | ||
374 | script prior to using the form to apply for compatibility as described | ||
375 | in the previous section. You need to achieve a "PASS" result in order to | ||
376 | have your application form successfully processed. | ||
377 | |||
378 | The script divides tests into three areas: COMMON, BSP, and DISTRO. For | ||
379 | example, given a distribution layer (DISTRO), the layer must pass both | ||
380 | the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP | ||
381 | layer, the layer must pass the COMMON and BSP set of tests. | ||
382 | |||
383 | To execute the script, enter the following commands from your build | ||
384 | directory: | ||
385 | :: | ||
386 | |||
387 | $ source oe-init-build-env | ||
388 | $ yocto-check-layer your_layer_directory | ||
389 | |||
390 | Be sure to provide the actual directory for your | ||
391 | layer as part of the command. | ||
392 | |||
393 | Entering the command causes the script to determine the type of layer | ||
394 | and then to execute a set of specific tests against the layer. The | ||
395 | following list overviews the test: | ||
396 | |||
397 | - ``common.test_readme``: Tests if a ``README`` file exists in the | ||
398 | layer and the file is not empty. | ||
399 | |||
400 | - ``common.test_parse``: Tests to make sure that BitBake can parse the | ||
401 | files without error (i.e. ``bitbake -p``). | ||
402 | |||
403 | - ``common.test_show_environment``: Tests that the global or per-recipe | ||
404 | environment is in order without errors (i.e. ``bitbake -e``). | ||
405 | |||
406 | - ``common.test_world``: Verifies that ``bitbake world`` works. | ||
407 | |||
408 | - ``common.test_signatures``: Tests to be sure that BSP and DISTRO | ||
409 | layers do not come with recipes that change signatures. | ||
410 | |||
411 | - ``common.test_layerseries_compat``: Verifies layer compatibility is | ||
412 | set properly. | ||
413 | |||
414 | - ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine | ||
415 | configurations. | ||
416 | |||
417 | - ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not | ||
418 | set the machine when the layer is added. | ||
419 | |||
420 | - ``bsp.test_machine_world``: Verifies that ``bitbake world`` works | ||
421 | regardless of which machine is selected. | ||
422 | |||
423 | - ``bsp.test_machine_signatures``: Verifies that building for a | ||
424 | particular machine affects only the signature of tasks specific to | ||
425 | that machine. | ||
426 | |||
427 | - ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has | ||
428 | distro configurations. | ||
429 | |||
430 | - ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer | ||
431 | does not set the distribution when the layer is added. | ||
432 | |||
433 | Enabling Your Layer | ||
434 | ------------------- | ||
435 | |||
436 | Before the OpenEmbedded build system can use your new layer, you need to | ||
437 | enable it. To enable your layer, simply add your layer's path to the | ||
438 | ``BBLAYERS`` variable in your ``conf/bblayers.conf`` file, which is | ||
439 | found in the :term:`Build Directory`. | ||
440 | The following example shows how to enable a layer named | ||
441 | ``meta-mylayer``: | ||
442 | :: | ||
443 | |||
444 | # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf | ||
445 | # changes incompatibly | ||
446 | POKY_BBLAYERS_CONF_VERSION = "2" | ||
447 | BBPATH = "${TOPDIR}" | ||
448 | BBFILES ?= "" | ||
449 | BBLAYERS ?= " \ | ||
450 | /home/user/poky/meta \ | ||
451 | /home/user/poky/meta-poky \ | ||
452 | /home/user/poky/meta-yocto-bsp \ | ||
453 | /home/user/poky/meta-mylayer \ | ||
454 | " | ||
455 | |||
456 | BitBake parses each ``conf/layer.conf`` file from the top down as | ||
457 | specified in the ``BBLAYERS`` variable within the ``conf/bblayers.conf`` | ||
458 | file. During the processing of each ``conf/layer.conf`` file, BitBake | ||
459 | adds the recipes, classes and configurations contained within the | ||
460 | particular layer to the source directory. | ||
461 | |||
462 | .. _using-bbappend-files: | ||
463 | |||
464 | Using .bbappend Files in Your Layer | ||
465 | ----------------------------------- | ||
466 | |||
467 | A recipe that appends Metadata to another recipe is called a BitBake | ||
468 | append file. A BitBake append file uses the ``.bbappend`` file type | ||
469 | suffix, while the corresponding recipe to which Metadata is being | ||
470 | appended uses the ``.bb`` file type suffix. | ||
471 | |||
472 | You can use a ``.bbappend`` file in your layer to make additions or | ||
473 | changes to the content of another layer's recipe without having to copy | ||
474 | the other layer's recipe into your layer. Your ``.bbappend`` file | ||
475 | resides in your layer, while the main ``.bb`` recipe file to which you | ||
476 | are appending Metadata resides in a different layer. | ||
477 | |||
478 | Being able to append information to an existing recipe not only avoids | ||
479 | duplication, but also automatically applies recipe changes from a | ||
480 | different layer into your layer. If you were copying recipes, you would | ||
481 | have to manually merge changes as they occur. | ||
482 | |||
483 | When you create an append file, you must use the same root name as the | ||
484 | corresponding recipe file. For example, the append file | ||
485 | ``someapp_DISTRO.bbappend`` must apply to ``someapp_DISTRO.bb``. This | ||
486 | means the original recipe and append file names are version | ||
487 | number-specific. If the corresponding recipe is renamed to update to a | ||
488 | newer version, you must also rename and possibly update the | ||
489 | corresponding ``.bbappend`` as well. During the build process, BitBake | ||
490 | displays an error on starting if it detects a ``.bbappend`` file that | ||
491 | does not have a corresponding recipe with a matching name. See the | ||
492 | :term:`BB_DANGLINGAPPENDS_WARNONLY` | ||
493 | variable for information on how to handle this error. | ||
494 | |||
495 | As an example, consider the main formfactor recipe and a corresponding | ||
496 | formfactor append file both from the :term:`Source Directory`. | ||
497 | Here is the main | ||
498 | formfactor recipe, which is named ``formfactor_0.0.bb`` and located in | ||
499 | the "meta" layer at ``meta/recipes-bsp/formfactor``: | ||
500 | :: | ||
501 | |||
502 | SUMMARY = "Device formfactor information" | ||
503 | SECTION = "base" | ||
504 | LICENSE = "MIT" | ||
505 | LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" | ||
506 | PR = "r45" | ||
507 | |||
508 | SRC_URI = "file://config file://machconfig" | ||
509 | S = "${WORKDIR}" | ||
510 | |||
511 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
512 | INHIBIT_DEFAULT_DEPS = "1" | ||
513 | |||
514 | do_install() { | ||
515 | # Install file only if it has contents | ||
516 | install -d ${D}${sysconfdir}/formfactor/ | ||
517 | install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ | ||
518 | if [ -s "${S}/machconfig" ]; then | ||
519 | install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ | ||
520 | fi | ||
521 | } | ||
522 | |||
523 | In the main recipe, note the :term:`SRC_URI` | ||
524 | variable, which tells the OpenEmbedded build system where to find files | ||
525 | during the build. | ||
526 | |||
527 | Following is the append file, which is named ``formfactor_0.0.bbappend`` | ||
528 | and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The | ||
529 | file is in the layer at ``recipes-bsp/formfactor``: | ||
530 | :: | ||
531 | |||
532 | FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" | ||
533 | |||
534 | By default, the build system uses the | ||
535 | :term:`FILESPATH` variable to | ||
536 | locate files. This append file extends the locations by setting the | ||
537 | :term:`FILESEXTRAPATHS` | ||
538 | variable. Setting this variable in the ``.bbappend`` file is the most | ||
539 | reliable and recommended method for adding directories to the search | ||
540 | path used by the build system to find files. | ||
541 | |||
542 | The statement in this example extends the directories to include | ||
543 | ``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``, | ||
544 | which resolves to a directory named ``formfactor`` in the same directory | ||
545 | in which the append file resides (i.e. | ||
546 | ``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must | ||
547 | have the supporting directory structure set up that will contain any | ||
548 | files or patches you will be including from the layer. | ||
549 | |||
550 | Using the immediate expansion assignment operator ``:=`` is important | ||
551 | because of the reference to ``THISDIR``. The trailing colon character is | ||
552 | important as it ensures that items in the list remain colon-separated. | ||
553 | |||
554 | .. note:: | ||
555 | |||
556 | BitBake automatically defines the ``THISDIR`` variable. You should | ||
557 | never set this variable yourself. Using "_prepend" as part of the | ||
558 | ``FILESEXTRAPATHS`` ensures your path will be searched prior to other | ||
559 | paths in the final list. | ||
560 | |||
561 | Also, not all append files add extra files. Many append files simply | ||
562 | exist to add build options (e.g. ``systemd``). For these cases, your | ||
563 | append file would not even use the ``FILESEXTRAPATHS`` statement. | ||
564 | |||
565 | Prioritizing Your Layer | ||
566 | ----------------------- | ||
567 | |||
568 | Each layer is assigned a priority value. Priority values control which | ||
569 | layer takes precedence if there are recipe files with the same name in | ||
570 | multiple layers. For these cases, the recipe file from the layer with a | ||
571 | higher priority number takes precedence. Priority values also affect the | ||
572 | order in which multiple ``.bbappend`` files for the same recipe are | ||
573 | applied. You can either specify the priority manually, or allow the | ||
574 | build system to calculate it based on the layer's dependencies. | ||
575 | |||
576 | To specify the layer's priority manually, use the | ||
577 | :term:`BBFILE_PRIORITY` | ||
578 | variable and append the layer's root name: | ||
579 | :: | ||
580 | |||
581 | BBFILE_PRIORITY_mylayer = "1" | ||
582 | |||
583 | .. note:: | ||
584 | |||
585 | It is possible for a recipe with a lower version number | ||
586 | :term:`PV` in a layer that has a higher | ||
587 | priority to take precedence. | ||
588 | |||
589 | Also, the layer priority does not currently affect the precedence | ||
590 | order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake | ||
591 | might address this. | ||
592 | |||
593 | Managing Layers | ||
594 | --------------- | ||
595 | |||
596 | You can use the BitBake layer management tool ``bitbake-layers`` to | ||
597 | provide a view into the structure of recipes across a multi-layer | ||
598 | project. Being able to generate output that reports on configured layers | ||
599 | with their paths and priorities and on ``.bbappend`` files and their | ||
600 | applicable recipes can help to reveal potential problems. | ||
601 | |||
602 | For help on the BitBake layer management tool, use the following | ||
603 | command: | ||
604 | :: | ||
605 | |||
606 | $ bitbake-layers --help NOTE: Starting bitbake server... usage: | ||
607 | NOTE: Starting bitbake server... | ||
608 | usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ... | ||
609 | |||
610 | BitBake layers utility | ||
611 | |||
612 | optional arguments: | ||
613 | -d, --debug Enable debug output | ||
614 | -q, --quiet Print only errors | ||
615 | -F, --force Force add without recipe parse verification | ||
616 | --color COLOR Colorize output (where COLOR is auto, always, never) | ||
617 | -h, --help show this help message and exit | ||
618 | |||
619 | subcommands: | ||
620 | <subcommand> | ||
621 | layerindex-fetch Fetches a layer from a layer index along with its | ||
622 | dependent layers, and adds them to conf/bblayers.conf. | ||
623 | layerindex-show-depends | ||
624 | Find layer dependencies from layer index. | ||
625 | add-layer Add one or more layers to bblayers.conf. | ||
626 | remove-layer Remove one or more layers from bblayers.conf. | ||
627 | flatten flatten layer configuration into a separate output | ||
628 | directory. | ||
629 | show-layers show current configured layers. | ||
630 | show-overlayed list overlayed recipes (where the same recipe exists | ||
631 | in another layer) | ||
632 | show-recipes list available recipes, showing the layer they are | ||
633 | provided by | ||
634 | show-appends list bbappend files and recipe files they apply to | ||
635 | show-cross-depends Show dependencies between recipes that cross layer | ||
636 | boundaries. | ||
637 | create-layer Create a basic layer | ||
638 | |||
639 | Use bitbake-layers <subcommand> --help to get help on a specific command | ||
640 | |||
641 | The following list describes the available commands: | ||
642 | |||
643 | - ``help:`` Displays general help or help on a specified command. | ||
644 | |||
645 | - ``show-layers:`` Shows the current configured layers. | ||
646 | |||
647 | - ``show-overlayed:`` Lists overlayed recipes. A recipe is overlayed | ||
648 | when a recipe with the same name exists in another layer that has a | ||
649 | higher layer priority. | ||
650 | |||
651 | - ``show-recipes:`` Lists available recipes and the layers that | ||
652 | provide them. | ||
653 | |||
654 | - ``show-appends:`` Lists ``.bbappend`` files and the recipe files to | ||
655 | which they apply. | ||
656 | |||
657 | - ``show-cross-depends:`` Lists dependency relationships between | ||
658 | recipes that cross layer boundaries. | ||
659 | |||
660 | - ``add-layer:`` Adds a layer to ``bblayers.conf``. | ||
661 | |||
662 | - ``remove-layer:`` Removes a layer from ``bblayers.conf`` | ||
663 | |||
664 | - ``flatten:`` Flattens the layer configuration into a separate | ||
665 | output directory. Flattening your layer configuration builds a | ||
666 | "flattened" directory that contains the contents of all layers, with | ||
667 | any overlayed recipes removed and any ``.bbappend`` files appended to | ||
668 | the corresponding recipes. You might have to perform some manual | ||
669 | cleanup of the flattened layer as follows: | ||
670 | |||
671 | - Non-recipe files (such as patches) are overwritten. The flatten | ||
672 | command shows a warning for these files. | ||
673 | |||
674 | - Anything beyond the normal layer setup has been added to the | ||
675 | ``layer.conf`` file. Only the lowest priority layer's | ||
676 | ``layer.conf`` is used. | ||
677 | |||
678 | - Overridden and appended items from ``.bbappend`` files need to be | ||
679 | cleaned up. The contents of each ``.bbappend`` end up in the | ||
680 | flattened recipe. However, if there are appended or changed | ||
681 | variable values, you need to tidy these up yourself. Consider the | ||
682 | following example. Here, the ``bitbake-layers`` command adds the | ||
683 | line ``#### bbappended ...`` so that you know where the following | ||
684 | lines originate: | ||
685 | :: | ||
686 | |||
687 | ... | ||
688 | DESCRIPTION = "A useful utility" | ||
689 | ... | ||
690 | EXTRA_OECONF = "--enable-something" | ||
691 | ... | ||
692 | |||
693 | #### bbappended from meta-anotherlayer #### | ||
694 | |||
695 | DESCRIPTION = "Customized utility" | ||
696 | EXTRA_OECONF += "--enable-somethingelse" | ||
697 | |||
698 | |||
699 | Ideally, you would tidy up these utilities as follows: | ||
700 | :: | ||
701 | |||
702 | ... | ||
703 | DESCRIPTION = "Customized utility" | ||
704 | ... | ||
705 | EXTRA_OECONF = "--enable-something --enable-somethingelse" | ||
706 | ... | ||
707 | |||
708 | - ``layerindex-fetch``: Fetches a layer from a layer index, along | ||
709 | with its dependent layers, and adds the layers to the | ||
710 | ``conf/bblayers.conf`` file. | ||
711 | |||
712 | - ``layerindex-show-depends``: Finds layer dependencies from the | ||
713 | layer index. | ||
714 | |||
715 | - ``create-layer``: Creates a basic layer. | ||
716 | |||
717 | Creating a General Layer Using the ``bitbake-layers`` Script | ||
718 | ------------------------------------------------------------ | ||
719 | |||
720 | The ``bitbake-layers`` script with the ``create-layer`` subcommand | ||
721 | simplifies creating a new general layer. | ||
722 | |||
723 | .. note:: | ||
724 | |||
725 | - For information on BSP layers, see the ":ref:`bsp-guide/bsp:bsp layers`" | ||
726 | section in the Yocto | ||
727 | Project Board Specific (BSP) Developer's Guide. | ||
728 | |||
729 | - In order to use a layer with the OpenEmbedded build system, you | ||
730 | need to add the layer to your ``bblayers.conf`` configuration | ||
731 | file. See the ":ref:`dev-manual/dev-manual-common-tasks:adding a layer using the \`\`bitbake-layers\`\` script`" | ||
732 | section for more information. | ||
733 | |||
734 | The default mode of the script's operation with this subcommand is to | ||
735 | create a layer with the following: | ||
736 | |||
737 | - A layer priority of 6. | ||
738 | |||
739 | - A ``conf`` subdirectory that contains a ``layer.conf`` file. | ||
740 | |||
741 | - A ``recipes-example`` subdirectory that contains a further | ||
742 | subdirectory named ``example``, which contains an ``example.bb`` | ||
743 | recipe file. | ||
744 | |||
745 | - A ``COPYING.MIT``, which is the license statement for the layer. The | ||
746 | script assumes you want to use the MIT license, which is typical for | ||
747 | most layers, for the contents of the layer itself. | ||
748 | |||
749 | - A ``README`` file, which is a file describing the contents of your | ||
750 | new layer. | ||
751 | |||
752 | In its simplest form, you can use the following command form to create a | ||
753 | layer. The command creates a layer whose name corresponds to | ||
754 | your_layer_name in the current directory: $ bitbake-layers create-layer | ||
755 | your_layer_name As an example, the following command creates a layer | ||
756 | named ``meta-scottrif`` in your home directory: | ||
757 | :: | ||
758 | |||
759 | $ cd /usr/home | ||
760 | $ bitbake-layers create-layer meta-scottrif | ||
761 | NOTE: Starting bitbake server... | ||
762 | Add your new layer with 'bitbake-layers add-layer meta-scottrif' | ||
763 | |||
764 | If you want to set the priority of the layer to other than the default | ||
765 | value of "6", you can either use the ``DASHDASHpriority`` option or you | ||
766 | can edit the | ||
767 | :term:`BBFILE_PRIORITY` value | ||
768 | in the ``conf/layer.conf`` after the script creates it. Furthermore, if | ||
769 | you want to give the example recipe file some name other than the | ||
770 | default, you can use the ``DASHDASHexample-recipe-name`` option. | ||
771 | |||
772 | The easiest way to see how the ``bitbake-layers create-layer`` command | ||
773 | works is to experiment with the script. You can also read the usage | ||
774 | information by entering the following: | ||
775 | :: | ||
776 | |||
777 | $ bitbake-layers create-layer --help | ||
778 | NOTE: Starting bitbake server... | ||
779 | usage: bitbake-layers create-layer [-h] [--priority PRIORITY] | ||
780 | [--example-recipe-name EXAMPLERECIPE] | ||
781 | layerdir | ||
782 | |||
783 | Create a basic layer | ||
784 | |||
785 | positional arguments: | ||
786 | layerdir Layer directory to create | ||
787 | |||
788 | optional arguments: | ||
789 | -h, --help show this help message and exit | ||
790 | --priority PRIORITY, -p PRIORITY | ||
791 | Layer directory to create | ||
792 | --example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE | ||
793 | Filename of the example recipe | ||
794 | |||
795 | Adding a Layer Using the ``bitbake-layers`` Script | ||
796 | -------------------------------------------------- | ||
797 | |||
798 | Once you create your general layer, you must add it to your | ||
799 | ``bblayers.conf`` file. Adding the layer to this configuration file | ||
800 | makes the OpenEmbedded build system aware of your layer so that it can | ||
801 | search it for metadata. | ||
802 | |||
803 | Add your layer by using the ``bitbake-layers add-layer`` command: | ||
804 | :: | ||
805 | |||
806 | $ bitbake-layers add-layer your_layer_name | ||
807 | |||
808 | Here is an example that adds a | ||
809 | layer named ``meta-scottrif`` to the configuration file. Following the | ||
810 | command that adds the layer is another ``bitbake-layers`` command that | ||
811 | shows the layers that are in your ``bblayers.conf`` file: | ||
812 | :: | ||
813 | |||
814 | $ bitbake-layers add-layer meta-scottrif | ||
815 | NOTE: Starting bitbake server... | ||
816 | Parsing recipes: 100% |##########################################################| Time: 0:00:49 | ||
817 | Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors. | ||
818 | $ bitbake-layers show-layers | ||
819 | NOTE: Starting bitbake server... | ||
820 | layer path priority | ||
821 | ========================================================================== | ||
822 | meta /home/scottrif/poky/meta 5 | ||
823 | meta-poky /home/scottrif/poky/meta-poky 5 | ||
824 | meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5 | ||
825 | workspace /home/scottrif/poky/build/workspace 99 | ||
826 | meta-scottrif /home/scottrif/poky/build/meta-scottrif 6 | ||
827 | |||
828 | |||
829 | Adding the layer to this file | ||
830 | enables the build system to locate the layer during the build. | ||
831 | |||
832 | .. note:: | ||
833 | |||
834 | During a build, the OpenEmbedded build system looks in the layers | ||
835 | from the top of the list down to the bottom in that order. | ||
836 | |||
837 | .. _usingpoky-extend-customimage: | ||
838 | |||
839 | Customizing Images | ||
840 | ================== | ||
841 | |||
842 | You can customize images to satisfy particular requirements. This | ||
843 | section describes several methods and provides guidelines for each. | ||
844 | |||
845 | .. _usingpoky-extend-customimage-localconf: | ||
846 | |||
847 | Customizing Images Using ``local.conf`` | ||
848 | --------------------------------------- | ||
849 | |||
850 | Probably the easiest way to customize an image is to add a package by | ||
851 | way of the ``local.conf`` configuration file. Because it is limited to | ||
852 | local use, this method generally only allows you to add packages and is | ||
853 | not as flexible as creating your own customized image. When you add | ||
854 | packages using local variables this way, you need to realize that these | ||
855 | variable changes are in effect for every build and consequently affect | ||
856 | all images, which might not be what you require. | ||
857 | |||
858 | To add a package to your image using the local configuration file, use | ||
859 | the ``IMAGE_INSTALL`` variable with the ``_append`` operator: | ||
860 | :: | ||
861 | |||
862 | IMAGE_INSTALL_append = " strace" | ||
863 | |||
864 | Use of the syntax is important - | ||
865 | specifically, the space between the quote and the package name, which is | ||
866 | ``strace`` in this example. This space is required since the ``_append`` | ||
867 | operator does not add the space. | ||
868 | |||
869 | Furthermore, you must use ``_append`` instead of the ``+=`` operator if | ||
870 | you want to avoid ordering issues. The reason for this is because doing | ||
871 | so unconditionally appends to the variable and avoids ordering problems | ||
872 | due to the variable being set in image recipes and ``.bbclass`` files | ||
873 | with operators like ``?=``. Using ``_append`` ensures the operation | ||
874 | takes affect. | ||
875 | |||
876 | As shown in its simplest use, ``IMAGE_INSTALL_append`` affects all | ||
877 | images. It is possible to extend the syntax so that the variable applies | ||
878 | to a specific image only. Here is an example: | ||
879 | IMAGE_INSTALL_append_pn-core-image-minimal = " strace" This example adds | ||
880 | ``strace`` to the ``core-image-minimal`` image only. | ||
881 | |||
882 | You can add packages using a similar approach through the | ||
883 | ``CORE_IMAGE_EXTRA_INSTALL`` variable. If you use this variable, only | ||
884 | ``core-image-*`` images are affected. | ||
885 | |||
886 | .. _usingpoky-extend-customimage-imagefeatures: | ||
887 | |||
888 | Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES`` | ||
889 | ------------------------------------------------------------------------------- | ||
890 | |||
891 | Another method for customizing your image is to enable or disable | ||
892 | high-level image features by using the | ||
893 | :term:`IMAGE_FEATURES` and | ||
894 | :term:`EXTRA_IMAGE_FEATURES` | ||
895 | variables. Although the functions for both variables are nearly | ||
896 | equivalent, best practices dictate using ``IMAGE_FEATURES`` from within | ||
897 | a recipe and using ``EXTRA_IMAGE_FEATURES`` from within your | ||
898 | ``local.conf`` file, which is found in the | ||
899 | :term:`Build Directory`. | ||
900 | |||
901 | To understand how these features work, the best reference is | ||
902 | ``meta/classes/core-image.bbclass``. This class lists out the available | ||
903 | ``IMAGE_FEATURES`` of which most map to package groups while some, such | ||
904 | as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general | ||
905 | configuration settings. | ||
906 | |||
907 | In summary, the file looks at the contents of the ``IMAGE_FEATURES`` | ||
908 | variable and then maps or configures the feature accordingly. Based on | ||
909 | this information, the build system automatically adds the appropriate | ||
910 | packages or configurations to the | ||
911 | :term:`IMAGE_INSTALL` variable. | ||
912 | Effectively, you are enabling extra features by extending the class or | ||
913 | creating a custom class for use with specialized image ``.bb`` files. | ||
914 | |||
915 | Use the ``EXTRA_IMAGE_FEATURES`` variable from within your local | ||
916 | configuration file. Using a separate area from which to enable features | ||
917 | with this variable helps you avoid overwriting the features in the image | ||
918 | recipe that are enabled with ``IMAGE_FEATURES``. The value of | ||
919 | ``EXTRA_IMAGE_FEATURES`` is added to ``IMAGE_FEATURES`` within | ||
920 | ``meta/conf/bitbake.conf``. | ||
921 | |||
922 | To illustrate how you can use these variables to modify your image, | ||
923 | consider an example that selects the SSH server. The Yocto Project ships | ||
924 | with two SSH servers you can use with your images: Dropbear and OpenSSH. | ||
925 | Dropbear is a minimal SSH server appropriate for resource-constrained | ||
926 | environments, while OpenSSH is a well-known standard SSH server | ||
927 | implementation. By default, the ``core-image-sato`` image is configured | ||
928 | to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb`` | ||
929 | images both include OpenSSH. The ``core-image-minimal`` image does not | ||
930 | contain an SSH server. | ||
931 | |||
932 | You can customize your image and change these defaults. Edit the | ||
933 | ``IMAGE_FEATURES`` variable in your recipe or use the | ||
934 | ``EXTRA_IMAGE_FEATURES`` in your ``local.conf`` file so that it | ||
935 | configures the image you are working with to include | ||
936 | ``ssh-server-dropbear`` or ``ssh-server-openssh``. | ||
937 | |||
938 | .. note:: | ||
939 | |||
940 | See the " | ||
941 | Images | ||
942 | " section in the Yocto Project Reference Manual for a complete list | ||
943 | of image features that ship with the Yocto Project. | ||
944 | |||
945 | .. _usingpoky-extend-customimage-custombb: | ||
946 | |||
947 | Customizing Images Using Custom .bb Files | ||
948 | ----------------------------------------- | ||
949 | |||
950 | You can also customize an image by creating a custom recipe that defines | ||
951 | additional software as part of the image. The following example shows | ||
952 | the form for the two lines you need: | ||
953 | :: | ||
954 | |||
955 | IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2" | ||
956 | inherit core-image | ||
957 | |||
958 | Defining the software using a custom recipe gives you total control over | ||
959 | the contents of the image. It is important to use the correct names of | ||
960 | packages in the ``IMAGE_INSTALL`` variable. You must use the | ||
961 | OpenEmbedded notation and not the Debian notation for the names (e.g. | ||
962 | ``glibc-dev`` instead of ``libc6-dev``). | ||
963 | |||
964 | The other method for creating a custom image is to base it on an | ||
965 | existing image. For example, if you want to create an image based on | ||
966 | ``core-image-sato`` but add the additional package ``strace`` to the | ||
967 | image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new | ||
968 | ``.bb`` and add the following line to the end of the copy: | ||
969 | :: | ||
970 | |||
971 | IMAGE_INSTALL += "strace" | ||
972 | |||
973 | .. _usingpoky-extend-customimage-customtasks: | ||
974 | |||
975 | Customizing Images Using Custom Package Groups | ||
976 | ---------------------------------------------- | ||
977 | |||
978 | For complex custom images, the best approach for customizing an image is | ||
979 | to create a custom package group recipe that is used to build the image | ||
980 | or images. A good example of a package group recipe is | ||
981 | ``meta/recipes-core/packagegroups/packagegroup-base.bb``. | ||
982 | |||
983 | If you examine that recipe, you see that the ``PACKAGES`` variable lists | ||
984 | the package group packages to produce. The ``inherit packagegroup`` | ||
985 | statement sets appropriate default values and automatically adds | ||
986 | ``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each | ||
987 | package specified in the ``PACKAGES`` statement. | ||
988 | |||
989 | .. note:: | ||
990 | |||
991 | The | ||
992 | inherit packagegroup | ||
993 | line should be located near the top of the recipe, certainly before | ||
994 | the | ||
995 | PACKAGES | ||
996 | statement. | ||
997 | |||
998 | For each package you specify in ``PACKAGES``, you can use ``RDEPENDS`` | ||
999 | and ``RRECOMMENDS`` entries to provide a list of packages the parent | ||
1000 | task package should contain. You can see examples of these further down | ||
1001 | in the ``packagegroup-base.bb`` recipe. | ||
1002 | |||
1003 | Here is a short, fabricated example showing the same basic pieces for a | ||
1004 | hypothetical packagegroup defined in ``packagegroup-custom.bb``, where | ||
1005 | the variable ``PN`` is the standard way to abbreviate the reference to | ||
1006 | the full packagegroup name ``packagegroup-custom``: | ||
1007 | :: | ||
1008 | |||
1009 | DESCRIPTION = "My Custom Package Groups" | ||
1010 | |||
1011 | inherit packagegroup | ||
1012 | |||
1013 | PACKAGES = "\ | ||
1014 | ${PN}-apps \ | ||
1015 | ${PN}-tools \ | ||
1016 | " | ||
1017 | |||
1018 | RDEPENDS_${PN}-apps = "\ | ||
1019 | dropbear \ | ||
1020 | portmap \ | ||
1021 | psplash" | ||
1022 | |||
1023 | RDEPENDS_${PN}-tools = "\ | ||
1024 | oprofile \ | ||
1025 | oprofileui-server \ | ||
1026 | lttng-tools" | ||
1027 | |||
1028 | RRECOMMENDS_${PN}-tools = "\ | ||
1029 | kernel-module-oprofile" | ||
1030 | |||
1031 | In the previous example, two package group packages are created with | ||
1032 | their dependencies and their recommended package dependencies listed: | ||
1033 | ``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To | ||
1034 | build an image using these package group packages, you need to add | ||
1035 | ``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to | ||
1036 | ``IMAGE_INSTALL``. For other forms of image dependencies see the other | ||
1037 | areas of this section. | ||
1038 | |||
1039 | .. _usingpoky-extend-customimage-image-name: | ||
1040 | |||
1041 | Customizing an Image Hostname | ||
1042 | ----------------------------- | ||
1043 | |||
1044 | By default, the configured hostname (i.e. ``/etc/hostname``) in an image | ||
1045 | is the same as the machine name. For example, if | ||
1046 | :term:`MACHINE` equals "qemux86", the | ||
1047 | configured hostname written to ``/etc/hostname`` is "qemux86". | ||
1048 | |||
1049 | You can customize this name by altering the value of the "hostname" | ||
1050 | variable in the ``base-files`` recipe using either an append file or a | ||
1051 | configuration file. Use the following in an append file: | ||
1052 | :: | ||
1053 | |||
1054 | hostname = "myhostname" | ||
1055 | |||
1056 | Use the following in a configuration file: | ||
1057 | :: | ||
1058 | |||
1059 | hostname_pn-base-files = "myhostname" | ||
1060 | |||
1061 | Changing the default value of the variable "hostname" can be useful in | ||
1062 | certain situations. For example, suppose you need to do extensive | ||
1063 | testing on an image and you would like to easily identify the image | ||
1064 | under test from existing images with typical default hostnames. In this | ||
1065 | situation, you could change the default hostname to "testme", which | ||
1066 | results in all the images using the name "testme". Once testing is | ||
1067 | complete and you do not need to rebuild the image for test any longer, | ||
1068 | you can easily reset the default hostname. | ||
1069 | |||
1070 | Another point of interest is that if you unset the variable, the image | ||
1071 | will have no default hostname in the filesystem. Here is an example that | ||
1072 | unsets the variable in a configuration file: | ||
1073 | :: | ||
1074 | |||
1075 | hostname_pn-base-files = "" | ||
1076 | |||
1077 | Having no default hostname in the filesystem is suitable for | ||
1078 | environments that use dynamic hostnames such as virtual machines. | ||
1079 | |||
1080 | .. _new-recipe-writing-a-new-recipe: | ||
1081 | |||
1082 | Writing a New Recipe | ||
1083 | ==================== | ||
1084 | |||
1085 | Recipes (``.bb`` files) are fundamental components in the Yocto Project | ||
1086 | environment. Each software component built by the OpenEmbedded build | ||
1087 | system requires a recipe to define the component. This section describes | ||
1088 | how to create, write, and test a new recipe. | ||
1089 | |||
1090 | .. note:: | ||
1091 | |||
1092 | For information on variables that are useful for recipes and for | ||
1093 | information about recipe naming issues, see the " | ||
1094 | Required | ||
1095 | " section of the Yocto Project Reference Manual. | ||
1096 | |||
1097 | .. _new-recipe-overview: | ||
1098 | |||
1099 | Overview | ||
1100 | -------- | ||
1101 | |||
1102 | The following figure shows the basic process for creating a new recipe. | ||
1103 | The remainder of the section provides details for the steps. | ||
1104 | |||
1105 | .. image:: figures/recipe-workflow.png | ||
1106 | :align: center | ||
1107 | |||
1108 | .. _new-recipe-locate-or-automatically-create-a-base-recipe: | ||
1109 | |||
1110 | Locate or Automatically Create a Base Recipe | ||
1111 | -------------------------------------------- | ||
1112 | |||
1113 | You can always write a recipe from scratch. However, three choices exist | ||
1114 | that can help you quickly get a start on a new recipe: | ||
1115 | |||
1116 | - ``devtool add``: A command that assists in creating a recipe and an | ||
1117 | environment conducive to development. | ||
1118 | |||
1119 | - ``recipetool create``: A command provided by the Yocto Project that | ||
1120 | automates creation of a base recipe based on the source files. | ||
1121 | |||
1122 | - *Existing Recipes:* Location and modification of an existing recipe | ||
1123 | that is similar in function to the recipe you need. | ||
1124 | |||
1125 | .. note:: | ||
1126 | |||
1127 | For information on recipe syntax, see the " | ||
1128 | Recipe Syntax | ||
1129 | " section. | ||
1130 | |||
1131 | .. _new-recipe-creating-the-base-recipe-using-devtool: | ||
1132 | |||
1133 | Creating the Base Recipe Using ``devtool add`` | ||
1134 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1135 | |||
1136 | The ``devtool add`` command uses the same logic for auto-creating the | ||
1137 | recipe as ``recipetool create``, which is listed below. Additionally, | ||
1138 | however, ``devtool add`` sets up an environment that makes it easy for | ||
1139 | you to patch the source and to make changes to the recipe as is often | ||
1140 | necessary when adding a recipe to build a new piece of software to be | ||
1141 | included in a build. | ||
1142 | |||
1143 | You can find a complete description of the ``devtool add`` command in | ||
1144 | the ":ref:`sdk-a-closer-look-at-devtool-add`" section | ||
1145 | in the Yocto Project Application Development and the Extensible Software | ||
1146 | Development Kit (eSDK) manual. | ||
1147 | |||
1148 | .. _new-recipe-creating-the-base-recipe-using-recipetool: | ||
1149 | |||
1150 | Creating the Base Recipe Using ``recipetool create`` | ||
1151 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1152 | |||
1153 | ``recipetool create`` automates creation of a base recipe given a set of | ||
1154 | source code files. As long as you can extract or point to the source | ||
1155 | files, the tool will construct a recipe and automatically configure all | ||
1156 | pre-build information into the recipe. For example, suppose you have an | ||
1157 | application that builds using Autotools. Creating the base recipe using | ||
1158 | ``recipetool`` results in a recipe that has the pre-build dependencies, | ||
1159 | license requirements, and checksums configured. | ||
1160 | |||
1161 | To run the tool, you just need to be in your | ||
1162 | :term:`Build Directory` and have sourced the | ||
1163 | build environment setup script (i.e. | ||
1164 | `:ref:`structure-core-script`). | ||
1165 | To get help on the tool, use the following command: | ||
1166 | :: | ||
1167 | |||
1168 | $ recipetool -h | ||
1169 | NOTE: Starting bitbake server... | ||
1170 | usage: recipetool [-d] [-q] [--color COLOR] [-h] <subcommand> ... | ||
1171 | |||
1172 | OpenEmbedded recipe tool | ||
1173 | |||
1174 | options: | ||
1175 | -d, --debug Enable debug output | ||
1176 | -q, --quiet Print only errors | ||
1177 | --color COLOR Colorize output (where COLOR is auto, always, never) | ||
1178 | -h, --help show this help message and exit | ||
1179 | |||
1180 | subcommands: | ||
1181 | create Create a new recipe | ||
1182 | newappend Create a bbappend for the specified target in the specified | ||
1183 | layer | ||
1184 | setvar Set a variable within a recipe | ||
1185 | appendfile Create/update a bbappend to replace a target file | ||
1186 | appendsrcfiles Create/update a bbappend to add or replace source files | ||
1187 | appendsrcfile Create/update a bbappend to add or replace a source file | ||
1188 | Use recipetool <subcommand> --help to get help on a specific command | ||
1189 | |||
1190 | Running ``recipetool create -o`` OUTFILE creates the base recipe and | ||
1191 | locates it properly in the layer that contains your source files. | ||
1192 | Following are some syntax examples: | ||
1193 | |||
1194 | Use this syntax to generate a recipe based on source. Once generated, | ||
1195 | the recipe resides in the existing source code layer: | ||
1196 | :: | ||
1197 | |||
1198 | recipetool create -o OUTFILE source | ||
1199 | |||
1200 | Use this syntax to generate a recipe using code that | ||
1201 | you extract from source. The extracted code is placed in its own layer | ||
1202 | defined by EXTERNALSRC. | ||
1203 | :: | ||
1204 | |||
1205 | recipetool create -o OUTFILE -x EXTERNALSRC source | ||
1206 | |||
1207 | Use this syntax to generate a recipe based on source. The options | ||
1208 | direct ``recipetool`` to generate debugging information. Once generated, | ||
1209 | the recipe resides in the existing source code layer: | ||
1210 | :: | ||
1211 | |||
1212 | recipetool create -d -o OUTFILE source | ||
1213 | |||
1214 | .. _new-recipe-locating-and-using-a-similar-recipe: | ||
1215 | |||
1216 | Locating and Using a Similar Recipe | ||
1217 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
1218 | |||
1219 | Before writing a recipe from scratch, it is often useful to discover | ||
1220 | whether someone else has already written one that meets (or comes close | ||
1221 | to meeting) your needs. The Yocto Project and OpenEmbedded communities | ||
1222 | maintain many recipes that might be candidates for what you are doing. | ||
1223 | You can find a good central index of these recipes in the `OpenEmbedded | ||
1224 | Layer Index <http://layers.openembedded.org>`__. | ||
1225 | |||
1226 | Working from an existing recipe or a skeleton recipe is the best way to | ||
1227 | get started. Here are some points on both methods: | ||
1228 | |||
1229 | - *Locate and modify a recipe that is close to what you want to do:* | ||
1230 | This method works when you are familiar with the current recipe | ||
1231 | space. The method does not work so well for those new to the Yocto | ||
1232 | Project or writing recipes. | ||
1233 | |||
1234 | Some risks associated with this method are using a recipe that has | ||
1235 | areas totally unrelated to what you are trying to accomplish with | ||
1236 | your recipe, not recognizing areas of the recipe that you might have | ||
1237 | to add from scratch, and so forth. All these risks stem from | ||
1238 | unfamiliarity with the existing recipe space. | ||
1239 | |||
1240 | - *Use and modify the following skeleton recipe:* If for some reason | ||
1241 | you do not want to use ``recipetool`` and you cannot find an existing | ||
1242 | recipe that is close to meeting your needs, you can use the following | ||
1243 | structure to provide the fundamental areas of a new recipe. | ||
1244 | :: | ||
1245 | |||
1246 | DESCRIPTION = "" | ||
1247 | HOMEPAGE = "" | ||
1248 | LICENSE = "" | ||
1249 | SECTION = "" | ||
1250 | DEPENDS = "" | ||
1251 | LIC_FILES_CHKSUM = "" | ||
1252 | |||
1253 | SRC_URI = "" | ||
1254 | |||
1255 | .. _new-recipe-storing-and-naming-the-recipe: | ||
1256 | |||
1257 | Storing and Naming the Recipe | ||
1258 | ----------------------------- | ||
1259 | |||
1260 | Once you have your base recipe, you should put it in your own layer and | ||
1261 | name it appropriately. Locating it correctly ensures that the | ||
1262 | OpenEmbedded build system can find it when you use BitBake to process | ||
1263 | the recipe. | ||
1264 | |||
1265 | - *Storing Your Recipe:* The OpenEmbedded build system locates your | ||
1266 | recipe through the layer's ``conf/layer.conf`` file and the | ||
1267 | :term:`BBFILES` variable. This | ||
1268 | variable sets up a path from which the build system can locate | ||
1269 | recipes. Here is the typical use: | ||
1270 | :: | ||
1271 | |||
1272 | BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ | ||
1273 | ${LAYERDIR}/recipes-*/*/*.bbappend" | ||
1274 | |||
1275 | Consequently, you need to be sure you locate your new recipe inside | ||
1276 | your layer such that it can be found. | ||
1277 | |||
1278 | You can find more information on how layers are structured in the | ||
1279 | "`Understanding and Creating | ||
1280 | Layers <#understanding-and-creating-layers>`__" section. | ||
1281 | |||
1282 | - *Naming Your Recipe:* When you name your recipe, you need to follow | ||
1283 | this naming convention: basename_version.bb Use lower-cased | ||
1284 | characters and do not include the reserved suffixes ``-native``, | ||
1285 | ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use them | ||
1286 | as part of your recipe name unless the string applies). Here are some | ||
1287 | examples: | ||
1288 | :: | ||
1289 | |||
1290 | cups_1.7.0.bb | ||
1291 | gawk_4.0.2.bb | ||
1292 | irssi_0.8.16-rc1.bb | ||
1293 | |||
1294 | .. _new-recipe-running-a-build-on-the-recipe: | ||
1295 | |||
1296 | Running a Build on the Recipe | ||
1297 | ----------------------------- | ||
1298 | |||
1299 | Creating a new recipe is usually an iterative process that requires | ||
1300 | using BitBake to process the recipe multiple times in order to | ||
1301 | progressively discover and add information to the recipe file. | ||
1302 | |||
1303 | Assuming you have sourced the build environment setup script (i.e. | ||
1304 | :ref:`structure-core-script`) and you are in | ||
1305 | the :term:`Build Directory`, use | ||
1306 | BitBake to process your recipe. All you need to provide is the | ||
1307 | ``basename`` of the recipe as described in the previous section: | ||
1308 | :: | ||
1309 | |||
1310 | $ bitbake basename | ||
1311 | |||
1312 | During the build, the OpenEmbedded build system creates a temporary work | ||
1313 | directory for each recipe | ||
1314 | (``${``\ :term:`WORKDIR`\ ``}``) | ||
1315 | where it keeps extracted source files, log files, intermediate | ||
1316 | compilation and packaging files, and so forth. | ||
1317 | |||
1318 | The path to the per-recipe temporary work directory depends on the | ||
1319 | context in which it is being built. The quickest way to find this path | ||
1320 | is to have BitBake return it by running the following: | ||
1321 | :: | ||
1322 | |||
1323 | $ bitbake -e basename \| grep ^WORKDIR= | ||
1324 | |||
1325 | As an example, assume a Source Directory | ||
1326 | top-level folder named ``poky``, a default Build Directory at | ||
1327 | ``poky/build``, and a ``qemux86-poky-linux`` machine target system. | ||
1328 | Furthermore, suppose your recipe is named ``foo_1.3.0.bb``. In this | ||
1329 | case, the work directory the build system uses to build the package | ||
1330 | would be as follows: poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 | ||
1331 | Inside this directory you can find sub-directories such as ``image``, | ||
1332 | ``packages-split``, and ``temp``. After the build, you can examine these | ||
1333 | to determine how well the build went. | ||
1334 | |||
1335 | .. note:: | ||
1336 | |||
1337 | You can find log files for each task in the recipe's | ||
1338 | temp | ||
1339 | directory (e.g. | ||
1340 | poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp | ||
1341 | ). Log files are named | ||
1342 | log. | ||
1343 | taskname | ||
1344 | (e.g. | ||
1345 | log.do_configure | ||
1346 | , | ||
1347 | log.do_fetch | ||
1348 | , and | ||
1349 | log.do_compile | ||
1350 | ). | ||
1351 | |||
1352 | You can find more information about the build process in | ||
1353 | ":doc:`../overview-manual/overview-manual-development-environment`" | ||
1354 | chapter of the Yocto Project Overview and Concepts Manual. | ||
1355 | |||
1356 | .. _new-recipe-fetching-code: | ||
1357 | |||
1358 | Fetching Code | ||
1359 | ------------- | ||
1360 | |||
1361 | The first thing your recipe must do is specify how to fetch the source | ||
1362 | files. Fetching is controlled mainly through the | ||
1363 | :term:`SRC_URI` variable. Your recipe | ||
1364 | must have a ``SRC_URI`` variable that points to where the source is | ||
1365 | located. For a graphical representation of source locations, see the | ||
1366 | ":ref:`sources-dev-environment`" section in | ||
1367 | the Yocto Project Overview and Concepts Manual. | ||
1368 | |||
1369 | The :ref:`ref-tasks-fetch` task uses | ||
1370 | the prefix of each entry in the ``SRC_URI`` variable value to determine | ||
1371 | which :ref:`fetcher <bitbake:bb-fetchers>` to use to get your | ||
1372 | source files. It is the ``SRC_URI`` variable that triggers the fetcher. | ||
1373 | The :ref:`ref-tasks-patch` task uses | ||
1374 | the variable after source is fetched to apply patches. The OpenEmbedded | ||
1375 | build system uses | ||
1376 | :term:`FILESOVERRIDES` for | ||
1377 | scanning directory locations for local files in ``SRC_URI``. | ||
1378 | |||
1379 | The ``SRC_URI`` variable in your recipe must define each unique location | ||
1380 | for your source files. It is good practice to not hard-code version | ||
1381 | numbers in a URL used in ``SRC_URI``. Rather than hard-code these | ||
1382 | values, use ``${``\ :term:`PV`\ ``}``, | ||
1383 | which causes the fetch process to use the version specified in the | ||
1384 | recipe filename. Specifying the version in this manner means that | ||
1385 | upgrading the recipe to a future version is as simple as renaming the | ||
1386 | recipe to match the new version. | ||
1387 | |||
1388 | Here is a simple example from the | ||
1389 | ``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source | ||
1390 | comes from a single tarball. Notice the use of the | ||
1391 | :term:`PV` variable: | ||
1392 | :: | ||
1393 | |||
1394 | SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \\ | ||
1395 | |||
1396 | Files mentioned in ``SRC_URI`` whose names end in a typical archive | ||
1397 | extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so | ||
1398 | forth), are automatically extracted during the | ||
1399 | :ref:`ref-tasks-unpack` task. For | ||
1400 | another example that specifies these types of files, see the | ||
1401 | "`Autotooled Package <#new-recipe-autotooled-package>`__" section. | ||
1402 | |||
1403 | Another way of specifying source is from an SCM. For Git repositories, | ||
1404 | you must specify :term:`SRCREV` and | ||
1405 | you should specify :term:`PV` to include | ||
1406 | the revision with :term:`SRCPV`. Here | ||
1407 | is an example from the recipe | ||
1408 | ``meta/recipes-kernel/blktrace/blktrace_git.bb``: | ||
1409 | :: | ||
1410 | |||
1411 | SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385" | ||
1412 | |||
1413 | PR = "r6" | ||
1414 | PV = "1.0.5+git${SRCPV}" | ||
1415 | |||
1416 | SRC_URI = "git://git.kernel.dk/blktrace.git \ | ||
1417 | file://ldflags.patch" | ||
1418 | |||
1419 | If your ``SRC_URI`` statement includes URLs pointing to individual files | ||
1420 | fetched from a remote server other than a version control system, | ||
1421 | BitBake attempts to verify the files against checksums defined in your | ||
1422 | recipe to ensure they have not been tampered with or otherwise modified | ||
1423 | since the recipe was written. Two checksums are used: | ||
1424 | ``SRC_URI[md5sum]`` and ``SRC_URI[sha256sum]``. | ||
1425 | |||
1426 | If your ``SRC_URI`` variable points to more than a single URL (excluding | ||
1427 | SCM URLs), you need to provide the ``md5`` and ``sha256`` checksums for | ||
1428 | each URL. For these cases, you provide a name for each URL as part of | ||
1429 | the ``SRC_URI`` and then reference that name in the subsequent checksum | ||
1430 | statements. Here is an example combining lines from the files | ||
1431 | ``git.inc`` and ``git_2.24.1.bb``: | ||
1432 | :: | ||
1433 | |||
1434 | SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \ | ||
1435 | ${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages" | ||
1436 | |||
1437 | SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b" | ||
1438 | SRC_URI[tarball.sha256sum] = "ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02" | ||
1439 | SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d" | ||
1440 | SRC_URI[manpages.sha256sum] = "9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230" | ||
1441 | |||
1442 | Proper values for ``md5`` and ``sha256`` checksums might be available | ||
1443 | with other signatures on the download page for the upstream source (e.g. | ||
1444 | ``md5``, ``sha1``, ``sha256``, ``GPG``, and so forth). Because the | ||
1445 | OpenEmbedded build system only deals with ``sha256sum`` and ``md5sum``, | ||
1446 | you should verify all the signatures you find by hand. | ||
1447 | |||
1448 | If no ``SRC_URI`` checksums are specified when you attempt to build the | ||
1449 | recipe, or you provide an incorrect checksum, the build will produce an | ||
1450 | error for each missing or incorrect checksum. As part of the error | ||
1451 | message, the build system provides the checksum string corresponding to | ||
1452 | the fetched file. Once you have the correct checksums, you can copy and | ||
1453 | paste them into your recipe and then run the build again to continue. | ||
1454 | |||
1455 | .. note:: | ||
1456 | |||
1457 | As mentioned, if the upstream source provides signatures for | ||
1458 | verifying the downloaded source code, you should verify those | ||
1459 | manually before setting the checksum values in the recipe and | ||
1460 | continuing with the build. | ||
1461 | |||
1462 | This final example is a bit more complicated and is from the | ||
1463 | ``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The | ||
1464 | example's ``SRC_URI`` statement identifies multiple files as the source | ||
1465 | files for the recipe: a tarball, a patch file, a desktop file, and an | ||
1466 | icon. | ||
1467 | :: | ||
1468 | |||
1469 | SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ | ||
1470 | file://xwc.patch \ | ||
1471 | file://rxvt.desktop \ | ||
1472 | file://rxvt.png" | ||
1473 | |||
1474 | When you specify local files using the ``file://`` URI protocol, the | ||
1475 | build system fetches files from the local machine. The path is relative | ||
1476 | to the :term:`FILESPATH` variable | ||
1477 | and searches specific directories in a certain order: | ||
1478 | ``${``\ :term:`BP`\ ``}``, | ||
1479 | ``${``\ :term:`BPN`\ ``}``, and | ||
1480 | ``files``. The directories are assumed to be subdirectories of the | ||
1481 | directory in which the recipe or append file resides. For another | ||
1482 | example that specifies these types of files, see the "`Single .c File | ||
1483 | Package (Hello | ||
1484 | World!) <#new-recipe-single-c-file-package-hello-world>`__" section. | ||
1485 | |||
1486 | The previous example also specifies a patch file. Patch files are files | ||
1487 | whose names usually end in ``.patch`` or ``.diff`` but can end with | ||
1488 | compressed suffixes such as ``diff.gz`` and ``patch.bz2``, for example. | ||
1489 | The build system automatically applies patches as described in the | ||
1490 | "`Patching Code <#new-recipe-patching-code>`__" section. | ||
1491 | |||
1492 | .. _new-recipe-unpacking-code: | ||
1493 | |||
1494 | Unpacking Code | ||
1495 | -------------- | ||
1496 | |||
1497 | During the build, the | ||
1498 | :ref:`ref-tasks-unpack` task unpacks | ||
1499 | the source with ``${``\ :term:`S`\ ``}`` | ||
1500 | pointing to where it is unpacked. | ||
1501 | |||
1502 | If you are fetching your source files from an upstream source archived | ||
1503 | tarball and the tarball's internal structure matches the common | ||
1504 | convention of a top-level subdirectory named | ||
1505 | ``${``\ :term:`BPN`\ ``}-${``\ :term:`PV`\ ``}``, | ||
1506 | then you do not need to set ``S``. However, if ``SRC_URI`` specifies to | ||
1507 | fetch source from an archive that does not use this convention, or from | ||
1508 | an SCM like Git or Subversion, your recipe needs to define ``S``. | ||
1509 | |||
1510 | If processing your recipe using BitBake successfully unpacks the source | ||
1511 | files, you need to be sure that the directory pointed to by ``${S}`` | ||
1512 | matches the structure of the source. | ||
1513 | |||
1514 | .. _new-recipe-patching-code: | ||
1515 | |||
1516 | Patching Code | ||
1517 | ------------- | ||
1518 | |||
1519 | Sometimes it is necessary to patch code after it has been fetched. Any | ||
1520 | files mentioned in ``SRC_URI`` whose names end in ``.patch`` or | ||
1521 | ``.diff`` or compressed versions of these suffixes (e.g. ``diff.gz`` are | ||
1522 | treated as patches. The | ||
1523 | :ref:`ref-tasks-patch` task | ||
1524 | automatically applies these patches. | ||
1525 | |||
1526 | The build system should be able to apply patches with the "-p1" option | ||
1527 | (i.e. one directory level in the path will be stripped off). If your | ||
1528 | patch needs to have more directory levels stripped off, specify the | ||
1529 | number of levels using the "striplevel" option in the ``SRC_URI`` entry | ||
1530 | for the patch. Alternatively, if your patch needs to be applied in a | ||
1531 | specific subdirectory that is not specified in the patch file, use the | ||
1532 | "patchdir" option in the entry. | ||
1533 | |||
1534 | As with all local files referenced in | ||
1535 | :term:`SRC_URI` using ``file://``, | ||
1536 | you should place patch files in a directory next to the recipe either | ||
1537 | named the same as the base name of the recipe | ||
1538 | (:term:`BP` and | ||
1539 | :term:`BPN`) or "files". | ||
1540 | |||
1541 | .. _new-recipe-licensing: | ||
1542 | |||
1543 | Licensing | ||
1544 | --------- | ||
1545 | |||
1546 | Your recipe needs to have both the | ||
1547 | :term:`LICENSE` and | ||
1548 | :term:`LIC_FILES_CHKSUM` | ||
1549 | variables: | ||
1550 | |||
1551 | - ``LICENSE``: This variable specifies the license for the software. | ||
1552 | If you do not know the license under which the software you are | ||
1553 | building is distributed, you should go to the source code and look | ||
1554 | for that information. Typical files containing this information | ||
1555 | include ``COPYING``, ``LICENSE``, and ``README`` files. You could | ||
1556 | also find the information near the top of a source file. For example, | ||
1557 | given a piece of software licensed under the GNU General Public | ||
1558 | License version 2, you would set ``LICENSE`` as follows: | ||
1559 | :: | ||
1560 | |||
1561 | LICENSE = "GPLv2" | ||
1562 | |||
1563 | The licenses you specify within ``LICENSE`` can have any name as long | ||
1564 | as you do not use spaces, since spaces are used as separators between | ||
1565 | license names. For standard licenses, use the names of the files in | ||
1566 | ``meta/files/common-licenses/`` or the ``SPDXLICENSEMAP`` flag names | ||
1567 | defined in ``meta/conf/licenses.conf``. | ||
1568 | |||
1569 | - ``LIC_FILES_CHKSUM``: The OpenEmbedded build system uses this | ||
1570 | variable to make sure the license text has not changed. If it has, | ||
1571 | the build produces an error and it affords you the chance to figure | ||
1572 | it out and correct the problem. | ||
1573 | |||
1574 | You need to specify all applicable licensing files for the software. | ||
1575 | At the end of the configuration step, the build process will compare | ||
1576 | the checksums of the files to be sure the text has not changed. Any | ||
1577 | differences result in an error with the message containing the | ||
1578 | current checksum. For more explanation and examples of how to set the | ||
1579 | ``LIC_FILES_CHKSUM`` variable, see the "`Tracking License | ||
1580 | Changes <#>`__" section. | ||
1581 | |||
1582 | To determine the correct checksum string, you can list the | ||
1583 | appropriate files in the ``LIC_FILES_CHKSUM`` variable with incorrect | ||
1584 | md5 strings, attempt to build the software, and then note the | ||
1585 | resulting error messages that will report the correct md5 strings. | ||
1586 | See the "`Fetching Code <#new-recipe-fetching-code>`__" section for | ||
1587 | additional information. | ||
1588 | |||
1589 | Here is an example that assumes the software has a ``COPYING`` file: | ||
1590 | :: | ||
1591 | |||
1592 | LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" | ||
1593 | |||
1594 | When you try to build the | ||
1595 | software, the build system will produce an error and give you the | ||
1596 | correct string that you can substitute into the recipe file for a | ||
1597 | subsequent build. | ||
1598 | |||
1599 | .. _new-dependencies: | ||
1600 | |||
1601 | Dependencies | ||
1602 | ------------ | ||
1603 | |||
1604 | Most software packages have a short list of other packages that they | ||
1605 | require, which are called dependencies. These dependencies fall into two | ||
1606 | main categories: build-time dependencies, which are required when the | ||
1607 | software is built; and runtime dependencies, which are required to be | ||
1608 | installed on the target in order for the software to run. | ||
1609 | |||
1610 | Within a recipe, you specify build-time dependencies using the | ||
1611 | :term:`DEPENDS` variable. Although | ||
1612 | nuances exist, items specified in ``DEPENDS`` should be names of other | ||
1613 | recipes. It is important that you specify all build-time dependencies | ||
1614 | explicitly. If you do not, due to the parallel nature of BitBake's | ||
1615 | execution, you can end up with a race condition where the dependency is | ||
1616 | present for one task of a recipe (e.g. | ||
1617 | :ref:`ref-tasks-configure`) and | ||
1618 | then gone when the next task runs (e.g. | ||
1619 | :ref:`ref-tasks-compile`). | ||
1620 | |||
1621 | Another consideration is that configure scripts might automatically | ||
1622 | check for optional dependencies and enable corresponding functionality | ||
1623 | if those dependencies are found. This behavior means that to ensure | ||
1624 | deterministic results and thus avoid more race conditions, you need to | ||
1625 | either explicitly specify these dependencies as well, or tell the | ||
1626 | configure script explicitly to disable the functionality. If you wish to | ||
1627 | make a recipe that is more generally useful (e.g. publish the recipe in | ||
1628 | a layer for others to use), instead of hard-disabling the functionality, | ||
1629 | you can use the | ||
1630 | :term:`PACKAGECONFIG` variable | ||
1631 | to allow functionality and the corresponding dependencies to be enabled | ||
1632 | and disabled easily by other users of the recipe. | ||
1633 | |||
1634 | Similar to build-time dependencies, you specify runtime dependencies | ||
1635 | through a variable - | ||
1636 | :term:`RDEPENDS`, which is | ||
1637 | package-specific. All variables that are package-specific need to have | ||
1638 | the name of the package added to the end as an override. Since the main | ||
1639 | package for a recipe has the same name as the recipe, and the recipe's | ||
1640 | name can be found through the | ||
1641 | ``${``\ :term:`PN`\ ``}`` variable, then | ||
1642 | you specify the dependencies for the main package by setting | ||
1643 | ``RDEPENDS_${PN}``. If the package were named ``${PN}-tools``, then you | ||
1644 | would set ``RDEPENDS_${PN}-tools``, and so forth. | ||
1645 | |||
1646 | Some runtime dependencies will be set automatically at packaging time. | ||
1647 | These dependencies include any shared library dependencies (i.e. if a | ||
1648 | package "example" contains "libexample" and another package "mypackage" | ||
1649 | contains a binary that links to "libexample" then the OpenEmbedded build | ||
1650 | system will automatically add a runtime dependency to "mypackage" on | ||
1651 | "example"). See the | ||
1652 | ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`" | ||
1653 | section in the Yocto Project Overview and Concepts Manual for further | ||
1654 | details. | ||
1655 | |||
1656 | .. _new-recipe-configuring-the-recipe: | ||
1657 | |||
1658 | Configuring the Recipe | ||
1659 | ---------------------- | ||
1660 | |||
1661 | Most software provides some means of setting build-time configuration | ||
1662 | options before compilation. Typically, setting these options is | ||
1663 | accomplished by running a configure script with options, or by modifying | ||
1664 | a build configuration file. | ||
1665 | |||
1666 | .. note:: | ||
1667 | |||
1668 | As of Yocto Project Release 1.7, some of the core recipes that | ||
1669 | package binary configuration scripts now disable the scripts due to | ||
1670 | the scripts previously requiring error-prone path substitution. The | ||
1671 | OpenEmbedded build system uses | ||
1672 | pkg-config | ||
1673 | now, which is much more robust. You can find a list of the | ||
1674 | \*-config | ||
1675 | scripts that are disabled list in the " | ||
1676 | Binary Configuration Scripts Disabled | ||
1677 | " section in the Yocto Project Reference Manual. | ||
1678 | |||
1679 | A major part of build-time configuration is about checking for | ||
1680 | build-time dependencies and possibly enabling optional functionality as | ||
1681 | a result. You need to specify any build-time dependencies for the | ||
1682 | software you are building in your recipe's | ||
1683 | :term:`DEPENDS` value, in terms of | ||
1684 | other recipes that satisfy those dependencies. You can often find | ||
1685 | build-time or runtime dependencies described in the software's | ||
1686 | documentation. | ||
1687 | |||
1688 | The following list provides configuration items of note based on how | ||
1689 | your software is built: | ||
1690 | |||
1691 | - *Autotools:* If your source files have a ``configure.ac`` file, then | ||
1692 | your software is built using Autotools. If this is the case, you just | ||
1693 | need to worry about modifying the configuration. | ||
1694 | |||
1695 | When using Autotools, your recipe needs to inherit the | ||
1696 | :ref:`autotools <ref-classes-autotools>` class | ||
1697 | and your recipe does not have to contain a | ||
1698 | :ref:`ref-tasks-configure` task. | ||
1699 | However, you might still want to make some adjustments. For example, | ||
1700 | you can set | ||
1701 | :term:`EXTRA_OECONF` or | ||
1702 | :term:`PACKAGECONFIG_CONFARGS` | ||
1703 | to pass any needed configure options that are specific to the recipe. | ||
1704 | |||
1705 | - *CMake:* If your source files have a ``CMakeLists.txt`` file, then | ||
1706 | your software is built using CMake. If this is the case, you just | ||
1707 | need to worry about modifying the configuration. | ||
1708 | |||
1709 | When you use CMake, your recipe needs to inherit the | ||
1710 | :ref:`cmake <ref-classes-cmake>` class and your | ||
1711 | recipe does not have to contain a | ||
1712 | :ref:`ref-tasks-configure` task. | ||
1713 | You can make some adjustments by setting | ||
1714 | :term:`EXTRA_OECMAKE` to | ||
1715 | pass any needed configure options that are specific to the recipe. | ||
1716 | |||
1717 | .. note:: | ||
1718 | |||
1719 | If you need to install one or more custom CMake toolchain files | ||
1720 | that are supplied by the application you are building, install the | ||
1721 | files to | ||
1722 | ${D}${datadir}/cmake/ | ||
1723 | Modules during | ||
1724 | do_install | ||
1725 | . | ||
1726 | |||
1727 | - *Other:* If your source files do not have a ``configure.ac`` or | ||
1728 | ``CMakeLists.txt`` file, then your software is built using some | ||
1729 | method other than Autotools or CMake. If this is the case, you | ||
1730 | normally need to provide a | ||
1731 | :ref:`ref-tasks-configure` task | ||
1732 | in your recipe unless, of course, there is nothing to configure. | ||
1733 | |||
1734 | Even if your software is not being built by Autotools or CMake, you | ||
1735 | still might not need to deal with any configuration issues. You need | ||
1736 | to determine if configuration is even a required step. You might need | ||
1737 | to modify a Makefile or some configuration file used for the build to | ||
1738 | specify necessary build options. Or, perhaps you might need to run a | ||
1739 | provided, custom configure script with the appropriate options. | ||
1740 | |||
1741 | For the case involving a custom configure script, you would run | ||
1742 | ``./configure --help`` and look for the options you need to set. | ||
1743 | |||
1744 | Once configuration succeeds, it is always good practice to look at the | ||
1745 | ``log.do_configure`` file to ensure that the appropriate options have | ||
1746 | been enabled and no additional build-time dependencies need to be added | ||
1747 | to ``DEPENDS``. For example, if the configure script reports that it | ||
1748 | found something not mentioned in ``DEPENDS``, or that it did not find | ||
1749 | something that it needed for some desired optional functionality, then | ||
1750 | you would need to add those to ``DEPENDS``. Looking at the log might | ||
1751 | also reveal items being checked for, enabled, or both that you do not | ||
1752 | want, or items not being found that are in ``DEPENDS``, in which case | ||
1753 | you would need to look at passing extra options to the configure script | ||
1754 | as needed. For reference information on configure options specific to | ||
1755 | the software you are building, you can consult the output of the | ||
1756 | ``./configure --help`` command within ``${S}`` or consult the software's | ||
1757 | upstream documentation. | ||
1758 | |||
1759 | .. _new-recipe-using-headers-to-interface-with-devices: | ||
1760 | |||
1761 | Using Headers to Interface with Devices | ||
1762 | --------------------------------------- | ||
1763 | |||
1764 | If your recipe builds an application that needs to communicate with some | ||
1765 | device or needs an API into a custom kernel, you will need to provide | ||
1766 | appropriate header files. Under no circumstances should you ever modify | ||
1767 | the existing | ||
1768 | ``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc`` file. | ||
1769 | These headers are used to build ``libc`` and must not be compromised | ||
1770 | with custom or machine-specific header information. If you customize | ||
1771 | ``libc`` through modified headers all other applications that use | ||
1772 | ``libc`` thus become affected. | ||
1773 | |||
1774 | .. note:: | ||
1775 | |||
1776 | Never copy and customize the | ||
1777 | libc | ||
1778 | header file (i.e. | ||
1779 | meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc | ||
1780 | ). | ||
1781 | |||
1782 | The correct way to interface to a device or custom kernel is to use a | ||
1783 | separate package that provides the additional headers for the driver or | ||
1784 | other unique interfaces. When doing so, your application also becomes | ||
1785 | responsible for creating a dependency on that specific provider. | ||
1786 | |||
1787 | Consider the following: | ||
1788 | |||
1789 | - Never modify ``linux-libc-headers.inc``. Consider that file to be | ||
1790 | part of the ``libc`` system, and not something you use to access the | ||
1791 | kernel directly. You should access ``libc`` through specific ``libc`` | ||
1792 | calls. | ||
1793 | |||
1794 | - Applications that must talk directly to devices should either provide | ||
1795 | necessary headers themselves, or establish a dependency on a special | ||
1796 | headers package that is specific to that driver. | ||
1797 | |||
1798 | For example, suppose you want to modify an existing header that adds I/O | ||
1799 | control or network support. If the modifications are used by a small | ||
1800 | number programs, providing a unique version of a header is easy and has | ||
1801 | little impact. When doing so, bear in mind the guidelines in the | ||
1802 | previous list. | ||
1803 | |||
1804 | .. note:: | ||
1805 | |||
1806 | If for some reason your changes need to modify the behavior of the | ||
1807 | libc | ||
1808 | , and subsequently all other applications on the system, use a | ||
1809 | .bbappend | ||
1810 | to modify the | ||
1811 | linux-kernel-headers.inc | ||
1812 | file. However, take care to not make the changes machine specific. | ||
1813 | |||
1814 | Consider a case where your kernel is older and you need an older | ||
1815 | ``libc`` ABI. The headers installed by your recipe should still be a | ||
1816 | standard mainline kernel, not your own custom one. | ||
1817 | |||
1818 | When you use custom kernel headers you need to get them from | ||
1819 | :term:`STAGING_KERNEL_DIR`, | ||
1820 | which is the directory with kernel headers that are required to build | ||
1821 | out-of-tree modules. Your recipe will also need the following: | ||
1822 | :: | ||
1823 | |||
1824 | do_configure[depends] += "virtual/kernel:do_shared_workdir" | ||
1825 | |||
1826 | .. _new-recipe-compilation: | ||
1827 | |||
1828 | Compilation | ||
1829 | ----------- | ||
1830 | |||
1831 | During a build, the ``do_compile`` task happens after source is fetched, | ||
1832 | unpacked, and configured. If the recipe passes through ``do_compile`` | ||
1833 | successfully, nothing needs to be done. | ||
1834 | |||
1835 | However, if the compile step fails, you need to diagnose the failure. | ||
1836 | Here are some common issues that cause failures. | ||
1837 | |||
1838 | .. note:: | ||
1839 | |||
1840 | For cases where improper paths are detected for configuration files | ||
1841 | or for when libraries/headers cannot be found, be sure you are using | ||
1842 | the more robust | ||
1843 | pkg-config | ||
1844 | . See the note in section " | ||
1845 | Configuring the Recipe | ||
1846 | " for additional information. | ||
1847 | |||
1848 | - *Parallel build failures:* These failures manifest themselves as | ||
1849 | intermittent errors, or errors reporting that a file or directory | ||
1850 | that should be created by some other part of the build process could | ||
1851 | not be found. This type of failure can occur even if, upon | ||
1852 | inspection, the file or directory does exist after the build has | ||
1853 | failed, because that part of the build process happened in the wrong | ||
1854 | order. | ||
1855 | |||
1856 | To fix the problem, you need to either satisfy the missing dependency | ||
1857 | in the Makefile or whatever script produced the Makefile, or (as a | ||
1858 | workaround) set :term:`PARALLEL_MAKE` to an empty string: | ||
1859 | :: | ||
1860 | |||
1861 | PARALLEL_MAKE = "" | ||
1862 | |||
1863 | For information on parallel Makefile issues, see the "`Debugging | ||
1864 | Parallel Make Races <#debugging-parallel-make-races>`__" section. | ||
1865 | |||
1866 | - *Improper host path usage:* This failure applies to recipes building | ||
1867 | for the target or ``nativesdk`` only. The failure occurs when the | ||
1868 | compilation process uses improper headers, libraries, or other files | ||
1869 | from the host system when cross-compiling for the target. | ||
1870 | |||
1871 | To fix the problem, examine the ``log.do_compile`` file to identify | ||
1872 | the host paths being used (e.g. ``/usr/include``, ``/usr/lib``, and | ||
1873 | so forth) and then either add configure options, apply a patch, or do | ||
1874 | both. | ||
1875 | |||
1876 | - *Failure to find required libraries/headers:* If a build-time | ||
1877 | dependency is missing because it has not been declared in | ||
1878 | :term:`DEPENDS`, or because the | ||
1879 | dependency exists but the path used by the build process to find the | ||
1880 | file is incorrect and the configure step did not detect it, the | ||
1881 | compilation process could fail. For either of these failures, the | ||
1882 | compilation process notes that files could not be found. In these | ||
1883 | cases, you need to go back and add additional options to the | ||
1884 | configure script as well as possibly add additional build-time | ||
1885 | dependencies to ``DEPENDS``. | ||
1886 | |||
1887 | Occasionally, it is necessary to apply a patch to the source to | ||
1888 | ensure the correct paths are used. If you need to specify paths to | ||
1889 | find files staged into the sysroot from other recipes, use the | ||
1890 | variables that the OpenEmbedded build system provides (e.g. | ||
1891 | ``STAGING_BINDIR``, ``STAGING_INCDIR``, ``STAGING_DATADIR``, and so | ||
1892 | forth). | ||
1893 | |||
1894 | .. _new-recipe-installing: | ||
1895 | |||
1896 | Installing | ||
1897 | ---------- | ||
1898 | |||
1899 | During ``do_install``, the task copies the built files along with their | ||
1900 | hierarchy to locations that would mirror their locations on the target | ||
1901 | device. The installation process copies files from the | ||
1902 | ``${``\ :term:`S`\ ``}``, | ||
1903 | ``${``\ :term:`B`\ ``}``, and | ||
1904 | ``${``\ :term:`WORKDIR`\ ``}`` | ||
1905 | directories to the ``${``\ :term:`D`\ ``}`` | ||
1906 | directory to create the structure as it should appear on the target | ||
1907 | system. | ||
1908 | |||
1909 | How your software is built affects what you must do to be sure your | ||
1910 | software is installed correctly. The following list describes what you | ||
1911 | must do for installation depending on the type of build system used by | ||
1912 | the software being built: | ||
1913 | |||
1914 | - *Autotools and CMake:* If the software your recipe is building uses | ||
1915 | Autotools or CMake, the OpenEmbedded build system understands how to | ||
1916 | install the software. Consequently, you do not have to have a | ||
1917 | ``do_install`` task as part of your recipe. You just need to make | ||
1918 | sure the install portion of the build completes with no issues. | ||
1919 | However, if you wish to install additional files not already being | ||
1920 | installed by ``make install``, you should do this using a | ||
1921 | ``do_install_append`` function using the install command as described | ||
1922 | in the "Manual" bulleted item later in this list. | ||
1923 | |||
1924 | - Other (using ``make install``): You need to define a ``do_install`` | ||
1925 | function in your recipe. The function should call | ||
1926 | ``oe_runmake install`` and will likely need to pass in the | ||
1927 | destination directory as well. How you pass that path is dependent on | ||
1928 | how the ``Makefile`` being run is written (e.g. ``DESTDIR=${D}``, | ||
1929 | ``PREFIX=${D}``, ``INSTALLROOT=${D}``, and so forth). | ||
1930 | |||
1931 | For an example recipe using ``make install``, see the | ||
1932 | "`Makefile-Based Package <#new-recipe-makefile-based-package>`__" | ||
1933 | section. | ||
1934 | |||
1935 | - *Manual:* You need to define a ``do_install`` function in your | ||
1936 | recipe. The function must first use ``install -d`` to create the | ||
1937 | directories under | ||
1938 | ``${``\ :term:`D`\ ``}``. Once the | ||
1939 | directories exist, your function can use ``install`` to manually | ||
1940 | install the built software into the directories. | ||
1941 | |||
1942 | You can find more information on ``install`` at | ||
1943 | http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html. | ||
1944 | |||
1945 | For the scenarios that do not use Autotools or CMake, you need to track | ||
1946 | the installation and diagnose and fix any issues until everything | ||
1947 | installs correctly. You need to look in the default location of | ||
1948 | ``${D}``, which is ``${WORKDIR}/image``, to be sure your files have been | ||
1949 | installed correctly. | ||
1950 | |||
1951 | .. note:: | ||
1952 | |||
1953 | - During the installation process, you might need to modify some of | ||
1954 | the installed files to suit the target layout. For example, you | ||
1955 | might need to replace hard-coded paths in an initscript with | ||
1956 | values of variables provided by the build system, such as | ||
1957 | replacing ``/usr/bin/`` with ``${bindir}``. If you do perform such | ||
1958 | modifications during ``do_install``, be sure to modify the | ||
1959 | destination file after copying rather than before copying. | ||
1960 | Modifying after copying ensures that the build system can | ||
1961 | re-execute ``do_install`` if needed. | ||
1962 | |||
1963 | - ``oe_runmake install``, which can be run directly or can be run | ||
1964 | indirectly by the | ||
1965 | :ref:`autotools <ref-classes-autotools>` and | ||
1966 | :ref:`cmake <ref-classes-cmake>` classes, | ||
1967 | runs ``make install`` in parallel. Sometimes, a Makefile can have | ||
1968 | missing dependencies between targets that can result in race | ||
1969 | conditions. If you experience intermittent failures during | ||
1970 | ``do_install``, you might be able to work around them by disabling | ||
1971 | parallel Makefile installs by adding the following to the recipe: | ||
1972 | PARALLEL_MAKEINST = "" See | ||
1973 | :term:`PARALLEL_MAKEINST` | ||
1974 | for additional information. | ||
1975 | |||
1976 | - If you need to install one or more custom CMake toolchain files | ||
1977 | that are supplied by the application you are building, install the | ||
1978 | files to ``${D}${datadir}/cmake/`` Modules during | ||
1979 | :ref:`ref-tasks-install`. | ||
1980 | |||
1981 | .. _new-recipe-enabling-system-services: | ||
1982 | |||
1983 | Enabling System Services | ||
1984 | ------------------------ | ||
1985 | |||
1986 | If you want to install a service, which is a process that usually starts | ||
1987 | on boot and runs in the background, then you must include some | ||
1988 | additional definitions in your recipe. | ||
1989 | |||
1990 | If you are adding services and the service initialization script or the | ||
1991 | service file itself is not installed, you must provide for that | ||
1992 | installation in your recipe using a ``do_install_append`` function. If | ||
1993 | your recipe already has a ``do_install`` function, update the function | ||
1994 | near its end rather than adding an additional ``do_install_append`` | ||
1995 | function. | ||
1996 | |||
1997 | When you create the installation for your services, you need to | ||
1998 | accomplish what is normally done by ``make install``. In other words, | ||
1999 | make sure your installation arranges the output similar to how it is | ||
2000 | arranged on the target system. | ||
2001 | |||
2002 | The OpenEmbedded build system provides support for starting services two | ||
2003 | different ways: | ||
2004 | |||
2005 | - *SysVinit:* SysVinit is a system and service manager that manages the | ||
2006 | init system used to control the very basic functions of your system. | ||
2007 | The init program is the first program started by the Linux kernel | ||
2008 | when the system boots. Init then controls the startup, running and | ||
2009 | shutdown of all other programs. | ||
2010 | |||
2011 | To enable a service using SysVinit, your recipe needs to inherit the | ||
2012 | :ref:`update-rc.d <ref-classes-update-rc.d>` | ||
2013 | class. The class helps facilitate safely installing the package on | ||
2014 | the target. | ||
2015 | |||
2016 | You will need to set the | ||
2017 | :term:`INITSCRIPT_PACKAGES`, | ||
2018 | :term:`INITSCRIPT_NAME`, | ||
2019 | and | ||
2020 | :term:`INITSCRIPT_PARAMS` | ||
2021 | variables within your recipe. | ||
2022 | |||
2023 | - *systemd:* System Management Daemon (systemd) was designed to replace | ||
2024 | SysVinit and to provide enhanced management of services. For more | ||
2025 | information on systemd, see the systemd homepage at | ||
2026 | http://freedesktop.org/wiki/Software/systemd/. | ||
2027 | |||
2028 | To enable a service using systemd, your recipe needs to inherit the | ||
2029 | :ref:`systemd <ref-classes-systemd>` class. See | ||
2030 | the ``systemd.bbclass`` file located in your :term:`Source Directory` | ||
2031 | section for | ||
2032 | more information. | ||
2033 | |||
2034 | .. _new-recipe-packaging: | ||
2035 | |||
2036 | Packaging | ||
2037 | --------- | ||
2038 | |||
2039 | Successful packaging is a combination of automated processes performed | ||
2040 | by the OpenEmbedded build system and some specific steps you need to | ||
2041 | take. The following list describes the process: | ||
2042 | |||
2043 | - *Splitting Files*: The ``do_package`` task splits the files produced | ||
2044 | by the recipe into logical components. Even software that produces a | ||
2045 | single binary might still have debug symbols, documentation, and | ||
2046 | other logical components that should be split out. The ``do_package`` | ||
2047 | task ensures that files are split up and packaged correctly. | ||
2048 | |||
2049 | - *Running QA Checks*: The | ||
2050 | :ref:`insane <ref-classes-insane>` class adds a | ||
2051 | step to the package generation process so that output quality | ||
2052 | assurance checks are generated by the OpenEmbedded build system. This | ||
2053 | step performs a range of checks to be sure the build's output is free | ||
2054 | of common problems that show up during runtime. For information on | ||
2055 | these checks, see the | ||
2056 | :ref:`insane <ref-classes-insane>` class and | ||
2057 | the ":ref:`ref-manual/ref-qa-checks:qa error and warning messages`" | ||
2058 | chapter in the Yocto Project Reference Manual. | ||
2059 | |||
2060 | - *Hand-Checking Your Packages*: After you build your software, you | ||
2061 | need to be sure your packages are correct. Examine the | ||
2062 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
2063 | directory and make sure files are where you expect them to be. If you | ||
2064 | discover problems, you can set | ||
2065 | :term:`PACKAGES`, | ||
2066 | :term:`FILES`, | ||
2067 | ``do_install(_append)``, and so forth as needed. | ||
2068 | |||
2069 | - *Splitting an Application into Multiple Packages*: If you need to | ||
2070 | split an application into several packages, see the "`Splitting an | ||
2071 | Application into Multiple | ||
2072 | Packages <#splitting-an-application-into-multiple-packages>`__" | ||
2073 | section for an example. | ||
2074 | |||
2075 | - *Installing a Post-Installation Script*: For an example showing how | ||
2076 | to install a post-installation script, see the "`Post-Installation | ||
2077 | Scripts <#new-recipe-post-installation-scripts>`__" section. | ||
2078 | |||
2079 | - *Marking Package Architecture*: Depending on what your recipe is | ||
2080 | building and how it is configured, it might be important to mark the | ||
2081 | packages produced as being specific to a particular machine, or to | ||
2082 | mark them as not being specific to a particular machine or | ||
2083 | architecture at all. | ||
2084 | |||
2085 | By default, packages apply to any machine with the same architecture | ||
2086 | as the target machine. When a recipe produces packages that are | ||
2087 | machine-specific (e.g. the | ||
2088 | :term:`MACHINE` value is passed | ||
2089 | into the configure script or a patch is applied only for a particular | ||
2090 | machine), you should mark them as such by adding the following to the | ||
2091 | recipe: | ||
2092 | :: | ||
2093 | |||
2094 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
2095 | |||
2096 | On the other hand, if the recipe produces packages that do not | ||
2097 | contain anything specific to the target machine or architecture at | ||
2098 | all (e.g. recipes that simply package script files or configuration | ||
2099 | files), you should use the | ||
2100 | :ref:`allarch <ref-classes-allarch>` class to | ||
2101 | do this for you by adding this to your recipe: | ||
2102 | :: | ||
2103 | |||
2104 | inherit allarch | ||
2105 | |||
2106 | Ensuring that the package architecture is correct is not critical | ||
2107 | while you are doing the first few builds of your recipe. However, it | ||
2108 | is important in order to ensure that your recipe rebuilds (or does | ||
2109 | not rebuild) appropriately in response to changes in configuration, | ||
2110 | and to ensure that you get the appropriate packages installed on the | ||
2111 | target machine, particularly if you run separate builds for more than | ||
2112 | one target machine. | ||
2113 | |||
2114 | .. _new-sharing-files-between-recipes: | ||
2115 | |||
2116 | Sharing Files Between Recipes | ||
2117 | ----------------------------- | ||
2118 | |||
2119 | Recipes often need to use files provided by other recipes on the build | ||
2120 | host. For example, an application linking to a common library needs | ||
2121 | access to the library itself and its associated headers. The way this | ||
2122 | access is accomplished is by populating a sysroot with files. Each | ||
2123 | recipe has two sysroots in its work directory, one for target files | ||
2124 | (``recipe-sysroot``) and one for files that are native to the build host | ||
2125 | (``recipe-sysroot-native``). | ||
2126 | |||
2127 | .. note:: | ||
2128 | |||
2129 | You could find the term "staging" used within the Yocto project | ||
2130 | regarding files populating sysroots (e.g. the | ||
2131 | STAGING_DIR | ||
2132 | variable). | ||
2133 | |||
2134 | Recipes should never populate the sysroot directly (i.e. write files | ||
2135 | into sysroot). Instead, files should be installed into standard | ||
2136 | locations during the | ||
2137 | :ref:`ref-tasks-install` task within | ||
2138 | the ``${``\ :term:`D`\ ``}`` directory. The | ||
2139 | reason for this limitation is that almost all files that populate the | ||
2140 | sysroot are cataloged in manifests in order to ensure the files can be | ||
2141 | removed later when a recipe is either modified or removed. Thus, the | ||
2142 | sysroot is able to remain free from stale files. | ||
2143 | |||
2144 | A subset of the files installed by the | ||
2145 | :ref:`ref-tasks-install` task are | ||
2146 | used by the | ||
2147 | :ref:`ref-tasks-populate_sysroot` | ||
2148 | task as defined by the the | ||
2149 | :term:`SYSROOT_DIRS` variable to | ||
2150 | automatically populate the sysroot. It is possible to modify the list of | ||
2151 | directories that populate the sysroot. The following example shows how | ||
2152 | you could add the ``/opt`` directory to the list of directories within a | ||
2153 | recipe: | ||
2154 | :: | ||
2155 | |||
2156 | SYSROOT_DIRS += "/opt" | ||
2157 | |||
2158 | For a more complete description of the | ||
2159 | :ref:`ref-tasks-populate_sysroot` | ||
2160 | task and its associated functions, see the | ||
2161 | :ref:`staging <ref-classes-staging>` class. | ||
2162 | |||
2163 | .. _metadata-virtual-providers: | ||
2164 | |||
2165 | Using Virtual Providers | ||
2166 | ----------------------- | ||
2167 | |||
2168 | Prior to a build, if you know that several different recipes provide the | ||
2169 | same functionality, you can use a virtual provider (i.e. ``virtual/*``) | ||
2170 | as a placeholder for the actual provider. The actual provider is | ||
2171 | determined at build-time. | ||
2172 | |||
2173 | A common scenario where a virtual provider is used would be for the | ||
2174 | kernel recipe. Suppose you have three kernel recipes whose | ||
2175 | :term:`PN` values map to ``kernel-big``, | ||
2176 | ``kernel-mid``, and ``kernel-small``. Furthermore, each of these recipes | ||
2177 | in some way uses a :term:`PROVIDES` | ||
2178 | statement that essentially identifies itself as being able to provide | ||
2179 | ``virtual/kernel``. Here is one way through the | ||
2180 | :ref:`kernel <ref-classes-kernel>` class: | ||
2181 | :: | ||
2182 | |||
2183 | PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }" | ||
2184 | |||
2185 | Any recipe that inherits the ``kernel`` class is | ||
2186 | going to utilize a ``PROVIDES`` statement that identifies that recipe as | ||
2187 | being able to provide the ``virtual/kernel`` item. | ||
2188 | |||
2189 | Now comes the time to actually build an image and you need a kernel | ||
2190 | recipe, but which one? You can configure your build to call out the | ||
2191 | kernel recipe you want by using the | ||
2192 | :term:`PREFERRED_PROVIDER` | ||
2193 | variable. As an example, consider the | ||
2194 | `x86-base.inc <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc>`_ | ||
2195 | include file, which is a machine (i.e. | ||
2196 | :term:`MACHINE`) configuration file. | ||
2197 | This include file is the reason all x86-based machines use the | ||
2198 | ``linux-yocto`` kernel. Here are the relevant lines from the include | ||
2199 | file: | ||
2200 | :: | ||
2201 | |||
2202 | PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto" | ||
2203 | PREFERRED_VERSION_linux-yocto ??= "4.15%" | ||
2204 | |||
2205 | When you use a virtual provider, you do not have to "hard code" a recipe | ||
2206 | name as a build dependency. You can use the | ||
2207 | :term:`DEPENDS` variable to state the | ||
2208 | build is dependent on ``virtual/kernel`` for example: DEPENDS = | ||
2209 | "virtual/kernel" During the build, the OpenEmbedded build system picks | ||
2210 | the correct recipe needed for the ``virtual/kernel`` dependency based on | ||
2211 | the ``PREFERRED_PROVIDER`` variable. If you want to use the small kernel | ||
2212 | mentioned at the beginning of this section, configure your build as | ||
2213 | follows: PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small" | ||
2214 | |||
2215 | .. note:: | ||
2216 | |||
2217 | Any recipe that | ||
2218 | PROVIDES | ||
2219 | a | ||
2220 | virtual/\* | ||
2221 | item that is ultimately not selected through | ||
2222 | PREFERRED_PROVIDER | ||
2223 | does not get built. Preventing these recipes from building is usually | ||
2224 | the desired behavior since this mechanism's purpose is to select | ||
2225 | between mutually exclusive alternative providers. | ||
2226 | |||
2227 | The following lists specific examples of virtual providers: | ||
2228 | |||
2229 | - ``virtual/kernel``: Provides the name of the kernel recipe to use | ||
2230 | when building a kernel image. | ||
2231 | |||
2232 | - ``virtual/bootloader``: Provides the name of the bootloader to use | ||
2233 | when building an image. | ||
2234 | |||
2235 | - ``virtual/libgbm``: Provides ``gbm.pc``. | ||
2236 | |||
2237 | - ``virtual/egl``: Provides ``egl.pc`` and possibly ``wayland-egl.pc``. | ||
2238 | |||
2239 | - ``virtual/libgl``: Provides ``gl.pc`` (i.e. libGL). | ||
2240 | |||
2241 | - ``virtual/libgles1``: Provides ``glesv1_cm.pc`` (i.e. libGLESv1_CM). | ||
2242 | |||
2243 | - ``virtual/libgles2``: Provides ``glesv2.pc`` (i.e. libGLESv2). | ||
2244 | |||
2245 | .. note:: | ||
2246 | |||
2247 | Virtual providers only apply to build time dependencies specified with | ||
2248 | :term:`PROVIDES` and :term:`DEPENDS`. They do not apply to runtime | ||
2249 | dependencies specified with :term:`RPROVIDES` and :term:`RDEPENDS`. | ||
2250 | |||
2251 | Properly Versioning Pre-Release Recipes | ||
2252 | --------------------------------------- | ||
2253 | |||
2254 | Sometimes the name of a recipe can lead to versioning problems when the | ||
2255 | recipe is upgraded to a final release. For example, consider the | ||
2256 | ``irssi_0.8.16-rc1.bb`` recipe file in the list of example recipes in | ||
2257 | the "`Storing and Naming the | ||
2258 | Recipe <#new-recipe-storing-and-naming-the-recipe>`__" section. This | ||
2259 | recipe is at a release candidate stage (i.e. "rc1"). When the recipe is | ||
2260 | released, the recipe filename becomes ``irssi_0.8.16.bb``. The version | ||
2261 | change from ``0.8.16-rc1`` to ``0.8.16`` is seen as a decrease by the | ||
2262 | build system and package managers, so the resulting packages will not | ||
2263 | correctly trigger an upgrade. | ||
2264 | |||
2265 | In order to ensure the versions compare properly, the recommended | ||
2266 | convention is to set :term:`PV` within the | ||
2267 | recipe to "previous_version+current_version". You can use an additional | ||
2268 | variable so that you can use the current version elsewhere. Here is an | ||
2269 | example: | ||
2270 | :: | ||
2271 | |||
2272 | REALPV = "0.8.16-rc1" | ||
2273 | PV = "0.8.15+${REALPV}" | ||
2274 | |||
2275 | .. _new-recipe-post-installation-scripts: | ||
2276 | |||
2277 | Post-Installation Scripts | ||
2278 | ------------------------- | ||
2279 | |||
2280 | Post-installation scripts run immediately after installing a package on | ||
2281 | the target or during image creation when a package is included in an | ||
2282 | image. To add a post-installation script to a package, add a | ||
2283 | ``pkg_postinst_``\ PACKAGENAME\ ``()`` function to the recipe file | ||
2284 | (``.bb``) and replace PACKAGENAME with the name of the package you want | ||
2285 | to attach to the ``postinst`` script. To apply the post-installation | ||
2286 | script to the main package for the recipe, which is usually what is | ||
2287 | required, specify | ||
2288 | ``${``\ :term:`PN`\ ``}`` in place of | ||
2289 | PACKAGENAME. | ||
2290 | |||
2291 | A post-installation function has the following structure: | ||
2292 | pkg_postinst_PACKAGENAME() { # Commands to carry out } | ||
2293 | |||
2294 | The script defined in the post-installation function is called when the | ||
2295 | root filesystem is created. If the script succeeds, the package is | ||
2296 | marked as installed. | ||
2297 | |||
2298 | .. note:: | ||
2299 | |||
2300 | Any RPM post-installation script that runs on the target should | ||
2301 | return a 0 exit code. RPM does not allow non-zero exit codes for | ||
2302 | these scripts, and the RPM package manager will cause the package to | ||
2303 | fail installation on the target. | ||
2304 | |||
2305 | Sometimes it is necessary for the execution of a post-installation | ||
2306 | script to be delayed until the first boot. For example, the script might | ||
2307 | need to be executed on the device itself. To delay script execution | ||
2308 | until boot time, you must explicitly mark post installs to defer to the | ||
2309 | target. You can use ``pkg_postinst_ontarget()`` or call | ||
2310 | ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``. Any | ||
2311 | failure of a ``pkg_postinst()`` script (including exit 1) triggers an | ||
2312 | error during the | ||
2313 | :ref:`ref-tasks-rootfs` task. | ||
2314 | |||
2315 | If you have recipes that use ``pkg_postinst`` function and they require | ||
2316 | the use of non-standard native tools that have dependencies during | ||
2317 | rootfs construction, you need to use the | ||
2318 | :term:`PACKAGE_WRITE_DEPS` | ||
2319 | variable in your recipe to list these tools. If you do not use this | ||
2320 | variable, the tools might be missing and execution of the | ||
2321 | post-installation script is deferred until first boot. Deferring the | ||
2322 | script to first boot is undesirable and for read-only rootfs impossible. | ||
2323 | |||
2324 | .. note:: | ||
2325 | |||
2326 | Equivalent support for pre-install, pre-uninstall, and post-uninstall | ||
2327 | scripts exist by way of | ||
2328 | pkg_preinst | ||
2329 | , | ||
2330 | pkg_prerm | ||
2331 | , and | ||
2332 | pkg_postrm | ||
2333 | , respectively. These scrips work in exactly the same way as does | ||
2334 | pkg_postinst | ||
2335 | with the exception that they run at different times. Also, because of | ||
2336 | when they run, they are not applicable to being run at image creation | ||
2337 | time like | ||
2338 | pkg_postinst | ||
2339 | . | ||
2340 | |||
2341 | .. _new-recipe-testing: | ||
2342 | |||
2343 | Testing | ||
2344 | ------- | ||
2345 | |||
2346 | The final step for completing your recipe is to be sure that the | ||
2347 | software you built runs correctly. To accomplish runtime testing, add | ||
2348 | the build's output packages to your image and test them on the target. | ||
2349 | |||
2350 | For information on how to customize your image by adding specific | ||
2351 | packages, see the "`Customizing | ||
2352 | Images <#usingpoky-extend-customimage>`__" section. | ||
2353 | |||
2354 | .. _new-recipe-testing-examples: | ||
2355 | |||
2356 | Examples | ||
2357 | -------- | ||
2358 | |||
2359 | To help summarize how to write a recipe, this section provides some | ||
2360 | examples given various scenarios: | ||
2361 | |||
2362 | - Recipes that use local files | ||
2363 | |||
2364 | - Using an Autotooled package | ||
2365 | |||
2366 | - Using a Makefile-based package | ||
2367 | |||
2368 | - Splitting an application into multiple packages | ||
2369 | |||
2370 | - Adding binaries to an image | ||
2371 | |||
2372 | .. _new-recipe-single-c-file-package-hello-world: | ||
2373 | |||
2374 | Single .c File Package (Hello World!) | ||
2375 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2376 | |||
2377 | Building an application from a single file that is stored locally (e.g. | ||
2378 | under ``files``) requires a recipe that has the file listed in the | ||
2379 | ``SRC_URI`` variable. Additionally, you need to manually write the | ||
2380 | ``do_compile`` and ``do_install`` tasks. The ``S`` variable defines the | ||
2381 | directory containing the source code, which is set to | ||
2382 | :term:`WORKDIR` in this case - the | ||
2383 | directory BitBake uses for the build. | ||
2384 | :: | ||
2385 | |||
2386 | SUMMARY = "Simple helloworld application" | ||
2387 | SECTION = "examples" | ||
2388 | LICENSE = "MIT" | ||
2389 | LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" | ||
2390 | |||
2391 | SRC_URI = "file://helloworld.c" | ||
2392 | |||
2393 | S = "${WORKDIR}" | ||
2394 | |||
2395 | do_compile() { | ||
2396 | ${CC} helloworld.c -o helloworld | ||
2397 | } | ||
2398 | |||
2399 | do_install() { | ||
2400 | install -d ${D}${bindir} | ||
2401 | install -m 0755 helloworld ${D}${bindir} | ||
2402 | } | ||
2403 | |||
2404 | By default, the ``helloworld``, ``helloworld-dbg``, and | ||
2405 | ``helloworld-dev`` packages are built. For information on how to | ||
2406 | customize the packaging process, see the "`Splitting an Application into | ||
2407 | Multiple Packages <#splitting-an-application-into-multiple-packages>`__" | ||
2408 | section. | ||
2409 | |||
2410 | .. _new-recipe-autotooled-package: | ||
2411 | |||
2412 | Autotooled Package | ||
2413 | ~~~~~~~~~~~~~~~~~~ | ||
2414 | |||
2415 | Applications that use Autotools such as ``autoconf`` and ``automake`` | ||
2416 | require a recipe that has a source archive listed in ``SRC_URI`` and | ||
2417 | also inherit the | ||
2418 | :ref:`autotools <ref-classes-autotools>` class, | ||
2419 | which contains the definitions of all the steps needed to build an | ||
2420 | Autotool-based application. The result of the build is automatically | ||
2421 | packaged. And, if the application uses NLS for localization, packages | ||
2422 | with local information are generated (one package per language). | ||
2423 | Following is one example: (``hello_2.3.bb``) | ||
2424 | :: | ||
2425 | |||
2426 | SUMMARY = "GNU Helloworld application" | ||
2427 | SECTION = "examples" | ||
2428 | LICENSE = "GPLv2+" | ||
2429 | LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" | ||
2430 | |||
2431 | SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz" | ||
2432 | |||
2433 | inherit autotools gettext | ||
2434 | |||
2435 | The variable ``LIC_FILES_CHKSUM`` is used to track source license | ||
2436 | changes as described in the "`Tracking License | ||
2437 | Changes <#usingpoky-configuring-LIC_FILES_CHKSUM>`__" section in the | ||
2438 | Yocto Project Overview and Concepts Manual. You can quickly create | ||
2439 | Autotool-based recipes in a manner similar to the previous example. | ||
2440 | |||
2441 | .. _new-recipe-makefile-based-package: | ||
2442 | |||
2443 | Makefile-Based Package | ||
2444 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
2445 | |||
2446 | Applications that use GNU ``make`` also require a recipe that has the | ||
2447 | source archive listed in ``SRC_URI``. You do not need to add a | ||
2448 | ``do_compile`` step since by default BitBake starts the ``make`` command | ||
2449 | to compile the application. If you need additional ``make`` options, you | ||
2450 | should store them in the | ||
2451 | :term:`EXTRA_OEMAKE` or | ||
2452 | :term:`PACKAGECONFIG_CONFARGS` | ||
2453 | variables. BitBake passes these options into the GNU ``make`` | ||
2454 | invocation. Note that a ``do_install`` task is still required. | ||
2455 | Otherwise, BitBake runs an empty ``do_install`` task by default. | ||
2456 | |||
2457 | Some applications might require extra parameters to be passed to the | ||
2458 | compiler. For example, the application might need an additional header | ||
2459 | path. You can accomplish this by adding to the ``CFLAGS`` variable. The | ||
2460 | following example shows this: | ||
2461 | :: | ||
2462 | |||
2463 | CFLAGS_prepend = "-I ${S}/include " | ||
2464 | |||
2465 | In the following example, ``mtd-utils`` is a makefile-based package: | ||
2466 | :: | ||
2467 | |||
2468 | SUMMARY = "Tools for managing memory technology devices" | ||
2469 | SECTION = "base" | ||
2470 | DEPENDS = "zlib lzo e2fsprogs util-linux" | ||
2471 | HOMEPAGE = "http://www.linux-mtd.infradead.org/" | ||
2472 | LICENSE = "GPLv2+" | ||
2473 | LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \ | ||
2474 | file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c" | ||
2475 | # Use the latest version at 26 Oct, 2013 | ||
2476 | SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b" | ||
2477 | SRC_URI = "git://git.infradead.org/mtd-utils.git \ | ||
2478 | file://add-exclusion-to-mkfs-jffs2-git-2.patch \ | ||
2479 | " | ||
2480 | PV = "1.5.1+git${SRCPV}" | ||
2481 | S = "${WORKDIR}/git" | ||
2482 | EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'" | ||
2483 | do_install () { | ||
2484 | oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir} | ||
2485 | } | ||
2486 | PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc" | ||
2487 | FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool" | ||
2488 | FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*" | ||
2489 | FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image" | ||
2490 | PARALLEL_MAKE = "" | ||
2491 | BBCLASSEXTEND = "native" | ||
2492 | |||
2493 | Splitting an Application into Multiple Packages | ||
2494 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2495 | |||
2496 | You can use the variables ``PACKAGES`` and ``FILES`` to split an | ||
2497 | application into multiple packages. | ||
2498 | |||
2499 | Following is an example that uses the ``libxpm`` recipe. By default, | ||
2500 | this recipe generates a single package that contains the library along | ||
2501 | with a few binaries. You can modify the recipe to split the binaries | ||
2502 | into separate packages: | ||
2503 | :: | ||
2504 | |||
2505 | require xorg-lib-common.inc | ||
2506 | SUMMARY = "Xpm: X Pixmap extension library" | ||
2507 | LICENSE = "BSD" | ||
2508 | LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" | ||
2509 | DEPENDS += "libxext libsm libxt" | ||
2510 | PE = "1" | ||
2511 | XORG_PN = "libXpm" | ||
2512 | PACKAGES =+ "sxpm cxpm" | ||
2513 | FILES_cxpm = "${bindir}/cxpm" | ||
2514 | FILES_sxpm = "${bindir}/sxpm" | ||
2515 | |||
2516 | In the previous example, we want to ship the ``sxpm`` and ``cxpm`` | ||
2517 | binaries in separate packages. Since ``bindir`` would be packaged into | ||
2518 | the main ``PN`` package by default, we prepend the ``PACKAGES`` variable | ||
2519 | so additional package names are added to the start of list. This results | ||
2520 | in the extra ``FILES_*`` variables then containing information that | ||
2521 | define which files and directories go into which packages. Files | ||
2522 | included by earlier packages are skipped by latter packages. Thus, the | ||
2523 | main ``PN`` package does not include the above listed files. | ||
2524 | |||
2525 | Packaging Externally Produced Binaries | ||
2526 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
2527 | |||
2528 | Sometimes, you need to add pre-compiled binaries to an image. For | ||
2529 | example, suppose that binaries for proprietary code exist, which are | ||
2530 | created by a particular division of a company. Your part of the company | ||
2531 | needs to use those binaries as part of an image that you are building | ||
2532 | using the OpenEmbedded build system. Since you only have the binaries | ||
2533 | and not the source code, you cannot use a typical recipe that expects to | ||
2534 | fetch the source specified in | ||
2535 | :term:`SRC_URI` and then compile it. | ||
2536 | |||
2537 | One method is to package the binaries and then install them as part of | ||
2538 | the image. Generally, it is not a good idea to package binaries since, | ||
2539 | among other things, it can hinder the ability to reproduce builds and | ||
2540 | could lead to compatibility problems with ABI in the future. However, | ||
2541 | sometimes you have no choice. | ||
2542 | |||
2543 | The easiest solution is to create a recipe that uses the | ||
2544 | :ref:`bin_package <ref-classes-bin-package>` class | ||
2545 | and to be sure that you are using default locations for build artifacts. | ||
2546 | In most cases, the ``bin_package`` class handles "skipping" the | ||
2547 | configure and compile steps as well as sets things up to grab packages | ||
2548 | from the appropriate area. In particular, this class sets ``noexec`` on | ||
2549 | both the :ref:`ref-tasks-configure` | ||
2550 | and :ref:`ref-tasks-compile` tasks, | ||
2551 | sets ``FILES_${PN}`` to "/" so that it picks up all files, and sets up a | ||
2552 | :ref:`ref-tasks-install` task, which | ||
2553 | effectively copies all files from ``${S}`` to ``${D}``. The | ||
2554 | ``bin_package`` class works well when the files extracted into ``${S}`` | ||
2555 | are already laid out in the way they should be laid out on the target. | ||
2556 | For more information on these variables, see the | ||
2557 | :term:`FILES`, | ||
2558 | :term:`PN`, | ||
2559 | :term:`S`, and | ||
2560 | :term:`D` variables in the Yocto Project | ||
2561 | Reference Manual's variable glossary. | ||
2562 | |||
2563 | .. note:: | ||
2564 | |||
2565 | - Using :term:`DEPENDS` is a good | ||
2566 | idea even for components distributed in binary form, and is often | ||
2567 | necessary for shared libraries. For a shared library, listing the | ||
2568 | library dependencies in ``DEPENDS`` makes sure that the libraries | ||
2569 | are available in the staging sysroot when other recipes link | ||
2570 | against the library, which might be necessary for successful | ||
2571 | linking. | ||
2572 | |||
2573 | - Using ``DEPENDS`` also allows runtime dependencies between | ||
2574 | packages to be added automatically. See the | ||
2575 | ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`" | ||
2576 | section in the Yocto Project Overview and Concepts Manual for more | ||
2577 | information. | ||
2578 | |||
2579 | If you cannot use the ``bin_package`` class, you need to be sure you are | ||
2580 | doing the following: | ||
2581 | |||
2582 | - Create a recipe where the | ||
2583 | :ref:`ref-tasks-configure` and | ||
2584 | :ref:`ref-tasks-compile` tasks do | ||
2585 | nothing: It is usually sufficient to just not define these tasks in | ||
2586 | the recipe, because the default implementations do nothing unless a | ||
2587 | Makefile is found in | ||
2588 | ``${``\ :term:`S`\ ``}``. | ||
2589 | |||
2590 | If ``${S}`` might contain a Makefile, or if you inherit some class | ||
2591 | that replaces ``do_configure`` and ``do_compile`` with custom | ||
2592 | versions, then you can use the | ||
2593 | ``[``\ :ref:`noexec <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]`` | ||
2594 | flag to turn the tasks into no-ops, as follows: | ||
2595 | :: | ||
2596 | |||
2597 | do_configure[noexec] = "1" | ||
2598 | do_compile[noexec] = "1" | ||
2599 | |||
2600 | Unlike | ||
2601 | :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:deleting a task`, | ||
2602 | using the flag preserves the dependency chain from the | ||
2603 | :ref:`ref-tasks-fetch`, | ||
2604 | :ref:`ref-tasks-unpack`, and | ||
2605 | :ref:`ref-tasks-patch` tasks to the | ||
2606 | :ref:`ref-tasks-install` task. | ||
2607 | |||
2608 | - Make sure your ``do_install`` task installs the binaries | ||
2609 | appropriately. | ||
2610 | |||
2611 | - Ensure that you set up :term:`FILES` | ||
2612 | (usually | ||
2613 | ``FILES_${``\ :term:`PN`\ ``}``) to | ||
2614 | point to the files you have installed, which of course depends on | ||
2615 | where you have installed them and whether those files are in | ||
2616 | different locations than the defaults. | ||
2617 | |||
2618 | Following Recipe Style Guidelines | ||
2619 | --------------------------------- | ||
2620 | |||
2621 | When writing recipes, it is good to conform to existing style | ||
2622 | guidelines. The `OpenEmbedded | ||
2623 | Styleguide <http://www.openembedded.org/wiki/Styleguide>`__ wiki page | ||
2624 | provides rough guidelines for preferred recipe style. | ||
2625 | |||
2626 | It is common for existing recipes to deviate a bit from this style. | ||
2627 | However, aiming for at least a consistent style is a good idea. Some | ||
2628 | practices, such as omitting spaces around ``=`` operators in assignments | ||
2629 | or ordering recipe components in an erratic way, are widely seen as poor | ||
2630 | style. | ||
2631 | |||
2632 | Recipe Syntax | ||
2633 | ------------- | ||
2634 | |||
2635 | Understanding recipe file syntax is important for writing recipes. The | ||
2636 | following list overviews the basic items that make up a BitBake recipe | ||
2637 | file. For more complete BitBake syntax descriptions, see the | ||
2638 | ":doc:`bitbake-user-manual/bitbake-user-manual-metadata`" | ||
2639 | chapter of the BitBake User Manual. | ||
2640 | |||
2641 | - *Variable Assignments and Manipulations:* Variable assignments allow | ||
2642 | a value to be assigned to a variable. The assignment can be static | ||
2643 | text or might include the contents of other variables. In addition to | ||
2644 | the assignment, appending and prepending operations are also | ||
2645 | supported. | ||
2646 | |||
2647 | The following example shows some of the ways you can use variables in | ||
2648 | recipes: | ||
2649 | :: | ||
2650 | |||
2651 | S = "${WORKDIR}/postfix-${PV}" | ||
2652 | CFLAGS += "-DNO_ASM" | ||
2653 | SRC_URI_append = " file://fixup.patch" | ||
2654 | |||
2655 | - *Functions:* Functions provide a series of actions to be performed. | ||
2656 | You usually use functions to override the default implementation of a | ||
2657 | task function or to complement a default function (i.e. append or | ||
2658 | prepend to an existing function). Standard functions use ``sh`` shell | ||
2659 | syntax, although access to OpenEmbedded variables and internal | ||
2660 | methods are also available. | ||
2661 | |||
2662 | The following is an example function from the ``sed`` recipe: | ||
2663 | :: | ||
2664 | |||
2665 | do_install () { | ||
2666 | autotools_do_install | ||
2667 | install -d ${D}${base_bindir} | ||
2668 | mv ${D}${bindir}/sed ${D}${base_bindir}/sed | ||
2669 | rmdir ${D}${bindir}/ | ||
2670 | } | ||
2671 | |||
2672 | It is | ||
2673 | also possible to implement new functions that are called between | ||
2674 | existing tasks as long as the new functions are not replacing or | ||
2675 | complementing the default functions. You can implement functions in | ||
2676 | Python instead of shell. Both of these options are not seen in the | ||
2677 | majority of recipes. | ||
2678 | |||
2679 | - *Keywords:* BitBake recipes use only a few keywords. You use keywords | ||
2680 | to include common functions (``inherit``), load parts of a recipe | ||
2681 | from other files (``include`` and ``require``) and export variables | ||
2682 | to the environment (``export``). | ||
2683 | |||
2684 | The following example shows the use of some of these keywords: | ||
2685 | :: | ||
2686 | |||
2687 | export POSTCONF = "${STAGING_BINDIR}/postconf" | ||
2688 | inherit autoconf | ||
2689 | require otherfile.inc | ||
2690 | |||
2691 | - *Comments (#):* Any lines that begin with the hash character (``#``) | ||
2692 | are treated as comment lines and are ignored: | ||
2693 | :: | ||
2694 | |||
2695 | # This is a comment | ||
2696 | |||
2697 | This next list summarizes the most important and most commonly used | ||
2698 | parts of the recipe syntax. For more information on these parts of the | ||
2699 | syntax, you can reference the | ||
2700 | :doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata` chapter | ||
2701 | in the BitBake User Manual. | ||
2702 | |||
2703 | - *Line Continuation (\):* Use the backward slash (``\``) character to | ||
2704 | split a statement over multiple lines. Place the slash character at | ||
2705 | the end of the line that is to be continued on the next line: | ||
2706 | :: | ||
2707 | |||
2708 | VAR = "A really long \ | ||
2709 | line" | ||
2710 | |||
2711 | .. note:: | ||
2712 | |||
2713 | You cannot have any characters including spaces or tabs after the | ||
2714 | slash character. | ||
2715 | |||
2716 | - *Using Variables (${VARNAME}):* Use the ``${VARNAME}`` syntax to | ||
2717 | access the contents of a variable: | ||
2718 | :: | ||
2719 | |||
2720 | SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz" | ||
2721 | |||
2722 | .. note:: | ||
2723 | |||
2724 | It is important to understand that the value of a variable | ||
2725 | expressed in this form does not get substituted automatically. The | ||
2726 | expansion of these expressions happens on-demand later (e.g. | ||
2727 | usually when a function that makes reference to the variable | ||
2728 | executes). This behavior ensures that the values are most | ||
2729 | appropriate for the context in which they are finally used. On the | ||
2730 | rare occasion that you do need the variable expression to be | ||
2731 | expanded immediately, you can use the | ||
2732 | := | ||
2733 | operator instead of | ||
2734 | = | ||
2735 | when you make the assignment, but this is not generally needed. | ||
2736 | |||
2737 | - *Quote All Assignments ("value"):* Use double quotes around values in | ||
2738 | all variable assignments (e.g. ``"value"``). Following is an example: | ||
2739 | :: | ||
2740 | |||
2741 | VAR1 = "${OTHERVAR}" | ||
2742 | VAR2 = "The version is ${PV}" | ||
2743 | |||
2744 | - *Conditional Assignment (?=):* Conditional assignment is used to | ||
2745 | assign a value to a variable, but only when the variable is currently | ||
2746 | unset. Use the question mark followed by the equal sign (``?=``) to | ||
2747 | make a "soft" assignment used for conditional assignment. Typically, | ||
2748 | "soft" assignments are used in the ``local.conf`` file for variables | ||
2749 | that are allowed to come through from the external environment. | ||
2750 | |||
2751 | Here is an example where ``VAR1`` is set to "New value" if it is | ||
2752 | currently empty. However, if ``VAR1`` has already been set, it | ||
2753 | remains unchanged: VAR1 ?= "New value" In this next example, ``VAR1`` | ||
2754 | is left with the value "Original value": | ||
2755 | :: | ||
2756 | |||
2757 | VAR1 = "Original value" | ||
2758 | VAR1 ?= "New value" | ||
2759 | |||
2760 | - *Appending (+=):* Use the plus character followed by the equals sign | ||
2761 | (``+=``) to append values to existing variables. | ||
2762 | |||
2763 | .. note:: | ||
2764 | |||
2765 | This operator adds a space between the existing content of the | ||
2766 | variable and the new content. | ||
2767 | |||
2768 | Here is an example: | ||
2769 | :: | ||
2770 | |||
2771 | SRC_URI += "file://fix-makefile.patch" | ||
2772 | |||
2773 | - *Prepending (=+):* Use the equals sign followed by the plus character | ||
2774 | (``=+``) to prepend values to existing variables. | ||
2775 | |||
2776 | .. note:: | ||
2777 | |||
2778 | This operator adds a space between the new content and the | ||
2779 | existing content of the variable. | ||
2780 | |||
2781 | Here is an example: | ||
2782 | :: | ||
2783 | |||
2784 | VAR =+ "Starts" | ||
2785 | |||
2786 | - *Appending (_append):* Use the ``_append`` operator to append values | ||
2787 | to existing variables. This operator does not add any additional | ||
2788 | space. Also, the operator is applied after all the ``+=``, and ``=+`` | ||
2789 | operators have been applied and after all ``=`` assignments have | ||
2790 | occurred. | ||
2791 | |||
2792 | The following example shows the space being explicitly added to the | ||
2793 | start to ensure the appended value is not merged with the existing | ||
2794 | value: | ||
2795 | :: | ||
2796 | |||
2797 | SRC_URI_append = " file://fix-makefile.patch" | ||
2798 | |||
2799 | You can also use | ||
2800 | the ``_append`` operator with overrides, which results in the actions | ||
2801 | only being performed for the specified target or machine: | ||
2802 | :: | ||
2803 | |||
2804 | SRC_URI_append_sh4 = " file://fix-makefile.patch" | ||
2805 | |||
2806 | - *Prepending (_prepend):* Use the ``_prepend`` operator to prepend | ||
2807 | values to existing variables. This operator does not add any | ||
2808 | additional space. Also, the operator is applied after all the ``+=``, | ||
2809 | and ``=+`` operators have been applied and after all ``=`` | ||
2810 | assignments have occurred. | ||
2811 | |||
2812 | The following example shows the space being explicitly added to the | ||
2813 | end to ensure the prepended value is not merged with the existing | ||
2814 | value: | ||
2815 | :: | ||
2816 | |||
2817 | CFLAGS_prepend = "-I${S}/myincludes " | ||
2818 | |||
2819 | You can also use the | ||
2820 | ``_prepend`` operator with overrides, which results in the actions | ||
2821 | only being performed for the specified target or machine: | ||
2822 | :: | ||
2823 | |||
2824 | CFLAGS_prepend_sh4 = "-I${S}/myincludes " | ||
2825 | |||
2826 | - *Overrides:* You can use overrides to set a value conditionally, | ||
2827 | typically based on how the recipe is being built. For example, to set | ||
2828 | the :term:`KBRANCH` variable's | ||
2829 | value to "standard/base" for any target | ||
2830 | :term:`MACHINE`, except for | ||
2831 | qemuarm where it should be set to "standard/arm-versatile-926ejs", | ||
2832 | you would do the following: | ||
2833 | :: | ||
2834 | |||
2835 | KBRANCH = "standard/base" | ||
2836 | KBRANCH_qemuarm = "standard/arm-versatile-926ejs" | ||
2837 | |||
2838 | Overrides are also used to separate | ||
2839 | alternate values of a variable in other situations. For example, when | ||
2840 | setting variables such as | ||
2841 | :term:`FILES` and | ||
2842 | :term:`RDEPENDS` that are | ||
2843 | specific to individual packages produced by a recipe, you should | ||
2844 | always use an override that specifies the name of the package. | ||
2845 | |||
2846 | - *Indentation:* Use spaces for indentation rather than than tabs. For | ||
2847 | shell functions, both currently work. However, it is a policy | ||
2848 | decision of the Yocto Project to use tabs in shell functions. Realize | ||
2849 | that some layers have a policy to use spaces for all indentation. | ||
2850 | |||
2851 | - *Using Python for Complex Operations:* For more advanced processing, | ||
2852 | it is possible to use Python code during variable assignments (e.g. | ||
2853 | search and replacement on a variable). | ||
2854 | |||
2855 | You indicate Python code using the ``${@python_code}`` syntax for the | ||
2856 | variable assignment: | ||
2857 | :: | ||
2858 | |||
2859 | SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz | ||
2860 | |||
2861 | - *Shell Function Syntax:* Write shell functions as if you were writing | ||
2862 | a shell script when you describe a list of actions to take. You | ||
2863 | should ensure that your script works with a generic ``sh`` and that | ||
2864 | it does not require any ``bash`` or other shell-specific | ||
2865 | functionality. The same considerations apply to various system | ||
2866 | utilities (e.g. ``sed``, ``grep``, ``awk``, and so forth) that you | ||
2867 | might wish to use. If in doubt, you should check with multiple | ||
2868 | implementations - including those from BusyBox. | ||
2869 | |||
2870 | .. _platdev-newmachine: | ||
2871 | |||
2872 | Adding a New Machine | ||
2873 | ==================== | ||
2874 | |||
2875 | Adding a new machine to the Yocto Project is a straightforward process. | ||
2876 | This section describes how to add machines that are similar to those | ||
2877 | that the Yocto Project already supports. | ||
2878 | |||
2879 | .. note:: | ||
2880 | |||
2881 | Although well within the capabilities of the Yocto Project, adding a | ||
2882 | totally new architecture might require changes to | ||
2883 | gcc/glibc | ||
2884 | and to the site information, which is beyond the scope of this | ||
2885 | manual. | ||
2886 | |||
2887 | For a complete example that shows how to add a new machine, see the | ||
2888 | ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`" | ||
2889 | section in the Yocto Project Board Support Package (BSP) Developer's | ||
2890 | Guide. | ||
2891 | |||
2892 | .. _platdev-newmachine-conffile: | ||
2893 | |||
2894 | Adding the Machine Configuration File | ||
2895 | ------------------------------------- | ||
2896 | |||
2897 | To add a new machine, you need to add a new machine configuration file | ||
2898 | to the layer's ``conf/machine`` directory. This configuration file | ||
2899 | provides details about the device you are adding. | ||
2900 | |||
2901 | The OpenEmbedded build system uses the root name of the machine | ||
2902 | configuration file to reference the new machine. For example, given a | ||
2903 | machine configuration file named ``crownbay.conf``, the build system | ||
2904 | recognizes the machine as "crownbay". | ||
2905 | |||
2906 | The most important variables you must set in your machine configuration | ||
2907 | file or include from a lower-level configuration file are as follows: | ||
2908 | |||
2909 | - ``TARGET_ARCH`` (e.g. "arm") | ||
2910 | |||
2911 | - ``PREFERRED_PROVIDER_virtual/kernel`` | ||
2912 | |||
2913 | - ``MACHINE_FEATURES`` (e.g. "apm screen wifi") | ||
2914 | |||
2915 | You might also need these variables: | ||
2916 | |||
2917 | - ``SERIAL_CONSOLES`` (e.g. "115200;ttyS0 115200;ttyS1") | ||
2918 | |||
2919 | - ``KERNEL_IMAGETYPE`` (e.g. "zImage") | ||
2920 | |||
2921 | - ``IMAGE_FSTYPES`` (e.g. "tar.gz jffs2") | ||
2922 | |||
2923 | You can find full details on these variables in the reference section. | ||
2924 | You can leverage existing machine ``.conf`` files from | ||
2925 | ``meta-yocto-bsp/conf/machine/``. | ||
2926 | |||
2927 | .. _platdev-newmachine-kernel: | ||
2928 | |||
2929 | Adding a Kernel for the Machine | ||
2930 | ------------------------------- | ||
2931 | |||
2932 | The OpenEmbedded build system needs to be able to build a kernel for the | ||
2933 | machine. You need to either create a new kernel recipe for this machine, | ||
2934 | or extend an existing kernel recipe. You can find several kernel recipe | ||
2935 | examples in the Source Directory at ``meta/recipes-kernel/linux`` that | ||
2936 | you can use as references. | ||
2937 | |||
2938 | If you are creating a new kernel recipe, normal recipe-writing rules | ||
2939 | apply for setting up a ``SRC_URI``. Thus, you need to specify any | ||
2940 | necessary patches and set ``S`` to point at the source code. You need to | ||
2941 | create a ``do_configure`` task that configures the unpacked kernel with | ||
2942 | a ``defconfig`` file. You can do this by using a ``make defconfig`` | ||
2943 | command or, more commonly, by copying in a suitable ``defconfig`` file | ||
2944 | and then running ``make oldconfig``. By making use of ``inherit kernel`` | ||
2945 | and potentially some of the ``linux-*.inc`` files, most other | ||
2946 | functionality is centralized and the defaults of the class normally work | ||
2947 | well. | ||
2948 | |||
2949 | If you are extending an existing kernel recipe, it is usually a matter | ||
2950 | of adding a suitable ``defconfig`` file. The file needs to be added into | ||
2951 | a location similar to ``defconfig`` files used for other machines in a | ||
2952 | given kernel recipe. A possible way to do this is by listing the file in | ||
2953 | the ``SRC_URI`` and adding the machine to the expression in | ||
2954 | ``COMPATIBLE_MACHINE``: | ||
2955 | :: | ||
2956 | |||
2957 | COMPATIBLE_MACHINE = '(qemux86|qemumips)' | ||
2958 | |||
2959 | For more information on ``defconfig`` files, see the | ||
2960 | ":ref:`kernel-dev/kernel-dev-common:changing the configuration`" | ||
2961 | section in the Yocto Project Linux Kernel Development Manual. | ||
2962 | |||
2963 | .. _platdev-newmachine-formfactor: | ||
2964 | |||
2965 | Adding a Formfactor Configuration File | ||
2966 | -------------------------------------- | ||
2967 | |||
2968 | A formfactor configuration file provides information about the target | ||
2969 | hardware for which the image is being built and information that the | ||
2970 | build system cannot obtain from other sources such as the kernel. Some | ||
2971 | examples of information contained in a formfactor configuration file | ||
2972 | include framebuffer orientation, whether or not the system has a | ||
2973 | keyboard, the positioning of the keyboard in relation to the screen, and | ||
2974 | the screen resolution. | ||
2975 | |||
2976 | The build system uses reasonable defaults in most cases. However, if | ||
2977 | customization is necessary, you need to create a ``machconfig`` file in | ||
2978 | the ``meta/recipes-bsp/formfactor/files`` directory. This directory | ||
2979 | contains directories for specific machines such as ``qemuarm`` and | ||
2980 | ``qemux86``. For information about the settings available and the | ||
2981 | defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file | ||
2982 | found in the same area. | ||
2983 | |||
2984 | Following is an example for "qemuarm" machine: | ||
2985 | :: | ||
2986 | |||
2987 | HAVE_TOUCHSCREEN=1 | ||
2988 | HAVE_KEYBOARD=1 | ||
2989 | DISPLAY_CAN_ROTATE=0 | ||
2990 | DISPLAY_ORIENTATION=0 | ||
2991 | #DISPLAY_WIDTH_PIXELS=640 | ||
2992 | #DISPLAY_HEIGHT_PIXELS=480 | ||
2993 | #DISPLAY_BPP=16 | ||
2994 | DISPLAY_DPI=150 | ||
2995 | DISPLAY_SUBPIXEL_ORDER=vrgb | ||
2996 | |||
2997 | .. _gs-upgrading-recipes: | ||
2998 | |||
2999 | Upgrading Recipes | ||
3000 | ================= | ||
3001 | |||
3002 | Over time, upstream developers publish new versions for software built | ||
3003 | by layer recipes. It is recommended to keep recipes up-to-date with | ||
3004 | upstream version releases. | ||
3005 | |||
3006 | While several methods exist that allow you upgrade a recipe, you might | ||
3007 | consider checking on the upgrade status of a recipe first. You can do so | ||
3008 | using the ``devtool check-upgrade-status`` command. See the | ||
3009 | ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" | ||
3010 | section in the Yocto Project Reference Manual for more information. | ||
3011 | |||
3012 | The remainder of this section describes three ways you can upgrade a | ||
3013 | recipe. You can use the Automated Upgrade Helper (AUH) to set up | ||
3014 | automatic version upgrades. Alternatively, you can use | ||
3015 | ``devtool upgrade`` to set up semi-automatic version upgrades. Finally, | ||
3016 | you can manually upgrade a recipe by editing the recipe itself. | ||
3017 | |||
3018 | .. _gs-using-the-auto-upgrade-helper: | ||
3019 | |||
3020 | Using the Auto Upgrade Helper (AUH) | ||
3021 | ----------------------------------- | ||
3022 | |||
3023 | The AUH utility works in conjunction with the OpenEmbedded build system | ||
3024 | in order to automatically generate upgrades for recipes based on new | ||
3025 | versions being published upstream. Use AUH when you want to create a | ||
3026 | service that performs the upgrades automatically and optionally sends | ||
3027 | you an email with the results. | ||
3028 | |||
3029 | AUH allows you to update several recipes with a single use. You can also | ||
3030 | optionally perform build and integration tests using images with the | ||
3031 | results saved to your hard drive and emails of results optionally sent | ||
3032 | to recipe maintainers. Finally, AUH creates Git commits with appropriate | ||
3033 | commit messages in the layer's tree for the changes made to recipes. | ||
3034 | |||
3035 | .. note:: | ||
3036 | |||
3037 | Conditions do exist when you should not use AUH to upgrade recipes | ||
3038 | and you should instead use either | ||
3039 | devtool upgrade | ||
3040 | or upgrade your recipes manually: | ||
3041 | |||
3042 | - When AUH cannot complete the upgrade sequence. This situation | ||
3043 | usually results because custom patches carried by the recipe | ||
3044 | cannot be automatically rebased to the new version. In this case, | ||
3045 | ``devtool upgrade`` allows you to manually resolve conflicts. | ||
3046 | |||
3047 | - When for any reason you want fuller control over the upgrade | ||
3048 | process. For example, when you want special arrangements for | ||
3049 | testing. | ||
3050 | |||
3051 | The following steps describe how to set up the AUH utility: | ||
3052 | |||
3053 | 1. *Be Sure the Development Host is Set Up:* You need to be sure that | ||
3054 | your development host is set up to use the Yocto Project. For | ||
3055 | information on how to set up your host, see the "`Preparing the Build | ||
3056 | Host <#dev-preparing-the-build-host>`__" section. | ||
3057 | |||
3058 | 2. *Make Sure Git is Configured:* The AUH utility requires Git to be | ||
3059 | configured because AUH uses Git to save upgrades. Thus, you must have | ||
3060 | Git user and email configured. The following command shows your | ||
3061 | configurations: | ||
3062 | |||
3063 | $ git config --list | ||
3064 | |||
3065 | If you do not have the user and | ||
3066 | email configured, you can use the following commands to do so: | ||
3067 | :: | ||
3068 | |||
3069 | $ git config --global user.name some_name | ||
3070 | $ git config --global user.email username@domain.com | ||
3071 | |||
3072 | 3. *Clone the AUH Repository:* To use AUH, you must clone the repository | ||
3073 | onto your development host. The following command uses Git to create | ||
3074 | a local copy of the repository on your system: | ||
3075 | :: | ||
3076 | |||
3077 | $ git clone git://git.yoctoproject.org/auto-upgrade-helper | ||
3078 | Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. | ||
3079 | remote: Compressing objects: 100% (300/300), done. | ||
3080 | remote: Total 768 (delta 499), reused 703 (delta 434) | ||
3081 | Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. | ||
3082 | Resolving deltas: 100% (499/499), done. | ||
3083 | Checking connectivity... done. | ||
3084 | |||
3085 | AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or | ||
3086 | :term:`Poky` repositories. | ||
3087 | |||
3088 | 4. *Create a Dedicated Build Directory:* Run the | ||
3089 | :ref:`structure-core-script` | ||
3090 | script to create a fresh build directory that you use exclusively for | ||
3091 | running the AUH utility: | ||
3092 | :: | ||
3093 | |||
3094 | $ cd ~/poky | ||
3095 | $ source oe-init-build-env | ||
3096 | |||
3097 | your_AUH_build_directory Re-using an existing build directory and its | ||
3098 | configurations is not recommended as existing settings could cause | ||
3099 | AUH to fail or behave undesirably. | ||
3100 | |||
3101 | 5. *Make Configurations in Your Local Configuration File:* Several | ||
3102 | settings need to exist in the ``local.conf`` file in the build | ||
3103 | directory you just created for AUH. Make these following | ||
3104 | configurations: | ||
3105 | |||
3106 | - If you want to enable :ref:`Build | ||
3107 | History <dev-manual/dev-manual-common-tasks:maintaining build output quality>`, | ||
3108 | which is optional, you need the following lines in the | ||
3109 | ``conf/local.conf`` file: | ||
3110 | :: | ||
3111 | |||
3112 | INHERIT =+ "buildhistory" | ||
3113 | BUILDHISTORY_COMMIT = "1" | ||
3114 | |||
3115 | With this configuration and a successful | ||
3116 | upgrade, a build history "diff" file appears in the | ||
3117 | ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in | ||
3118 | your build directory. | ||
3119 | |||
3120 | - If you want to enable testing through the | ||
3121 | :ref:`testimage <ref-classes-testimage*>` | ||
3122 | class, which is optional, you need to have the following set in | ||
3123 | your ``conf/local.conf`` file: INHERIT += "testimage" | ||
3124 | |||
3125 | .. note:: | ||
3126 | |||
3127 | If your distro does not enable by default ptest, which Poky | ||
3128 | does, you need the following in your | ||
3129 | local.conf | ||
3130 | file: | ||
3131 | :: | ||
3132 | |||
3133 | DISTRO_FEATURES_append = " ptest" | ||
3134 | |||
3135 | |||
3136 | 6. *Optionally Start a vncserver:* If you are running in a server | ||
3137 | without an X11 session, you need to start a vncserver: | ||
3138 | :: | ||
3139 | |||
3140 | $ vncserver :1 | ||
3141 | $ export DISPLAY=:1 | ||
3142 | |||
3143 | 7. *Create and Edit an AUH Configuration File:* You need to have the | ||
3144 | ``upgrade-helper/upgrade-helper.conf`` configuration file in your | ||
3145 | build directory. You can find a sample configuration file in the `AUH | ||
3146 | source | ||
3147 | repository <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/>`__. | ||
3148 | |||
3149 | Read through the sample file and make configurations as needed. For | ||
3150 | example, if you enabled build history in your ``local.conf`` as | ||
3151 | described earlier, you must enable it in ``upgrade-helper.conf``. | ||
3152 | |||
3153 | Also, if you are using the default ``maintainers.inc`` file supplied | ||
3154 | with Poky and located in ``meta-yocto`` and you do not set a | ||
3155 | "maintainers_whitelist" or "global_maintainer_override" in the | ||
3156 | ``upgrade-helper.conf`` configuration, and you specify "-e all" on | ||
3157 | the AUH command-line, the utility automatically sends out emails to | ||
3158 | all the default maintainers. Please avoid this. | ||
3159 | |||
3160 | This next set of examples describes how to use the AUH: | ||
3161 | |||
3162 | - *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the | ||
3163 | following form: $ upgrade-helper.py recipe_name For example, this | ||
3164 | command upgrades the ``xmodmap`` recipe: | ||
3165 | :: | ||
3166 | |||
3167 | $ upgrade-helper.py xmodmap | ||
3168 | |||
3169 | - *Upgrading a Specific Recipe to a Particular Version:* To upgrade a | ||
3170 | specific recipe to a particular version, use the following form: $ | ||
3171 | upgrade-helper.py recipe_name -t version For example, this command | ||
3172 | upgrades the ``xmodmap`` recipe to version 1.2.3: | ||
3173 | :: | ||
3174 | |||
3175 | $ upgrade-helper.py xmodmap -t 1.2.3 | ||
3176 | |||
3177 | - *Upgrading all Recipes to the Latest Versions and Suppressing Email | ||
3178 | Notifications:* To upgrade all recipes to their most recent versions | ||
3179 | and suppress the email notifications, use the following command: | ||
3180 | :: | ||
3181 | |||
3182 | $ upgrade-helper.py all | ||
3183 | |||
3184 | - *Upgrading all Recipes to the Latest Versions and Send Email | ||
3185 | Notifications:* To upgrade all recipes to their most recent versions | ||
3186 | and send email messages to maintainers for each attempted recipe as | ||
3187 | well as a status email, use the following command: | ||
3188 | :: | ||
3189 | |||
3190 | $ upgrade-helper.py -e all | ||
3191 | |||
3192 | Once you have run the AUH utility, you can find the results in the AUH | ||
3193 | build directory: | ||
3194 | :: | ||
3195 | |||
3196 | ${BUILDDIR}/upgrade-helper/timestamp | ||
3197 | |||
3198 | The AUH utility | ||
3199 | also creates recipe update commits from successful upgrade attempts in | ||
3200 | the layer tree. | ||
3201 | |||
3202 | You can easily set up to run the AUH utility on a regular basis by using | ||
3203 | a cron job. See the | ||
3204 | `weeklyjob.sh <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh>`_ | ||
3205 | file distributed with the utility for an example. | ||
3206 | |||
3207 | .. _gs-using-devtool-upgrade: | ||
3208 | |||
3209 | Using ``devtool upgrade`` | ||
3210 | ------------------------- | ||
3211 | |||
3212 | As mentioned earlier, an alternative method for upgrading recipes to | ||
3213 | newer versions is to use | ||
3214 | :doc:`devtool upgrade <../ref-manual/ref-devtool-reference>`. | ||
3215 | You can read about ``devtool upgrade`` in general in the | ||
3216 | ":ref:`sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software`" | ||
3217 | section in the Yocto Project Application Development and the Extensible | ||
3218 | Software Development Kit (eSDK) Manual. | ||
3219 | |||
3220 | To see all the command-line options available with ``devtool upgrade``, | ||
3221 | use the following help command: | ||
3222 | :: | ||
3223 | |||
3224 | $ devtool upgrade -h | ||
3225 | |||
3226 | If you want to find out what version a recipe is currently at upstream | ||
3227 | without any attempt to upgrade your local version of the recipe, you can | ||
3228 | use the following command: | ||
3229 | :: | ||
3230 | |||
3231 | $ devtool latest-version recipe_name | ||
3232 | |||
3233 | As mentioned in the previous section describing AUH, ``devtool upgrade`` | ||
3234 | works in a less-automated manner than AUH. Specifically, | ||
3235 | ``devtool upgrade`` only works on a single recipe that you name on the | ||
3236 | command line, cannot perform build and integration testing using images, | ||
3237 | and does not automatically generate commits for changes in the source | ||
3238 | tree. Despite all these "limitations", ``devtool upgrade`` updates the | ||
3239 | recipe file to the new upstream version and attempts to rebase custom | ||
3240 | patches contained by the recipe as needed. | ||
3241 | |||
3242 | .. note:: | ||
3243 | |||
3244 | AUH uses much of | ||
3245 | devtool upgrade | ||
3246 | behind the scenes making AUH somewhat of a "wrapper" application for | ||
3247 | devtool upgrade | ||
3248 | . | ||
3249 | |||
3250 | A typical scenario involves having used Git to clone an upstream | ||
3251 | repository that you use during build operations. Because you are (or | ||
3252 | have) built the recipe in the past, the layer is likely added to your | ||
3253 | configuration already. If for some reason, the layer is not added, you | ||
3254 | could add it easily using the | ||
3255 | ":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" | ||
3256 | script. For example, suppose you use the ``nano.bb`` recipe from the | ||
3257 | ``meta-oe`` layer in the ``meta-openembedded`` repository. For this | ||
3258 | example, assume that the layer has been cloned into following area: | ||
3259 | :: | ||
3260 | |||
3261 | /home/scottrif/meta-openembedded | ||
3262 | |||
3263 | The following command from your | ||
3264 | :term:`Build Directory` adds the layer to | ||
3265 | your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``): | ||
3266 | :: | ||
3267 | |||
3268 | $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe | ||
3269 | NOTE: Starting bitbake server... | ||
3270 | Parsing recipes: 100% |##########################################| Time: 0:00:55 | ||
3271 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
3272 | Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 | ||
3273 | Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 | ||
3274 | Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 | ||
3275 | Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 | ||
3276 | |||
3277 | For this example, assume that the ``nano.bb`` recipe that | ||
3278 | is upstream has a 2.9.3 version number. However, the version in the | ||
3279 | local repository is 2.7.4. The following command from your build | ||
3280 | directory automatically upgrades the recipe for you: | ||
3281 | |||
3282 | .. note:: | ||
3283 | |||
3284 | Using the | ||
3285 | -V | ||
3286 | option is not necessary. Omitting the version number causes | ||
3287 | devtool upgrade | ||
3288 | to upgrade the recipe to the most recent version. | ||
3289 | |||
3290 | :: | ||
3291 | |||
3292 | $ devtool upgrade nano -V 2.9.3 | ||
3293 | NOTE: Starting bitbake server... | ||
3294 | NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace | ||
3295 | Parsing recipes: 100% |##########################################| Time: 0:00:46 | ||
3296 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
3297 | NOTE: Extracting current version source... | ||
3298 | NOTE: Resolving any missing task queue dependencies | ||
3299 | . | ||
3300 | . | ||
3301 | . | ||
3302 | NOTE: Executing SetScene Tasks | ||
3303 | NOTE: Executing RunQueue Tasks | ||
3304 | NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. | ||
3305 | Adding changed files: 100% |#####################################| Time: 0:00:00 | ||
3306 | NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano | ||
3307 | NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb | ||
3308 | |||
3309 | Continuing with this example, you can use ``devtool build`` to build the | ||
3310 | newly upgraded recipe: | ||
3311 | :: | ||
3312 | |||
3313 | $ devtool build nano | ||
3314 | NOTE: Starting bitbake server... | ||
3315 | Loading cache: 100% |################################################################################################| Time: 0:00:01 | ||
3316 | Loaded 2040 entries from dependency cache. | ||
3317 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 | ||
3318 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
3319 | NOTE: Resolving any missing task queue dependencies | ||
3320 | . | ||
3321 | . | ||
3322 | . | ||
3323 | NOTE: Executing SetScene Tasks | ||
3324 | NOTE: Executing RunQueue Tasks | ||
3325 | NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano | ||
3326 | NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. | ||
3327 | |||
3328 | Within the ``devtool upgrade`` workflow, opportunity | ||
3329 | exists to deploy and test your rebuilt software. For this example, | ||
3330 | however, running ``devtool finish`` cleans up the workspace once the | ||
3331 | source in your workspace is clean. This usually means using Git to stage | ||
3332 | and submit commits for the changes generated by the upgrade process. | ||
3333 | |||
3334 | Once the tree is clean, you can clean things up in this example with the | ||
3335 | following command from the ``${BUILDDIR}/workspace/sources/nano`` | ||
3336 | directory: | ||
3337 | :: | ||
3338 | |||
3339 | $ devtool finish nano meta-oe | ||
3340 | NOTE: Starting bitbake server... | ||
3341 | Loading cache: 100% |################################################################################################| Time: 0:00:00 | ||
3342 | Loaded 2040 entries from dependency cache. | ||
3343 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 | ||
3344 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
3345 | NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch | ||
3346 | NOTE: Updating recipe nano_2.9.3.bb | ||
3347 | NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb | ||
3348 | NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano | ||
3349 | NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually | ||
3350 | |||
3351 | |||
3352 | Using the ``devtool finish`` command cleans up the workspace and creates a patch | ||
3353 | file based on your commits. The tool puts all patch files back into the | ||
3354 | source directory in a sub-directory named ``nano`` in this case. | ||
3355 | |||
3356 | .. _dev-manually-upgrading-a-recipe: | ||
3357 | |||
3358 | Manually Upgrading a Recipe | ||
3359 | --------------------------- | ||
3360 | |||
3361 | If for some reason you choose not to upgrade recipes using the `Auto | ||
3362 | Upgrade Helper (AUH) <#gs-using-the-auto-upgrade-helper>`__ or by using | ||
3363 | ```devtool upgrade`` <#gs-using-devtool-upgrade>`__, you can manually | ||
3364 | edit the recipe files to upgrade the versions. | ||
3365 | |||
3366 | .. note:: | ||
3367 | |||
3368 | Manually updating multiple recipes scales poorly and involves many | ||
3369 | steps. The recommendation to upgrade recipe versions is through AUH | ||
3370 | or | ||
3371 | devtool upgrade | ||
3372 | , both of which automate some steps and provide guidance for others | ||
3373 | needed for the manual process. | ||
3374 | |||
3375 | To manually upgrade recipe versions, follow these general steps: | ||
3376 | |||
3377 | 1. *Change the Version:* Rename the recipe such that the version (i.e. | ||
3378 | the :term:`PV` part of the recipe name) | ||
3379 | changes appropriately. If the version is not part of the recipe name, | ||
3380 | change the value as it is set for ``PV`` within the recipe itself. | ||
3381 | |||
3382 | 2. Update ``SRCREV`` if Needed: If the source code your recipe builds | ||
3383 | is fetched from Git or some other version control system, update | ||
3384 | :term:`SRCREV` to point to the | ||
3385 | commit hash that matches the new version. | ||
3386 | |||
3387 | 3. *Build the Software:* Try to build the recipe using BitBake. Typical | ||
3388 | build failures include the following: | ||
3389 | |||
3390 | - License statements were updated for the new version. For this | ||
3391 | case, you need to review any changes to the license and update the | ||
3392 | values of :term:`LICENSE` and | ||
3393 | :term:`LIC_FILES_CHKSUM` | ||
3394 | as needed. | ||
3395 | |||
3396 | .. note:: | ||
3397 | |||
3398 | License changes are often inconsequential. For example, the | ||
3399 | license text's copyright year might have changed. | ||
3400 | |||
3401 | - Custom patches carried by the older version of the recipe might | ||
3402 | fail to apply to the new version. For these cases, you need to | ||
3403 | review the failures. Patches might not be necessary for the new | ||
3404 | version of the software if the upgraded version has fixed those | ||
3405 | issues. If a patch is necessary and failing, you need to rebase it | ||
3406 | into the new version. | ||
3407 | |||
3408 | 4. *Optionally Attempt to Build for Several Architectures:* Once you | ||
3409 | successfully build the new software for a given architecture, you | ||
3410 | could test the build for other architectures by changing the | ||
3411 | :term:`MACHINE` variable and | ||
3412 | rebuilding the software. This optional step is especially important | ||
3413 | if the recipe is to be released publicly. | ||
3414 | |||
3415 | 5. *Check the Upstream Change Log or Release Notes:* Checking both these | ||
3416 | reveals if new features exist that could break | ||
3417 | backwards-compatibility. If so, you need to take steps to mitigate or | ||
3418 | eliminate that situation. | ||
3419 | |||
3420 | 6. *Optionally Create a Bootable Image and Test:* If you want, you can | ||
3421 | test the new software by booting it onto actual hardware. | ||
3422 | |||
3423 | 7. *Create a Commit with the Change in the Layer Repository:* After all | ||
3424 | builds work and any testing is successful, you can create commits for | ||
3425 | any changes in the layer holding your upgraded recipe. | ||
3426 | |||
3427 | .. _finding-the-temporary-source-code: | ||
3428 | |||
3429 | Finding Temporary Source Code | ||
3430 | ============================= | ||
3431 | |||
3432 | You might find it helpful during development to modify the temporary | ||
3433 | source code used by recipes to build packages. For example, suppose you | ||
3434 | are developing a patch and you need to experiment a bit to figure out | ||
3435 | your solution. After you have initially built the package, you can | ||
3436 | iteratively tweak the source code, which is located in the | ||
3437 | :term:`Build Directory`, and then you can | ||
3438 | force a re-compile and quickly test your altered code. Once you settle | ||
3439 | on a solution, you can then preserve your changes in the form of | ||
3440 | patches. | ||
3441 | |||
3442 | During a build, the unpacked temporary source code used by recipes to | ||
3443 | build packages is available in the Build Directory as defined by the | ||
3444 | :term:`S` variable. Below is the default | ||
3445 | value for the ``S`` variable as defined in the | ||
3446 | ``meta/conf/bitbake.conf`` configuration file in the | ||
3447 | :term:`Source Directory`: | ||
3448 | :: | ||
3449 | |||
3450 | S = "${WORKDIR}/${BP}" | ||
3451 | |||
3452 | You should be aware that many recipes override the | ||
3453 | ``S`` variable. For example, recipes that fetch their source from Git | ||
3454 | usually set ``S`` to ``${WORKDIR}/git``. | ||
3455 | |||
3456 | .. note:: | ||
3457 | |||
3458 | The | ||
3459 | BP | ||
3460 | represents the base recipe name, which consists of the name and | ||
3461 | version: | ||
3462 | :: | ||
3463 | |||
3464 | BP = "${BPN}-${PV}" | ||
3465 | |||
3466 | |||
3467 | The path to the work directory for the recipe | ||
3468 | (:term:`WORKDIR`) is defined as | ||
3469 | follows: | ||
3470 | ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} The | ||
3471 | actual directory depends on several things: | ||
3472 | |||
3473 | - :term:`TMPDIR`: The top-level build | ||
3474 | output directory. | ||
3475 | |||
3476 | - :term:`MULTIMACH_TARGET_SYS`: | ||
3477 | The target system identifier. | ||
3478 | |||
3479 | - :term:`PN`: The recipe name. | ||
3480 | |||
3481 | - :term:`EXTENDPE`: The epoch - (if | ||
3482 | :term:`PE` is not specified, which is | ||
3483 | usually the case for most recipes, then ``EXTENDPE`` is blank). | ||
3484 | |||
3485 | - :term:`PV`: The recipe version. | ||
3486 | |||
3487 | - :term:`PR`: The recipe revision. | ||
3488 | |||
3489 | As an example, assume a Source Directory top-level folder named | ||
3490 | ``poky``, a default Build Directory at ``poky/build``, and a | ||
3491 | ``qemux86-poky-linux`` machine target system. Furthermore, suppose your | ||
3492 | recipe is named ``foo_1.3.0.bb``. In this case, the work directory the | ||
3493 | build system uses to build the package would be as follows: | ||
3494 | :: | ||
3495 | |||
3496 | poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 | ||
3497 | |||
3498 | .. _using-a-quilt-workflow: | ||
3499 | |||
3500 | Using Quilt in Your Workflow | ||
3501 | ============================ | ||
3502 | |||
3503 | `Quilt <http://savannah.nongnu.org/projects/quilt>`__ is a powerful tool | ||
3504 | that allows you to capture source code changes without having a clean | ||
3505 | source tree. This section outlines the typical workflow you can use to | ||
3506 | modify source code, test changes, and then preserve the changes in the | ||
3507 | form of a patch all using Quilt. | ||
3508 | |||
3509 | .. note:: | ||
3510 | |||
3511 | With regard to preserving changes to source files, if you clean a | ||
3512 | recipe or have | ||
3513 | rm_work | ||
3514 | enabled, the | ||
3515 | devtool | ||
3516 | workflow | ||
3517 | as described in the Yocto Project Application Development and the | ||
3518 | Extensible Software Development Kit (eSDK) manual is a safer | ||
3519 | development flow than the flow that uses Quilt. | ||
3520 | |||
3521 | Follow these general steps: | ||
3522 | |||
3523 | 1. *Find the Source Code:* Temporary source code used by the | ||
3524 | OpenEmbedded build system is kept in the | ||
3525 | :term:`Build Directory`. See the | ||
3526 | "`Finding Temporary Source | ||
3527 | Code <#finding-the-temporary-source-code>`__" section to learn how to | ||
3528 | locate the directory that has the temporary source code for a | ||
3529 | particular package. | ||
3530 | |||
3531 | 2. *Change Your Working Directory:* You need to be in the directory that | ||
3532 | has the temporary source code. That directory is defined by the | ||
3533 | :term:`S` variable. | ||
3534 | |||
3535 | 3. *Create a New Patch:* Before modifying source code, you need to | ||
3536 | create a new patch. To create a new patch file, use ``quilt new`` as | ||
3537 | below: | ||
3538 | :; | ||
3539 | |||
3540 | $ quilt new my_changes.patch | ||
3541 | |||
3542 | 4. *Notify Quilt and Add Files:* After creating the patch, you need to | ||
3543 | notify Quilt about the files you plan to edit. You notify Quilt by | ||
3544 | adding the files to the patch you just created: | ||
3545 | :: | ||
3546 | |||
3547 | $ quilt add file1.c file2.c file3.c | ||
3548 | |||
3549 | 5. *Edit the Files:* Make your changes in the source code to the files | ||
3550 | you added to the patch. | ||
3551 | |||
3552 | 6. *Test Your Changes:* Once you have modified the source code, the | ||
3553 | easiest way to test your changes is by calling the ``do_compile`` | ||
3554 | task as shown in the following example: | ||
3555 | :: | ||
3556 | |||
3557 | $ bitbake -c compile -f package | ||
3558 | |||
3559 | The ``-f`` or ``--force`` option forces the specified task to | ||
3560 | execute. If you find problems with your code, you can just keep | ||
3561 | editing and re-testing iteratively until things work as expected. | ||
3562 | |||
3563 | .. note:: | ||
3564 | |||
3565 | All the modifications you make to the temporary source code | ||
3566 | disappear once you run the | ||
3567 | do_clean | ||
3568 | or | ||
3569 | do_cleanall | ||
3570 | tasks using BitBake (i.e. | ||
3571 | bitbake -c clean | ||
3572 | package | ||
3573 | and | ||
3574 | bitbake -c cleanall | ||
3575 | package | ||
3576 | ). Modifications will also disappear if you use the | ||
3577 | rm_work | ||
3578 | feature as described in the " | ||
3579 | Conserving Disk Space During Builds | ||
3580 | " section. | ||
3581 | |||
3582 | 7. *Generate the Patch:* Once your changes work as expected, you need to | ||
3583 | use Quilt to generate the final patch that contains all your | ||
3584 | modifications. | ||
3585 | :: | ||
3586 | |||
3587 | $ quilt refresh | ||
3588 | |||
3589 | At this point, the | ||
3590 | ``my_changes.patch`` file has all your edits made to the ``file1.c``, | ||
3591 | ``file2.c``, and ``file3.c`` files. | ||
3592 | |||
3593 | You can find the resulting patch file in the ``patches/`` | ||
3594 | subdirectory of the source (``S``) directory. | ||
3595 | |||
3596 | 8. *Copy the Patch File:* For simplicity, copy the patch file into a | ||
3597 | directory named ``files``, which you can create in the same directory | ||
3598 | that holds the recipe (``.bb``) file or the append (``.bbappend``) | ||
3599 | file. Placing the patch here guarantees that the OpenEmbedded build | ||
3600 | system will find the patch. Next, add the patch into the ``SRC_URI`` | ||
3601 | of the recipe. Here is an example: | ||
3602 | :: | ||
3603 | |||
3604 | SRC_URI += "file://my_changes.patch" | ||
3605 | |||
3606 | .. _platdev-appdev-devshell: | ||
3607 | |||
3608 | Using a Development Shell | ||
3609 | ========================= | ||
3610 | |||
3611 | When debugging certain commands or even when just editing packages, | ||
3612 | ``devshell`` can be a useful tool. When you invoke ``devshell``, all | ||
3613 | tasks up to and including | ||
3614 | :ref:`ref-tasks-patch` are run for the | ||
3615 | specified target. Then, a new terminal is opened and you are placed in | ||
3616 | ``${``\ :term:`S`\ ``}``, the source | ||
3617 | directory. In the new terminal, all the OpenEmbedded build-related | ||
3618 | environment variables are still defined so you can use commands such as | ||
3619 | ``configure`` and ``make``. The commands execute just as if the | ||
3620 | OpenEmbedded build system were executing them. Consequently, working | ||
3621 | this way can be helpful when debugging a build or preparing software to | ||
3622 | be used with the OpenEmbedded build system. | ||
3623 | |||
3624 | Following is an example that uses ``devshell`` on a target named | ||
3625 | ``matchbox-desktop``: | ||
3626 | :: | ||
3627 | |||
3628 | $ bitbake matchbox-desktop -c devshell | ||
3629 | |||
3630 | This command spawns a terminal with a shell prompt within the | ||
3631 | OpenEmbedded build environment. The | ||
3632 | :term:`OE_TERMINAL` variable | ||
3633 | controls what type of shell is opened. | ||
3634 | |||
3635 | For spawned terminals, the following occurs: | ||
3636 | |||
3637 | - The ``PATH`` variable includes the cross-toolchain. | ||
3638 | |||
3639 | - The ``pkgconfig`` variables find the correct ``.pc`` files. | ||
3640 | |||
3641 | - The ``configure`` command finds the Yocto Project site files as well | ||
3642 | as any other necessary files. | ||
3643 | |||
3644 | Within this environment, you can run configure or compile commands as if | ||
3645 | they were being run by the OpenEmbedded build system itself. As noted | ||
3646 | earlier, the working directory also automatically changes to the Source | ||
3647 | Directory (:term:`S`). | ||
3648 | |||
3649 | To manually run a specific task using ``devshell``, run the | ||
3650 | corresponding ``run.*`` script in the | ||
3651 | ``${``\ :term:`WORKDIR`\ ``}/temp`` | ||
3652 | directory (e.g., ``run.do_configure.``\ pid). If a task's script does | ||
3653 | not exist, which would be the case if the task was skipped by way of the | ||
3654 | sstate cache, you can create the task by first running it outside of the | ||
3655 | ``devshell``: | ||
3656 | :: | ||
3657 | |||
3658 | $ bitbake -c task | ||
3659 | |||
3660 | .. note:: | ||
3661 | |||
3662 | - Execution of a task's ``run.*`` script and BitBake's execution of | ||
3663 | a task are identical. In other words, running the script re-runs | ||
3664 | the task just as it would be run using the ``bitbake -c`` command. | ||
3665 | |||
3666 | - Any ``run.*`` file that does not have a ``.pid`` extension is a | ||
3667 | symbolic link (symlink) to the most recent version of that file. | ||
3668 | |||
3669 | Remember, that the ``devshell`` is a mechanism that allows you to get | ||
3670 | into the BitBake task execution environment. And as such, all commands | ||
3671 | must be called just as BitBake would call them. That means you need to | ||
3672 | provide the appropriate options for cross-compilation and so forth as | ||
3673 | applicable. | ||
3674 | |||
3675 | When you are finished using ``devshell``, exit the shell or close the | ||
3676 | terminal window. | ||
3677 | |||
3678 | .. note:: | ||
3679 | |||
3680 | - It is worth remembering that when using ``devshell`` you need to | ||
3681 | use the full compiler name such as ``arm-poky-linux-gnueabi-gcc`` | ||
3682 | instead of just using ``gcc``. The same applies to other | ||
3683 | applications such as ``binutils``, ``libtool`` and so forth. | ||
3684 | BitBake sets up environment variables such as ``CC`` to assist | ||
3685 | applications, such as ``make`` to find the correct tools. | ||
3686 | |||
3687 | - It is also worth noting that ``devshell`` still works over X11 | ||
3688 | forwarding and similar situations. | ||
3689 | |||
3690 | .. _platdev-appdev-devpyshell: | ||
3691 | |||
3692 | Using a Development Python Shell | ||
3693 | ================================ | ||
3694 | |||
3695 | Similar to working within a development shell as described in the | ||
3696 | previous section, you can also spawn and work within an interactive | ||
3697 | Python development shell. When debugging certain commands or even when | ||
3698 | just editing packages, ``devpyshell`` can be a useful tool. When you | ||
3699 | invoke ``devpyshell``, all tasks up to and including | ||
3700 | :ref:`ref-tasks-patch` are run for the | ||
3701 | specified target. Then a new terminal is opened. Additionally, key | ||
3702 | Python objects and code are available in the same way they are to | ||
3703 | BitBake tasks, in particular, the data store 'd'. So, commands such as | ||
3704 | the following are useful when exploring the data store and running | ||
3705 | functions: | ||
3706 | :: | ||
3707 | |||
3708 | pydevshell> d.getVar("STAGING_DIR") | ||
3709 | '/media/build1/poky/build/tmp/sysroots' | ||
3710 | pydevshell> d.getVar("STAGING_DIR") | ||
3711 | '${TMPDIR}/sysroots' | ||
3712 | pydevshell> d.setVar("FOO", "bar") | ||
3713 | pydevshell> d.getVar("FOO") | ||
3714 | 'bar' | ||
3715 | pydevshell> d.delVar("FOO") | ||
3716 | pydevshell> d.getVar("FOO") | ||
3717 | pydevshell> bb.build.exec_func("do_unpack", d) | ||
3718 | pydevshell> | ||
3719 | |||
3720 | The commands execute just as if the OpenEmbedded build | ||
3721 | system were executing them. Consequently, working this way can be | ||
3722 | helpful when debugging a build or preparing software to be used with the | ||
3723 | OpenEmbedded build system. | ||
3724 | |||
3725 | Following is an example that uses ``devpyshell`` on a target named | ||
3726 | ``matchbox-desktop``: | ||
3727 | :: | ||
3728 | |||
3729 | $ bitbake matchbox-desktop -c devpyshell | ||
3730 | |||
3731 | This command spawns a terminal and places you in an interactive Python | ||
3732 | interpreter within the OpenEmbedded build environment. The | ||
3733 | :term:`OE_TERMINAL` variable | ||
3734 | controls what type of shell is opened. | ||
3735 | |||
3736 | When you are finished using ``devpyshell``, you can exit the shell | ||
3737 | either by using Ctrl+d or closing the terminal window. | ||
3738 | |||
3739 | .. _dev-building: | ||
3740 | |||
3741 | Building | ||
3742 | ======== | ||
3743 | |||
3744 | This section describes various build procedures. For example, the steps | ||
3745 | needed for a simple build, a target that uses multiple configurations, | ||
3746 | building an image for more than one machine, and so forth. | ||
3747 | |||
3748 | .. _dev-building-a-simple-image: | ||
3749 | |||
3750 | Building a Simple Image | ||
3751 | ----------------------- | ||
3752 | |||
3753 | In the development environment, you need to build an image whenever you | ||
3754 | change hardware support, add or change system libraries, or add or | ||
3755 | change services that have dependencies. Several methods exist that allow | ||
3756 | you to build an image within the Yocto Project. This section presents | ||
3757 | the basic steps you need to build a simple image using BitBake from a | ||
3758 | build host running Linux. | ||
3759 | |||
3760 | .. note:: | ||
3761 | |||
3762 | - For information on how to build an image using | ||
3763 | :term:`Toaster`, see the | ||
3764 | :doc:`../toaster-manual/toaster-manual`. | ||
3765 | |||
3766 | - For information on how to use ``devtool`` to build images, see the | ||
3767 | ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
3768 | section in the Yocto Project Application Development and the | ||
3769 | Extensible Software Development Kit (eSDK) manual. | ||
3770 | |||
3771 | - For a quick example on how to build an image using the | ||
3772 | OpenEmbedded build system, see the | ||
3773 | :doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document. | ||
3774 | |||
3775 | The build process creates an entire Linux distribution from source and | ||
3776 | places it in your :term:`Build Directory` under | ||
3777 | ``tmp/deploy/images``. For detailed information on the build process | ||
3778 | using BitBake, see the ":ref:`images-dev-environment`" section in the | ||
3779 | Yocto Project Overview and Concepts Manual. | ||
3780 | |||
3781 | The following figure and list overviews the build process: | ||
3782 | |||
3783 | .. image:: figures/bitbake-build-flow.png | ||
3784 | :align: center | ||
3785 | |||
3786 | 1. *Set up Your Host Development System to Support Development Using the | ||
3787 | Yocto Project*: See the "`Setting Up to Use the Yocto | ||
3788 | Project <#dev-manual-start>`__" section for options on how to get a | ||
3789 | build host ready to use the Yocto Project. | ||
3790 | |||
3791 | 2. *Initialize the Build Environment:* Initialize the build environment | ||
3792 | by sourcing the build environment script (i.e. | ||
3793 | :ref:`structure-core-script`): | ||
3794 | :: | ||
3795 | |||
3796 | $ source oe-init-build-env [build_dir] | ||
3797 | |||
3798 | When you use the initialization script, the OpenEmbedded build system | ||
3799 | uses ``build`` as the default Build Directory in your current work | ||
3800 | directory. You can use a build_dir argument with the script to | ||
3801 | specify a different build directory. | ||
3802 | |||
3803 | .. note:: | ||
3804 | |||
3805 | A common practice is to use a different Build Directory for | ||
3806 | different targets. For example, | ||
3807 | ~/build/x86 | ||
3808 | for a | ||
3809 | qemux86 | ||
3810 | target, and | ||
3811 | ~/build/arm | ||
3812 | for a | ||
3813 | qemuarm | ||
3814 | target. | ||
3815 | |||
3816 | 3. Make Sure Your ``local.conf`` File is Correct: Ensure the | ||
3817 | ``conf/local.conf`` configuration file, which is found in the Build | ||
3818 | Directory, is set up how you want it. This file defines many aspects | ||
3819 | of the build environment including the target machine architecture | ||
3820 | through the ``MACHINE`` variable, the packaging format used during | ||
3821 | the build | ||
3822 | (:term:`PACKAGE_CLASSES`), | ||
3823 | and a centralized tarball download directory through the | ||
3824 | :term:`DL_DIR` variable. | ||
3825 | |||
3826 | 4. *Build the Image:* Build the image using the ``bitbake`` command: | ||
3827 | :: | ||
3828 | |||
3829 | $ bitbake target | ||
3830 | |||
3831 | .. note:: | ||
3832 | |||
3833 | For information on BitBake, see the | ||
3834 | BitBake User Manual | ||
3835 | . | ||
3836 | |||
3837 | The target is the name of the recipe you want to build. Common | ||
3838 | targets are the images in ``meta/recipes-core/images``, | ||
3839 | ``meta/recipes-sato/images``, and so forth all found in the | ||
3840 | :term:`Source Directory`. Or, the target | ||
3841 | can be the name of a recipe for a specific piece of software such as | ||
3842 | BusyBox. For more details about the images the OpenEmbedded build | ||
3843 | system supports, see the | ||
3844 | ":ref:`ref-manual/ref-images:Images`" chapter in the Yocto | ||
3845 | Project Reference Manual. | ||
3846 | |||
3847 | As an example, the following command builds the | ||
3848 | ``core-image-minimal`` image: | ||
3849 | :: | ||
3850 | |||
3851 | $ bitbake core-image-minimal | ||
3852 | |||
3853 | Once an | ||
3854 | image has been built, it often needs to be installed. The images and | ||
3855 | kernels built by the OpenEmbedded build system are placed in the | ||
3856 | Build Directory in ``tmp/deploy/images``. For information on how to | ||
3857 | run pre-built images such as ``qemux86`` and ``qemuarm``, see the | ||
3858 | :doc:`../sdk-manual/sdk-manual` manual. For | ||
3859 | information about how to install these images, see the documentation | ||
3860 | for your particular board or machine. | ||
3861 | |||
3862 | .. _dev-building-images-for-multiple-targets-using-multiple-configurations: | ||
3863 | |||
3864 | Building Images for Multiple Targets Using Multiple Configurations | ||
3865 | ------------------------------------------------------------------ | ||
3866 | |||
3867 | You can use a single ``bitbake`` command to build multiple images or | ||
3868 | packages for different targets where each image or package requires a | ||
3869 | different configuration (multiple configuration builds). The builds, in | ||
3870 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
3871 | section uses that term throughout. | ||
3872 | |||
3873 | This section describes how to set up for multiple configuration builds | ||
3874 | and how to account for cross-build dependencies between the | ||
3875 | multiconfigs. | ||
3876 | |||
3877 | .. _dev-setting-up-and-running-a-multiple-configuration-build: | ||
3878 | |||
3879 | Setting Up and Running a Multiple Configuration Build | ||
3880 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
3881 | |||
3882 | To accomplish a multiple configuration build, you must define each | ||
3883 | target's configuration separately using a parallel configuration file in | ||
3884 | the :term:`Build Directory`, and you | ||
3885 | must follow a required file hierarchy. Additionally, you must enable the | ||
3886 | multiple configuration builds in your ``local.conf`` file. | ||
3887 | |||
3888 | Follow these steps to set up and execute multiple configuration builds: | ||
3889 | |||
3890 | - *Create Separate Configuration Files*: You need to create a single | ||
3891 | configuration file for each build target (each multiconfig). | ||
3892 | Minimally, each configuration file must define the machine and the | ||
3893 | temporary directory BitBake uses for the build. Suggested practice | ||
3894 | dictates that you do not overlap the temporary directories used | ||
3895 | during the builds. However, it is possible that you can share the | ||
3896 | temporary directory | ||
3897 | (:term:`TMPDIR`). For example, | ||
3898 | consider a scenario with two different multiconfigs for the same | ||
3899 | :term:`MACHINE`: "qemux86" built | ||
3900 | for two distributions such as "poky" and "poky-lsb". In this case, | ||
3901 | you might want to use the same ``TMPDIR``. | ||
3902 | |||
3903 | Here is an example showing the minimal statements needed in a | ||
3904 | configuration file for a "qemux86" target whose temporary build | ||
3905 | directory is ``tmpmultix86``: | ||
3906 | :: | ||
3907 | |||
3908 | MACHINE = "qemux86" | ||
3909 | TMPDIR = "${TOPDIR}/tmpmultix86" | ||
3910 | |||
3911 | The location for these multiconfig configuration files is specific. | ||
3912 | They must reside in the current build directory in a sub-directory of | ||
3913 | ``conf`` named ``multiconfig``. Following is an example that defines | ||
3914 | two configuration files for the "x86" and "arm" multiconfigs: | ||
3915 | |||
3916 | .. image:: figures/multiconfig_files.png | ||
3917 | :align: center | ||
3918 | |||
3919 | The reason for this required file hierarchy is because the ``BBPATH`` | ||
3920 | variable is not constructed until the layers are parsed. | ||
3921 | Consequently, using the configuration file as a pre-configuration | ||
3922 | file is not possible unless it is located in the current working | ||
3923 | directory. | ||
3924 | |||
3925 | - *Add the BitBake Multi-configuration Variable to the Local | ||
3926 | Configuration File*: Use the | ||
3927 | :term:`BBMULTICONFIG` | ||
3928 | variable in your ``conf/local.conf`` configuration file to specify | ||
3929 | each multiconfig. Continuing with the example from the previous | ||
3930 | figure, the ``BBMULTICONFIG`` variable needs to enable two | ||
3931 | multiconfigs: "x86" and "arm" by specifying each configuration file: | ||
3932 | :: | ||
3933 | |||
3934 | BBMULTICONFIG = "x86 arm" | ||
3935 | |||
3936 | .. note:: | ||
3937 | |||
3938 | A "default" configuration already exists by definition. This | ||
3939 | configuration is named: "" (i.e. empty string) and is defined by | ||
3940 | the variables coming from your | ||
3941 | local.conf | ||
3942 | file. Consequently, the previous example actually adds two | ||
3943 | additional configurations to your build: "arm" and "x86" along | ||
3944 | with "". | ||
3945 | |||
3946 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
3947 | the multiple configuration build: | ||
3948 | :: | ||
3949 | |||
3950 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
3951 | |||
3952 | For the example in this section, the following command applies: | ||
3953 | :: | ||
3954 | |||
3955 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
3956 | |||
3957 | The previous BitBake command builds a ``core-image-minimal`` image | ||
3958 | that is configured through the ``x86.conf`` configuration file, a | ||
3959 | ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
3960 | configuration file and a ``core-image-base`` that is configured | ||
3961 | through your ``local.conf`` configuration file. | ||
3962 | |||
3963 | .. note:: | ||
3964 | |||
3965 | Support for multiple configuration builds in the Yocto Project DISTRO | ||
3966 | (DISTRO_NAME) Release does not include Shared State (sstate) | ||
3967 | optimizations. Consequently, if a build uses the same object twice | ||
3968 | in, for example, two different | ||
3969 | TMPDIR | ||
3970 | directories, the build either loads from an existing sstate cache for | ||
3971 | that build at the start or builds the object fresh. | ||
3972 | |||
3973 | .. _dev-enabling-multiple-configuration-build-dependencies: | ||
3974 | |||
3975 | Enabling Multiple Configuration Build Dependencies | ||
3976 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
3977 | |||
3978 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
3979 | multiple configuration build. For example, suppose that in order to | ||
3980 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
3981 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
3982 | essentially that the | ||
3983 | :ref:`ref-tasks-image` task in the | ||
3984 | ``core-image-sato`` recipe depends on the completion of the | ||
3985 | :ref:`ref-tasks-rootfs` task of the | ||
3986 | ``core-image-minimal`` recipe. | ||
3987 | |||
3988 | To enable dependencies in a multiple configuration build, you must | ||
3989 | declare the dependencies in the recipe using the following statement | ||
3990 | form: | ||
3991 | :: | ||
3992 | |||
3993 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
3994 | |||
3995 | To better show how to use this statement, consider the example scenario | ||
3996 | from the first paragraph of this section. The following statement needs | ||
3997 | to be added to the recipe that builds the ``core-image-sato`` image: | ||
3998 | :: | ||
3999 | |||
4000 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
4001 | |||
4002 | In this example, the from_multiconfig is "x86". The to_multiconfig is "arm". The | ||
4003 | task on which the ``do_image`` task in the recipe depends is the | ||
4004 | ``do_rootfs`` task from the ``core-image-minimal`` recipe associated | ||
4005 | with the "arm" multiconfig. | ||
4006 | |||
4007 | Once you set up this dependency, you can build the "x86" multiconfig | ||
4008 | using a BitBake command as follows: | ||
4009 | :: | ||
4010 | |||
4011 | $ bitbake mc:x86:core-image-sato | ||
4012 | |||
4013 | This command executes all the tasks needed to create the | ||
4014 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
4015 | dependency, BitBake also executes through the ``do_rootfs`` task for the | ||
4016 | "arm" multiconfig build. | ||
4017 | |||
4018 | Having a recipe depend on the root filesystem of another build might not | ||
4019 | seem that useful. Consider this change to the statement in the | ||
4020 | ``core-image-sato`` recipe: | ||
4021 | :: | ||
4022 | |||
4023 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
4024 | |||
4025 | In this case, BitBake must | ||
4026 | create the ``core-image-minimal`` image for the "arm" build since the | ||
4027 | "x86" build depends on it. | ||
4028 | |||
4029 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
4030 | and have separate configuration files, BitBake places the artifacts for | ||
4031 | each build in the respective temporary build directories (i.e. | ||
4032 | :term:`TMPDIR`). | ||
4033 | |||
4034 | .. _building-an-initramfs-image: | ||
4035 | |||
4036 | Building an Initial RAM Filesystem (initramfs) Image | ||
4037 | ---------------------------------------------------- | ||
4038 | |||
4039 | An initial RAM filesystem (initramfs) image provides a temporary root | ||
4040 | filesystem used for early system initialization (e.g. loading of modules | ||
4041 | needed to locate and mount the "real" root filesystem). | ||
4042 | |||
4043 | .. note:: | ||
4044 | |||
4045 | The initramfs image is the successor of initial RAM disk (initrd). It | ||
4046 | is a "copy in and out" (cpio) archive of the initial filesystem that | ||
4047 | gets loaded into memory during the Linux startup process. Because | ||
4048 | Linux uses the contents of the archive during initialization, the | ||
4049 | initramfs image needs to contain all of the device drivers and tools | ||
4050 | needed to mount the final root filesystem. | ||
4051 | |||
4052 | Follow these steps to create an initramfs image: | ||
4053 | |||
4054 | 1. *Create the initramfs Image Recipe:* You can reference the | ||
4055 | ``core-image-minimal-initramfs.bb`` recipe found in the | ||
4056 | ``meta/recipes-core`` directory of the :term:`Source Directory` | ||
4057 | as an example | ||
4058 | from which to work. | ||
4059 | |||
4060 | 2. *Decide if You Need to Bundle the initramfs Image Into the Kernel | ||
4061 | Image:* If you want the initramfs image that is built to be bundled | ||
4062 | in with the kernel image, set the | ||
4063 | :term:`INITRAMFS_IMAGE_BUNDLE` | ||
4064 | variable to "1" in your ``local.conf`` configuration file and set the | ||
4065 | :term:`INITRAMFS_IMAGE` | ||
4066 | variable in the recipe that builds the kernel image. | ||
4067 | |||
4068 | .. note:: | ||
4069 | |||
4070 | It is recommended that you do bundle the initramfs image with the | ||
4071 | kernel image to avoid circular dependencies between the kernel | ||
4072 | recipe and the initramfs recipe should the initramfs image include | ||
4073 | kernel modules. | ||
4074 | |||
4075 | Setting the ``INITRAMFS_IMAGE_BUNDLE`` flag causes the initramfs | ||
4076 | image to be unpacked into the ``${B}/usr/`` directory. The unpacked | ||
4077 | initramfs image is then passed to the kernel's ``Makefile`` using the | ||
4078 | :term:`CONFIG_INITRAMFS_SOURCE` | ||
4079 | variable, allowing the initramfs image to be built into the kernel | ||
4080 | normally. | ||
4081 | |||
4082 | .. note:: | ||
4083 | |||
4084 | If you choose to not bundle the initramfs image with the kernel | ||
4085 | image, you are essentially using an | ||
4086 | Initial RAM Disk (initrd) | ||
4087 | . Creating an initrd is handled primarily through the | ||
4088 | INITRD_IMAGE | ||
4089 | , | ||
4090 | INITRD_LIVE | ||
4091 | , and | ||
4092 | INITRD_IMAGE_LIVE | ||
4093 | variables. For more information, see the | ||
4094 | image-live.bbclass | ||
4095 | file. | ||
4096 | |||
4097 | 3. *Optionally Add Items to the initramfs Image Through the initramfs | ||
4098 | Image Recipe:* If you add items to the initramfs image by way of its | ||
4099 | recipe, you should use | ||
4100 | :term:`PACKAGE_INSTALL` | ||
4101 | rather than | ||
4102 | :term:`IMAGE_INSTALL`. | ||
4103 | ``PACKAGE_INSTALL`` gives more direct control of what is added to the | ||
4104 | image as compared to the defaults you might not necessarily want that | ||
4105 | are set by the :ref:`image <ref-classes-image>` | ||
4106 | or :ref:`core-image <ref-classes-core-image>` | ||
4107 | classes. | ||
4108 | |||
4109 | 4. *Build the Kernel Image and the initramfs Image:* Build your kernel | ||
4110 | image using BitBake. Because the initramfs image recipe is a | ||
4111 | dependency of the kernel image, the initramfs image is built as well | ||
4112 | and bundled with the kernel image if you used the | ||
4113 | :term:`INITRAMFS_IMAGE_BUNDLE` | ||
4114 | variable described earlier. | ||
4115 | |||
4116 | Building a Tiny System | ||
4117 | ---------------------- | ||
4118 | |||
4119 | Very small distributions have some significant advantages such as | ||
4120 | requiring less on-die or in-package memory (cheaper), better performance | ||
4121 | through efficient cache usage, lower power requirements due to less | ||
4122 | memory, faster boot times, and reduced development overhead. Some | ||
4123 | real-world examples where a very small distribution gives you distinct | ||
4124 | advantages are digital cameras, medical devices, and small headless | ||
4125 | systems. | ||
4126 | |||
4127 | This section presents information that shows you how you can trim your | ||
4128 | distribution to even smaller sizes than the ``poky-tiny`` distribution, | ||
4129 | which is around 5 Mbytes, that can be built out-of-the-box using the | ||
4130 | Yocto Project. | ||
4131 | |||
4132 | .. _tiny-system-overview: | ||
4133 | |||
4134 | Tiny System Overview | ||
4135 | ~~~~~~~~~~~~~~~~~~~~ | ||
4136 | |||
4137 | The following list presents the overall steps you need to consider and | ||
4138 | perform to create distributions with smaller root filesystems, achieve | ||
4139 | faster boot times, maintain your critical functionality, and avoid | ||
4140 | initial RAM disks: | ||
4141 | |||
4142 | - `Determine your goals and guiding | ||
4143 | principles. <#goals-and-guiding-principles>`__ | ||
4144 | |||
4145 | - `Understand what contributes to your image | ||
4146 | size. <#understand-what-gives-your-image-size>`__ | ||
4147 | |||
4148 | - `Reduce the size of the root | ||
4149 | filesystem. <#trim-the-root-filesystem>`__ | ||
4150 | |||
4151 | - `Reduce the size of the kernel. <#trim-the-kernel>`__ | ||
4152 | |||
4153 | - `Eliminate packaging | ||
4154 | requirements. <#remove-package-management-requirements>`__ | ||
4155 | |||
4156 | - `Look for other ways to minimize | ||
4157 | size. <#look-for-other-ways-to-minimize-size>`__ | ||
4158 | |||
4159 | - `Iterate on the process. <#iterate-on-the-process>`__ | ||
4160 | |||
4161 | Goals and Guiding Principles | ||
4162 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4163 | |||
4164 | Before you can reach your destination, you need to know where you are | ||
4165 | going. Here is an example list that you can use as a guide when creating | ||
4166 | very small distributions: | ||
4167 | |||
4168 | - Determine how much space you need (e.g. a kernel that is 1 Mbyte or | ||
4169 | less and a root filesystem that is 3 Mbytes or less). | ||
4170 | |||
4171 | - Find the areas that are currently taking 90% of the space and | ||
4172 | concentrate on reducing those areas. | ||
4173 | |||
4174 | - Do not create any difficult "hacks" to achieve your goals. | ||
4175 | |||
4176 | - Leverage the device-specific options. | ||
4177 | |||
4178 | - Work in a separate layer so that you keep changes isolated. For | ||
4179 | information on how to create layers, see the "`Understanding and | ||
4180 | Creating Layers <#understanding-and-creating-layers>`__" section. | ||
4181 | |||
4182 | .. _understand-what-gives-your-image-size: | ||
4183 | |||
4184 | Understand What Contributes to Your Image Size | ||
4185 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4186 | |||
4187 | It is easiest to have something to start with when creating your own | ||
4188 | distribution. You can use the Yocto Project out-of-the-box to create the | ||
4189 | ``poky-tiny`` distribution. Ultimately, you will want to make changes in | ||
4190 | your own distribution that are likely modeled after ``poky-tiny``. | ||
4191 | |||
4192 | .. note:: | ||
4193 | |||
4194 | To use | ||
4195 | poky-tiny | ||
4196 | in your build, set the | ||
4197 | DISTRO | ||
4198 | variable in your | ||
4199 | local.conf | ||
4200 | file to "poky-tiny" as described in the " | ||
4201 | Creating Your Own Distribution | ||
4202 | " section. | ||
4203 | |||
4204 | Understanding some memory concepts will help you reduce the system size. | ||
4205 | Memory consists of static, dynamic, and temporary memory. Static memory | ||
4206 | is the TEXT (code), DATA (initialized data in the code), and BSS | ||
4207 | (uninitialized data) sections. Dynamic memory represents memory that is | ||
4208 | allocated at runtime: stacks, hash tables, and so forth. Temporary | ||
4209 | memory is recovered after the boot process. This memory consists of | ||
4210 | memory used for decompressing the kernel and for the ``__init__`` | ||
4211 | functions. | ||
4212 | |||
4213 | To help you see where you currently are with kernel and root filesystem | ||
4214 | sizes, you can use two tools found in the :term:`Source Directory` | ||
4215 | in the | ||
4216 | ``scripts/tiny/`` directory: | ||
4217 | |||
4218 | - ``ksize.py``: Reports component sizes for the kernel build objects. | ||
4219 | |||
4220 | - ``dirsize.py``: Reports component sizes for the root filesystem. | ||
4221 | |||
4222 | This next tool and command help you organize configuration fragments and | ||
4223 | view file dependencies in a human-readable form: | ||
4224 | |||
4225 | - ``merge_config.sh``: Helps you manage configuration files and | ||
4226 | fragments within the kernel. With this tool, you can merge individual | ||
4227 | configuration fragments together. The tool allows you to make | ||
4228 | overrides and warns you of any missing configuration options. The | ||
4229 | tool is ideal for allowing you to iterate on configurations, create | ||
4230 | minimal configurations, and create configuration files for different | ||
4231 | machines without having to duplicate your process. | ||
4232 | |||
4233 | The ``merge_config.sh`` script is part of the Linux Yocto kernel Git | ||
4234 | repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, | ||
4235 | ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` | ||
4236 | directory. | ||
4237 | |||
4238 | For more information on configuration fragments, see the | ||
4239 | ":ref:`creating-config-fragments`" | ||
4240 | section in the Yocto Project Linux Kernel Development Manual. | ||
4241 | |||
4242 | - ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command | ||
4243 | with these options brings up a Dependency Explorer from which you can | ||
4244 | view file dependencies. Understanding these dependencies allows you | ||
4245 | to make informed decisions when cutting out various pieces of the | ||
4246 | kernel and root filesystem. | ||
4247 | |||
4248 | Trim the Root Filesystem | ||
4249 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4250 | |||
4251 | The root filesystem is made up of packages for booting, libraries, and | ||
4252 | applications. To change things, you can configure how the packaging | ||
4253 | happens, which changes the way you build them. You can also modify the | ||
4254 | filesystem itself or select a different filesystem. | ||
4255 | |||
4256 | First, find out what is hogging your root filesystem by running the | ||
4257 | ``dirsize.py`` script from your root directory: | ||
4258 | :: | ||
4259 | |||
4260 | $ cd root-directory-of-image | ||
4261 | $ dirsize.py 100000 > dirsize-100k.log | ||
4262 | $ cat dirsize-100k.log | ||
4263 | |||
4264 | You can apply a filter to the script to ignore files | ||
4265 | under a certain size. The previous example filters out any files below | ||
4266 | 100 Kbytes. The sizes reported by the tool are uncompressed, and thus | ||
4267 | will be smaller by a relatively constant factor in a compressed root | ||
4268 | filesystem. When you examine your log file, you can focus on areas of | ||
4269 | the root filesystem that take up large amounts of memory. | ||
4270 | |||
4271 | You need to be sure that what you eliminate does not cripple the | ||
4272 | functionality you need. One way to see how packages relate to each other | ||
4273 | is by using the Dependency Explorer UI with the BitBake command: | ||
4274 | :: | ||
4275 | |||
4276 | $ cd image-directory | ||
4277 | $ bitbake -u taskexp -g image | ||
4278 | |||
4279 | Use the interface to | ||
4280 | select potential packages you wish to eliminate and see their dependency | ||
4281 | relationships. | ||
4282 | |||
4283 | When deciding how to reduce the size, get rid of packages that result in | ||
4284 | minimal impact on the feature set. For example, you might not need a VGA | ||
4285 | display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` | ||
4286 | instead of ``udev``. | ||
4287 | |||
4288 | Use your ``local.conf`` file to make changes. For example, to eliminate | ||
4289 | ``udev`` and ``glib``, set the following in the local configuration | ||
4290 | file: | ||
4291 | :: | ||
4292 | |||
4293 | VIRTUAL-RUNTIME_dev_manager = "" | ||
4294 | |||
4295 | Finally, you should consider exactly the type of root filesystem you | ||
4296 | need to meet your needs while also reducing its size. For example, | ||
4297 | consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an | ||
4298 | ``initramfs`` using ``initramfs``. Be aware that ``ext3`` requires a 1 | ||
4299 | Mbyte journal. If you are okay with running read-only, you do not need | ||
4300 | this journal. | ||
4301 | |||
4302 | .. note:: | ||
4303 | |||
4304 | After each round of elimination, you need to rebuild your system and | ||
4305 | then use the tools to see the effects of your reductions. | ||
4306 | |||
4307 | Trim the Kernel | ||
4308 | ~~~~~~~~~~~~~~~ | ||
4309 | |||
4310 | The kernel is built by including policies for hardware-independent | ||
4311 | aspects. What subsystems do you enable? For what architecture are you | ||
4312 | building? Which drivers do you build by default? | ||
4313 | |||
4314 | .. note:: | ||
4315 | |||
4316 | You can modify the kernel source if you want to help with boot time. | ||
4317 | |||
4318 | Run the ``ksize.py`` script from the top-level Linux build directory to | ||
4319 | get an idea of what is making up the kernel: | ||
4320 | :: | ||
4321 | |||
4322 | $ cd top-level-linux-build-directory | ||
4323 | $ ksize.py > ksize.log | ||
4324 | $ cat ksize.log | ||
4325 | |||
4326 | When you examine the log, you will see how much space is taken up with | ||
4327 | the built-in ``.o`` files for drivers, networking, core kernel files, | ||
4328 | filesystem, sound, and so forth. The sizes reported by the tool are | ||
4329 | uncompressed, and thus will be smaller by a relatively constant factor | ||
4330 | in a compressed kernel image. Look to reduce the areas that are large | ||
4331 | and taking up around the "90% rule." | ||
4332 | |||
4333 | To examine, or drill down, into any particular area, use the ``-d`` | ||
4334 | option with the script: | ||
4335 | :: | ||
4336 | |||
4337 | $ ksize.py -d > ksize.log | ||
4338 | |||
4339 | Using this option | ||
4340 | breaks out the individual file information for each area of the kernel | ||
4341 | (e.g. drivers, networking, and so forth). | ||
4342 | |||
4343 | Use your log file to see what you can eliminate from the kernel based on | ||
4344 | features you can let go. For example, if you are not going to need | ||
4345 | sound, you do not need any drivers that support sound. | ||
4346 | |||
4347 | After figuring out what to eliminate, you need to reconfigure the kernel | ||
4348 | to reflect those changes during the next build. You could run | ||
4349 | ``menuconfig`` and make all your changes at once. However, that makes it | ||
4350 | difficult to see the effects of your individual eliminations and also | ||
4351 | makes it difficult to replicate the changes for perhaps another target | ||
4352 | device. A better method is to start with no configurations using | ||
4353 | ``allnoconfig``, create configuration fragments for individual changes, | ||
4354 | and then manage the fragments into a single configuration file using | ||
4355 | ``merge_config.sh``. The tool makes it easy for you to iterate using the | ||
4356 | configuration change and build cycle. | ||
4357 | |||
4358 | Each time you make configuration changes, you need to rebuild the kernel | ||
4359 | and check to see what impact your changes had on the overall size. | ||
4360 | |||
4361 | Remove Package Management Requirements | ||
4362 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4363 | |||
4364 | Packaging requirements add size to the image. One way to reduce the size | ||
4365 | of the image is to remove all the packaging requirements from the image. | ||
4366 | This reduction includes both removing the package manager and its unique | ||
4367 | dependencies as well as removing the package management data itself. | ||
4368 | |||
4369 | To eliminate all the packaging requirements for an image, be sure that | ||
4370 | "package-management" is not part of your | ||
4371 | :term:`IMAGE_FEATURES` | ||
4372 | statement for the image. When you remove this feature, you are removing | ||
4373 | the package manager as well as its dependencies from the root | ||
4374 | filesystem. | ||
4375 | |||
4376 | Look for Other Ways to Minimize Size | ||
4377 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4378 | |||
4379 | Depending on your particular circumstances, other areas that you can | ||
4380 | trim likely exist. The key to finding these areas is through tools and | ||
4381 | methods described here combined with experimentation and iteration. Here | ||
4382 | are a couple of areas to experiment with: | ||
4383 | |||
4384 | - ``glibc``: In general, follow this process: | ||
4385 | |||
4386 | 1. Remove ``glibc`` features from | ||
4387 | :term:`DISTRO_FEATURES` | ||
4388 | that you think you do not need. | ||
4389 | |||
4390 | 2. Build your distribution. | ||
4391 | |||
4392 | 3. If the build fails due to missing symbols in a package, determine | ||
4393 | if you can reconfigure the package to not need those features. For | ||
4394 | example, change the configuration to not support wide character | ||
4395 | support as is done for ``ncurses``. Or, if support for those | ||
4396 | characters is needed, determine what ``glibc`` features provide | ||
4397 | the support and restore the configuration. | ||
4398 | |||
4399 | 4. Rebuild and repeat the process. | ||
4400 | |||
4401 | - ``busybox``: For BusyBox, use a process similar as described for | ||
4402 | ``glibc``. A difference is you will need to boot the resulting system | ||
4403 | to see if you are able to do everything you expect from the running | ||
4404 | system. You need to be sure to integrate configuration fragments into | ||
4405 | Busybox because BusyBox handles its own core features and then allows | ||
4406 | you to add configuration fragments on top. | ||
4407 | |||
4408 | Iterate on the Process | ||
4409 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
4410 | |||
4411 | If you have not reached your goals on system size, you need to iterate | ||
4412 | on the process. The process is the same. Use the tools and see just what | ||
4413 | is taking up 90% of the root filesystem and the kernel. Decide what you | ||
4414 | can eliminate without limiting your device beyond what you need. | ||
4415 | |||
4416 | Depending on your system, a good place to look might be Busybox, which | ||
4417 | provides a stripped down version of Unix tools in a single, executable | ||
4418 | file. You might be able to drop virtual terminal services or perhaps | ||
4419 | ipv6. | ||
4420 | |||
4421 | Building Images for More than One Machine | ||
4422 | ----------------------------------------- | ||
4423 | |||
4424 | A common scenario developers face is creating images for several | ||
4425 | different machines that use the same software environment. In this | ||
4426 | situation, it is tempting to set the tunings and optimization flags for | ||
4427 | each build specifically for the targeted hardware (i.e. "maxing out" the | ||
4428 | tunings). Doing so can considerably add to build times and package feed | ||
4429 | maintenance collectively for the machines. For example, selecting tunes | ||
4430 | that are extremely specific to a CPU core used in a system might enable | ||
4431 | some micro optimizations in GCC for that particular system but would | ||
4432 | otherwise not gain you much of a performance difference across the other | ||
4433 | systems as compared to using a more general tuning across all the builds | ||
4434 | (e.g. setting :term:`DEFAULTTUNE` | ||
4435 | specifically for each machine's build). Rather than "max out" each | ||
4436 | build's tunings, you can take steps that cause the OpenEmbedded build | ||
4437 | system to reuse software across the various machines where it makes | ||
4438 | sense. | ||
4439 | |||
4440 | If build speed and package feed maintenance are considerations, you | ||
4441 | should consider the points in this section that can help you optimize | ||
4442 | your tunings to best consider build times and package feed maintenance. | ||
4443 | |||
4444 | - *Share the Build Directory:* If at all possible, share the | ||
4445 | :term:`TMPDIR` across builds. The | ||
4446 | Yocto Project supports switching between different | ||
4447 | :term:`MACHINE` values in the same | ||
4448 | ``TMPDIR``. This practice is well supported and regularly used by | ||
4449 | developers when building for multiple machines. When you use the same | ||
4450 | ``TMPDIR`` for multiple machine builds, the OpenEmbedded build system | ||
4451 | can reuse the existing native and often cross-recipes for multiple | ||
4452 | machines. Thus, build time decreases. | ||
4453 | |||
4454 | .. note:: | ||
4455 | |||
4456 | If | ||
4457 | DISTRO | ||
4458 | settings change or fundamental configuration settings such as the | ||
4459 | filesystem layout, you need to work with a clean | ||
4460 | TMPDIR | ||
4461 | . Sharing | ||
4462 | TMPDIR | ||
4463 | under these circumstances might work but since it is not | ||
4464 | guaranteed, you should use a clean | ||
4465 | TMPDIR | ||
4466 | . | ||
4467 | |||
4468 | - *Enable the Appropriate Package Architecture:* By default, the | ||
4469 | OpenEmbedded build system enables three levels of package | ||
4470 | architectures: "all", "tune" or "package", and "machine". Any given | ||
4471 | recipe usually selects one of these package architectures (types) for | ||
4472 | its output. Depending for what a given recipe creates packages, | ||
4473 | making sure you enable the appropriate package architecture can | ||
4474 | directly impact the build time. | ||
4475 | |||
4476 | A recipe that just generates scripts can enable "all" architecture | ||
4477 | because there are no binaries to build. To specifically enable "all" | ||
4478 | architecture, be sure your recipe inherits the | ||
4479 | :ref:`allarch <ref-classes-allarch>` class. | ||
4480 | This class is useful for "all" architectures because it configures | ||
4481 | many variables so packages can be used across multiple architectures. | ||
4482 | |||
4483 | If your recipe needs to generate packages that are machine-specific | ||
4484 | or when one of the build or runtime dependencies is already | ||
4485 | machine-architecture dependent, which makes your recipe also | ||
4486 | machine-architecture dependent, make sure your recipe enables the | ||
4487 | "machine" package architecture through the | ||
4488 | :term:`MACHINE_ARCH` | ||
4489 | variable: | ||
4490 | :: | ||
4491 | |||
4492 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
4493 | |||
4494 | When you do not | ||
4495 | specifically enable a package architecture through the | ||
4496 | :term:`PACKAGE_ARCH`, The | ||
4497 | OpenEmbedded build system defaults to the | ||
4498 | :term:`TUNE_PKGARCH` setting: | ||
4499 | :: | ||
4500 | |||
4501 | PACKAGE_ARCH = "${TUNE_PKGARCH}" | ||
4502 | |||
4503 | - *Choose a Generic Tuning File if Possible:* Some tunes are more | ||
4504 | generic and can run on multiple targets (e.g. an ``armv5`` set of | ||
4505 | packages could run on ``armv6`` and ``armv7`` processors in most | ||
4506 | cases). Similarly, ``i486`` binaries could work on ``i586`` and | ||
4507 | higher processors. You should realize, however, that advances on | ||
4508 | newer processor versions would not be used. | ||
4509 | |||
4510 | If you select the same tune for several different machines, the | ||
4511 | OpenEmbedded build system reuses software previously built, thus | ||
4512 | speeding up the overall build time. Realize that even though a new | ||
4513 | sysroot for each machine is generated, the software is not recompiled | ||
4514 | and only one package feed exists. | ||
4515 | |||
4516 | - *Manage Granular Level Packaging:* Sometimes cases exist where | ||
4517 | injecting another level of package architecture beyond the three | ||
4518 | higher levels noted earlier can be useful. For example, consider how | ||
4519 | NXP (formerly Freescale) allows for the easy reuse of binary packages | ||
4520 | in their layer | ||
4521 | :yocto_git:`meta-freescale </cgit/cgit.cgi/meta-freescale/>`. | ||
4522 | In this example, the | ||
4523 | :yocto_git:`fsl-dynamic-packagearch </cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` | ||
4524 | class shares GPU packages for i.MX53 boards because all boards share | ||
4525 | the AMD GPU. The i.MX6-based boards can do the same because all | ||
4526 | boards share the Vivante GPU. This class inspects the BitBake | ||
4527 | datastore to identify if the package provides or depends on one of | ||
4528 | the sub-architecture values. If so, the class sets the | ||
4529 | :term:`PACKAGE_ARCH` value | ||
4530 | based on the ``MACHINE_SUBARCH`` value. If the package does not | ||
4531 | provide or depend on one of the sub-architecture values but it | ||
4532 | matches a value in the machine-specific filter, it sets | ||
4533 | :term:`MACHINE_ARCH`. This | ||
4534 | behavior reduces the number of packages built and saves build time by | ||
4535 | reusing binaries. | ||
4536 | |||
4537 | - *Use Tools to Debug Issues:* Sometimes you can run into situations | ||
4538 | where software is being rebuilt when you think it should not be. For | ||
4539 | example, the OpenEmbedded build system might not be using shared | ||
4540 | state between machines when you think it should be. These types of | ||
4541 | situations are usually due to references to machine-specific | ||
4542 | variables such as :term:`MACHINE`, | ||
4543 | :term:`SERIAL_CONSOLES`, | ||
4544 | :term:`XSERVER`, | ||
4545 | :term:`MACHINE_FEATURES`, | ||
4546 | and so forth in code that is supposed to only be tune-specific or | ||
4547 | when the recipe depends | ||
4548 | (:term:`DEPENDS`, | ||
4549 | :term:`RDEPENDS`, | ||
4550 | :term:`RRECOMMENDS`, | ||
4551 | :term:`RSUGGESTS`, and so forth) | ||
4552 | on some other recipe that already has | ||
4553 | :term:`PACKAGE_ARCH` defined | ||
4554 | as "${MACHINE_ARCH}". | ||
4555 | |||
4556 | .. note:: | ||
4557 | |||
4558 | Patches to fix any issues identified are most welcome as these | ||
4559 | issues occasionally do occur. | ||
4560 | |||
4561 | For such cases, you can use some tools to help you sort out the | ||
4562 | situation: | ||
4563 | |||
4564 | - *sstate-diff-machines.sh:* You can find this tool in the | ||
4565 | ``scripts`` directory of the Source Repositories. See the comments | ||
4566 | in the script for information on how to use the tool. | ||
4567 | |||
4568 | - *BitBake's "-S printdiff" Option:* Using this option causes | ||
4569 | BitBake to try to establish the closest signature match it can | ||
4570 | (e.g. in the shared state cache) and then run ``bitbake-diffsigs`` | ||
4571 | over the matches to determine the stamps and delta where these two | ||
4572 | stamp trees diverge. | ||
4573 | |||
4574 | Building Software from an External Source | ||
4575 | ----------------------------------------- | ||
4576 | |||
4577 | By default, the OpenEmbedded build system uses the | ||
4578 | :term:`Build Directory` when building source | ||
4579 | code. The build process involves fetching the source files, unpacking | ||
4580 | them, and then patching them if necessary before the build takes place. | ||
4581 | |||
4582 | Situations exist where you might want to build software from source | ||
4583 | files that are external to and thus outside of the OpenEmbedded build | ||
4584 | system. For example, suppose you have a project that includes a new BSP | ||
4585 | with a heavily customized kernel. And, you want to minimize exposing the | ||
4586 | build system to the development team so that they can focus on their | ||
4587 | project and maintain everyone's workflow as much as possible. In this | ||
4588 | case, you want a kernel source directory on the development machine | ||
4589 | where the development occurs. You want the recipe's | ||
4590 | :term:`SRC_URI` variable to point to | ||
4591 | the external directory and use it as is, not copy it. | ||
4592 | |||
4593 | To build from software that comes from an external source, all you need | ||
4594 | to do is inherit the | ||
4595 | :ref:`externalsrc <ref-classes-externalsrc>` class | ||
4596 | and then set the | ||
4597 | :term:`EXTERNALSRC` variable to | ||
4598 | point to your external source code. Here are the statements to put in | ||
4599 | your ``local.conf`` file: | ||
4600 | :: | ||
4601 | |||
4602 | INHERIT += "externalsrc" | ||
4603 | EXTERNALSRC_pn-myrecipe = "path-to-your-source-tree" | ||
4604 | |||
4605 | This next example shows how to accomplish the same thing by setting | ||
4606 | ``EXTERNALSRC`` in the recipe itself or in the recipe's append file: | ||
4607 | :: | ||
4608 | |||
4609 | EXTERNALSRC = "path" | ||
4610 | EXTERNALSRC_BUILD = "path" | ||
4611 | |||
4612 | .. note:: | ||
4613 | |||
4614 | In order for these settings to take effect, you must globally or | ||
4615 | locally inherit the | ||
4616 | externalsrc | ||
4617 | class. | ||
4618 | |||
4619 | By default, ``externalsrc.bbclass`` builds the source code in a | ||
4620 | directory separate from the external source directory as specified by | ||
4621 | :term:`EXTERNALSRC`. If you need | ||
4622 | to have the source built in the same directory in which it resides, or | ||
4623 | some other nominated directory, you can set | ||
4624 | :term:`EXTERNALSRC_BUILD` | ||
4625 | to point to that directory: | ||
4626 | :: | ||
4627 | |||
4628 | EXTERNALSRC_BUILD_pn-myrecipe = "path-to-your-source-tree" | ||
4629 | |||
4630 | Replicating a Build Offline | ||
4631 | --------------------------- | ||
4632 | |||
4633 | It can be useful to take a "snapshot" of upstream sources used in a | ||
4634 | build and then use that "snapshot" later to replicate the build offline. | ||
4635 | To do so, you need to first prepare and populate your downloads | ||
4636 | directory your "snapshot" of files. Once your downloads directory is | ||
4637 | ready, you can use it at any time and from any machine to replicate your | ||
4638 | build. | ||
4639 | |||
4640 | Follow these steps to populate your Downloads directory: | ||
4641 | |||
4642 | 1. *Create a Clean Downloads Directory:* Start with an empty downloads | ||
4643 | directory (:term:`DL_DIR`). You | ||
4644 | start with an empty downloads directory by either removing the files | ||
4645 | in the existing directory or by setting ``DL_DIR`` to point to either | ||
4646 | an empty location or one that does not yet exist. | ||
4647 | |||
4648 | 2. *Generate Tarballs of the Source Git Repositories:* Edit your | ||
4649 | ``local.conf`` configuration file as follows: | ||
4650 | :: | ||
4651 | |||
4652 | DL_DIR = "/home/your-download-dir/" | ||
4653 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
4654 | |||
4655 | During | ||
4656 | the fetch process in the next step, BitBake gathers the source files | ||
4657 | and creates tarballs in the directory pointed to by ``DL_DIR``. See | ||
4658 | the | ||
4659 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
4660 | variable for more information. | ||
4661 | |||
4662 | 3. *Populate Your Downloads Directory Without Building:* Use BitBake to | ||
4663 | fetch your sources but inhibit the build: | ||
4664 | :: | ||
4665 | |||
4666 | $ bitbake target --runonly=fetch | ||
4667 | |||
4668 | The downloads directory (i.e. ``${DL_DIR}``) now has | ||
4669 | a "snapshot" of the source files in the form of tarballs, which can | ||
4670 | be used for the build. | ||
4671 | |||
4672 | 4. *Optionally Remove Any Git or other SCM Subdirectories From the | ||
4673 | Downloads Directory:* If you want, you can clean up your downloads | ||
4674 | directory by removing any Git or other Source Control Management | ||
4675 | (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs | ||
4676 | already contain these subdirectories. | ||
4677 | |||
4678 | Once your downloads directory has everything it needs regarding source | ||
4679 | files, you can create your "own-mirror" and build your target. | ||
4680 | Understand that you can use the files to build the target offline from | ||
4681 | any machine and at any time. | ||
4682 | |||
4683 | Follow these steps to build your target using the files in the downloads | ||
4684 | directory: | ||
4685 | |||
4686 | 1. *Using Local Files Only:* Inside your ``local.conf`` file, add the | ||
4687 | :term:`SOURCE_MIRROR_URL` | ||
4688 | variable, inherit the | ||
4689 | :ref:`own-mirrors <ref-classes-own-mirrors>` | ||
4690 | class, and use the | ||
4691 | :term:`bitbake:BB_NO_NETWORK` | ||
4692 | variable to your ``local.conf``. | ||
4693 | :: | ||
4694 | |||
4695 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | ||
4696 | INHERIT += "own-mirrors" | ||
4697 | BB_NO_NETWORK = "1" | ||
4698 | |||
4699 | The ``SOURCE_MIRROR_URL`` and ``own-mirror`` | ||
4700 | class set up the system to use the downloads directory as your "own | ||
4701 | mirror". Using the ``BB_NO_NETWORK`` variable makes sure that | ||
4702 | BitBake's fetching process in step 3 stays local, which means files | ||
4703 | from your "own-mirror" are used. | ||
4704 | |||
4705 | 2. *Start With a Clean Build:* You can start with a clean build by | ||
4706 | removing the | ||
4707 | ``${``\ :term:`TMPDIR`\ ``}`` | ||
4708 | directory or using a new :term:`Build Directory`. | ||
4709 | |||
4710 | 3. *Build Your Target:* Use BitBake to build your target: | ||
4711 | :: | ||
4712 | |||
4713 | $ bitbake target | ||
4714 | |||
4715 | The build completes using the known local "snapshot" of source | ||
4716 | files from your mirror. The resulting tarballs for your "snapshot" of | ||
4717 | source files are in the downloads directory. | ||
4718 | |||
4719 | .. note:: | ||
4720 | |||
4721 | The offline build does not work if recipes attempt to find the | ||
4722 | latest version of software by setting | ||
4723 | :term:`SRCREV` to | ||
4724 | ``${``\ :term:`AUTOREV`\ ``}``: | ||
4725 | SRCREV = "${AUTOREV}" When a recipe sets ``SRCREV`` to | ||
4726 | ``${AUTOREV}``, the build system accesses the network in an | ||
4727 | attempt to determine the latest version of software from the SCM. | ||
4728 | Typically, recipes that use ``AUTOREV`` are custom or modified | ||
4729 | recipes. Recipes that reside in public repositories usually do not | ||
4730 | use ``AUTOREV``. | ||
4731 | |||
4732 | If you do have recipes that use ``AUTOREV``, you can take steps to | ||
4733 | still use the recipes in an offline build. Do the following: | ||
4734 | |||
4735 | 1. Use a configuration generated by enabling `build | ||
4736 | history <#maintaining-build-output-quality>`__. | ||
4737 | |||
4738 | 2. Use the ``buildhistory-collect-srcrevs`` command to collect the | ||
4739 | stored ``SRCREV`` values from the build's history. For more | ||
4740 | information on collecting these values, see the "`Build History | ||
4741 | Package Information <#build-history-package-information>`__" | ||
4742 | section. | ||
4743 | |||
4744 | 3. Once you have the correct source revisions, you can modify | ||
4745 | those recipes to to set ``SRCREV`` to specific versions of the | ||
4746 | software. | ||
4747 | |||
4748 | Speeding Up a Build | ||
4749 | =================== | ||
4750 | |||
4751 | Build time can be an issue. By default, the build system uses simple | ||
4752 | controls to try and maximize build efficiency. In general, the default | ||
4753 | settings for all the following variables result in the most efficient | ||
4754 | build times when dealing with single socket systems (i.e. a single CPU). | ||
4755 | If you have multiple CPUs, you might try increasing the default values | ||
4756 | to gain more speed. See the descriptions in the glossary for each | ||
4757 | variable for more information: | ||
4758 | |||
4759 | - :term:`BB_NUMBER_THREADS`: | ||
4760 | The maximum number of threads BitBake simultaneously executes. | ||
4761 | |||
4762 | - :term:`bitbake:BB_NUMBER_PARSE_THREADS`: | ||
4763 | The number of threads BitBake uses during parsing. | ||
4764 | |||
4765 | - :term:`PARALLEL_MAKE`: Extra | ||
4766 | options passed to the ``make`` command during the | ||
4767 | :ref:`ref-tasks-compile` task in | ||
4768 | order to specify parallel compilation on the local build host. | ||
4769 | |||
4770 | - :term:`PARALLEL_MAKEINST`: | ||
4771 | Extra options passed to the ``make`` command during the | ||
4772 | :ref:`ref-tasks-install` task in | ||
4773 | order to specify parallel installation on the local build host. | ||
4774 | |||
4775 | As mentioned, these variables all scale to the number of processor cores | ||
4776 | available on the build system. For single socket systems, this | ||
4777 | auto-scaling ensures that the build system fundamentally takes advantage | ||
4778 | of potential parallel operations during the build based on the build | ||
4779 | machine's capabilities. | ||
4780 | |||
4781 | Following are additional factors that can affect build speed: | ||
4782 | |||
4783 | - File system type: The file system type that the build is being | ||
4784 | performed on can also influence performance. Using ``ext4`` is | ||
4785 | recommended as compared to ``ext2`` and ``ext3`` due to ``ext4`` | ||
4786 | improved features such as extents. | ||
4787 | |||
4788 | - Disabling the updating of access time using ``noatime``: The | ||
4789 | ``noatime`` mount option prevents the build system from updating file | ||
4790 | and directory access times. | ||
4791 | |||
4792 | - Setting a longer commit: Using the "commit=" mount option increases | ||
4793 | the interval in seconds between disk cache writes. Changing this | ||
4794 | interval from the five second default to something longer increases | ||
4795 | the risk of data loss but decreases the need to write to the disk, | ||
4796 | thus increasing the build performance. | ||
4797 | |||
4798 | - Choosing the packaging backend: Of the available packaging backends, | ||
4799 | IPK is the fastest. Additionally, selecting a singular packaging | ||
4800 | backend also helps. | ||
4801 | |||
4802 | - Using ``tmpfs`` for :term:`TMPDIR` | ||
4803 | as a temporary file system: While this can help speed up the build, | ||
4804 | the benefits are limited due to the compiler using ``-pipe``. The | ||
4805 | build system goes to some lengths to avoid ``sync()`` calls into the | ||
4806 | file system on the principle that if there was a significant failure, | ||
4807 | the :term:`Build Directory` | ||
4808 | contents could easily be rebuilt. | ||
4809 | |||
4810 | - Inheriting the | ||
4811 | :ref:`rm_work <ref-classes-rm-work>` class: | ||
4812 | Inheriting this class has shown to speed up builds due to | ||
4813 | significantly lower amounts of data stored in the data cache as well | ||
4814 | as on disk. Inheriting this class also makes cleanup of | ||
4815 | :term:`TMPDIR` faster, at the | ||
4816 | expense of being easily able to dive into the source code. File | ||
4817 | system maintainers have recommended that the fastest way to clean up | ||
4818 | large numbers of files is to reformat partitions rather than delete | ||
4819 | files due to the linear nature of partitions. This, of course, | ||
4820 | assumes you structure the disk partitions and file systems in a way | ||
4821 | that this is practical. | ||
4822 | |||
4823 | Aside from the previous list, you should keep some trade offs in mind | ||
4824 | that can help you speed up the build: | ||
4825 | |||
4826 | - Remove items from | ||
4827 | :term:`DISTRO_FEATURES` | ||
4828 | that you might not need. | ||
4829 | |||
4830 | - Exclude debug symbols and other debug information: If you do not need | ||
4831 | these symbols and other debug information, disabling the ``*-dbg`` | ||
4832 | package generation can speed up the build. You can disable this | ||
4833 | generation by setting the | ||
4834 | :term:`INHIBIT_PACKAGE_DEBUG_SPLIT` | ||
4835 | variable to "1". | ||
4836 | |||
4837 | - Disable static library generation for recipes derived from | ||
4838 | ``autoconf`` or ``libtool``: Following is an example showing how to | ||
4839 | disable static libraries and still provide an override to handle | ||
4840 | exceptions: | ||
4841 | :: | ||
4842 | |||
4843 | STATICLIBCONF = "--disable-static" | ||
4844 | STATICLIBCONF_sqlite3-native = "" | ||
4845 | EXTRA_OECONF += "${STATICLIBCONF}" | ||
4846 | |||
4847 | .. note:: | ||
4848 | |||
4849 | - Some recipes need static libraries in order to work correctly | ||
4850 | (e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides, | ||
4851 | as in the previous example, account for these kinds of | ||
4852 | exceptions. | ||
4853 | |||
4854 | - Some packages have packaging code that assumes the presence of | ||
4855 | the static libraries. If so, you might need to exclude them as | ||
4856 | well. | ||
4857 | |||
4858 | .. _platdev-working-with-libraries: | ||
4859 | |||
4860 | Working With Libraries | ||
4861 | ====================== | ||
4862 | |||
4863 | Libraries are an integral part of your system. This section describes | ||
4864 | some common practices you might find helpful when working with libraries | ||
4865 | to build your system: | ||
4866 | |||
4867 | - `How to include static library | ||
4868 | files <#including-static-library-files>`__ | ||
4869 | |||
4870 | - `How to use the Multilib feature to combine multiple versions of | ||
4871 | library files into a single | ||
4872 | image <#combining-multiple-versions-library-files-into-one-image>`__ | ||
4873 | |||
4874 | - `How to install multiple versions of the same library in parallel on | ||
4875 | the same | ||
4876 | system <#installing-multiple-versions-of-the-same-library>`__ | ||
4877 | |||
4878 | Including Static Library Files | ||
4879 | ------------------------------ | ||
4880 | |||
4881 | If you are building a library and the library offers static linking, you | ||
4882 | can control which static library files (``*.a`` files) get included in | ||
4883 | the built library. | ||
4884 | |||
4885 | The :term:`PACKAGES` and | ||
4886 | :term:`FILES_* <FILES>` variables in the | ||
4887 | ``meta/conf/bitbake.conf`` configuration file define how files installed | ||
4888 | by the ``do_install`` task are packaged. By default, the ``PACKAGES`` | ||
4889 | variable includes ``${PN}-staticdev``, which represents all static | ||
4890 | library files. | ||
4891 | |||
4892 | .. note:: | ||
4893 | |||
4894 | Some previously released versions of the Yocto Project defined the | ||
4895 | static library files through | ||
4896 | ${PN}-dev | ||
4897 | . | ||
4898 | |||
4899 | Following is part of the BitBake configuration file, where you can see | ||
4900 | how the static library files are defined: | ||
4901 | :: | ||
4902 | |||
4903 | PACKAGE_BEFORE_PN ?= "" | ||
4904 | PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" | ||
4905 | PACKAGES_DYNAMIC = "^${PN}-locale-.*" | ||
4906 | FILES = "" | ||
4907 | |||
4908 | FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \ | ||
4909 | ${sysconfdir} ${sharedstatedir} ${localstatedir} \ | ||
4910 | ${base_bindir}/* ${base_sbindir}/* \ | ||
4911 | ${base_libdir}/*${SOLIBS} \ | ||
4912 | ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \ | ||
4913 | ${datadir}/${BPN} ${libdir}/${BPN}/* \ | ||
4914 | ${datadir}/pixmaps ${datadir}/applications \ | ||
4915 | ${datadir}/idl ${datadir}/omf ${datadir}/sounds \ | ||
4916 | ${libdir}/bonobo/servers" | ||
4917 | |||
4918 | FILES_${PN}-bin = "${bindir}/* ${sbindir}/*" | ||
4919 | |||
4920 | FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \ | ||
4921 | ${datadir}/gnome/help" | ||
4922 | SECTION_${PN}-doc = "doc" | ||
4923 | |||
4924 | FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}" | ||
4925 | FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \ | ||
4926 | ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \ | ||
4927 | ${datadir}/aclocal ${base_libdir}/*.o \ | ||
4928 | ${libdir}/${BPN}/*.la ${base_libdir}/*.la" | ||
4929 | SECTION_${PN}-dev = "devel" | ||
4930 | ALLOW_EMPTY_${PN}-dev = "1" | ||
4931 | RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" | ||
4932 | |||
4933 | FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" | ||
4934 | SECTION_${PN}-staticdev = "devel" | ||
4935 | RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})" | ||
4936 | |||
4937 | .. _combining-multiple-versions-library-files-into-one-image: | ||
4938 | |||
4939 | Combining Multiple Versions of Library Files into One Image | ||
4940 | ----------------------------------------------------------- | ||
4941 | |||
4942 | The build system offers the ability to build libraries with different | ||
4943 | target optimizations or architecture formats and combine these together | ||
4944 | into one system image. You can link different binaries in the image | ||
4945 | against the different libraries as needed for specific use cases. This | ||
4946 | feature is called "Multilib." | ||
4947 | |||
4948 | An example would be where you have most of a system compiled in 32-bit | ||
4949 | mode using 32-bit libraries, but you have something large, like a | ||
4950 | database engine, that needs to be a 64-bit application and uses 64-bit | ||
4951 | libraries. Multilib allows you to get the best of both 32-bit and 64-bit | ||
4952 | libraries. | ||
4953 | |||
4954 | While the Multilib feature is most commonly used for 32 and 64-bit | ||
4955 | differences, the approach the build system uses facilitates different | ||
4956 | target optimizations. You could compile some binaries to use one set of | ||
4957 | libraries and other binaries to use a different set of libraries. The | ||
4958 | libraries could differ in architecture, compiler options, or other | ||
4959 | optimizations. | ||
4960 | |||
4961 | Several examples exist in the ``meta-skeleton`` layer found in the | ||
4962 | :term:`Source Directory`: | ||
4963 | |||
4964 | - ``conf/multilib-example.conf`` configuration file | ||
4965 | |||
4966 | - ``conf/multilib-example2.conf`` configuration file | ||
4967 | |||
4968 | - ``recipes-multilib/images/core-image-multilib-example.bb`` recipe | ||
4969 | |||
4970 | Preparing to Use Multilib | ||
4971 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
4972 | |||
4973 | User-specific requirements drive the Multilib feature. Consequently, | ||
4974 | there is no one "out-of-the-box" configuration that likely exists to | ||
4975 | meet your needs. | ||
4976 | |||
4977 | In order to enable Multilib, you first need to ensure your recipe is | ||
4978 | extended to support multiple libraries. Many standard recipes are | ||
4979 | already extended and support multiple libraries. You can check in the | ||
4980 | ``meta/conf/multilib.conf`` configuration file in the | ||
4981 | :term:`Source Directory` to see how this is | ||
4982 | done using the | ||
4983 | :term:`BBCLASSEXTEND` variable. | ||
4984 | Eventually, all recipes will be covered and this list will not be | ||
4985 | needed. | ||
4986 | |||
4987 | For the most part, the Multilib class extension works automatically to | ||
4988 | extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where | ||
4989 | ``MLPREFIX`` is the particular multilib (e.g. "lib32-" or "lib64-"). | ||
4990 | Standard variables such as | ||
4991 | :term:`DEPENDS`, | ||
4992 | :term:`RDEPENDS`, | ||
4993 | :term:`RPROVIDES`, | ||
4994 | :term:`RRECOMMENDS`, | ||
4995 | :term:`PACKAGES`, and | ||
4996 | :term:`PACKAGES_DYNAMIC` are | ||
4997 | automatically extended by the system. If you are extending any manual | ||
4998 | code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure | ||
4999 | those names are extended correctly. This automatic extension code | ||
5000 | resides in ``multilib.bbclass``. | ||
5001 | |||
5002 | Using Multilib | ||
5003 | ~~~~~~~~~~~~~~ | ||
5004 | |||
5005 | After you have set up the recipes, you need to define the actual | ||
5006 | combination of multiple libraries you want to build. You accomplish this | ||
5007 | through your ``local.conf`` configuration file in the | ||
5008 | :term:`Build Directory`. An example | ||
5009 | configuration would be as follows: | ||
5010 | :: | ||
5011 | |||
5012 | MACHINE = "qemux86-64" | ||
5013 | require conf/multilib.conf | ||
5014 | MULTILIBS = "multilib:lib32" | ||
5015 | DEFAULTTUNE_virtclass-multilib-lib32 = "x86" | ||
5016 | IMAGE_INSTALL_append = "lib32-glib-2.0" | ||
5017 | |||
5018 | This example enables an additional library named | ||
5019 | ``lib32`` alongside the normal target packages. When combining these | ||
5020 | "lib32" alternatives, the example uses "x86" for tuning. For information | ||
5021 | on this particular tuning, see | ||
5022 | ``meta/conf/machine/include/ia32/arch-ia32.inc``. | ||
5023 | |||
5024 | The example then includes ``lib32-glib-2.0`` in all the images, which | ||
5025 | illustrates one method of including a multiple library dependency. You | ||
5026 | can use a normal image build to include this dependency, for example: | ||
5027 | :: | ||
5028 | |||
5029 | $ bitbake core-image-sato | ||
5030 | |||
5031 | You can also build Multilib packages | ||
5032 | specifically with a command like this: | ||
5033 | :: | ||
5034 | |||
5035 | $ bitbake lib32-glib-2.0 | ||
5036 | |||
5037 | Additional Implementation Details | ||
5038 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5039 | |||
5040 | Generic implementation details as well as details that are specific to | ||
5041 | package management systems exist. Following are implementation details | ||
5042 | that exist regardless of the package management system: | ||
5043 | |||
5044 | - The typical convention used for the class extension code as used by | ||
5045 | Multilib assumes that all package names specified in | ||
5046 | :term:`PACKAGES` that contain | ||
5047 | ``${PN}`` have ``${PN}`` at the start of the name. When that | ||
5048 | convention is not followed and ``${PN}`` appears at the middle or the | ||
5049 | end of a name, problems occur. | ||
5050 | |||
5051 | - The :term:`TARGET_VENDOR` | ||
5052 | value under Multilib will be extended to "-vendormlmultilib" (e.g. | ||
5053 | "-pokymllib32" for a "lib32" Multilib with Poky). The reason for this | ||
5054 | slightly unwieldy contraction is that any "-" characters in the | ||
5055 | vendor string presently break Autoconf's ``config.sub``, and other | ||
5056 | separators are problematic for different reasons. | ||
5057 | |||
5058 | For the RPM Package Management System, the following implementation | ||
5059 | details exist: | ||
5060 | |||
5061 | - A unique architecture is defined for the Multilib packages, along | ||
5062 | with creating a unique deploy folder under ``tmp/deploy/rpm`` in the | ||
5063 | :term:`Build Directory`. For | ||
5064 | example, consider ``lib32`` in a ``qemux86-64`` image. The possible | ||
5065 | architectures in the system are "all", "qemux86_64", | ||
5066 | "lib32_qemux86_64", and "lib32_x86". | ||
5067 | |||
5068 | - The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM | ||
5069 | packaging. The naming for a normal RPM package and a Multilib RPM | ||
5070 | package in a ``qemux86-64`` system resolves to something similar to | ||
5071 | ``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``, | ||
5072 | respectively. | ||
5073 | |||
5074 | - When installing a Multilib image, the RPM backend first installs the | ||
5075 | base image and then installs the Multilib libraries. | ||
5076 | |||
5077 | - The build system relies on RPM to resolve the identical files in the | ||
5078 | two (or more) Multilib packages. | ||
5079 | |||
5080 | For the IPK Package Management System, the following implementation | ||
5081 | details exist: | ||
5082 | |||
5083 | - The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK | ||
5084 | packaging. The naming for a normal RPM package and a Multilib IPK | ||
5085 | package in a ``qemux86-64`` system resolves to something like | ||
5086 | ``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw_x86.ipk``, | ||
5087 | respectively. | ||
5088 | |||
5089 | - The IPK deploy folder is not modified with ``${MLPREFIX}`` because | ||
5090 | packages with and without the Multilib feature can exist in the same | ||
5091 | folder due to the ``${PN}`` differences. | ||
5092 | |||
5093 | - IPK defines a sanity check for Multilib installation using certain | ||
5094 | rules for file comparison, overridden, etc. | ||
5095 | |||
5096 | Installing Multiple Versions of the Same Library | ||
5097 | ------------------------------------------------ | ||
5098 | |||
5099 | Situations can exist where you need to install and use multiple versions | ||
5100 | of the same library on the same system at the same time. These | ||
5101 | situations almost always exist when a library API changes and you have | ||
5102 | multiple pieces of software that depend on the separate versions of the | ||
5103 | library. To accommodate these situations, you can install multiple | ||
5104 | versions of the same library in parallel on the same system. | ||
5105 | |||
5106 | The process is straightforward as long as the libraries use proper | ||
5107 | versioning. With properly versioned libraries, all you need to do to | ||
5108 | individually specify the libraries is create separate, appropriately | ||
5109 | named recipes where the :term:`PN` part of | ||
5110 | the name includes a portion that differentiates each library version | ||
5111 | (e.g.the major part of the version number). Thus, instead of having a | ||
5112 | single recipe that loads one version of a library (e.g. ``clutter``), | ||
5113 | you provide multiple recipes that result in different versions of the | ||
5114 | libraries you want. As an example, the following two recipes would allow | ||
5115 | the two separate versions of the ``clutter`` library to co-exist on the | ||
5116 | same system: | ||
5117 | :: | ||
5118 | |||
5119 | clutter-1.6_1.6.20.bb | ||
5120 | clutter-1.8_1.8.4.bb | ||
5121 | |||
5122 | Additionally, if | ||
5123 | you have other recipes that depend on a given library, you need to use | ||
5124 | the :term:`DEPENDS` variable to | ||
5125 | create the dependency. Continuing with the same example, if you want to | ||
5126 | have a recipe depend on the 1.8 version of the ``clutter`` library, use | ||
5127 | the following in your recipe: | ||
5128 | :: | ||
5129 | |||
5130 | DEPENDS = "clutter-1.8" | ||
5131 | |||
5132 | Using x32 psABI | ||
5133 | =============== | ||
5134 | |||
5135 | x32 processor-specific Application Binary Interface (`x32 | ||
5136 | psABI <https://software.intel.com/en-us/node/628948>`__) is a native | ||
5137 | 32-bit processor-specific ABI for Intel 64 (x86-64) architectures. An | ||
5138 | ABI defines the calling conventions between functions in a processing | ||
5139 | environment. The interface determines what registers are used and what | ||
5140 | the sizes are for various C data types. | ||
5141 | |||
5142 | Some processing environments prefer using 32-bit applications even when | ||
5143 | running on Intel 64-bit platforms. Consider the i386 psABI, which is a | ||
5144 | very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not | ||
5145 | provide efficient use and access of the Intel 64-bit processor | ||
5146 | resources, leaving the system underutilized. Now consider the x86_64 | ||
5147 | psABI. This ABI is newer and uses 64-bits for data sizes and program | ||
5148 | pointers. The extra bits increase the footprint size of the programs, | ||
5149 | libraries, and also increases the memory and file system size | ||
5150 | requirements. Executing under the x32 psABI enables user programs to | ||
5151 | utilize CPU and system resources more efficiently while keeping the | ||
5152 | memory footprint of the applications low. Extra bits are used for | ||
5153 | registers but not for addressing mechanisms. | ||
5154 | |||
5155 | The Yocto Project supports the final specifications of x32 psABI as | ||
5156 | follows: | ||
5157 | |||
5158 | - You can create packages and images in x32 psABI format on x86_64 | ||
5159 | architecture targets. | ||
5160 | |||
5161 | - You can successfully build recipes with the x32 toolchain. | ||
5162 | |||
5163 | - You can create and boot ``core-image-minimal`` and | ||
5164 | ``core-image-sato`` images. | ||
5165 | |||
5166 | - RPM Package Manager (RPM) support exists for x32 binaries. | ||
5167 | |||
5168 | - Support for large images exists. | ||
5169 | |||
5170 | To use the x32 psABI, you need to edit your ``conf/local.conf`` | ||
5171 | configuration file as follows: | ||
5172 | :: | ||
5173 | |||
5174 | MACHINE = "qemux86-64" | ||
5175 | DEFAULTTUNE = "x86-64-x32" | ||
5176 | baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE') \ | ||
5177 | or 'INVALID')) or 'lib'}" | ||
5178 | |||
5179 | Once you have set | ||
5180 | up your configuration file, use BitBake to build an image that supports | ||
5181 | the x32 psABI. Here is an example: | ||
5182 | :: | ||
5183 | |||
5184 | $ bitbake core-image-sato | ||
5185 | |||
5186 | Enabling GObject Introspection Support | ||
5187 | ====================================== | ||
5188 | |||
5189 | `GObject | ||
5190 | introspection <https://wiki.gnome.org/Projects/GObjectIntrospection>`__ | ||
5191 | is the standard mechanism for accessing GObject-based software from | ||
5192 | runtime environments. GObject is a feature of the GLib library that | ||
5193 | provides an object framework for the GNOME desktop and related software. | ||
5194 | GObject Introspection adds information to GObject that allows objects | ||
5195 | created within it to be represented across different programming | ||
5196 | languages. If you want to construct GStreamer pipelines using Python, or | ||
5197 | control UPnP infrastructure using Javascript and GUPnP, GObject | ||
5198 | introspection is the only way to do it. | ||
5199 | |||
5200 | This section describes the Yocto Project support for generating and | ||
5201 | packaging GObject introspection data. GObject introspection data is a | ||
5202 | description of the API provided by libraries built on top of GLib | ||
5203 | framework, and, in particular, that framework's GObject mechanism. | ||
5204 | GObject Introspection Repository (GIR) files go to ``-dev`` packages, | ||
5205 | ``typelib`` files go to main packages as they are packaged together with | ||
5206 | libraries that are introspected. | ||
5207 | |||
5208 | The data is generated when building such a library, by linking the | ||
5209 | library with a small executable binary that asks the library to describe | ||
5210 | itself, and then executing the binary and processing its output. | ||
5211 | |||
5212 | Generating this data in a cross-compilation environment is difficult | ||
5213 | because the library is produced for the target architecture, but its | ||
5214 | code needs to be executed on the build host. This problem is solved with | ||
5215 | the OpenEmbedded build system by running the code through QEMU, which | ||
5216 | allows precisely that. Unfortunately, QEMU does not always work | ||
5217 | perfectly as mentioned in the "`Known Issues <#known-issues>`__" | ||
5218 | section. | ||
5219 | |||
5220 | Enabling the Generation of Introspection Data | ||
5221 | --------------------------------------------- | ||
5222 | |||
5223 | Enabling the generation of introspection data (GIR files) in your | ||
5224 | library package involves the following: | ||
5225 | |||
5226 | 1. Inherit the | ||
5227 | :ref:`gobject-introspection <ref-classes-gobject-introspection>` | ||
5228 | class. | ||
5229 | |||
5230 | 2. Make sure introspection is not disabled anywhere in the recipe or | ||
5231 | from anything the recipe includes. Also, make sure that | ||
5232 | "gobject-introspection-data" is not in | ||
5233 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED` | ||
5234 | and that "qemu-usermode" is not in | ||
5235 | :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`. | ||
5236 | If either of these conditions exist, nothing will happen. | ||
5237 | |||
5238 | 3. Try to build the recipe. If you encounter build errors that look like | ||
5239 | something is unable to find ``.so`` libraries, check where these | ||
5240 | libraries are located in the source tree and add the following to the | ||
5241 | recipe: | ||
5242 | :: | ||
5243 | |||
5244 | GIR_EXTRA_LIBS_PATH = "${B}/something/.libs" | ||
5245 | |||
5246 | .. note:: | ||
5247 | |||
5248 | See recipes in the | ||
5249 | oe-core | ||
5250 | repository that use that | ||
5251 | GIR_EXTRA_LIBS_PATH | ||
5252 | variable as an example. | ||
5253 | |||
5254 | 4. Look for any other errors, which probably mean that introspection | ||
5255 | support in a package is not entirely standard, and thus breaks down | ||
5256 | in a cross-compilation environment. For such cases, custom-made fixes | ||
5257 | are needed. A good place to ask and receive help in these cases is | ||
5258 | the :ref:`Yocto Project mailing | ||
5259 | lists <resources-mailinglist>`. | ||
5260 | |||
5261 | .. note:: | ||
5262 | |||
5263 | Using a library that no longer builds against the latest Yocto | ||
5264 | Project release and prints introspection related errors is a good | ||
5265 | candidate for the previous procedure. | ||
5266 | |||
5267 | Disabling the Generation of Introspection Data | ||
5268 | ---------------------------------------------- | ||
5269 | |||
5270 | You might find that you do not want to generate introspection data. Or, | ||
5271 | perhaps QEMU does not work on your build host and target architecture | ||
5272 | combination. If so, you can use either of the following methods to | ||
5273 | disable GIR file generations: | ||
5274 | |||
5275 | - Add the following to your distro configuration: | ||
5276 | :: | ||
5277 | |||
5278 | DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data" | ||
5279 | |||
5280 | Adding this statement disables generating introspection data using | ||
5281 | QEMU but will still enable building introspection tools and libraries | ||
5282 | (i.e. building them does not require the use of QEMU). | ||
5283 | |||
5284 | - Add the following to your machine configuration: | ||
5285 | :: | ||
5286 | |||
5287 | MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode" | ||
5288 | |||
5289 | Adding this statement disables the use of QEMU when building packages for your | ||
5290 | machine. Currently, this feature is used only by introspection | ||
5291 | recipes and has the same effect as the previously described option. | ||
5292 | |||
5293 | .. note:: | ||
5294 | |||
5295 | Future releases of the Yocto Project might have other features | ||
5296 | affected by this option. | ||
5297 | |||
5298 | If you disable introspection data, you can still obtain it through other | ||
5299 | means such as copying the data from a suitable sysroot, or by generating | ||
5300 | it on the target hardware. The OpenEmbedded build system does not | ||
5301 | currently provide specific support for these techniques. | ||
5302 | |||
5303 | Testing that Introspection Works in an Image | ||
5304 | -------------------------------------------- | ||
5305 | |||
5306 | Use the following procedure to test if generating introspection data is | ||
5307 | working in an image: | ||
5308 | |||
5309 | 1. Make sure that "gobject-introspection-data" is not in | ||
5310 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED` | ||
5311 | and that "qemu-usermode" is not in | ||
5312 | :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`. | ||
5313 | |||
5314 | 2. Build ``core-image-sato``. | ||
5315 | |||
5316 | 3. Launch a Terminal and then start Python in the terminal. | ||
5317 | |||
5318 | 4. Enter the following in the terminal: | ||
5319 | :: | ||
5320 | |||
5321 | >>> from gi.repository import GLib | ||
5322 | >>> GLib.get_host_name() | ||
5323 | |||
5324 | 5. For something a little more advanced, enter the following see: | ||
5325 | http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html | ||
5326 | |||
5327 | Known Issues | ||
5328 | ------------ | ||
5329 | |||
5330 | The following know issues exist for GObject Introspection Support: | ||
5331 | |||
5332 | - ``qemu-ppc64`` immediately crashes. Consequently, you cannot build | ||
5333 | introspection data on that architecture. | ||
5334 | |||
5335 | - x32 is not supported by QEMU. Consequently, introspection data is | ||
5336 | disabled. | ||
5337 | |||
5338 | - musl causes transient GLib binaries to crash on assertion failures. | ||
5339 | Consequently, generating introspection data is disabled. | ||
5340 | |||
5341 | - Because QEMU is not able to run the binaries correctly, introspection | ||
5342 | is disabled for some specific packages under specific architectures | ||
5343 | (e.g. ``gcr``, ``libsecret``, and ``webkit``). | ||
5344 | |||
5345 | - QEMU usermode might not work properly when running 64-bit binaries | ||
5346 | under 32-bit host machines. In particular, "qemumips64" is known to | ||
5347 | not work under i686. | ||
5348 | |||
5349 | .. _dev-optionally-using-an-external-toolchain: | ||
5350 | |||
5351 | Optionally Using an External Toolchain | ||
5352 | ====================================== | ||
5353 | |||
5354 | You might want to use an external toolchain as part of your development. | ||
5355 | If this is the case, the fundamental steps you need to accomplish are as | ||
5356 | follows: | ||
5357 | |||
5358 | - Understand where the installed toolchain resides. For cases where you | ||
5359 | need to build the external toolchain, you would need to take separate | ||
5360 | steps to build and install the toolchain. | ||
5361 | |||
5362 | - Make sure you add the layer that contains the toolchain to your | ||
5363 | ``bblayers.conf`` file through the | ||
5364 | :term:`BBLAYERS` variable. | ||
5365 | |||
5366 | - Set the ``EXTERNAL_TOOLCHAIN`` variable in your ``local.conf`` file | ||
5367 | to the location in which you installed the toolchain. | ||
5368 | |||
5369 | A good example of an external toolchain used with the Yocto Project is | ||
5370 | Mentor Graphics Sourcery G++ Toolchain. You can see information on how | ||
5371 | to use that particular layer in the ``README`` file at | ||
5372 | http://github.com/MentorEmbedded/meta-sourcery/. You can find | ||
5373 | further information by reading about the | ||
5374 | :term:`TCMODE` variable in the Yocto | ||
5375 | Project Reference Manual's variable glossary. | ||
5376 | |||
5377 | Creating Partitioned Images Using Wic | ||
5378 | ===================================== | ||
5379 | |||
5380 | Creating an image for a particular hardware target using the | ||
5381 | OpenEmbedded build system does not necessarily mean you can boot that | ||
5382 | image as is on your device. Physical devices accept and boot images in | ||
5383 | various ways depending on the specifics of the device. Usually, | ||
5384 | information about the hardware can tell you what image format the device | ||
5385 | requires. Should your device require multiple partitions on an SD card, | ||
5386 | flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to | ||
5387 | create the properly partitioned image. | ||
5388 | |||
5389 | The ``wic`` command generates partitioned images from existing | ||
5390 | OpenEmbedded build artifacts. Image generation is driven by partitioning | ||
5391 | commands contained in an Openembedded kickstart file (``.wks``) | ||
5392 | specified either directly on the command line or as one of a selection | ||
5393 | of canned kickstart files as shown with the ``wic list images`` command | ||
5394 | in the "`Using an Existing Kickstart | ||
5395 | File <#using-a-provided-kickstart-file>`__" section. When you apply the | ||
5396 | command to a given set of build artifacts, the result is an image or set | ||
5397 | of images that can be directly written onto media and used on a | ||
5398 | particular system. | ||
5399 | |||
5400 | .. note:: | ||
5401 | |||
5402 | For a kickstart file reference, see the " | ||
5403 | OpenEmbedded Kickstart ( | ||
5404 | .wks | ||
5405 | ) Reference | ||
5406 | " Chapter in the Yocto Project Reference Manual. | ||
5407 | |||
5408 | The ``wic`` command and the infrastructure it is based on is by | ||
5409 | definition incomplete. The purpose of the command is to allow the | ||
5410 | generation of customized images, and as such, was designed to be | ||
5411 | completely extensible through a plugin interface. See the "`Using the | ||
5412 | Wic PlugIn Interface <#wic-using-the-wic-plugin-interface>`__" section | ||
5413 | for information on these plugins. | ||
5414 | |||
5415 | This section provides some background information on Wic, describes what | ||
5416 | you need to have in place to run the tool, provides instruction on how | ||
5417 | to use the Wic utility, provides information on using the Wic plugins | ||
5418 | interface, and provides several examples that show how to use Wic. | ||
5419 | |||
5420 | .. _wic-background: | ||
5421 | |||
5422 | Background | ||
5423 | ---------- | ||
5424 | |||
5425 | This section provides some background on the Wic utility. While none of | ||
5426 | this information is required to use Wic, you might find it interesting. | ||
5427 | |||
5428 | - The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The | ||
5429 | "oe" diphthong in "oeic" was promoted to the letter "w", because | ||
5430 | "oeic" is both difficult to remember and to pronounce. | ||
5431 | |||
5432 | - Wic is loosely based on the Meego Image Creator (``mic``) framework. | ||
5433 | The Wic implementation has been heavily modified to make direct use | ||
5434 | of OpenEmbedded build artifacts instead of package installation and | ||
5435 | configuration, which are already incorporated within the OpenEmbedded | ||
5436 | artifacts. | ||
5437 | |||
5438 | - Wic is a completely independent standalone utility that initially | ||
5439 | provides easier-to-use and more flexible replacements for an existing | ||
5440 | functionality in OE-Core's | ||
5441 | :ref:`image-live <ref-classes-image-live>` | ||
5442 | class. The difference between Wic and those examples is that with Wic | ||
5443 | the functionality of those scripts is implemented by a | ||
5444 | general-purpose partitioning language, which is based on Redhat | ||
5445 | kickstart syntax. | ||
5446 | |||
5447 | .. _wic-requirements: | ||
5448 | |||
5449 | Requirements | ||
5450 | ------------ | ||
5451 | |||
5452 | In order to use the Wic utility with the OpenEmbedded Build system, your | ||
5453 | system needs to meet the following requirements: | ||
5454 | |||
5455 | - The Linux distribution on your development host must support the | ||
5456 | Yocto Project. See the ":ref:`detailed-supported-distros`" | ||
5457 | section in the Yocto Project Reference Manual for the list of | ||
5458 | distributions that support the Yocto Project. | ||
5459 | |||
5460 | - The standard system utilities, such as ``cp``, must be installed on | ||
5461 | your development host system. | ||
5462 | |||
5463 | - You must have sourced the build environment setup script (i.e. | ||
5464 | :ref:`structure-core-script`) found in the | ||
5465 | :term:`Build Directory`. | ||
5466 | |||
5467 | - You need to have the build artifacts already available, which | ||
5468 | typically means that you must have already created an image using the | ||
5469 | Openembedded build system (e.g. ``core-image-minimal``). While it | ||
5470 | might seem redundant to generate an image in order to create an image | ||
5471 | using Wic, the current version of Wic requires the artifacts in the | ||
5472 | form generated by the OpenEmbedded build system. | ||
5473 | |||
5474 | - You must build several native tools, which are built to run on the | ||
5475 | build system: $ bitbake parted-native dosfstools-native mtools-native | ||
5476 | |||
5477 | - Include "wic" as part of the | ||
5478 | :term:`IMAGE_FSTYPES` | ||
5479 | variable. | ||
5480 | |||
5481 | - Include the name of the :ref:`wic kickstart file <openembedded-kickstart-wks-reference>` | ||
5482 | as part of the :term:`WKS_FILE` variable | ||
5483 | |||
5484 | .. _wic-getting-help: | ||
5485 | |||
5486 | Getting Help | ||
5487 | ------------ | ||
5488 | |||
5489 | You can get general help for the ``wic`` command by entering the ``wic`` | ||
5490 | command by itself or by entering the command with a help argument as | ||
5491 | follows: | ||
5492 | :: | ||
5493 | |||
5494 | $ wic -h | ||
5495 | $ wic --help | ||
5496 | $ wic help | ||
5497 | |||
5498 | Currently, Wic supports seven commands: ``cp``, ``create``, ``help``, | ||
5499 | ``list``, ``ls``, ``rm``, and ``write``. You can get help for all these | ||
5500 | commands except "help" by using the following form: | ||
5501 | :: | ||
5502 | |||
5503 | $ wic help command | ||
5504 | |||
5505 | For example, the following command returns help for the ``write`` | ||
5506 | command: | ||
5507 | :: | ||
5508 | |||
5509 | $ wic help write | ||
5510 | |||
5511 | Wic supports help for three topics: ``overview``, ``plugins``, and | ||
5512 | ``kickstart``. You can get help for any topic using the following form: | ||
5513 | :: | ||
5514 | |||
5515 | $ wic help topic | ||
5516 | |||
5517 | For example, the following returns overview help for Wic: | ||
5518 | :: | ||
5519 | |||
5520 | $ wic help overview | ||
5521 | |||
5522 | One additional level of help exists for Wic. You can get help on | ||
5523 | individual images through the ``list`` command. You can use the ``list`` | ||
5524 | command to return the available Wic images as follows: | ||
5525 | :: | ||
5526 | |||
5527 | $ wic list images | ||
5528 | genericx86 Create an EFI disk image for genericx86* | ||
5529 | beaglebone-yocto Create SD card image for Beaglebone | ||
5530 | edgerouter Create SD card image for Edgerouter | ||
5531 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | ||
5532 | directdisk-gpt Create a 'pcbios' direct disk image | ||
5533 | mkefidisk Create an EFI disk image | ||
5534 | directdisk Create a 'pcbios' direct disk image | ||
5535 | systemd-bootdisk Create an EFI disk image with systemd-boot | ||
5536 | mkhybridiso Create a hybrid ISO image | ||
5537 | sdimage-bootpart Create SD card image with a boot partition | ||
5538 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | ||
5539 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | ||
5540 | |||
5541 | Once you know the list of available | ||
5542 | Wic images, you can use ``help`` with the command to get help on a | ||
5543 | particular image. For example, the following command returns help on the | ||
5544 | "beaglebone-yocto" image: | ||
5545 | :: | ||
5546 | |||
5547 | $ wic list beaglebone-yocto help | ||
5548 | |||
5549 | Creates a partitioned SD card image for Beaglebone. | ||
5550 | Boot files are located in the first vfat partition. | ||
5551 | |||
5552 | Operational Modes | ||
5553 | ----------------- | ||
5554 | |||
5555 | You can use Wic in two different modes, depending on how much control | ||
5556 | you need for specifying the Openembedded build artifacts that are used | ||
5557 | for creating the image: Raw and Cooked: | ||
5558 | |||
5559 | - *Raw Mode:* You explicitly specify build artifacts through Wic | ||
5560 | command-line arguments. | ||
5561 | |||
5562 | - *Cooked Mode:* The current | ||
5563 | :term:`MACHINE` setting and image | ||
5564 | name are used to automatically locate and provide the build | ||
5565 | artifacts. You just supply a kickstart file and the name of the image | ||
5566 | from which to use artifacts. | ||
5567 | |||
5568 | Regardless of the mode you use, you need to have the build artifacts | ||
5569 | ready and available. | ||
5570 | |||
5571 | Raw Mode | ||
5572 | ~~~~~~~~ | ||
5573 | |||
5574 | Running Wic in raw mode allows you to specify all the partitions through | ||
5575 | the ``wic`` command line. The primary use for raw mode is if you have | ||
5576 | built your kernel outside of the Yocto Project | ||
5577 | :term:`Build Directory`. In other words, you | ||
5578 | can point to arbitrary kernel, root filesystem locations, and so forth. | ||
5579 | Contrast this behavior with cooked mode where Wic looks in the Build | ||
5580 | Directory (e.g. ``tmp/deploy/images/``\ machine). | ||
5581 | |||
5582 | The general form of the ``wic`` command in raw mode is: | ||
5583 | :: | ||
5584 | |||
5585 | $ wic create wks_file options ... | ||
5586 | |||
5587 | Where: | ||
5588 | |||
5589 | wks_file: | ||
5590 | An OpenEmbedded kickstart file. You can provide | ||
5591 | your own custom file or use a file from a set of | ||
5592 | existing files as described by further options. | ||
5593 | |||
5594 | optional arguments: | ||
5595 | -h, --help show this help message and exit | ||
5596 | -o OUTDIR, --outdir OUTDIR | ||
5597 | name of directory to create image in | ||
5598 | -e IMAGE_NAME, --image-name IMAGE_NAME | ||
5599 | name of the image to use the artifacts from e.g. core- | ||
5600 | image-sato | ||
5601 | -r ROOTFS_DIR, --rootfs-dir ROOTFS_DIR | ||
5602 | path to the /rootfs dir to use as the .wks rootfs | ||
5603 | source | ||
5604 | -b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR | ||
5605 | path to the dir containing the boot artifacts (e.g. | ||
5606 | /EFI or /syslinux dirs) to use as the .wks bootimg | ||
5607 | source | ||
5608 | -k KERNEL_DIR, --kernel-dir KERNEL_DIR | ||
5609 | path to the dir containing the kernel to use in the | ||
5610 | .wks bootimg | ||
5611 | -n NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT | ||
5612 | path to the native sysroot containing the tools to use | ||
5613 | to build the image | ||
5614 | -s, --skip-build-check | ||
5615 | skip the build check | ||
5616 | -f, --build-rootfs build rootfs | ||
5617 | -c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz} | ||
5618 | compress image with specified compressor | ||
5619 | -m, --bmap generate .bmap | ||
5620 | --no-fstab-update Do not change fstab file. | ||
5621 | -v VARS_DIR, --vars VARS_DIR | ||
5622 | directory with <image>.env files that store bitbake | ||
5623 | variables | ||
5624 | -D, --debug output debug information | ||
5625 | |||
5626 | .. note:: | ||
5627 | |||
5628 | You do not need root privileges to run Wic. In fact, you should not | ||
5629 | run as root when using the utility. | ||
5630 | |||
5631 | Cooked Mode | ||
5632 | ~~~~~~~~~~~ | ||
5633 | |||
5634 | Running Wic in cooked mode leverages off artifacts in the Build | ||
5635 | Directory. In other words, you do not have to specify kernel or root | ||
5636 | filesystem locations as part of the command. All you need to provide is | ||
5637 | a kickstart file and the name of the image from which to use artifacts | ||
5638 | by using the "-e" option. Wic looks in the Build Directory (e.g. | ||
5639 | ``tmp/deploy/images/``\ machine) for artifacts. | ||
5640 | |||
5641 | The general form of the ``wic`` command using Cooked Mode is as follows: | ||
5642 | :: | ||
5643 | |||
5644 | $ wic create wks_file -e IMAGE_NAME | ||
5645 | |||
5646 | Where: | ||
5647 | |||
5648 | wks_file: | ||
5649 | An OpenEmbedded kickstart file. You can provide | ||
5650 | your own custom file or use a file from a set of | ||
5651 | existing files provided with the Yocto Project | ||
5652 | release. | ||
5653 | |||
5654 | required argument: | ||
5655 | -e IMAGE_NAME, --image-name IMAGE_NAME | ||
5656 | name of the image to use the artifacts from e.g. core- | ||
5657 | image-sato | ||
5658 | |||
5659 | .. _using-a-provided-kickstart-file: | ||
5660 | |||
5661 | Using an Existing Kickstart File | ||
5662 | -------------------------------- | ||
5663 | |||
5664 | If you do not want to create your own kickstart file, you can use an | ||
5665 | existing file provided by the Wic installation. As shipped, kickstart | ||
5666 | files can be found in the :ref:`overview-manual/overview-manual-development-environment:yocto project source repositories` in the | ||
5667 | following two locations: | ||
5668 | :: | ||
5669 | |||
5670 | poky/meta-yocto-bsp/wic | ||
5671 | poky/scripts/lib/wic/canned-wks | ||
5672 | |||
5673 | Use the following command to list the available kickstart files: | ||
5674 | :: | ||
5675 | |||
5676 | $ wic list images | ||
5677 | genericx86 Create an EFI disk image for genericx86* | ||
5678 | beaglebone-yocto Create SD card image for Beaglebone | ||
5679 | edgerouter Create SD card image for Edgerouter | ||
5680 | qemux86-directdisk Create a qemu machine 'pcbios' direct disk image | ||
5681 | directdisk-gpt Create a 'pcbios' direct disk image | ||
5682 | mkefidisk Create an EFI disk image | ||
5683 | directdisk Create a 'pcbios' direct disk image | ||
5684 | systemd-bootdisk Create an EFI disk image with systemd-boot | ||
5685 | mkhybridiso Create a hybrid ISO image | ||
5686 | sdimage-bootpart Create SD card image with a boot partition | ||
5687 | directdisk-multi-rootfs Create multi rootfs image using rootfs plugin | ||
5688 | directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config | ||
5689 | |||
5690 | When you use an existing file, you | ||
5691 | do not have to use the ``.wks`` extension. Here is an example in Raw | ||
5692 | Mode that uses the ``directdisk`` file: | ||
5693 | :: | ||
5694 | |||
5695 | $ wic create directdisk -r rootfs_dir -b bootimg_dir \ | ||
5696 | -k kernel_dir -n native_sysroot | ||
5697 | |||
5698 | Here are the actual partition language commands used in the | ||
5699 | ``genericx86.wks`` file to generate an image: | ||
5700 | :: | ||
5701 | |||
5702 | # short-description: Create an EFI disk image for genericx86* | ||
5703 | # long-description: Creates a partitioned EFI disk image for genericx86* machines | ||
5704 | part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 | ||
5705 | part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid | ||
5706 | part swap --ondisk sda --size 44 --label swap1 --fstype=swap | ||
5707 | |||
5708 | bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0" | ||
5709 | |||
5710 | .. _wic-using-the-wic-plugin-interface: | ||
5711 | |||
5712 | Using the Wic Plugin Interface | ||
5713 | ------------------------------ | ||
5714 | |||
5715 | You can extend and specialize Wic functionality by using Wic plugins. | ||
5716 | This section explains the Wic plugin interface. | ||
5717 | |||
5718 | .. note:: | ||
5719 | |||
5720 | Wic plugins consist of "source" and "imager" plugins. Imager plugins | ||
5721 | are beyond the scope of this section. | ||
5722 | |||
5723 | Source plugins provide a mechanism to customize partition content during | ||
5724 | the Wic image generation process. You can use source plugins to map | ||
5725 | values that you specify using ``--source`` commands in kickstart files | ||
5726 | (i.e. ``*.wks``) to a plugin implementation used to populate a given | ||
5727 | partition. | ||
5728 | |||
5729 | .. note:: | ||
5730 | |||
5731 | If you use plugins that have build-time dependencies (e.g. native | ||
5732 | tools, bootloaders, and so forth) when building a Wic image, you need | ||
5733 | to specify those dependencies using the | ||
5734 | WKS_FILE_DEPENDS | ||
5735 | variable. | ||
5736 | |||
5737 | Source plugins are subclasses defined in plugin files. As shipped, the | ||
5738 | Yocto Project provides several plugin files. You can see the source | ||
5739 | plugin files that ship with the Yocto Project | ||
5740 | :yocto_git:`here </cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source>`. | ||
5741 | Each of these plugin files contains source plugins that are designed to | ||
5742 | populate a specific Wic image partition. | ||
5743 | |||
5744 | Source plugins are subclasses of the ``SourcePlugin`` class, which is | ||
5745 | defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example, | ||
5746 | the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py`` | ||
5747 | file is a subclass of the ``SourcePlugin`` class, which is found in the | ||
5748 | ``pluginbase.py`` file. | ||
5749 | |||
5750 | You can also implement source plugins in a layer outside of the Source | ||
5751 | Repositories (external layer). To do so, be sure that your plugin files | ||
5752 | are located in a directory whose path is | ||
5753 | ``scripts/lib/wic/plugins/source/`` within your external layer. When the | ||
5754 | plugin files are located there, the source plugins they contain are made | ||
5755 | available to Wic. | ||
5756 | |||
5757 | When the Wic implementation needs to invoke a partition-specific | ||
5758 | implementation, it looks for the plugin with the same name as the | ||
5759 | ``--source`` parameter used in the kickstart file given to that | ||
5760 | partition. For example, if the partition is set up using the following | ||
5761 | command in a kickstart file: | ||
5762 | :: | ||
5763 | |||
5764 | part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 | ||
5765 | |||
5766 | The methods defined as class | ||
5767 | members of the matching source plugin (i.e. ``bootimg-pcbios``) in the | ||
5768 | ``bootimg-pcbios.py`` plugin file are used. | ||
5769 | |||
5770 | To be more concrete, here is the corresponding plugin definition from | ||
5771 | the ``bootimg-pcbios.py`` file for the previous command along with an | ||
5772 | example method called by the Wic implementation when it needs to prepare | ||
5773 | a partition using an implementation-specific function: | ||
5774 | :: | ||
5775 | |||
5776 | . | ||
5777 | . | ||
5778 | . | ||
5779 | class BootimgPcbiosPlugin(SourcePlugin): | ||
5780 | """ | ||
5781 | Create MBR boot partition and install syslinux on it. | ||
5782 | """ | ||
5783 | |||
5784 | name = 'bootimg-pcbios' | ||
5785 | . | ||
5786 | . | ||
5787 | . | ||
5788 | @classmethod | ||
5789 | def do_prepare_partition(cls, part, source_params, creator, cr_workdir, | ||
5790 | oe_builddir, bootimg_dir, kernel_dir, | ||
5791 | rootfs_dir, native_sysroot): | ||
5792 | """ | ||
5793 | Called to do the actual content population for a partition i.e. it | ||
5794 | 'prepares' the partition to be incorporated into the image. | ||
5795 | In this case, prepare content for legacy bios boot partition. | ||
5796 | """ | ||
5797 | . | ||
5798 | . | ||
5799 | . | ||
5800 | |||
5801 | If a | ||
5802 | subclass (plugin) itself does not implement a particular function, Wic | ||
5803 | locates and uses the default version in the superclass. It is for this | ||
5804 | reason that all source plugins are derived from the ``SourcePlugin`` | ||
5805 | class. | ||
5806 | |||
5807 | The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines | ||
5808 | a set of methods that source plugins can implement or override. Any | ||
5809 | plugins (subclass of ``SourcePlugin``) that do not implement a | ||
5810 | particular method inherit the implementation of the method from the | ||
5811 | ``SourcePlugin`` class. For more information, see the ``SourcePlugin`` | ||
5812 | class in the ``pluginbase.py`` file for details: | ||
5813 | |||
5814 | The following list describes the methods implemented in the | ||
5815 | ``SourcePlugin`` class: | ||
5816 | |||
5817 | - ``do_prepare_partition()``: Called to populate a partition with | ||
5818 | actual content. In other words, the method prepares the final | ||
5819 | partition image that is incorporated into the disk image. | ||
5820 | |||
5821 | - ``do_configure_partition()``: Called before | ||
5822 | ``do_prepare_partition()`` to create custom configuration files for a | ||
5823 | partition (e.g. syslinux or grub configuration files). | ||
5824 | |||
5825 | - ``do_install_disk()``: Called after all partitions have been | ||
5826 | prepared and assembled into a disk image. This method provides a hook | ||
5827 | to allow finalization of a disk image (e.g. writing an MBR). | ||
5828 | |||
5829 | - ``do_stage_partition()``: Special content-staging hook called | ||
5830 | before ``do_prepare_partition()``. This method is normally empty. | ||
5831 | |||
5832 | Typically, a partition just uses the passed-in parameters (e.g. the | ||
5833 | unmodified value of ``bootimg_dir``). However, in some cases, things | ||
5834 | might need to be more tailored. As an example, certain files might | ||
5835 | additionally need to be taken from ``bootimg_dir + /boot``. This hook | ||
5836 | allows those files to be staged in a customized fashion. | ||
5837 | |||
5838 | .. note:: | ||
5839 | |||
5840 | get_bitbake_var() | ||
5841 | allows you to access non-standard variables that you might want to | ||
5842 | use for this behavior. | ||
5843 | |||
5844 | You can extend the source plugin mechanism. To add more hooks, create | ||
5845 | more source plugin methods within ``SourcePlugin`` and the corresponding | ||
5846 | derived subclasses. The code that calls the plugin methods uses the | ||
5847 | ``plugin.get_source_plugin_methods()`` function to find the method or | ||
5848 | methods needed by the call. Retrieval of those methods is accomplished | ||
5849 | by filling up a dict with keys that contain the method names of | ||
5850 | interest. On success, these will be filled in with the actual methods. | ||
5851 | See the Wic implementation for examples and details. | ||
5852 | |||
5853 | .. _wic-usage-examples: | ||
5854 | |||
5855 | Wic Examples | ||
5856 | ------------ | ||
5857 | |||
5858 | This section provides several examples that show how to use the Wic | ||
5859 | utility. All the examples assume the list of requirements in the | ||
5860 | "`Requirements <#wic-requirements>`__" section have been met. The | ||
5861 | examples assume the previously generated image is | ||
5862 | ``core-image-minimal``. | ||
5863 | |||
5864 | .. _generate-an-image-using-a-provided-kickstart-file: | ||
5865 | |||
5866 | Generate an Image using an Existing Kickstart File | ||
5867 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5868 | |||
5869 | This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart | ||
5870 | file: | ||
5871 | :: | ||
5872 | |||
5873 | $ wic create mkefidisk -e core-image-minimal | ||
5874 | INFO: Building wic-tools... | ||
5875 | . | ||
5876 | . | ||
5877 | . | ||
5878 | INFO: The new image(s) can be found here: | ||
5879 | ./mkefidisk-201804191017-sda.direct | ||
5880 | |||
5881 | The following build artifacts were used to create the image(s): | ||
5882 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
5883 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
5884 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
5885 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5886 | |||
5887 | INFO: The image(s) were created using OE kickstart file: | ||
5888 | /home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks | ||
5889 | |||
5890 | The previous example shows the easiest way to create an image by running | ||
5891 | in cooked mode and supplying a kickstart file and the "-e" option to | ||
5892 | point to the existing build artifacts. Your ``local.conf`` file needs to | ||
5893 | have the :term:`MACHINE` variable set | ||
5894 | to the machine you are using, which is "qemux86" in this example. | ||
5895 | |||
5896 | Once the image builds, the output provides image location, artifact use, | ||
5897 | and kickstart file information. | ||
5898 | |||
5899 | .. note:: | ||
5900 | |||
5901 | You should always verify the details provided in the output to make | ||
5902 | sure that the image was indeed created exactly as expected. | ||
5903 | |||
5904 | Continuing with the example, you can now write the image from the Build | ||
5905 | Directory onto a USB stick, or whatever media for which you built your | ||
5906 | image, and boot from the media. You can write the image by using | ||
5907 | ``bmaptool`` or ``dd``: | ||
5908 | :: | ||
5909 | |||
5910 | $ oe-run-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sdX | ||
5911 | |||
5912 | or :: | ||
5913 | |||
5914 | $ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sdX | ||
5915 | |||
5916 | .. note:: | ||
5917 | |||
5918 | For more information on how to use the | ||
5919 | bmaptool | ||
5920 | to flash a device with an image, see the " | ||
5921 | Flashing Images Using | ||
5922 | bmaptool | ||
5923 | " section. | ||
5924 | |||
5925 | Using a Modified Kickstart File | ||
5926 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
5927 | |||
5928 | Because partitioned image creation is driven by the kickstart file, it | ||
5929 | is easy to affect image creation by changing the parameters in the file. | ||
5930 | This next example demonstrates that through modification of the | ||
5931 | ``directdisk-gpt`` kickstart file. | ||
5932 | |||
5933 | As mentioned earlier, you can use the command ``wic list images`` to | ||
5934 | show the list of existing kickstart files. The directory in which the | ||
5935 | ``directdisk-gpt.wks`` file resides is | ||
5936 | ``scripts/lib/image/canned-wks/``, which is located in the | ||
5937 | :term:`Source Directory` (e.g. ``poky``). | ||
5938 | Because available files reside in this directory, you can create and add | ||
5939 | your own custom files to the directory. Subsequent use of the | ||
5940 | ``wic list images`` command would then include your kickstart files. | ||
5941 | |||
5942 | In this example, the existing ``directdisk-gpt`` file already does most | ||
5943 | of what is needed. However, for the hardware in this example, the image | ||
5944 | will need to boot from ``sdb`` instead of ``sda``, which is what the | ||
5945 | ``directdisk-gpt`` kickstart file uses. | ||
5946 | |||
5947 | The example begins by making a copy of the ``directdisk-gpt.wks`` file | ||
5948 | in the ``scripts/lib/image/canned-wks`` directory and then by changing | ||
5949 | the lines that specify the target disk from which to boot. | ||
5950 | :: | ||
5951 | |||
5952 | $ cp /home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \ | ||
5953 | /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks | ||
5954 | |||
5955 | Next, the example modifies the ``directdisksdb-gpt.wks`` file and | ||
5956 | changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The | ||
5957 | example changes the following two lines and leaves the remaining lines | ||
5958 | untouched: | ||
5959 | :: | ||
5960 | |||
5961 | part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024 | ||
5962 | part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid | ||
5963 | |||
5964 | Once the lines are changed, the | ||
5965 | example generates the ``directdisksdb-gpt`` image. The command points | ||
5966 | the process at the ``core-image-minimal`` artifacts for the Next Unit of | ||
5967 | Computing (nuc) :term:`MACHINE` the | ||
5968 | ``local.conf``. | ||
5969 | :: | ||
5970 | |||
5971 | $ wic create directdisksdb-gpt -e core-image-minimal | ||
5972 | INFO: Building wic-tools... | ||
5973 | . | ||
5974 | . | ||
5975 | . | ||
5976 | Initialising tasks: 100% |#######################################| Time: 0:00:01 | ||
5977 | NOTE: Executing SetScene Tasks | ||
5978 | NOTE: Executing RunQueue Tasks | ||
5979 | NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded. | ||
5980 | INFO: Creating image(s)... | ||
5981 | |||
5982 | INFO: The new image(s) can be found here: | ||
5983 | ./directdisksdb-gpt-201710090938-sdb.direct | ||
5984 | |||
5985 | The following build artifacts were used to create the image(s): | ||
5986 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
5987 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
5988 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
5989 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
5990 | |||
5991 | INFO: The image(s) were created using OE kickstart file: | ||
5992 | /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks | ||
5993 | |||
5994 | Continuing with the example, you can now directly ``dd`` the image to a | ||
5995 | USB stick, or whatever media for which you built your image, and boot | ||
5996 | the resulting media: | ||
5997 | :: | ||
5998 | |||
5999 | $ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb | ||
6000 | 140966+0 records in | ||
6001 | 140966+0 records out | ||
6002 | 72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s | ||
6003 | $ sudo eject /dev/sdb | ||
6004 | |||
6005 | Using a Modified Kickstart File and Running in Raw Mode | ||
6006 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6007 | |||
6008 | This next example manually specifies each build artifact (runs in Raw | ||
6009 | Mode) and uses a modified kickstart file. The example also uses the | ||
6010 | ``-o`` option to cause Wic to create the output somewhere other than the | ||
6011 | default output directory, which is the current directory: | ||
6012 | :: | ||
6013 | |||
6014 | $ wic create /home/stephano/my_yocto/test.wks -o /home/stephano/testwic \ | ||
6015 | --rootfs-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \ | ||
6016 | --bootimg-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \ | ||
6017 | --kernel-dir /home/stephano/build/master/build/tmp/deploy/images/qemux86 \ | ||
6018 | --native-sysroot /home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
6019 | |||
6020 | INFO: Creating image(s)... | ||
6021 | |||
6022 | INFO: The new image(s) can be found here: | ||
6023 | /home/stephano/testwic/test-201710091445-sdb.direct | ||
6024 | |||
6025 | The following build artifacts were used to create the image(s): | ||
6026 | ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs | ||
6027 | BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share | ||
6028 | KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 | ||
6029 | NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native | ||
6030 | |||
6031 | INFO: The image(s) were created using OE kickstart file: | ||
6032 | /home/stephano/my_yocto/test.wks | ||
6033 | |||
6034 | For this example, | ||
6035 | :term:`MACHINE` did not have to be | ||
6036 | specified in the ``local.conf`` file since the artifact is manually | ||
6037 | specified. | ||
6038 | |||
6039 | Using Wic to Manipulate an Image | ||
6040 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6041 | |||
6042 | Wic image manipulation allows you to shorten turnaround time during | ||
6043 | image development. For example, you can use Wic to delete the kernel | ||
6044 | partition of a Wic image and then insert a newly built kernel. This | ||
6045 | saves you time from having to rebuild the entire image each time you | ||
6046 | modify the kernel. | ||
6047 | |||
6048 | .. note:: | ||
6049 | |||
6050 | In order to use Wic to manipulate a Wic image as in this example, | ||
6051 | your development machine must have the | ||
6052 | mtools | ||
6053 | package installed. | ||
6054 | |||
6055 | The following example examines the contents of the Wic image, deletes | ||
6056 | the existing kernel, and then inserts a new kernel: | ||
6057 | |||
6058 | 1. *List the Partitions:* Use the ``wic ls`` command to list all the | ||
6059 | partitions in the Wic image: | ||
6060 | :: | ||
6061 | |||
6062 | $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic | ||
6063 | Num Start End Size Fstype | ||
6064 | 1 1048576 25041919 23993344 fat16 | ||
6065 | 2 25165824 72157183 46991360 ext4 | ||
6066 | |||
6067 | The previous output shows two partitions in the | ||
6068 | ``core-image-minimal-qemux86.wic`` image. | ||
6069 | |||
6070 | 2. *Examine a Particular Partition:* Use the ``wic ls`` command again | ||
6071 | but in a different form to examine a particular partition. | ||
6072 | |||
6073 | .. note:: | ||
6074 | |||
6075 | You can get command usage on any Wic command using the following | ||
6076 | form: | ||
6077 | :: | ||
6078 | |||
6079 | $ wic help command | ||
6080 | |||
6081 | |||
6082 | For example, the following command shows you the various ways to | ||
6083 | use the | ||
6084 | wic ls | ||
6085 | command: | ||
6086 | :: | ||
6087 | |||
6088 | $ wic help ls | ||
6089 | |||
6090 | |||
6091 | The following command shows what is in Partition one: | ||
6092 | :: | ||
6093 | |||
6094 | $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1 | ||
6095 | Volume in drive : is boot | ||
6096 | Volume Serial Number is E894-1809 | ||
6097 | Directory for ::/ | ||
6098 | |||
6099 | libcom32 c32 186500 2017-10-09 16:06 | ||
6100 | libutil c32 24148 2017-10-09 16:06 | ||
6101 | syslinux cfg 220 2017-10-09 16:06 | ||
6102 | vesamenu c32 27104 2017-10-09 16:06 | ||
6103 | vmlinuz 6904608 2017-10-09 16:06 | ||
6104 | 5 files 7 142 580 bytes | ||
6105 | 16 582 656 bytes free | ||
6106 | |||
6107 | The previous output shows five files, with the | ||
6108 | ``vmlinuz`` being the kernel. | ||
6109 | |||
6110 | .. note:: | ||
6111 | |||
6112 | If you see the following error, you need to update or create a | ||
6113 | ~/.mtoolsrc | ||
6114 | file and be sure to have the line "mtools_skip_check=1" in the | ||
6115 | file. Then, run the Wic command again: | ||
6116 | :: | ||
6117 | |||
6118 | ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0 | ||
6119 | output: Total number of sectors (47824) not a multiple of sectors per track (32)! | ||
6120 | Add mtools_skip_check=1 to your .mtoolsrc file to skip this test | ||
6121 | |||
6122 | |||
6123 | 3. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the | ||
6124 | ``vmlinuz`` file (kernel): | ||
6125 | :: | ||
6126 | |||
6127 | $ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz | ||
6128 | |||
6129 | 4. *Add In the New Kernel:* Use the ``wic cp`` command to add the | ||
6130 | updated kernel to the Wic image. Depending on how you built your | ||
6131 | kernel, it could be in different places. If you used ``devtool`` and | ||
6132 | an SDK to build your kernel, it resides in the ``tmp/work`` directory | ||
6133 | of the extensible SDK. If you used ``make`` to build the kernel, the | ||
6134 | kernel will be in the ``workspace/sources`` area. | ||
6135 | |||
6136 | The following example assumes ``devtool`` was used to build the | ||
6137 | kernel: | ||
6138 | :: | ||
6139 | |||
6140 | cp ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \ | ||
6141 | ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz | ||
6142 | |||
6143 | Once the new kernel is added back into the image, you can use the | ||
6144 | ``dd`` command or ```bmaptool`` <#flashing-images-using-bmaptool>`__ | ||
6145 | to flash your wic image onto an SD card or USB stick and test your | ||
6146 | target. | ||
6147 | |||
6148 | .. note:: | ||
6149 | |||
6150 | Using | ||
6151 | bmaptool | ||
6152 | is generally 10 to 20 times faster than using | ||
6153 | dd | ||
6154 | . | ||
6155 | |||
6156 | Flashing Images Using ``bmaptool`` | ||
6157 | ================================== | ||
6158 | |||
6159 | A fast and easy way to flash an image to a bootable device is to use | ||
6160 | Bmaptool, which is integrated into the OpenEmbedded build system. | ||
6161 | Bmaptool is a generic tool that creates a file's block map (bmap) and | ||
6162 | then uses that map to copy the file. As compared to traditional tools | ||
6163 | such as dd or cp, Bmaptool can copy (or flash) large files like raw | ||
6164 | system image files much faster. | ||
6165 | |||
6166 | .. note:: | ||
6167 | |||
6168 | - If you are using Ubuntu or Debian distributions, you can install | ||
6169 | the ``bmap-tools`` package using the following command and then | ||
6170 | use the tool without specifying ``PATH`` even from the root | ||
6171 | account: $ sudo apt-get install bmap-tools | ||
6172 | |||
6173 | - If you are unable to install the ``bmap-tools`` package, you will | ||
6174 | need to build Bmaptool before using it. Use the following command: | ||
6175 | $ bitbake bmap-tools-native | ||
6176 | |||
6177 | Following, is an example that shows how to flash a Wic image. Realize | ||
6178 | that while this example uses a Wic image, you can use Bmaptool to flash | ||
6179 | any type of image. Use these steps to flash an image using Bmaptool: | ||
6180 | |||
6181 | 1. *Update your local.conf File:* You need to have the following set | ||
6182 | in your ``local.conf`` file before building your image: | ||
6183 | :: | ||
6184 | |||
6185 | IMAGE_FSTYPES += "wic wic.bmap" | ||
6186 | |||
6187 | 2. *Get Your Image:* Either have your image ready (pre-built with the | ||
6188 | :term:`IMAGE_FSTYPES` | ||
6189 | setting previously mentioned) or take the step to build the image: | ||
6190 | :: | ||
6191 | |||
6192 | $ bitbake image | ||
6193 | |||
6194 | 3. *Flash the Device:* Flash the device with the image by using Bmaptool | ||
6195 | depending on your particular setup. The following commands assume the | ||
6196 | image resides in the Build Directory's ``deploy/images/`` area: | ||
6197 | |||
6198 | - If you have write access to the media, use this command form: | ||
6199 | :: | ||
6200 | |||
6201 | $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | ||
6202 | |||
6203 | - If you do not have write access to the media, set your permissions | ||
6204 | first and then use the same command form: | ||
6205 | :: | ||
6206 | |||
6207 | $ sudo chmod 666 /dev/sdX | ||
6208 | $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX | ||
6209 | |||
6210 | For help on the ``bmaptool`` command, use the following command: | ||
6211 | :: | ||
6212 | |||
6213 | $ bmaptool --help | ||
6214 | |||
6215 | Making Images More Secure | ||
6216 | ========================= | ||
6217 | |||
6218 | Security is of increasing concern for embedded devices. Consider the | ||
6219 | issues and problems discussed in just this sampling of work found across | ||
6220 | the Internet: | ||
6221 | |||
6222 | - *"*\ `Security Risks of Embedded | ||
6223 | Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"* | ||
6224 | by Bruce Schneier | ||
6225 | |||
6226 | - *"*\ `Internet Census | ||
6227 | 2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna | ||
6228 | Botnet | ||
6229 | |||
6230 | - *"*\ `Security Issues for Embedded | ||
6231 | Devices <http://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"* | ||
6232 | by Jake Edge | ||
6233 | |||
6234 | When securing your image is of concern, there are steps, tools, and | ||
6235 | variables that you can consider to help you reach the security goals you | ||
6236 | need for your particular device. Not all situations are identical when | ||
6237 | it comes to making an image secure. Consequently, this section provides | ||
6238 | some guidance and suggestions for consideration when you want to make | ||
6239 | your image more secure. | ||
6240 | |||
6241 | .. note:: | ||
6242 | |||
6243 | Because the security requirements and risks are different for every | ||
6244 | type of device, this section cannot provide a complete reference on | ||
6245 | securing your custom OS. It is strongly recommended that you also | ||
6246 | consult other sources of information on embedded Linux system | ||
6247 | hardening and on security. | ||
6248 | |||
6249 | General Considerations | ||
6250 | ---------------------- | ||
6251 | |||
6252 | General considerations exist that help you create more secure images. | ||
6253 | You should consider the following suggestions to help make your device | ||
6254 | more secure: | ||
6255 | |||
6256 | - Scan additional code you are adding to the system (e.g. application | ||
6257 | code) by using static analysis tools. Look for buffer overflows and | ||
6258 | other potential security problems. | ||
6259 | |||
6260 | - Pay particular attention to the security for any web-based | ||
6261 | administration interface. | ||
6262 | |||
6263 | Web interfaces typically need to perform administrative functions and | ||
6264 | tend to need to run with elevated privileges. Thus, the consequences | ||
6265 | resulting from the interface's security becoming compromised can be | ||
6266 | serious. Look for common web vulnerabilities such as | ||
6267 | cross-site-scripting (XSS), unvalidated inputs, and so forth. | ||
6268 | |||
6269 | As with system passwords, the default credentials for accessing a | ||
6270 | web-based interface should not be the same across all devices. This | ||
6271 | is particularly true if the interface is enabled by default as it can | ||
6272 | be assumed that many end-users will not change the credentials. | ||
6273 | |||
6274 | - Ensure you can update the software on the device to mitigate | ||
6275 | vulnerabilities discovered in the future. This consideration | ||
6276 | especially applies when your device is network-enabled. | ||
6277 | |||
6278 | - Ensure you remove or disable debugging functionality before producing | ||
6279 | the final image. For information on how to do this, see the | ||
6280 | "`Considerations Specific to the OpenEmbedded Build | ||
6281 | System <#considerations-specific-to-the-openembedded-build-system>`__" | ||
6282 | section. | ||
6283 | |||
6284 | - Ensure you have no network services listening that are not needed. | ||
6285 | |||
6286 | - Remove any software from the image that is not needed. | ||
6287 | |||
6288 | - Enable hardware support for secure boot functionality when your | ||
6289 | device supports this functionality. | ||
6290 | |||
6291 | Security Flags | ||
6292 | -------------- | ||
6293 | |||
6294 | The Yocto Project has security flags that you can enable that help make | ||
6295 | your build output more secure. The security flags are in the | ||
6296 | ``meta/conf/distro/include/security_flags.inc`` file in your | ||
6297 | :term:`Source Directory` (e.g. ``poky``). | ||
6298 | |||
6299 | .. note:: | ||
6300 | |||
6301 | Depending on the recipe, certain security flags are enabled and | ||
6302 | disabled by default. | ||
6303 | |||
6304 | Use the following line in your ``local.conf`` file or in your custom | ||
6305 | distribution configuration file to enable the security compiler and | ||
6306 | linker flags for your build: | ||
6307 | :: | ||
6308 | |||
6309 | require conf/distro/include/security_flags.inc | ||
6310 | |||
6311 | Considerations Specific to the OpenEmbedded Build System | ||
6312 | -------------------------------------------------------- | ||
6313 | |||
6314 | You can take some steps that are specific to the OpenEmbedded build | ||
6315 | system to make your images more secure: | ||
6316 | |||
6317 | - Ensure "debug-tweaks" is not one of your selected | ||
6318 | :term:`IMAGE_FEATURES`. | ||
6319 | When creating a new project, the default is to provide you with an | ||
6320 | initial ``local.conf`` file that enables this feature using the | ||
6321 | :term:`EXTRA_IMAGE_FEATURES` | ||
6322 | variable with the line: | ||
6323 | :: | ||
6324 | |||
6325 | EXTRA_IMAGE_FEATURES = "debug-tweaks" | ||
6326 | |||
6327 | To disable that feature, simply comment out that line in your | ||
6328 | ``local.conf`` file, or make sure ``IMAGE_FEATURES`` does not contain | ||
6329 | "debug-tweaks" before producing your final image. Among other things, | ||
6330 | leaving this in place sets the root password as blank, which makes | ||
6331 | logging in for debugging or inspection easy during development but | ||
6332 | also means anyone can easily log in during production. | ||
6333 | |||
6334 | - It is possible to set a root password for the image and also to set | ||
6335 | passwords for any extra users you might add (e.g. administrative or | ||
6336 | service type users). When you set up passwords for multiple images or | ||
6337 | users, you should not duplicate passwords. | ||
6338 | |||
6339 | To set up passwords, use the | ||
6340 | :ref:`extrausers <ref-classes-extrausers>` | ||
6341 | class, which is the preferred method. For an example on how to set up | ||
6342 | both root and user passwords, see the | ||
6343 | ":ref:`extrausers.bbclass <ref-classes-extrausers>`" | ||
6344 | section. | ||
6345 | |||
6346 | .. note:: | ||
6347 | |||
6348 | When adding extra user accounts or setting a root password, be | ||
6349 | cautious about setting the same password on every device. If you | ||
6350 | do this, and the password you have set is exposed, then every | ||
6351 | device is now potentially compromised. If you need this access but | ||
6352 | want to ensure security, consider setting a different, random | ||
6353 | password for each device. Typically, you do this as a separate | ||
6354 | step after you deploy the image onto the device. | ||
6355 | |||
6356 | - Consider enabling a Mandatory Access Control (MAC) framework such as | ||
6357 | SMACK or SELinux and tuning it appropriately for your device's usage. | ||
6358 | You can find more information in the | ||
6359 | `meta-selinux <http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/>`__ | ||
6360 | layer. | ||
6361 | |||
6362 | Tools for Hardening Your Image | ||
6363 | ------------------------------ | ||
6364 | |||
6365 | The Yocto Project provides tools for making your image more secure. You | ||
6366 | can find these tools in the ``meta-security`` layer of the | ||
6367 | :yocto_git:`Yocto Project Source Repositories <>`. | ||
6368 | |||
6369 | Creating Your Own Distribution | ||
6370 | ============================== | ||
6371 | |||
6372 | When you build an image using the Yocto Project and do not alter any | ||
6373 | distribution :term:`Metadata`, you are | ||
6374 | creating a Poky distribution. If you wish to gain more control over | ||
6375 | package alternative selections, compile-time options, and other | ||
6376 | low-level configurations, you can create your own distribution. | ||
6377 | |||
6378 | To create your own distribution, the basic steps consist of creating | ||
6379 | your own distribution layer, creating your own distribution | ||
6380 | configuration file, and then adding any needed code and Metadata to the | ||
6381 | layer. The following steps provide some more detail: | ||
6382 | |||
6383 | - *Create a layer for your new distro:* Create your distribution layer | ||
6384 | so that you can keep your Metadata and code for the distribution | ||
6385 | separate. It is strongly recommended that you create and use your own | ||
6386 | layer for configuration and code. Using your own layer as compared to | ||
6387 | just placing configurations in a ``local.conf`` configuration file | ||
6388 | makes it easier to reproduce the same build configuration when using | ||
6389 | multiple build machines. See the | ||
6390 | ":ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`" | ||
6391 | section for information on how to quickly set up a layer. | ||
6392 | |||
6393 | - *Create the distribution configuration file:* The distribution | ||
6394 | configuration file needs to be created in the ``conf/distro`` | ||
6395 | directory of your layer. You need to name it using your distribution | ||
6396 | name (e.g. ``mydistro.conf``). | ||
6397 | |||
6398 | .. note:: | ||
6399 | |||
6400 | The | ||
6401 | DISTRO | ||
6402 | variable in your | ||
6403 | local.conf | ||
6404 | file determines the name of your distribution. | ||
6405 | |||
6406 | You can split out parts of your configuration file into include files | ||
6407 | and then "require" them from within your distribution configuration | ||
6408 | file. Be sure to place the include files in the | ||
6409 | ``conf/distro/include`` directory of your layer. A common example | ||
6410 | usage of include files would be to separate out the selection of | ||
6411 | desired version and revisions for individual recipes. | ||
6412 | |||
6413 | Your configuration file needs to set the following required | ||
6414 | variables: | ||
6415 | |||
6416 | - :term:`DISTRO_NAME` | ||
6417 | |||
6418 | - :term:`DISTRO_VERSION` | ||
6419 | |||
6420 | These following variables are optional and you typically set them | ||
6421 | from the distribution configuration file: | ||
6422 | |||
6423 | - :term:`DISTRO_FEATURES` | ||
6424 | |||
6425 | - :term:`DISTRO_EXTRA_RDEPENDS` | ||
6426 | |||
6427 | - :term:`DISTRO_EXTRA_RRECOMMENDS` | ||
6428 | |||
6429 | - :term:`TCLIBC` | ||
6430 | |||
6431 | .. tip:: | ||
6432 | |||
6433 | If you want to base your distribution configuration file on the | ||
6434 | very basic configuration from OE-Core, you can use | ||
6435 | conf/distro/defaultsetup.conf | ||
6436 | as a reference and just include variables that differ as compared | ||
6437 | to | ||
6438 | defaultsetup.conf | ||
6439 | . Alternatively, you can create a distribution configuration file | ||
6440 | from scratch using the | ||
6441 | defaultsetup.conf | ||
6442 | file or configuration files from other distributions such as Poky | ||
6443 | or Angstrom as references. | ||
6444 | |||
6445 | - *Provide miscellaneous variables:* Be sure to define any other | ||
6446 | variables for which you want to create a default or enforce as part | ||
6447 | of the distribution configuration. You can include nearly any | ||
6448 | variable from the ``local.conf`` file. The variables you use are not | ||
6449 | limited to the list in the previous bulleted item. | ||
6450 | |||
6451 | - *Point to Your distribution configuration file:* In your | ||
6452 | ``local.conf`` file in the :term:`Build Directory`, | ||
6453 | set your | ||
6454 | :term:`DISTRO` variable to point to | ||
6455 | your distribution's configuration file. For example, if your | ||
6456 | distribution's configuration file is named ``mydistro.conf``, then | ||
6457 | you point to it as follows: | ||
6458 | :: | ||
6459 | |||
6460 | DISTRO = "mydistro" | ||
6461 | |||
6462 | - *Add more to the layer if necessary:* Use your layer to hold other | ||
6463 | information needed for the distribution: | ||
6464 | |||
6465 | - Add recipes for installing distro-specific configuration files | ||
6466 | that are not already installed by another recipe. If you have | ||
6467 | distro-specific configuration files that are included by an | ||
6468 | existing recipe, you should add an append file (``.bbappend``) for | ||
6469 | those. For general information and recommendations on how to add | ||
6470 | recipes to your layer, see the "`Creating Your Own | ||
6471 | Layer <#creating-your-own-layer>`__" and "`Following Best | ||
6472 | Practices When Creating | ||
6473 | Layers <#best-practices-to-follow-when-creating-layers>`__" | ||
6474 | sections. | ||
6475 | |||
6476 | - Add any image recipes that are specific to your distribution. | ||
6477 | |||
6478 | - Add a ``psplash`` append file for a branded splash screen. For | ||
6479 | information on append files, see the "`Using .bbappend Files in | ||
6480 | Your Layer <#using-bbappend-files>`__" section. | ||
6481 | |||
6482 | - Add any other append files to make custom changes that are | ||
6483 | specific to individual recipes. | ||
6484 | |||
6485 | Creating a Custom Template Configuration Directory | ||
6486 | ================================================== | ||
6487 | |||
6488 | If you are producing your own customized version of the build system for | ||
6489 | use by other users, you might want to customize the message shown by the | ||
6490 | setup script or you might want to change the template configuration | ||
6491 | files (i.e. ``local.conf`` and ``bblayers.conf``) that are created in a | ||
6492 | new build directory. | ||
6493 | |||
6494 | The OpenEmbedded build system uses the environment variable | ||
6495 | ``TEMPLATECONF`` to locate the directory from which it gathers | ||
6496 | configuration information that ultimately ends up in the | ||
6497 | :term:`Build Directory` ``conf`` directory. | ||
6498 | By default, ``TEMPLATECONF`` is set as follows in the ``poky`` | ||
6499 | repository: | ||
6500 | :: | ||
6501 | |||
6502 | TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} | ||
6503 | |||
6504 | This is the | ||
6505 | directory used by the build system to find templates from which to build | ||
6506 | some key configuration files. If you look at this directory, you will | ||
6507 | see the ``bblayers.conf.sample``, ``local.conf.sample``, and | ||
6508 | ``conf-notes.txt`` files. The build system uses these files to form the | ||
6509 | respective ``bblayers.conf`` file, ``local.conf`` file, and display the | ||
6510 | list of BitBake targets when running the setup script. | ||
6511 | |||
6512 | To override these default configuration files with configurations you | ||
6513 | want used within every new Build Directory, simply set the | ||
6514 | ``TEMPLATECONF`` variable to your directory. The ``TEMPLATECONF`` | ||
6515 | variable is set in the ``.templateconf`` file, which is in the top-level | ||
6516 | :term:`Source Directory` folder | ||
6517 | (e.g. ``poky``). Edit the ``.templateconf`` so that it can locate your | ||
6518 | directory. | ||
6519 | |||
6520 | Best practices dictate that you should keep your template configuration | ||
6521 | directory in your custom distribution layer. For example, suppose you | ||
6522 | have a layer named ``meta-mylayer`` located in your home directory and | ||
6523 | you want your template configuration directory named ``myconf``. | ||
6524 | Changing the ``.templateconf`` as follows causes the OpenEmbedded build | ||
6525 | system to look in your directory and base its configuration files on the | ||
6526 | ``*.sample`` configuration files it finds. The final configuration files | ||
6527 | (i.e. ``local.conf`` and ``bblayers.conf`` ultimately still end up in | ||
6528 | your Build Directory, but they are based on your ``*.sample`` files. | ||
6529 | :: | ||
6530 | |||
6531 | TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf} | ||
6532 | |||
6533 | Aside from the ``*.sample`` configuration files, the ``conf-notes.txt`` | ||
6534 | also resides in the default ``meta-poky/conf`` directory. The script | ||
6535 | that sets up the build environment (i.e. | ||
6536 | :ref:`structure-core-script`) uses this file to | ||
6537 | display BitBake targets as part of the script output. Customizing this | ||
6538 | ``conf-notes.txt`` file is a good way to make sure your list of custom | ||
6539 | targets appears as part of the script's output. | ||
6540 | |||
6541 | Here is the default list of targets displayed as a result of running | ||
6542 | either of the setup scripts: | ||
6543 | :: | ||
6544 | |||
6545 | You can now run 'bitbake <target>' | ||
6546 | |||
6547 | Common targets are: | ||
6548 | core-image-minimal | ||
6549 | core-image-sato | ||
6550 | meta-toolchain | ||
6551 | meta-ide-support | ||
6552 | |||
6553 | Changing the listed common targets is as easy as editing your version of | ||
6554 | ``conf-notes.txt`` in your custom template configuration directory and | ||
6555 | making sure you have ``TEMPLATECONF`` set to your directory. | ||
6556 | |||
6557 | .. _dev-saving-memory-during-a-build: | ||
6558 | |||
6559 | Conserving Disk Space During Builds | ||
6560 | =================================== | ||
6561 | |||
6562 | To help conserve disk space during builds, you can add the following | ||
6563 | statement to your project's ``local.conf`` configuration file found in | ||
6564 | the :term:`Build Directory`: | ||
6565 | :: | ||
6566 | |||
6567 | INHERIT += "rm_work" | ||
6568 | |||
6569 | Adding this statement deletes the work directory used for | ||
6570 | building a recipe once the recipe is built. For more information on | ||
6571 | "rm_work", see the | ||
6572 | :ref:`rm_work <ref-classes-rm-work>` class in the | ||
6573 | Yocto Project Reference Manual. | ||
6574 | |||
6575 | Working with Packages | ||
6576 | ===================== | ||
6577 | |||
6578 | This section describes a few tasks that involve packages: | ||
6579 | |||
6580 | - `Excluding packages from an | ||
6581 | image <#excluding-packages-from-an-image>`__ | ||
6582 | |||
6583 | - `Incrementing a binary package | ||
6584 | version <#incrementing-a-binary-package-version>`__ | ||
6585 | |||
6586 | - `Handling optional module | ||
6587 | packaging <#handling-optional-module-packaging>`__ | ||
6588 | |||
6589 | - `Using runtime package | ||
6590 | management <#using-runtime-package-management>`__ | ||
6591 | |||
6592 | - `Generating and using signed | ||
6593 | packages <#generating-and-using-signed-packages>`__ | ||
6594 | |||
6595 | - `Setting up and running package test | ||
6596 | (ptest) <#testing-packages-with-ptest>`__ | ||
6597 | |||
6598 | - `Creating node package manager (NPM) | ||
6599 | packages <#creating-node-package-manager-npm-packages>`__ | ||
6600 | |||
6601 | - `Adding custom metadata to | ||
6602 | packages <#adding-custom-metadata-to-packages>`__ | ||
6603 | |||
6604 | Excluding Packages from an Image | ||
6605 | -------------------------------- | ||
6606 | |||
6607 | You might find it necessary to prevent specific packages from being | ||
6608 | installed into an image. If so, you can use several variables to direct | ||
6609 | the build system to essentially ignore installing recommended packages | ||
6610 | or to not install a package at all. | ||
6611 | |||
6612 | The following list introduces variables you can use to prevent packages | ||
6613 | from being installed into your image. Each of these variables only works | ||
6614 | with IPK and RPM package types. Support for Debian packages does not | ||
6615 | exist. Also, you can use these variables from your ``local.conf`` file | ||
6616 | or attach them to a specific image recipe by using a recipe name | ||
6617 | override. For more detail on the variables, see the descriptions in the | ||
6618 | Yocto Project Reference Manual's glossary chapter. | ||
6619 | |||
6620 | - :term:`BAD_RECOMMENDATIONS`: | ||
6621 | Use this variable to specify "recommended-only" packages that you do | ||
6622 | not want installed. | ||
6623 | |||
6624 | - :term:`NO_RECOMMENDATIONS`: | ||
6625 | Use this variable to prevent all "recommended-only" packages from | ||
6626 | being installed. | ||
6627 | |||
6628 | - :term:`PACKAGE_EXCLUDE`: | ||
6629 | Use this variable to prevent specific packages from being installed | ||
6630 | regardless of whether they are "recommended-only" or not. You need to | ||
6631 | realize that the build process could fail with an error when you | ||
6632 | prevent the installation of a package whose presence is required by | ||
6633 | an installed package. | ||
6634 | |||
6635 | .. _incrementing-a-binary-package-version: | ||
6636 | |||
6637 | Incrementing a Package Version | ||
6638 | ------------------------------ | ||
6639 | |||
6640 | This section provides some background on how binary package versioning | ||
6641 | is accomplished and presents some of the services, variables, and | ||
6642 | terminology involved. | ||
6643 | |||
6644 | In order to understand binary package versioning, you need to consider | ||
6645 | the following: | ||
6646 | |||
6647 | - Binary Package: The binary package that is eventually built and | ||
6648 | installed into an image. | ||
6649 | |||
6650 | - Binary Package Version: The binary package version is composed of two | ||
6651 | components - a version and a revision. | ||
6652 | |||
6653 | .. note:: | ||
6654 | |||
6655 | Technically, a third component, the "epoch" (i.e. | ||
6656 | PE | ||
6657 | ) is involved but this discussion for the most part ignores | ||
6658 | PE | ||
6659 | . | ||
6660 | |||
6661 | The version and revision are taken from the | ||
6662 | :term:`PV` and | ||
6663 | :term:`PR` variables, respectively. | ||
6664 | |||
6665 | - ``PV``: The recipe version. ``PV`` represents the version of the | ||
6666 | software being packaged. Do not confuse ``PV`` with the binary | ||
6667 | package version. | ||
6668 | |||
6669 | - ``PR``: The recipe revision. | ||
6670 | |||
6671 | - :term:`SRCPV`: The OpenEmbedded | ||
6672 | build system uses this string to help define the value of ``PV`` when | ||
6673 | the source code revision needs to be included in it. | ||
6674 | |||
6675 | - :yocto_wiki:`PR Service </wiki/PR_Service>`: A | ||
6676 | network-based service that helps automate keeping package feeds | ||
6677 | compatible with existing package manager applications such as RPM, | ||
6678 | APT, and OPKG. | ||
6679 | |||
6680 | Whenever the binary package content changes, the binary package version | ||
6681 | must change. Changing the binary package version is accomplished by | ||
6682 | changing or "bumping" the ``PR`` and/or ``PV`` values. Increasing these | ||
6683 | values occurs one of two ways: | ||
6684 | |||
6685 | - Automatically using a Package Revision Service (PR Service). | ||
6686 | |||
6687 | - Manually incrementing the ``PR`` and/or ``PV`` variables. | ||
6688 | |||
6689 | Given a primary challenge of any build system and its users is how to | ||
6690 | maintain a package feed that is compatible with existing package manager | ||
6691 | applications such as RPM, APT, and OPKG, using an automated system is | ||
6692 | much preferred over a manual system. In either system, the main | ||
6693 | requirement is that binary package version numbering increases in a | ||
6694 | linear fashion and that a number of version components exist that | ||
6695 | support that linear progression. For information on how to ensure | ||
6696 | package revisioning remains linear, see the "`Automatically Incrementing | ||
6697 | a Binary Package Revision | ||
6698 | Number <#automatically-incrementing-a-binary-package-revision-number>`__" | ||
6699 | section. | ||
6700 | |||
6701 | The following three sections provide related information on the PR | ||
6702 | Service, the manual method for "bumping" ``PR`` and/or ``PV``, and on | ||
6703 | how to ensure binary package revisioning remains linear. | ||
6704 | |||
6705 | Working With a PR Service | ||
6706 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6707 | |||
6708 | As mentioned, attempting to maintain revision numbers in the | ||
6709 | :term:`Metadata` is error prone, inaccurate, | ||
6710 | and causes problems for people submitting recipes. Conversely, the PR | ||
6711 | Service automatically generates increasing numbers, particularly the | ||
6712 | revision field, which removes the human element. | ||
6713 | |||
6714 | .. note:: | ||
6715 | |||
6716 | For additional information on using a PR Service, you can see the | ||
6717 | PR Service | ||
6718 | wiki page. | ||
6719 | |||
6720 | The Yocto Project uses variables in order of decreasing priority to | ||
6721 | facilitate revision numbering (i.e. | ||
6722 | :term:`PE`, | ||
6723 | :term:`PV`, and | ||
6724 | :term:`PR` for epoch, version, and | ||
6725 | revision, respectively). The values are highly dependent on the policies | ||
6726 | and procedures of a given distribution and package feed. | ||
6727 | |||
6728 | Because the OpenEmbedded build system uses | ||
6729 | ":ref:`signatures <overview-checksums>`", which are | ||
6730 | unique to a given build, the build system knows when to rebuild | ||
6731 | packages. All the inputs into a given task are represented by a | ||
6732 | signature, which can trigger a rebuild when different. Thus, the build | ||
6733 | system itself does not rely on the ``PR``, ``PV``, and ``PE`` numbers to | ||
6734 | trigger a rebuild. The signatures, however, can be used to generate | ||
6735 | these values. | ||
6736 | |||
6737 | The PR Service works with both ``OEBasic`` and ``OEBasicHash`` | ||
6738 | generators. The value of ``PR`` bumps when the checksum changes and the | ||
6739 | different generator mechanisms change signatures under different | ||
6740 | circumstances. | ||
6741 | |||
6742 | As implemented, the build system includes values from the PR Service | ||
6743 | into the ``PR`` field as an addition using the form "``.x``" so ``r0`` | ||
6744 | becomes ``r0.1``, ``r0.2`` and so forth. This scheme allows existing | ||
6745 | ``PR`` values to be used for whatever reasons, which include manual | ||
6746 | ``PR`` bumps, should it be necessary. | ||
6747 | |||
6748 | By default, the PR Service is not enabled or running. Thus, the packages | ||
6749 | generated are just "self consistent". The build system adds and removes | ||
6750 | packages and there are no guarantees about upgrade paths but images will | ||
6751 | be consistent and correct with the latest changes. | ||
6752 | |||
6753 | The simplest form for a PR Service is for it to exist for a single host | ||
6754 | development system that builds the package feed (building system). For | ||
6755 | this scenario, you can enable a local PR Service by setting | ||
6756 | :term:`PRSERV_HOST` in your | ||
6757 | ``local.conf`` file in the :term:`Build Directory`: | ||
6758 | :: | ||
6759 | |||
6760 | PRSERV_HOST = "localhost:0" | ||
6761 | |||
6762 | Once the service is started, packages will automatically | ||
6763 | get increasing ``PR`` values and BitBake takes care of starting and | ||
6764 | stopping the server. | ||
6765 | |||
6766 | If you have a more complex setup where multiple host development systems | ||
6767 | work against a common, shared package feed, you have a single PR Service | ||
6768 | running and it is connected to each building system. For this scenario, | ||
6769 | you need to start the PR Service using the ``bitbake-prserv`` command: | ||
6770 | :: | ||
6771 | |||
6772 | bitbake-prserv --host ip --port port --start | ||
6773 | |||
6774 | In addition to | ||
6775 | hand-starting the service, you need to update the ``local.conf`` file of | ||
6776 | each building system as described earlier so each system points to the | ||
6777 | server and port. | ||
6778 | |||
6779 | It is also recommended you use build history, which adds some sanity | ||
6780 | checks to binary package versions, in conjunction with the server that | ||
6781 | is running the PR Service. To enable build history, add the following to | ||
6782 | each building system's ``local.conf`` file: | ||
6783 | :: | ||
6784 | |||
6785 | # It is recommended to activate "buildhistory" for testing the PR service | ||
6786 | INHERIT += "buildhistory" | ||
6787 | BUILDHISTORY_COMMIT = "1" | ||
6788 | |||
6789 | For information on build | ||
6790 | history, see the "`Maintaining Build Output | ||
6791 | Quality <#maintaining-build-output-quality>`__" section. | ||
6792 | |||
6793 | .. note:: | ||
6794 | |||
6795 | The OpenEmbedded build system does not maintain ``PR`` information as | ||
6796 | part of the shared state (sstate) packages. If you maintain an sstate | ||
6797 | feed, its expected that either all your building systems that | ||
6798 | contribute to the sstate feed use a shared PR Service, or you do not | ||
6799 | run a PR Service on any of your building systems. Having some systems | ||
6800 | use a PR Service while others do not leads to obvious problems. | ||
6801 | |||
6802 | For more information on shared state, see the | ||
6803 | ":ref:`overview-manual/overview-manual-concepts:shared state cache`" | ||
6804 | section in the Yocto Project Overview and Concepts Manual. | ||
6805 | |||
6806 | Manually Bumping PR | ||
6807 | ~~~~~~~~~~~~~~~~~~~ | ||
6808 | |||
6809 | The alternative to setting up a PR Service is to manually "bump" the | ||
6810 | :term:`PR` variable. | ||
6811 | |||
6812 | If a committed change results in changing the package output, then the | ||
6813 | value of the PR variable needs to be increased (or "bumped") as part of | ||
6814 | that commit. For new recipes you should add the ``PR`` variable and set | ||
6815 | its initial value equal to "r0", which is the default. Even though the | ||
6816 | default value is "r0", the practice of adding it to a new recipe makes | ||
6817 | it harder to forget to bump the variable when you make changes to the | ||
6818 | recipe in future. | ||
6819 | |||
6820 | If you are sharing a common ``.inc`` file with multiple recipes, you can | ||
6821 | also use the ``INC_PR`` variable to ensure that the recipes sharing the | ||
6822 | ``.inc`` file are rebuilt when the ``.inc`` file itself is changed. The | ||
6823 | ``.inc`` file must set ``INC_PR`` (initially to "r0"), and all recipes | ||
6824 | referring to it should set ``PR`` to "${INC_PR}.0" initially, | ||
6825 | incrementing the last number when the recipe is changed. If the ``.inc`` | ||
6826 | file is changed then its ``INC_PR`` should be incremented. | ||
6827 | |||
6828 | When upgrading the version of a binary package, assuming the ``PV`` | ||
6829 | changes, the ``PR`` variable should be reset to "r0" (or "${INC_PR}.0" | ||
6830 | if you are using ``INC_PR``). | ||
6831 | |||
6832 | Usually, version increases occur only to binary packages. However, if | ||
6833 | for some reason ``PV`` changes but does not increase, you can increase | ||
6834 | the ``PE`` variable (Package Epoch). The ``PE`` variable defaults to | ||
6835 | "0". | ||
6836 | |||
6837 | Binary package version numbering strives to follow the `Debian Version | ||
6838 | Field Policy | ||
6839 | Guidelines <http://www.debian.org/doc/debian-policy/ch-controlfields.html>`__. | ||
6840 | These guidelines define how versions are compared and what "increasing" | ||
6841 | a version means. | ||
6842 | |||
6843 | .. _automatically-incrementing-a-binary-package-revision-number: | ||
6844 | |||
6845 | Automatically Incrementing a Package Version Number | ||
6846 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6847 | |||
6848 | When fetching a repository, BitBake uses the | ||
6849 | :term:`SRCREV` variable to determine | ||
6850 | the specific source code revision from which to build. You set the | ||
6851 | ``SRCREV`` variable to | ||
6852 | :term:`AUTOREV` to cause the | ||
6853 | OpenEmbedded build system to automatically use the latest revision of | ||
6854 | the software: | ||
6855 | :: | ||
6856 | |||
6857 | SRCREV = "${AUTOREV}" | ||
6858 | |||
6859 | Furthermore, you need to reference ``SRCPV`` in ``PV`` in order to | ||
6860 | automatically update the version whenever the revision of the source | ||
6861 | code changes. Here is an example: | ||
6862 | :: | ||
6863 | |||
6864 | PV = "1.0+git${SRCPV}" | ||
6865 | |||
6866 | The OpenEmbedded build system substitutes ``SRCPV`` with the following: | ||
6867 | :: | ||
6868 | |||
6869 | AUTOINC+source_code_revision | ||
6870 | |||
6871 | The build system replaces the ``AUTOINC`` | ||
6872 | with a number. The number used depends on the state of the PR Service: | ||
6873 | |||
6874 | - If PR Service is enabled, the build system increments the number, | ||
6875 | which is similar to the behavior of | ||
6876 | :term:`PR`. This behavior results in | ||
6877 | linearly increasing package versions, which is desirable. Here is an | ||
6878 | example: | ||
6879 | :: | ||
6880 | |||
6881 | hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk | ||
6882 | hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk | ||
6883 | |||
6884 | - If PR Service is not enabled, the build system replaces the | ||
6885 | ``AUTOINC`` placeholder with zero (i.e. "0"). This results in | ||
6886 | changing the package version since the source revision is included. | ||
6887 | However, package versions are not increased linearly. Here is an | ||
6888 | example: | ||
6889 | :: | ||
6890 | |||
6891 | hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk | ||
6892 | hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk | ||
6893 | |||
6894 | In summary, the OpenEmbedded build system does not track the history of | ||
6895 | binary package versions for this purpose. ``AUTOINC``, in this case, is | ||
6896 | comparable to ``PR``. If PR server is not enabled, ``AUTOINC`` in the | ||
6897 | package version is simply replaced by "0". If PR server is enabled, the | ||
6898 | build system keeps track of the package versions and bumps the number | ||
6899 | when the package revision changes. | ||
6900 | |||
6901 | Handling Optional Module Packaging | ||
6902 | ---------------------------------- | ||
6903 | |||
6904 | Many pieces of software split functionality into optional modules (or | ||
6905 | plugins) and the plugins that are built might depend on configuration | ||
6906 | options. To avoid having to duplicate the logic that determines what | ||
6907 | modules are available in your recipe or to avoid having to package each | ||
6908 | module by hand, the OpenEmbedded build system provides functionality to | ||
6909 | handle module packaging dynamically. | ||
6910 | |||
6911 | To handle optional module packaging, you need to do two things: | ||
6912 | |||
6913 | - Ensure the module packaging is actually done. | ||
6914 | |||
6915 | - Ensure that any dependencies on optional modules from other recipes | ||
6916 | are satisfied by your recipe. | ||
6917 | |||
6918 | Making Sure the Packaging is Done | ||
6919 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
6920 | |||
6921 | To ensure the module packaging actually gets done, you use the | ||
6922 | ``do_split_packages`` function within the ``populate_packages`` Python | ||
6923 | function in your recipe. The ``do_split_packages`` function searches for | ||
6924 | a pattern of files or directories under a specified path and creates a | ||
6925 | package for each one it finds by appending to the | ||
6926 | :term:`PACKAGES` variable and | ||
6927 | setting the appropriate values for ``FILES_packagename``, | ||
6928 | ``RDEPENDS_packagename``, ``DESCRIPTION_packagename``, and so forth. | ||
6929 | Here is an example from the ``lighttpd`` recipe: | ||
6930 | :: | ||
6931 | |||
6932 | python populate_packages_prepend () { | ||
6933 | lighttpd_libdir = d.expand('${libdir}') | ||
6934 | do_split_packages(d, lighttpd_libdir, '^mod_(.*).so$', | ||
6935 | 'lighttpd-module-%s', 'Lighttpd module for %s', | ||
6936 | extra_depends='') | ||
6937 | } | ||
6938 | |||
6939 | The previous example specifies a number of things in the call to | ||
6940 | ``do_split_packages``. | ||
6941 | |||
6942 | - A directory within the files installed by your recipe through | ||
6943 | ``do_install`` in which to search. | ||
6944 | |||
6945 | - A regular expression used to match module files in that directory. In | ||
6946 | the example, note the parentheses () that mark the part of the | ||
6947 | expression from which the module name should be derived. | ||
6948 | |||
6949 | - A pattern to use for the package names. | ||
6950 | |||
6951 | - A description for each package. | ||
6952 | |||
6953 | - An empty string for ``extra_depends``, which disables the default | ||
6954 | dependency on the main ``lighttpd`` package. Thus, if a file in | ||
6955 | ``${libdir}`` called ``mod_alias.so`` is found, a package called | ||
6956 | ``lighttpd-module-alias`` is created for it and the | ||
6957 | :term:`DESCRIPTION` is set to | ||
6958 | "Lighttpd module for alias". | ||
6959 | |||
6960 | Often, packaging modules is as simple as the previous example. However, | ||
6961 | more advanced options exist that you can use within | ||
6962 | ``do_split_packages`` to modify its behavior. And, if you need to, you | ||
6963 | can add more logic by specifying a hook function that is called for each | ||
6964 | package. It is also perfectly acceptable to call ``do_split_packages`` | ||
6965 | multiple times if you have more than one set of modules to package. | ||
6966 | |||
6967 | For more examples that show how to use ``do_split_packages``, see the | ||
6968 | ``connman.inc`` file in the ``meta/recipes-connectivity/connman/`` | ||
6969 | directory of the ``poky`` :ref:`source repository <yocto-project-repositories>`. You can | ||
6970 | also find examples in ``meta/classes/kernel.bbclass``. | ||
6971 | |||
6972 | Following is a reference that shows ``do_split_packages`` mandatory and | ||
6973 | optional arguments: | ||
6974 | :: | ||
6975 | |||
6976 | Mandatory arguments | ||
6977 | |||
6978 | root | ||
6979 | The path in which to search | ||
6980 | file_regex | ||
6981 | Regular expression to match searched files. | ||
6982 | Use parentheses () to mark the part of this | ||
6983 | expression that should be used to derive the | ||
6984 | module name (to be substituted where %s is | ||
6985 | used in other function arguments as noted below) | ||
6986 | output_pattern | ||
6987 | Pattern to use for the package names. Must | ||
6988 | include %s. | ||
6989 | description | ||
6990 | Description to set for each package. Must | ||
6991 | include %s. | ||
6992 | |||
6993 | Optional arguments | ||
6994 | |||
6995 | postinst | ||
6996 | Postinstall script to use for all packages | ||
6997 | (as a string) | ||
6998 | recursive | ||
6999 | True to perform a recursive search - default | ||
7000 | False | ||
7001 | hook | ||
7002 | A hook function to be called for every match. | ||
7003 | The function will be called with the following | ||
7004 | arguments (in the order listed): | ||
7005 | |||
7006 | f | ||
7007 | Full path to the file/directory match | ||
7008 | pkg | ||
7009 | The package name | ||
7010 | file_regex | ||
7011 | As above | ||
7012 | output_pattern | ||
7013 | As above | ||
7014 | modulename | ||
7015 | The module name derived using file_regex | ||
7016 | extra_depends | ||
7017 | Extra runtime dependencies (RDEPENDS) to be | ||
7018 | set for all packages. The default value of None | ||
7019 | causes a dependency on the main package | ||
7020 | (${PN}) - if you do not want this, pass empty | ||
7021 | string '' for this parameter. | ||
7022 | aux_files_pattern | ||
7023 | Extra item(s) to be added to FILES for each | ||
7024 | package. Can be a single string item or a list | ||
7025 | of strings for multiple items. Must include %s. | ||
7026 | postrm | ||
7027 | postrm script to use for all packages (as a | ||
7028 | string) | ||
7029 | allow_dirs | ||
7030 | True to allow directories to be matched - | ||
7031 | default False | ||
7032 | prepend | ||
7033 | If True, prepend created packages to PACKAGES | ||
7034 | instead of the default False which appends them | ||
7035 | match_path | ||
7036 | match file_regex on the whole relative path to | ||
7037 | the root rather than just the file name | ||
7038 | aux_files_pattern_verbatim | ||
7039 | Extra item(s) to be added to FILES for each | ||
7040 | package, using the actual derived module name | ||
7041 | rather than converting it to something legal | ||
7042 | for a package name. Can be a single string item | ||
7043 | or a list of strings for multiple items. Must | ||
7044 | include %s. | ||
7045 | allow_links | ||
7046 | True to allow symlinks to be matched - default | ||
7047 | False | ||
7048 | summary | ||
7049 | Summary to set for each package. Must include %s; | ||
7050 | defaults to description if not set. | ||
7051 | |||
7052 | |||
7053 | |||
7054 | Satisfying Dependencies | ||
7055 | ~~~~~~~~~~~~~~~~~~~~~~~ | ||
7056 | |||
7057 | The second part for handling optional module packaging is to ensure that | ||
7058 | any dependencies on optional modules from other recipes are satisfied by | ||
7059 | your recipe. You can be sure these dependencies are satisfied by using | ||
7060 | the :term:`PACKAGES_DYNAMIC` | ||
7061 | variable. Here is an example that continues with the ``lighttpd`` recipe | ||
7062 | shown earlier: | ||
7063 | :: | ||
7064 | |||
7065 | PACKAGES_DYNAMIC = "lighttpd-module-.*" | ||
7066 | |||
7067 | The name | ||
7068 | specified in the regular expression can of course be anything. In this | ||
7069 | example, it is ``lighttpd-module-`` and is specified as the prefix to | ||
7070 | ensure that any :term:`RDEPENDS` and | ||
7071 | :term:`RRECOMMENDS` on a package | ||
7072 | name starting with the prefix are satisfied during build time. If you | ||
7073 | are using ``do_split_packages`` as described in the previous section, | ||
7074 | the value you put in ``PACKAGES_DYNAMIC`` should correspond to the name | ||
7075 | pattern specified in the call to ``do_split_packages``. | ||
7076 | |||
7077 | Using Runtime Package Management | ||
7078 | -------------------------------- | ||
7079 | |||
7080 | During a build, BitBake always transforms a recipe into one or more | ||
7081 | packages. For example, BitBake takes the ``bash`` recipe and produces a | ||
7082 | number of packages (e.g. ``bash``, ``bash-bashbug``, | ||
7083 | ``bash-completion``, ``bash-completion-dbg``, ``bash-completion-dev``, | ||
7084 | ``bash-completion-extra``, ``bash-dbg``, and so forth). Not all | ||
7085 | generated packages are included in an image. | ||
7086 | |||
7087 | In several situations, you might need to update, add, remove, or query | ||
7088 | the packages on a target device at runtime (i.e. without having to | ||
7089 | generate a new image). Examples of such situations include: | ||
7090 | |||
7091 | - You want to provide in-the-field updates to deployed devices (e.g. | ||
7092 | security updates). | ||
7093 | |||
7094 | - You want to have a fast turn-around development cycle for one or more | ||
7095 | applications that run on your device. | ||
7096 | |||
7097 | - You want to temporarily install the "debug" packages of various | ||
7098 | applications on your device so that debugging can be greatly improved | ||
7099 | by allowing access to symbols and source debugging. | ||
7100 | |||
7101 | - You want to deploy a more minimal package selection of your device | ||
7102 | but allow in-the-field updates to add a larger selection for | ||
7103 | customization. | ||
7104 | |||
7105 | In all these situations, you have something similar to a more | ||
7106 | traditional Linux distribution in that in-field devices are able to | ||
7107 | receive pre-compiled packages from a server for installation or update. | ||
7108 | Being able to install these packages on a running, in-field device is | ||
7109 | what is termed "runtime package management". | ||
7110 | |||
7111 | In order to use runtime package management, you need a host or server | ||
7112 | machine that serves up the pre-compiled packages plus the required | ||
7113 | metadata. You also need package manipulation tools on the target. The | ||
7114 | build machine is a likely candidate to act as the server. However, that | ||
7115 | machine does not necessarily have to be the package server. The build | ||
7116 | machine could push its artifacts to another machine that acts as the | ||
7117 | server (e.g. Internet-facing). In fact, doing so is advantageous for a | ||
7118 | production environment as getting the packages away from the development | ||
7119 | system's build directory prevents accidental overwrites. | ||
7120 | |||
7121 | A simple build that targets just one device produces more than one | ||
7122 | package database. In other words, the packages produced by a build are | ||
7123 | separated out into a couple of different package groupings based on | ||
7124 | criteria such as the target's CPU architecture, the target board, or the | ||
7125 | C library used on the target. For example, a build targeting the | ||
7126 | ``qemux86`` device produces the following three package databases: | ||
7127 | ``noarch``, ``i586``, and ``qemux86``. If you wanted your ``qemux86`` | ||
7128 | device to be aware of all the packages that were available to it, you | ||
7129 | would need to point it to each of these databases individually. In a | ||
7130 | similar way, a traditional Linux distribution usually is configured to | ||
7131 | be aware of a number of software repositories from which it retrieves | ||
7132 | packages. | ||
7133 | |||
7134 | Using runtime package management is completely optional and not required | ||
7135 | for a successful build or deployment in any way. But if you want to make | ||
7136 | use of runtime package management, you need to do a couple things above | ||
7137 | and beyond the basics. The remainder of this section describes what you | ||
7138 | need to do. | ||
7139 | |||
7140 | .. _runtime-package-management-build: | ||
7141 | |||
7142 | Build Considerations | ||
7143 | ~~~~~~~~~~~~~~~~~~~~ | ||
7144 | |||
7145 | This section describes build considerations of which you need to be | ||
7146 | aware in order to provide support for runtime package management. | ||
7147 | |||
7148 | When BitBake generates packages, it needs to know what format or formats | ||
7149 | to use. In your configuration, you use the | ||
7150 | :term:`PACKAGE_CLASSES` | ||
7151 | variable to specify the format: | ||
7152 | |||
7153 | 1. Open the ``local.conf`` file inside your | ||
7154 | :term:`Build Directory` (e.g. | ||
7155 | ``~/poky/build/conf/local.conf``). | ||
7156 | |||
7157 | 2. Select the desired package format as follows: | ||
7158 | :: | ||
7159 | |||
7160 | PACKAGE_CLASSES ?= "package_packageformat" | ||
7161 | |||
7162 | where packageformat can be "ipk", "rpm", | ||
7163 | "deb", or "tar" which are the supported package formats. | ||
7164 | |||
7165 | .. note:: | ||
7166 | |||
7167 | Because the Yocto Project supports four different package formats, | ||
7168 | you can set the variable with more than one argument. However, the | ||
7169 | OpenEmbedded build system only uses the first argument when | ||
7170 | creating an image or Software Development Kit (SDK). | ||
7171 | |||
7172 | If you would like your image to start off with a basic package database | ||
7173 | containing the packages in your current build as well as to have the | ||
7174 | relevant tools available on the target for runtime package management, | ||
7175 | you can include "package-management" in the | ||
7176 | :term:`IMAGE_FEATURES` | ||
7177 | variable. Including "package-management" in this configuration variable | ||
7178 | ensures that when the image is assembled for your target, the image | ||
7179 | includes the currently-known package databases as well as the | ||
7180 | target-specific tools required for runtime package management to be | ||
7181 | performed on the target. However, this is not strictly necessary. You | ||
7182 | could start your image off without any databases but only include the | ||
7183 | required on-target package tool(s). As an example, you could include | ||
7184 | "opkg" in your | ||
7185 | :term:`IMAGE_INSTALL` variable | ||
7186 | if you are using the IPK package format. You can then initialize your | ||
7187 | target's package database(s) later once your image is up and running. | ||
7188 | |||
7189 | Whenever you perform any sort of build step that can potentially | ||
7190 | generate a package or modify existing package, it is always a good idea | ||
7191 | to re-generate the package index after the build by using the following | ||
7192 | command: | ||
7193 | :: | ||
7194 | |||
7195 | $ bitbake package-index | ||
7196 | |||
7197 | It might be tempting to build the | ||
7198 | package and the package index at the same time with a command such as | ||
7199 | the following: | ||
7200 | :: | ||
7201 | |||
7202 | $ bitbake some-package package-index | ||
7203 | |||
7204 | Do not do this as | ||
7205 | BitBake does not schedule the package index for after the completion of | ||
7206 | the package you are building. Consequently, you cannot be sure of the | ||
7207 | package index including information for the package you just built. | ||
7208 | Thus, be sure to run the package update step separately after building | ||
7209 | any packages. | ||
7210 | |||
7211 | You can use the | ||
7212 | :term:`PACKAGE_FEED_ARCHS`, | ||
7213 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7214 | and | ||
7215 | :term:`PACKAGE_FEED_URIS` | ||
7216 | variables to pre-configure target images to use a package feed. If you | ||
7217 | do not define these variables, then manual steps as described in the | ||
7218 | subsequent sections are necessary to configure the target. You should | ||
7219 | set these variables before building the image in order to produce a | ||
7220 | correctly configured image. | ||
7221 | |||
7222 | When your build is complete, your packages reside in the | ||
7223 | ``${TMPDIR}/deploy/packageformat`` directory. For example, if | ||
7224 | ``${``\ :term:`TMPDIR`\ ``}`` is | ||
7225 | ``tmp`` and your selected package type is RPM, then your RPM packages | ||
7226 | are available in ``tmp/deploy/rpm``. | ||
7227 | |||
7228 | .. _runtime-package-management-server: | ||
7229 | |||
7230 | Host or Server Machine Setup | ||
7231 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7232 | |||
7233 | Although other protocols are possible, a server using HTTP typically | ||
7234 | serves packages. If you want to use HTTP, then set up and configure a | ||
7235 | web server such as Apache 2, lighttpd, or SimpleHTTPServer on the | ||
7236 | machine serving the packages. | ||
7237 | |||
7238 | To keep things simple, this section describes how to set up a | ||
7239 | SimpleHTTPServer web server to share package feeds from the developer's | ||
7240 | machine. Although this server might not be the best for a production | ||
7241 | environment, the setup is simple and straight forward. Should you want | ||
7242 | to use a different server more suited for production (e.g. Apache 2, | ||
7243 | Lighttpd, or Nginx), take the appropriate steps to do so. | ||
7244 | |||
7245 | From within the build directory where you have built an image based on | ||
7246 | your packaging choice (i.e. the | ||
7247 | :term:`PACKAGE_CLASSES` | ||
7248 | setting), simply start the server. The following example assumes a build | ||
7249 | directory of ``~/poky/build/tmp/deploy/rpm`` and a ``PACKAGE_CLASSES`` | ||
7250 | setting of "package_rpm": | ||
7251 | :: | ||
7252 | |||
7253 | $ cd ~/poky/build/tmp/deploy/rpm | ||
7254 | $ python -m SimpleHTTPServer | ||
7255 | |||
7256 | .. _runtime-package-management-target: | ||
7257 | |||
7258 | Target Setup | ||
7259 | ~~~~~~~~~~~~ | ||
7260 | |||
7261 | Setting up the target differs depending on the package management | ||
7262 | system. This section provides information for RPM, IPK, and DEB. | ||
7263 | |||
7264 | .. _runtime-package-management-target-rpm: | ||
7265 | |||
7266 | Using RPM | ||
7267 | ^^^^^^^^^ | ||
7268 | |||
7269 | The `Dandified Packaging | ||
7270 | Tool <https://en.wikipedia.org/wiki/DNF_(software)>`__ (DNF) performs | ||
7271 | runtime package management of RPM packages. In order to use DNF for | ||
7272 | runtime package management, you must perform an initial setup on the | ||
7273 | target machine for cases where the ``PACKAGE_FEED_*`` variables were not | ||
7274 | set as part of the image that is running on the target. This means if | ||
7275 | you built your image and did not not use these variables as part of the | ||
7276 | build and your image is now running on the target, you need to perform | ||
7277 | the steps in this section if you want to use runtime package management. | ||
7278 | |||
7279 | .. note:: | ||
7280 | |||
7281 | For information on the PACKAGE_FEED_* variables, see | ||
7282 | PACKAGE_FEED_ARCHS | ||
7283 | , | ||
7284 | PACKAGE_FEED_BASE_PATHS | ||
7285 | , and | ||
7286 | PACKAGE_FEED_URIS | ||
7287 | in the Yocto Project Reference Manual variables glossary. | ||
7288 | |||
7289 | On the target, you must inform DNF that package databases are available. | ||
7290 | You do this by creating a file named | ||
7291 | ``/etc/yum.repos.d/oe-packages.repo`` and defining the ``oe-packages``. | ||
7292 | |||
7293 | As an example, assume the target is able to use the following package | ||
7294 | databases: ``all``, ``i586``, and ``qemux86`` from a server named | ||
7295 | ``my.server``. The specifics for setting up the web server are up to | ||
7296 | you. The critical requirement is that the URIs in the target repository | ||
7297 | configuration point to the correct remote location for the feeds. | ||
7298 | |||
7299 | .. note:: | ||
7300 | |||
7301 | For development purposes, you can point the web server to the build | ||
7302 | system's | ||
7303 | deploy | ||
7304 | directory. However, for production use, it is better to copy the | ||
7305 | package directories to a location outside of the build area and use | ||
7306 | that location. Doing so avoids situations where the build system | ||
7307 | overwrites or changes the | ||
7308 | deploy | ||
7309 | directory. | ||
7310 | |||
7311 | When telling DNF where to look for the package databases, you must | ||
7312 | declare individual locations per architecture or a single location used | ||
7313 | for all architectures. You cannot do both: | ||
7314 | |||
7315 | - *Create an Explicit List of Architectures:* Define individual base | ||
7316 | URLs to identify where each package database is located: | ||
7317 | :: | ||
7318 | |||
7319 | [oe-packages] | ||
7320 | baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all | ||
7321 | |||
7322 | This example | ||
7323 | informs DNF about individual package databases for all three | ||
7324 | architectures. | ||
7325 | |||
7326 | - *Create a Single (Full) Package Index:* Define a single base URL that | ||
7327 | identifies where a full package database is located: | ||
7328 | :: | ||
7329 | |||
7330 | [oe-packages] | ||
7331 | baseurl=http://my.server/rpm | ||
7332 | |||
7333 | This example informs DNF about a single | ||
7334 | package database that contains all the package index information for | ||
7335 | all supported architectures. | ||
7336 | |||
7337 | Once you have informed DNF where to find the package databases, you need | ||
7338 | to fetch them: | ||
7339 | :: | ||
7340 | |||
7341 | # dnf makecache | ||
7342 | |||
7343 | DNF is now able to find, install, and | ||
7344 | upgrade packages from the specified repository or repositories. | ||
7345 | |||
7346 | .. note:: | ||
7347 | |||
7348 | See the | ||
7349 | DNF documentation | ||
7350 | for additional information. | ||
7351 | |||
7352 | .. _runtime-package-management-target-ipk: | ||
7353 | |||
7354 | Using IPK | ||
7355 | ^^^^^^^^^ | ||
7356 | |||
7357 | The ``opkg`` application performs runtime package management of IPK | ||
7358 | packages. You must perform an initial setup for ``opkg`` on the target | ||
7359 | machine if the | ||
7360 | :term:`PACKAGE_FEED_ARCHS`, | ||
7361 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7362 | and | ||
7363 | :term:`PACKAGE_FEED_URIS` | ||
7364 | variables have not been set or the target image was built before the | ||
7365 | variables were set. | ||
7366 | |||
7367 | The ``opkg`` application uses configuration files to find available | ||
7368 | package databases. Thus, you need to create a configuration file inside | ||
7369 | the ``/etc/opkg/`` direction, which informs ``opkg`` of any repository | ||
7370 | you want to use. | ||
7371 | |||
7372 | As an example, suppose you are serving packages from a ``ipk/`` | ||
7373 | directory containing the ``i586``, ``all``, and ``qemux86`` databases | ||
7374 | through an HTTP server named ``my.server``. On the target, create a | ||
7375 | configuration file (e.g. ``my_repo.conf``) inside the ``/etc/opkg/`` | ||
7376 | directory containing the following: | ||
7377 | :: | ||
7378 | |||
7379 | src/gz all http://my.server/ipk/all | ||
7380 | src/gz i586 http://my.server/ipk/i586 | ||
7381 | src/gz qemux86 http://my.server/ipk/qemux86 | ||
7382 | |||
7383 | Next, instruct ``opkg`` to fetch the | ||
7384 | repository information: # opkg update The ``opkg`` application is now | ||
7385 | able to find, install, and upgrade packages from the specified | ||
7386 | repository. | ||
7387 | |||
7388 | .. _runtime-package-management-target-deb: | ||
7389 | |||
7390 | Using DEB | ||
7391 | ^^^^^^^^^ | ||
7392 | |||
7393 | The ``apt`` application performs runtime package management of DEB | ||
7394 | packages. This application uses a source list file to find available | ||
7395 | package databases. You must perform an initial setup for ``apt`` on the | ||
7396 | target machine if the | ||
7397 | :term:`PACKAGE_FEED_ARCHS`, | ||
7398 | :term:`PACKAGE_FEED_BASE_PATHS`, | ||
7399 | and | ||
7400 | :term:`PACKAGE_FEED_URIS` | ||
7401 | variables have not been set or the target image was built before the | ||
7402 | variables were set. | ||
7403 | |||
7404 | To inform ``apt`` of the repository you want to use, you might create a | ||
7405 | list file (e.g. ``my_repo.list``) inside the | ||
7406 | ``/etc/apt/sources.list.d/`` directory. As an example, suppose you are | ||
7407 | serving packages from a ``deb/`` directory containing the ``i586``, | ||
7408 | ``all``, and ``qemux86`` databases through an HTTP server named | ||
7409 | ``my.server``. The list file should contain: | ||
7410 | :: | ||
7411 | |||
7412 | deb http://my.server/deb/all ./ | ||
7413 | deb http://my.server/deb/i586 ./ | ||
7414 | deb http://my.server/deb/qemux86 ./ | ||
7415 | |||
7416 | Next, instruct the ``apt`` application | ||
7417 | to fetch the repository information: | ||
7418 | :: | ||
7419 | |||
7420 | # apt-get update | ||
7421 | |||
7422 | After this step, | ||
7423 | ``apt`` is able to find, install, and upgrade packages from the | ||
7424 | specified repository. | ||
7425 | |||
7426 | Generating and Using Signed Packages | ||
7427 | ------------------------------------ | ||
7428 | |||
7429 | In order to add security to RPM packages used during a build, you can | ||
7430 | take steps to securely sign them. Once a signature is verified, the | ||
7431 | OpenEmbedded build system can use the package in the build. If security | ||
7432 | fails for a signed package, the build system aborts the build. | ||
7433 | |||
7434 | This section describes how to sign RPM packages during a build and how | ||
7435 | to use signed package feeds (repositories) when doing a build. | ||
7436 | |||
7437 | Signing RPM Packages | ||
7438 | ~~~~~~~~~~~~~~~~~~~~ | ||
7439 | |||
7440 | To enable signing RPM packages, you must set up the following | ||
7441 | configurations in either your ``local.config`` or ``distro.config`` | ||
7442 | file: | ||
7443 | :: | ||
7444 | |||
7445 | # Inherit sign_rpm.bbclass to enable signing functionality | ||
7446 | INHERIT += " sign_rpm" | ||
7447 | # Define the GPG key that will be used for signing. | ||
7448 | RPM_GPG_NAME = "key_name" | ||
7449 | # Provide passphrase for the key | ||
7450 | RPM_GPG_PASSPHRASE = "passphrase" | ||
7451 | |||
7452 | .. note:: | ||
7453 | |||
7454 | Be sure to supply appropriate values for both | ||
7455 | key_name | ||
7456 | and | ||
7457 | passphrase | ||
7458 | |||
7459 | Aside from the ``RPM_GPG_NAME`` and ``RPM_GPG_PASSPHRASE`` variables in | ||
7460 | the previous example, two optional variables related to signing exist: | ||
7461 | |||
7462 | - *GPG_BIN:* Specifies a ``gpg`` binary/wrapper that is executed | ||
7463 | when the package is signed. | ||
7464 | |||
7465 | - *GPG_PATH:* Specifies the ``gpg`` home directory used when the | ||
7466 | package is signed. | ||
7467 | |||
7468 | Processing Package Feeds | ||
7469 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7470 | |||
7471 | In addition to being able to sign RPM packages, you can also enable | ||
7472 | signed package feeds for IPK and RPM packages. | ||
7473 | |||
7474 | The steps you need to take to enable signed package feed use are similar | ||
7475 | to the steps used to sign RPM packages. You must define the following in | ||
7476 | your ``local.config`` or ``distro.config`` file: | ||
7477 | :: | ||
7478 | |||
7479 | INHERIT += "sign_package_feed" | ||
7480 | PACKAGE_FEED_GPG_NAME = "key_name" | ||
7481 | PACKAGE_FEED_GPG_PASSPHRASE_FILE = "path_to_file_containing_passphrase" | ||
7482 | |||
7483 | For signed package feeds, the passphrase must exist in a separate file, | ||
7484 | which is pointed to by the ``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` | ||
7485 | variable. Regarding security, keeping a plain text passphrase out of the | ||
7486 | configuration is more secure. | ||
7487 | |||
7488 | Aside from the ``PACKAGE_FEED_GPG_NAME`` and | ||
7489 | ``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` variables, three optional variables | ||
7490 | related to signed package feeds exist: | ||
7491 | |||
7492 | - *GPG_BIN* Specifies a ``gpg`` binary/wrapper that is executed | ||
7493 | when the package is signed. | ||
7494 | |||
7495 | - *GPG_PATH:* Specifies the ``gpg`` home directory used when the | ||
7496 | package is signed. | ||
7497 | |||
7498 | - *PACKAGE_FEED_GPG_SIGNATURE_TYPE:* Specifies the type of ``gpg`` | ||
7499 | signature. This variable applies only to RPM and IPK package feeds. | ||
7500 | Allowable values for the ``PACKAGE_FEED_GPG_SIGNATURE_TYPE`` are | ||
7501 | "ASC", which is the default and specifies ascii armored, and "BIN", | ||
7502 | which specifies binary. | ||
7503 | |||
7504 | Testing Packages With ptest | ||
7505 | --------------------------- | ||
7506 | |||
7507 | A Package Test (ptest) runs tests against packages built by the | ||
7508 | OpenEmbedded build system on the target machine. A ptest contains at | ||
7509 | least two items: the actual test, and a shell script (``run-ptest``) | ||
7510 | that starts the test. The shell script that starts the test must not | ||
7511 | contain the actual test - the script only starts the test. On the other | ||
7512 | hand, the test can be anything from a simple shell script that runs a | ||
7513 | binary and checks the output to an elaborate system of test binaries and | ||
7514 | data files. | ||
7515 | |||
7516 | The test generates output in the format used by Automake: | ||
7517 | :: | ||
7518 | |||
7519 | result: testname | ||
7520 | |||
7521 | where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and | ||
7522 | the testname can be any identifying string. | ||
7523 | |||
7524 | For a list of Yocto Project recipes that are already enabled with ptest, | ||
7525 | see the :yocto_wiki:`Ptest </wiki/Ptest>` wiki page. | ||
7526 | |||
7527 | .. note:: | ||
7528 | |||
7529 | A recipe is "ptest-enabled" if it inherits the | ||
7530 | ptest | ||
7531 | class. | ||
7532 | |||
7533 | Adding ptest to Your Build | ||
7534 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7535 | |||
7536 | To add package testing to your build, add the | ||
7537 | :term:`DISTRO_FEATURES` and | ||
7538 | :term:`EXTRA_IMAGE_FEATURES` | ||
7539 | variables to your ``local.conf`` file, which is found in the | ||
7540 | :term:`Build Directory`: | ||
7541 | :: | ||
7542 | |||
7543 | DISTRO_FEATURES_append = " ptest" | ||
7544 | EXTRA_IMAGE_FEATURES += "ptest-pkgs" | ||
7545 | |||
7546 | Once your build is complete, the ptest files are installed into the | ||
7547 | ``/usr/lib/package/ptest`` directory within the image, where ``package`` | ||
7548 | is the name of the package. | ||
7549 | |||
7550 | Running ptest | ||
7551 | ~~~~~~~~~~~~~ | ||
7552 | |||
7553 | The ``ptest-runner`` package installs a shell script that loops through | ||
7554 | all installed ptest test suites and runs them in sequence. Consequently, | ||
7555 | you might want to add this package to your image. | ||
7556 | |||
7557 | Getting Your Package Ready | ||
7558 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7559 | |||
7560 | In order to enable a recipe to run installed ptests on target hardware, | ||
7561 | you need to prepare the recipes that build the packages you want to | ||
7562 | test. Here is what you have to do for each recipe: | ||
7563 | |||
7564 | - *Be sure the recipe inherits | ||
7565 | the*\ :ref:`ptest <ref-classes-ptest>`\ *class:* | ||
7566 | Include the following line in each recipe: | ||
7567 | :: | ||
7568 | |||
7569 | inherit ptest | ||
7570 | |||
7571 | - *Create run-ptest:* This script starts your test. Locate the | ||
7572 | script where you will refer to it using | ||
7573 | :term:`SRC_URI`. Here is an | ||
7574 | example that starts a test for ``dbus``: | ||
7575 | :: | ||
7576 | |||
7577 | #!/bin/sh | ||
7578 | cd test | ||
7579 | make -k runtest-TESTS | ||
7580 | |||
7581 | - *Ensure dependencies are met:* If the test adds build or runtime | ||
7582 | dependencies that normally do not exist for the package (such as | ||
7583 | requiring "make" to run the test suite), use the | ||
7584 | :term:`DEPENDS` and | ||
7585 | :term:`RDEPENDS` variables in | ||
7586 | your recipe in order for the package to meet the dependencies. Here | ||
7587 | is an example where the package has a runtime dependency on "make": | ||
7588 | :: | ||
7589 | |||
7590 | RDEPENDS_${PN}-ptest += "make" | ||
7591 | |||
7592 | - *Add a function to build the test suite:* Not many packages support | ||
7593 | cross-compilation of their test suites. Consequently, you usually | ||
7594 | need to add a cross-compilation function to the package. | ||
7595 | |||
7596 | Many packages based on Automake compile and run the test suite by | ||
7597 | using a single command such as ``make check``. However, the host | ||
7598 | ``make check`` builds and runs on the same computer, while | ||
7599 | cross-compiling requires that the package is built on the host but | ||
7600 | executed for the target architecture (though often, as in the case | ||
7601 | for ptest, the execution occurs on the host). The built version of | ||
7602 | Automake that ships with the Yocto Project includes a patch that | ||
7603 | separates building and execution. Consequently, packages that use the | ||
7604 | unaltered, patched version of ``make check`` automatically | ||
7605 | cross-compiles. | ||
7606 | |||
7607 | Regardless, you still must add a ``do_compile_ptest`` function to | ||
7608 | build the test suite. Add a function similar to the following to your | ||
7609 | recipe: | ||
7610 | :: | ||
7611 | |||
7612 | do_compile_ptest() { | ||
7613 | oe_runmake buildtest-TESTS | ||
7614 | } | ||
7615 | |||
7616 | - *Ensure special configurations are set:* If the package requires | ||
7617 | special configurations prior to compiling the test code, you must | ||
7618 | insert a ``do_configure_ptest`` function into the recipe. | ||
7619 | |||
7620 | - *Install the test suite:* The ``ptest`` class automatically copies | ||
7621 | the file ``run-ptest`` to the target and then runs make | ||
7622 | ``install-ptest`` to run the tests. If this is not enough, you need | ||
7623 | to create a ``do_install_ptest`` function and make sure it gets | ||
7624 | called after the "make install-ptest" completes. | ||
7625 | |||
7626 | Creating Node Package Manager (NPM) Packages | ||
7627 | -------------------------------------------- | ||
7628 | |||
7629 | `NPM <https://en.wikipedia.org/wiki/Npm_(software)>`__ is a package | ||
7630 | manager for the JavaScript programming language. The Yocto Project | ||
7631 | supports the NPM :ref:`fetcher <bitbake:bb-fetchers>`. You can | ||
7632 | use this fetcher in combination with | ||
7633 | :doc:```devtool`` <../ref-manual/ref-devtool-reference>` to create | ||
7634 | recipes that produce NPM packages. | ||
7635 | |||
7636 | Two workflows exist that allow you to create NPM packages using | ||
7637 | ``devtool``: the NPM registry modules method and the NPM project code | ||
7638 | method. | ||
7639 | |||
7640 | .. note:: | ||
7641 | |||
7642 | While it is possible to create NPM recipes manually, using | ||
7643 | devtool | ||
7644 | is far simpler. | ||
7645 | |||
7646 | Additionally, some requirements and caveats exist. | ||
7647 | |||
7648 | .. _npm-package-creation-requirements: | ||
7649 | |||
7650 | Requirements and Caveats | ||
7651 | ~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7652 | |||
7653 | You need to be aware of the following before using ``devtool`` to create | ||
7654 | NPM packages: | ||
7655 | |||
7656 | - Of the two methods that you can use ``devtool`` to create NPM | ||
7657 | packages, the registry approach is slightly simpler. However, you | ||
7658 | might consider the project approach because you do not have to | ||
7659 | publish your module in the NPM registry | ||
7660 | (`npm-registry <https://docs.npmjs.com/misc/registry>`_), which | ||
7661 | is NPM's public registry. | ||
7662 | |||
7663 | - Be familiar with | ||
7664 | :doc:```devtool`` <../ref-manual/ref-devtool-reference>`. | ||
7665 | |||
7666 | - The NPM host tools need the native ``nodejs-npm`` package, which is | ||
7667 | part of the OpenEmbedded environment. You need to get the package by | ||
7668 | cloning the https://github.com/openembedded/meta-openembedded | ||
7669 | repository out of GitHub. Be sure to add the path to your local copy | ||
7670 | to your ``bblayers.conf`` file. | ||
7671 | |||
7672 | - ``devtool`` cannot detect native libraries in module dependencies. | ||
7673 | Consequently, you must manually add packages to your recipe. | ||
7674 | |||
7675 | - While deploying NPM packages, ``devtool`` cannot determine which | ||
7676 | dependent packages are missing on the target (e.g. the node runtime | ||
7677 | ``nodejs``). Consequently, you need to find out what files are | ||
7678 | missing and be sure they are on the target. | ||
7679 | |||
7680 | - Although you might not need NPM to run your node package, it is | ||
7681 | useful to have NPM on your target. The NPM package name is | ||
7682 | ``nodejs-npm``. | ||
7683 | |||
7684 | .. _npm-using-the-registry-modules-method: | ||
7685 | |||
7686 | Using the Registry Modules Method | ||
7687 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7688 | |||
7689 | This section presents an example that uses the ``cute-files`` module, | ||
7690 | which is a file browser web application. | ||
7691 | |||
7692 | .. note:: | ||
7693 | |||
7694 | You must know the | ||
7695 | cute-files | ||
7696 | module version. | ||
7697 | |||
7698 | The first thing you need to do is use ``devtool`` and the NPM fetcher to | ||
7699 | create the recipe: | ||
7700 | :: | ||
7701 | |||
7702 | $ devtool add "npm://registry.npmjs.org;package=cute-files;version=1.0.2" | ||
7703 | |||
7704 | The | ||
7705 | ``devtool add`` command runs ``recipetool create`` and uses the same | ||
7706 | fetch URI to download each dependency and capture license details where | ||
7707 | possible. The result is a generated recipe. | ||
7708 | |||
7709 | The recipe file is fairly simple and contains every license that | ||
7710 | ``recipetool`` finds and includes the licenses in the recipe's | ||
7711 | :term:`LIC_FILES_CHKSUM` | ||
7712 | variables. You need to examine the variables and look for those with | ||
7713 | "unknown" in the :term:`LICENSE` | ||
7714 | field. You need to track down the license information for "unknown" | ||
7715 | modules and manually add the information to the recipe. | ||
7716 | |||
7717 | ``recipetool`` creates a "shrinkwrap" file for your recipe. Shrinkwrap | ||
7718 | files capture the version of all dependent modules. Many packages do not | ||
7719 | provide shrinkwrap files. ``recipetool`` create a shrinkwrap file as it | ||
7720 | runs. | ||
7721 | |||
7722 | .. note:: | ||
7723 | |||
7724 | A package is created for each sub-module. This policy is the only | ||
7725 | practical way to have the licenses for all of the dependencies | ||
7726 | represented in the license manifest of the image. | ||
7727 | |||
7728 | The ``devtool edit-recipe`` command lets you take a look at the recipe: | ||
7729 | :: | ||
7730 | |||
7731 | $ devtool edit-recipe cute-files | ||
7732 | SUMMARY = "Turn any folder on your computer into a cute file browser, available on the local network." | ||
7733 | LICENSE = "MIT & ISC & Unknown" | ||
7734 | LIC_FILES_CHKSUM = "file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \ | ||
7735 | file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502 \ | ||
7736 | file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99 \ | ||
7737 | ... | ||
7738 | SRC_URI = " \ | ||
7739 | npm://registry.npmjs.org/;package=cute-files;version=${PV} \ | ||
7740 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ | ||
7741 | " | ||
7742 | S = "${WORKDIR}/npm" | ||
7743 | inherit npm LICENSE_${PN} = "MIT" | ||
7744 | LICENSE_${PN}-accepts = "MIT" | ||
7745 | LICENSE_${PN}-array-flatten = "MIT" | ||
7746 | ... | ||
7747 | LICENSE_${PN}-vary = "MIT" | ||
7748 | |||
7749 | Three key points exist in the previous example: | ||
7750 | |||
7751 | - :term:`SRC_URI` uses the NPM | ||
7752 | scheme so that the NPM fetcher is used. | ||
7753 | |||
7754 | - ``recipetool`` collects all the license information. If a | ||
7755 | sub-module's license is unavailable, the sub-module's name appears in | ||
7756 | the comments. | ||
7757 | |||
7758 | - The ``inherit npm`` statement causes the | ||
7759 | :ref:`npm <ref-classes-npm>` class to package | ||
7760 | up all the modules. | ||
7761 | |||
7762 | You can run the following command to build the ``cute-files`` package: | ||
7763 | :: | ||
7764 | |||
7765 | $ devtool build cute-files | ||
7766 | |||
7767 | Remember that ``nodejs`` must be installed on | ||
7768 | the target before your package. | ||
7769 | |||
7770 | Assuming 192.168.7.2 for the target's IP address, use the following | ||
7771 | command to deploy your package: | ||
7772 | :: | ||
7773 | |||
7774 | $ devtool deploy-target -s cute-files root@192.168.7.2 | ||
7775 | |||
7776 | Once the package is installed on the target, you can | ||
7777 | test the application: | ||
7778 | |||
7779 | .. note:: | ||
7780 | |||
7781 | Because of a know issue, you cannot simply run | ||
7782 | cute-files | ||
7783 | as you would if you had run | ||
7784 | npm install | ||
7785 | . | ||
7786 | |||
7787 | :: | ||
7788 | |||
7789 | $ cd /usr/lib/node_modules/cute-files | ||
7790 | $ node cute-files.js | ||
7791 | |||
7792 | On a browser, | ||
7793 | go to ``http://192.168.7.2:3000`` and you see the following: | ||
7794 | |||
7795 | .. image:: figures/cute-files-npm-example.png | ||
7796 | :align: center | ||
7797 | |||
7798 | You can find the recipe in ``workspace/recipes/cute-files``. You can use | ||
7799 | the recipe in any layer you choose. | ||
7800 | |||
7801 | .. _npm-using-the-npm-projects-method: | ||
7802 | |||
7803 | Using the NPM Projects Code Method | ||
7804 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
7805 | |||
7806 | Although it is useful to package modules already in the NPM registry, | ||
7807 | adding ``node.js`` projects under development is a more common developer | ||
7808 | use case. | ||
7809 | |||
7810 | This section covers the NPM projects code method, which is very similar | ||
7811 | to the "registry" approach described in the previous section. In the NPM | ||
7812 | projects method, you provide ``devtool`` with an URL that points to the | ||
7813 | source files. | ||
7814 | |||
7815 | Replicating the same example, (i.e. ``cute-files``) use the following | ||
7816 | command: | ||
7817 | :: | ||
7818 | |||
7819 | $ devtool add https://github.com/martinaglv/cute-files.git | ||
7820 | |||
7821 | The | ||
7822 | recipe this command generates is very similar to the recipe created in | ||
7823 | the previous section. However, the ``SRC_URI`` looks like the following: | ||
7824 | :: | ||
7825 | |||
7826 | SRC_URI = " \ | ||
7827 | git://github.com/martinaglv/cute-files.git;protocol=https \ | ||
7828 | npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ | ||
7829 | " | ||
7830 | |||
7831 | In this example, | ||
7832 | the main module is taken from the Git repository and dependents are | ||
7833 | taken from the NPM registry. Other than those differences, the recipe is | ||
7834 | basically the same between the two methods. You can build and deploy the | ||
7835 | package exactly as described in the previous section that uses the | ||
7836 | registry modules method. | ||
7837 | |||
7838 | Adding custom metadata to packages | ||
7839 | ---------------------------------- | ||
7840 | |||
7841 | The variable | ||
7842 | :term:`PACKAGE_ADD_METADATA` | ||
7843 | can be used to add additional metadata to packages. This is reflected in | ||
7844 | the package control/spec file. To take the ipk format for example, the | ||
7845 | CONTROL file stored inside would contain the additional metadata as | ||
7846 | additional lines. | ||
7847 | |||
7848 | The variable can be used in multiple ways, including using suffixes to | ||
7849 | set it for a specific package type and/or package. Note that the order | ||
7850 | of precedence is the same as this list: | ||
7851 | |||
7852 | - ``PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>`` | ||
7853 | |||
7854 | - ``PACKAGE_ADD_METADATA_<PKGTYPE>`` | ||
7855 | |||
7856 | - ``PACKAGE_ADD_METADATA_<PN>`` | ||
7857 | |||
7858 | - ``PACKAGE_ADD_METADATA`` | ||
7859 | |||
7860 | <PKGTYPE> is a parameter and expected to be a distinct name of specific | ||
7861 | package type: | ||
7862 | |||
7863 | - IPK for .ipk packages | ||
7864 | |||
7865 | - DEB for .deb packages | ||
7866 | |||
7867 | - RPM for .rpm packages | ||
7868 | |||
7869 | <PN> is a parameter and expected to be a package name. | ||
7870 | |||
7871 | The variable can contain multiple [one-line] metadata fields separated | ||
7872 | by the literal sequence '\n'. The separator can be redefined using the | ||
7873 | variable flag ``separator``. | ||
7874 | |||
7875 | The following is an example that adds two custom fields for ipk | ||
7876 | packages: PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup: | ||
7877 | Applications/Spreadsheets" | ||
7878 | |||
7879 | Efficiently Fetching Source Files During a Build | ||
7880 | ================================================ | ||
7881 | |||
7882 | The OpenEmbedded build system works with source files located through | ||
7883 | the :term:`SRC_URI` variable. When | ||
7884 | you build something using BitBake, a big part of the operation is | ||
7885 | locating and downloading all the source tarballs. For images, | ||
7886 | downloading all the source for various packages can take a significant | ||
7887 | amount of time. | ||
7888 | |||
7889 | This section shows you how you can use mirrors to speed up fetching | ||
7890 | source files and how you can pre-fetch files all of which leads to more | ||
7891 | efficient use of resources and time. | ||
7892 | |||
7893 | Setting up Effective Mirrors | ||
7894 | ---------------------------- | ||
7895 | |||
7896 | A good deal that goes into a Yocto Project build is simply downloading | ||
7897 | all of the source tarballs. Maybe you have been working with another | ||
7898 | build system (OpenEmbedded or Angstrom) for which you have built up a | ||
7899 | sizable directory of source tarballs. Or, perhaps someone else has such | ||
7900 | a directory for which you have read access. If so, you can save time by | ||
7901 | adding statements to your configuration file so that the build process | ||
7902 | checks local directories first for existing tarballs before checking the | ||
7903 | Internet. | ||
7904 | |||
7905 | Here is an efficient way to set it up in your ``local.conf`` file: | ||
7906 | :: | ||
7907 | |||
7908 | SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" | ||
7909 | INHERIT += "own-mirrors" | ||
7910 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
7911 | # BB_NO_NETWORK = "1" | ||
7912 | |||
7913 | In the previous example, the | ||
7914 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
7915 | variable causes the OpenEmbedded build system to generate tarballs of | ||
7916 | the Git repositories and store them in the | ||
7917 | :term:`DL_DIR` directory. Due to | ||
7918 | performance reasons, generating and storing these tarballs is not the | ||
7919 | build system's default behavior. | ||
7920 | |||
7921 | You can also use the | ||
7922 | :term:`PREMIRRORS` variable. For | ||
7923 | an example, see the variable's glossary entry in the Yocto Project | ||
7924 | Reference Manual. | ||
7925 | |||
7926 | Getting Source Files and Suppressing the Build | ||
7927 | ---------------------------------------------- | ||
7928 | |||
7929 | Another technique you can use to ready yourself for a successive string | ||
7930 | of build operations, is to pre-fetch all the source files without | ||
7931 | actually starting a build. This technique lets you work through any | ||
7932 | download issues and ultimately gathers all the source files into your | ||
7933 | download directory :ref:`structure-build-downloads`, | ||
7934 | which is located with :term:`DL_DIR`. | ||
7935 | |||
7936 | Use the following BitBake command form to fetch all the necessary | ||
7937 | sources without starting the build: | ||
7938 | :: | ||
7939 | |||
7940 | $ bitbake target --runall=fetch | ||
7941 | |||
7942 | This | ||
7943 | variation of the BitBake command guarantees that you have all the | ||
7944 | sources for that BitBake target should you disconnect from the Internet | ||
7945 | and want to do the build later offline. | ||
7946 | |||
7947 | Selecting an Initialization Manager | ||
7948 | =================================== | ||
7949 | |||
7950 | By default, the Yocto Project uses SysVinit as the initialization | ||
7951 | manager. However, support also exists for systemd, which is a full | ||
7952 | replacement for init with parallel starting of services, reduced shell | ||
7953 | overhead and other features that are used by many distributions. | ||
7954 | |||
7955 | Within the system, SysVinit treats system components as services. These | ||
7956 | services are maintained as shell scripts stored in the ``/etc/init.d/`` | ||
7957 | directory. Services organize into different run levels. This | ||
7958 | organization is maintained by putting links to the services in the | ||
7959 | ``/etc/rcN.d/`` directories, where N/ is one of the following options: | ||
7960 | "S", "0", "1", "2", "3", "4", "5", or "6". | ||
7961 | |||
7962 | .. note:: | ||
7963 | |||
7964 | Each runlevel has a dependency on the previous runlevel. This | ||
7965 | dependency allows the services to work properly. | ||
7966 | |||
7967 | In comparison, systemd treats components as units. Using units is a | ||
7968 | broader concept as compared to using a service. A unit includes several | ||
7969 | different types of entities. Service is one of the types of entities. | ||
7970 | The runlevel concept in SysVinit corresponds to the concept of a target | ||
7971 | in systemd, where target is also a type of supported unit. | ||
7972 | |||
7973 | In a SysVinit-based system, services load sequentially (i.e. one by one) | ||
7974 | during and parallelization is not supported. With systemd, services | ||
7975 | start in parallel. Needless to say, the method can have an impact on | ||
7976 | system startup performance. | ||
7977 | |||
7978 | If you want to use SysVinit, you do not have to do anything. But, if you | ||
7979 | want to use systemd, you must take some steps as described in the | ||
7980 | following sections. | ||
7981 | |||
7982 | Using systemd Exclusively | ||
7983 | ------------------------- | ||
7984 | |||
7985 | Set these variables in your distribution configuration file as follows: | ||
7986 | :: | ||
7987 | |||
7988 | DISTRO_FEATURES_append = " systemd" | ||
7989 | VIRTUAL-RUNTIME_init_manager = "systemd" | ||
7990 | |||
7991 | You can also prevent the SysVinit distribution feature from | ||
7992 | being automatically enabled as follows: | ||
7993 | :: | ||
7994 | |||
7995 | DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" | ||
7996 | |||
7997 | Doing so removes any | ||
7998 | redundant SysVinit scripts. | ||
7999 | |||
8000 | To remove initscripts from your image altogether, set this variable | ||
8001 | also: | ||
8002 | :: | ||
8003 | |||
8004 | VIRTUAL-RUNTIME_initscripts = "" | ||
8005 | |||
8006 | For information on the backfill variable, see | ||
8007 | :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`. | ||
8008 | |||
8009 | Using systemd for the Main Image and Using SysVinit for the Rescue Image | ||
8010 | ------------------------------------------------------------------------ | ||
8011 | |||
8012 | Set these variables in your distribution configuration file as follows: | ||
8013 | :: | ||
8014 | |||
8015 | DISTRO_FEATURES_append = " systemd" | ||
8016 | VIRTUAL-RUNTIME_init_manager = "systemd" | ||
8017 | |||
8018 | Doing so causes your main image to use the | ||
8019 | ``packagegroup-core-boot.bb`` recipe and systemd. The rescue/minimal | ||
8020 | image cannot use this package group. However, it can install SysVinit | ||
8021 | and the appropriate packages will have support for both systemd and | ||
8022 | SysVinit. | ||
8023 | |||
8024 | .. _selecting-dev-manager: | ||
8025 | |||
8026 | Selecting a Device Manager | ||
8027 | ========================== | ||
8028 | |||
8029 | The Yocto Project provides multiple ways to manage the device manager | ||
8030 | (``/dev``): | ||
8031 | |||
8032 | - Persistent and Pre-Populated\ ``/dev``: For this case, the ``/dev`` | ||
8033 | directory is persistent and the required device nodes are created | ||
8034 | during the build. | ||
8035 | |||
8036 | - Use ``devtmpfs`` with a Device Manager: For this case, the ``/dev`` | ||
8037 | directory is provided by the kernel as an in-memory file system and | ||
8038 | is automatically populated by the kernel at runtime. Additional | ||
8039 | configuration of device nodes is done in user space by a device | ||
8040 | manager like ``udev`` or ``busybox-mdev``. | ||
8041 | |||
8042 | .. _static-dev-management: | ||
8043 | |||
8044 | Using Persistent and Pre-Populated\ ``/dev`` | ||
8045 | -------------------------------------------- | ||
8046 | |||
8047 | To use the static method for device population, you need to set the | ||
8048 | :term:`USE_DEVFS` variable to "0" | ||
8049 | as follows: | ||
8050 | :: | ||
8051 | |||
8052 | USE_DEVFS = "0" | ||
8053 | |||
8054 | The content of the resulting ``/dev`` directory is defined in a Device | ||
8055 | Table file. The | ||
8056 | :term:`IMAGE_DEVICE_TABLES` | ||
8057 | variable defines the Device Table to use and should be set in the | ||
8058 | machine or distro configuration file. Alternatively, you can set this | ||
8059 | variable in your ``local.conf`` configuration file. | ||
8060 | |||
8061 | If you do not define the ``IMAGE_DEVICE_TABLES`` variable, the default | ||
8062 | ``device_table-minimal.txt`` is used: | ||
8063 | :: | ||
8064 | |||
8065 | IMAGE_DEVICE_TABLES = "device_table-mymachine.txt" | ||
8066 | |||
8067 | The population is handled by the ``makedevs`` utility during image | ||
8068 | creation: | ||
8069 | |||
8070 | .. _devtmpfs-dev-management: | ||
8071 | |||
8072 | Using ``devtmpfs`` and a Device Manager | ||
8073 | --------------------------------------- | ||
8074 | |||
8075 | To use the dynamic method for device population, you need to use (or be | ||
8076 | sure to set) the :term:`USE_DEVFS` | ||
8077 | variable to "1", which is the default: | ||
8078 | :: | ||
8079 | |||
8080 | USE_DEVFS = "1" | ||
8081 | |||
8082 | With this | ||
8083 | setting, the resulting ``/dev`` directory is populated by the kernel | ||
8084 | using ``devtmpfs``. Make sure the corresponding kernel configuration | ||
8085 | variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux | ||
8086 | kernel. | ||
8087 | |||
8088 | All devices created by ``devtmpfs`` will be owned by ``root`` and have | ||
8089 | permissions ``0600``. | ||
8090 | |||
8091 | To have more control over the device nodes, you can use a device manager | ||
8092 | like ``udev`` or ``busybox-mdev``. You choose the device manager by | ||
8093 | defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or | ||
8094 | distro configuration file. Alternatively, you can set this variable in | ||
8095 | your ``local.conf`` configuration file: | ||
8096 | :: | ||
8097 | |||
8098 | VIRTUAL-RUNTIME_dev_manager = "udev" | ||
8099 | |||
8100 | # Some alternative values | ||
8101 | # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev" | ||
8102 | # VIRTUAL-RUNTIME_dev_manager = "systemd" | ||
8103 | |||
8104 | .. _platdev-appdev-srcrev: | ||
8105 | |||
8106 | Using an External SCM | ||
8107 | ===================== | ||
8108 | |||
8109 | If you're working on a recipe that pulls from an external Source Code | ||
8110 | Manager (SCM), it is possible to have the OpenEmbedded build system | ||
8111 | notice new recipe changes added to the SCM and then build the resulting | ||
8112 | packages that depend on the new recipes by using the latest versions. | ||
8113 | This only works for SCMs from which it is possible to get a sensible | ||
8114 | revision number for changes. Currently, you can do this with Apache | ||
8115 | Subversion (SVN), Git, and Bazaar (BZR) repositories. | ||
8116 | |||
8117 | To enable this behavior, the :term:`PV` of | ||
8118 | the recipe needs to reference | ||
8119 | :term:`SRCPV`. Here is an example: | ||
8120 | :: | ||
8121 | |||
8122 | PV = "1.2.3+git${SRCPV}" | ||
8123 | |||
8124 | Then, you can add the following to your | ||
8125 | ``local.conf``: | ||
8126 | :: | ||
8127 | |||
8128 | SRCREV_pn-PN = "${AUTOREV}" | ||
8129 | |||
8130 | :term:`PN` is the name of the recipe for | ||
8131 | which you want to enable automatic source revision updating. | ||
8132 | |||
8133 | If you do not want to update your local configuration file, you can add | ||
8134 | the following directly to the recipe to finish enabling the feature: | ||
8135 | :: | ||
8136 | |||
8137 | SRCREV = "${AUTOREV}" | ||
8138 | |||
8139 | The Yocto Project provides a distribution named ``poky-bleeding``, whose | ||
8140 | configuration file contains the line: | ||
8141 | :: | ||
8142 | |||
8143 | require conf/distro/include/poky-floating-revisions.inc | ||
8144 | |||
8145 | This line pulls in the | ||
8146 | listed include file that contains numerous lines of exactly that form: | ||
8147 | :: | ||
8148 | |||
8149 | #SRCREV_pn-opkg-native ?= "${AUTOREV}" | ||
8150 | #SRCREV_pn-opkg-sdk ?= "${AUTOREV}" | ||
8151 | #SRCREV_pn-opkg ?= "${AUTOREV}" | ||
8152 | #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" | ||
8153 | #SRCREV_pn-opkg-utils ?= "${AUTOREV}" | ||
8154 | SRCREV_pn-gconf-dbus ?= "${AUTOREV}" | ||
8155 | SRCREV_pn-matchbox-common ?= "${AUTOREV}" | ||
8156 | SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}" | ||
8157 | SRCREV_pn-matchbox-desktop ?= "${AUTOREV}" | ||
8158 | SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" | ||
8159 | SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}" | ||
8160 | SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}" | ||
8161 | SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" | ||
8162 | SRCREV_pn-matchbox-wm ?= "${AUTOREV}" | ||
8163 | SRCREV_pn-settings-daemon ?= "${AUTOREV}" | ||
8164 | SRCREV_pn-screenshot ?= "${AUTOREV}" | ||
8165 | . . . | ||
8166 | |||
8167 | These lines allow you to | ||
8168 | experiment with building a distribution that tracks the latest | ||
8169 | development source for numerous packages. | ||
8170 | |||
8171 | .. note:: | ||
8172 | |||
8173 | The | ||
8174 | poky-bleeding | ||
8175 | distribution is not tested on a regular basis. Keep this in mind if | ||
8176 | you use it. | ||
8177 | |||
8178 | Creating a Read-Only Root Filesystem | ||
8179 | ==================================== | ||
8180 | |||
8181 | Suppose, for security reasons, you need to disable your target device's | ||
8182 | root filesystem's write permissions (i.e. you need a read-only root | ||
8183 | filesystem). Or, perhaps you are running the device's operating system | ||
8184 | from a read-only storage device. For either case, you can customize your | ||
8185 | image for that behavior. | ||
8186 | |||
8187 | .. note:: | ||
8188 | |||
8189 | Supporting a read-only root filesystem requires that the system and | ||
8190 | applications do not try to write to the root filesystem. You must | ||
8191 | configure all parts of the target system to write elsewhere, or to | ||
8192 | gracefully fail in the event of attempting to write to the root | ||
8193 | filesystem. | ||
8194 | |||
8195 | Creating the Root Filesystem | ||
8196 | ---------------------------- | ||
8197 | |||
8198 | To create the read-only root filesystem, simply add the | ||
8199 | "read-only-rootfs" feature to your image, normally in one of two ways. | ||
8200 | The first way is to add the "read-only-rootfs" image feature in the | ||
8201 | image's recipe file via the ``IMAGE_FEATURES`` variable: | ||
8202 | :: | ||
8203 | |||
8204 | IMAGE_FEATURES += "read-only-rootfs" | ||
8205 | |||
8206 | As an alternative, you can add the same feature | ||
8207 | from within your build directory's ``local.conf`` file with the | ||
8208 | associated ``EXTRA_IMAGE_FEATURES`` variable, as in: | ||
8209 | :: | ||
8210 | |||
8211 | EXTRA_IMAGE_FEATURES = "read-only-rootfs" | ||
8212 | |||
8213 | For more information on how to use these variables, see the | ||
8214 | ":ref:`usingpoky-extend-customimage-imagefeatures`" | ||
8215 | section. For information on the variables, see | ||
8216 | :term:`IMAGE_FEATURES` and | ||
8217 | :term:`EXTRA_IMAGE_FEATURES`. | ||
8218 | |||
8219 | Post-Installation Scripts and Read-Only Root Filesystem | ||
8220 | ------------------------------------------------------- | ||
8221 | |||
8222 | It is very important that you make sure all post-Installation | ||
8223 | (``pkg_postinst``) scripts for packages that are installed into the | ||
8224 | image can be run at the time when the root filesystem is created during | ||
8225 | the build on the host system. These scripts cannot attempt to run during | ||
8226 | first-boot on the target device. With the "read-only-rootfs" feature | ||
8227 | enabled, the build system checks during root filesystem creation to make | ||
8228 | sure all post-installation scripts succeed. If any of these scripts | ||
8229 | still need to be run after the root filesystem is created, the build | ||
8230 | immediately fails. These build-time checks ensure that the build fails | ||
8231 | rather than the target device fails later during its initial boot | ||
8232 | operation. | ||
8233 | |||
8234 | Most of the common post-installation scripts generated by the build | ||
8235 | system for the out-of-the-box Yocto Project are engineered so that they | ||
8236 | can run during root filesystem creation (e.g. post-installation scripts | ||
8237 | for caching fonts). However, if you create and add custom scripts, you | ||
8238 | need to be sure they can be run during this file system creation. | ||
8239 | |||
8240 | Here are some common problems that prevent post-installation scripts | ||
8241 | from running during root filesystem creation: | ||
8242 | |||
8243 | - *Not using $D in front of absolute paths:* The build system defines | ||
8244 | ``$``\ :term:`D` when the root | ||
8245 | filesystem is created. Furthermore, ``$D`` is blank when the script | ||
8246 | is run on the target device. This implies two purposes for ``$D``: | ||
8247 | ensuring paths are valid in both the host and target environments, | ||
8248 | and checking to determine which environment is being used as a method | ||
8249 | for taking appropriate actions. | ||
8250 | |||
8251 | - *Attempting to run processes that are specific to or dependent on the | ||
8252 | target architecture:* You can work around these attempts by using | ||
8253 | native tools, which run on the host system, to accomplish the same | ||
8254 | tasks, or by alternatively running the processes under QEMU, which | ||
8255 | has the ``qemu_run_binary`` function. For more information, see the | ||
8256 | :ref:`qemu <ref-classes-qemu>` class. | ||
8257 | |||
8258 | Areas With Write Access | ||
8259 | ----------------------- | ||
8260 | |||
8261 | With the "read-only-rootfs" feature enabled, any attempt by the target | ||
8262 | to write to the root filesystem at runtime fails. Consequently, you must | ||
8263 | make sure that you configure processes and applications that attempt | ||
8264 | these types of writes do so to directories with write access (e.g. | ||
8265 | ``/tmp`` or ``/var/run``). | ||
8266 | |||
8267 | Maintaining Build Output Quality | ||
8268 | ================================ | ||
8269 | |||
8270 | Many factors can influence the quality of a build. For example, if you | ||
8271 | upgrade a recipe to use a new version of an upstream software package or | ||
8272 | you experiment with some new configuration options, subtle changes can | ||
8273 | occur that you might not detect until later. Consider the case where | ||
8274 | your recipe is using a newer version of an upstream package. In this | ||
8275 | case, a new version of a piece of software might introduce an optional | ||
8276 | dependency on another library, which is auto-detected. If that library | ||
8277 | has already been built when the software is building, the software will | ||
8278 | link to the built library and that library will be pulled into your | ||
8279 | image along with the new software even if you did not want the library. | ||
8280 | |||
8281 | The :ref:`buildhistory <ref-classes-buildhistory>` | ||
8282 | class exists to help you maintain the quality of your build output. You | ||
8283 | can use the class to highlight unexpected and possibly unwanted changes | ||
8284 | in the build output. When you enable build history, it records | ||
8285 | information about the contents of each package and image and then | ||
8286 | commits that information to a local Git repository where you can examine | ||
8287 | the information. | ||
8288 | |||
8289 | The remainder of this section describes the following: | ||
8290 | |||
8291 | - How you can enable and disable build history | ||
8292 | |||
8293 | - How to understand what the build history contains | ||
8294 | |||
8295 | - How to limit the information used for build history | ||
8296 | |||
8297 | - How to examine the build history from both a command-line and web | ||
8298 | interface | ||
8299 | |||
8300 | Enabling and Disabling Build History | ||
8301 | ------------------------------------ | ||
8302 | |||
8303 | Build history is disabled by default. To enable it, add the following | ||
8304 | ``INHERIT`` statement and set the | ||
8305 | :term:`BUILDHISTORY_COMMIT` | ||
8306 | variable to "1" at the end of your ``conf/local.conf`` file found in the | ||
8307 | :term:`Build Directory`: | ||
8308 | :: | ||
8309 | |||
8310 | INHERIT += "buildhistory" | ||
8311 | BUILDHISTORY_COMMIT = "1" | ||
8312 | |||
8313 | Enabling build history as | ||
8314 | previously described causes the OpenEmbedded build system to collect | ||
8315 | build output information and commit it as a single commit to a local | ||
8316 | :ref:`overview-manual/overview-manual-development-environment:git` repository. | ||
8317 | |||
8318 | .. note:: | ||
8319 | |||
8320 | Enabling build history increases your build times slightly, | ||
8321 | particularly for images, and increases the amount of disk space used | ||
8322 | during the build. | ||
8323 | |||
8324 | You can disable build history by removing the previous statements from | ||
8325 | your ``conf/local.conf`` file. | ||
8326 | |||
8327 | Understanding What the Build History Contains | ||
8328 | --------------------------------------------- | ||
8329 | |||
8330 | Build history information is kept in | ||
8331 | ``${``\ :term:`TOPDIR`\ ``}/buildhistory`` | ||
8332 | in the Build Directory as defined by the | ||
8333 | :term:`BUILDHISTORY_DIR` | ||
8334 | variable. The following is an example abbreviated listing: | ||
8335 | |||
8336 | .. image:: figures/buildhistory.png | ||
8337 | :align: center | ||
8338 | |||
8339 | At the top level, a ``metadata-revs`` file exists that lists the | ||
8340 | revisions of the repositories for the enabled layers when the build was | ||
8341 | produced. The rest of the data splits into separate ``packages``, | ||
8342 | ``images`` and ``sdk`` directories, the contents of which are described | ||
8343 | as follows. | ||
8344 | |||
8345 | Build History Package Information | ||
8346 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8347 | |||
8348 | The history for each package contains a text file that has name-value | ||
8349 | pairs with information about the package. For example, | ||
8350 | ``buildhistory/packages/i586-poky-linux/busybox/busybox/latest`` | ||
8351 | contains the following: | ||
8352 | :: | ||
8353 | |||
8354 | PV = 1.22.1 | ||
8355 | PR = r32 | ||
8356 | RPROVIDES = | ||
8357 | RDEPENDS = glibc (>= 2.20) update-alternatives-opkg | ||
8358 | RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d | ||
8359 | PKGSIZE = 540168 | ||
8360 | FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \ | ||
8361 | /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \ | ||
8362 | /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \ | ||
8363 | /usr/share/pixmaps /usr/share/applications /usr/share/idl \ | ||
8364 | /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers | ||
8365 | FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \ | ||
8366 | /etc/busybox.links.nosuid /etc/busybox.links.suid | ||
8367 | |||
8368 | Most of these | ||
8369 | name-value pairs correspond to variables used to produce the package. | ||
8370 | The exceptions are ``FILELIST``, which is the actual list of files in | ||
8371 | the package, and ``PKGSIZE``, which is the total size of files in the | ||
8372 | package in bytes. | ||
8373 | |||
8374 | A file also exists that corresponds to the recipe from which the package | ||
8375 | came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``): | ||
8376 | :: | ||
8377 | |||
8378 | PV = 1.22.1 | ||
8379 | PR = r32 | ||
8380 | DEPENDS = initscripts kern-tools-native update-rc.d-native \ | ||
8381 | virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \ | ||
8382 | virtual/libc virtual/update-alternatives | ||
8383 | PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \ | ||
8384 | busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \ | ||
8385 | busybox-staticdev busybox-dev busybox-doc busybox-locale busybox | ||
8386 | |||
8387 | Finally, for those recipes fetched from a version control system (e.g., | ||
8388 | Git), a file exists that lists source revisions that are specified in | ||
8389 | the recipe and lists the actual revisions used during the build. Listed | ||
8390 | and actual revisions might differ when | ||
8391 | :term:`SRCREV` is set to | ||
8392 | ${:term:`AUTOREV`}. Here is an | ||
8393 | example assuming | ||
8394 | ``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``): | ||
8395 | :: | ||
8396 | |||
8397 | # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8398 | SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8399 | # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8400 | SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8401 | |||
8402 | You can use the | ||
8403 | ``buildhistory-collect-srcrevs`` command with the ``-a`` option to | ||
8404 | collect the stored ``SRCREV`` values from build history and report them | ||
8405 | in a format suitable for use in global configuration (e.g., | ||
8406 | ``local.conf`` or a distro include file) to override floating | ||
8407 | ``AUTOREV`` values to a fixed set of revisions. Here is some example | ||
8408 | output from this command: | ||
8409 | :: | ||
8410 | |||
8411 | $ buildhistory-collect-srcrevs -a | ||
8412 | # i586-poky-linux | ||
8413 | SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8414 | SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8415 | SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a" | ||
8416 | SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" | ||
8417 | # x86_64-linux | ||
8418 | SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa" | ||
8419 | SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" | ||
8420 | SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11" | ||
8421 | SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072" | ||
8422 | SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3" | ||
8423 | SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca" | ||
8424 | SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a" | ||
8425 | SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" | ||
8426 | SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" | ||
8427 | # qemux86-poky-linux | ||
8428 | SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" | ||
8429 | SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f" | ||
8430 | # all-poky-linux | ||
8431 | SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11" | ||
8432 | |||
8433 | .. note:: | ||
8434 | |||
8435 | Here are some notes on using the | ||
8436 | buildhistory-collect-srcrevs | ||
8437 | command: | ||
8438 | |||
8439 | - By default, only values where the ``SRCREV`` was not hardcoded | ||
8440 | (usually when ``AUTOREV`` is used) are reported. Use the ``-a`` | ||
8441 | option to see all ``SRCREV`` values. | ||
8442 | |||
8443 | - The output statements might not have any effect if overrides are | ||
8444 | applied elsewhere in the build system configuration. Use the | ||
8445 | ``-f`` option to add the ``forcevariable`` override to each output | ||
8446 | line if you need to work around this restriction. | ||
8447 | |||
8448 | - The script does apply special handling when building for multiple | ||
8449 | machines. However, the script does place a comment before each set | ||
8450 | of values that specifies which triplet to which they belong as | ||
8451 | previously shown (e.g., ``i586-poky-linux``). | ||
8452 | |||
8453 | Build History Image Information | ||
8454 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8455 | |||
8456 | The files produced for each image are as follows: | ||
8457 | |||
8458 | - ``image-files:`` A directory containing selected files from the root | ||
8459 | filesystem. The files are defined by | ||
8460 | :term:`BUILDHISTORY_IMAGE_FILES`. | ||
8461 | |||
8462 | - ``build-id.txt:`` Human-readable information about the build | ||
8463 | configuration and metadata source revisions. This file contains the | ||
8464 | full build header as printed by BitBake. | ||
8465 | |||
8466 | - ``*.dot:`` Dependency graphs for the image that are compatible with | ||
8467 | ``graphviz``. | ||
8468 | |||
8469 | - ``files-in-image.txt:`` A list of files in the image with | ||
8470 | permissions, owner, group, size, and symlink information. | ||
8471 | |||
8472 | - ``image-info.txt:`` A text file containing name-value pairs with | ||
8473 | information about the image. See the following listing example for | ||
8474 | more information. | ||
8475 | |||
8476 | - ``installed-package-names.txt:`` A list of installed packages by name | ||
8477 | only. | ||
8478 | |||
8479 | - ``installed-package-sizes.txt:`` A list of installed packages ordered | ||
8480 | by size. | ||
8481 | |||
8482 | - ``installed-packages.txt:`` A list of installed packages with full | ||
8483 | package filenames. | ||
8484 | |||
8485 | .. note:: | ||
8486 | |||
8487 | Installed package information is able to be gathered and produced | ||
8488 | even if package management is disabled for the final image. | ||
8489 | |||
8490 | Here is an example of ``image-info.txt``: | ||
8491 | :: | ||
8492 | |||
8493 | DISTRO = poky | ||
8494 | DISTRO_VERSION = 1.7 | ||
8495 | USER_CLASSES = buildstats image-mklibs image-prelink | ||
8496 | IMAGE_CLASSES = image_types | ||
8497 | IMAGE_FEATURES = debug-tweaks | ||
8498 | IMAGE_LINGUAS = | ||
8499 | IMAGE_INSTALL = packagegroup-core-boot run-postinsts | ||
8500 | BAD_RECOMMENDATIONS = | ||
8501 | NO_RECOMMENDATIONS = | ||
8502 | PACKAGE_EXCLUDE = | ||
8503 | ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \ | ||
8504 | write_image_manifest ; buildhistory_list_installed_image ; \ | ||
8505 | buildhistory_get_image_installed ; ssh_allow_empty_password; \ | ||
8506 | postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ; | ||
8507 | IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ; | ||
8508 | IMAGESIZE = 6900 | ||
8509 | |||
8510 | Other than ``IMAGESIZE``, | ||
8511 | which is the total size of the files in the image in Kbytes, the | ||
8512 | name-value pairs are variables that may have influenced the content of | ||
8513 | the image. This information is often useful when you are trying to | ||
8514 | determine why a change in the package or file listings has occurred. | ||
8515 | |||
8516 | Using Build History to Gather Image Information Only | ||
8517 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8518 | |||
8519 | As you can see, build history produces image information, including | ||
8520 | dependency graphs, so you can see why something was pulled into the | ||
8521 | image. If you are just interested in this information and not interested | ||
8522 | in collecting specific package or SDK information, you can enable | ||
8523 | writing only image information without any history by adding the | ||
8524 | following to your ``conf/local.conf`` file found in the | ||
8525 | :term:`Build Directory`: | ||
8526 | :: | ||
8527 | |||
8528 | INHERIT += "buildhistory" | ||
8529 | BUILDHISTORY_COMMIT = "0" | ||
8530 | BUILDHISTORY_FEATURES = "image" | ||
8531 | |||
8532 | Here, you set the | ||
8533 | :term:`BUILDHISTORY_FEATURES` | ||
8534 | variable to use the image feature only. | ||
8535 | |||
8536 | Build History SDK Information | ||
8537 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8538 | |||
8539 | Build history collects similar information on the contents of SDKs (e.g. | ||
8540 | ``bitbake -c populate_sdk imagename``) as compared to information it | ||
8541 | collects for images. Furthermore, this information differs depending on | ||
8542 | whether an extensible or standard SDK is being produced. | ||
8543 | |||
8544 | The following list shows the files produced for SDKs: | ||
8545 | |||
8546 | - ``files-in-sdk.txt:`` A list of files in the SDK with permissions, | ||
8547 | owner, group, size, and symlink information. This list includes both | ||
8548 | the host and target parts of the SDK. | ||
8549 | |||
8550 | - ``sdk-info.txt:`` A text file containing name-value pairs with | ||
8551 | information about the SDK. See the following listing example for more | ||
8552 | information. | ||
8553 | |||
8554 | - ``sstate-task-sizes.txt:`` A text file containing name-value pairs | ||
8555 | with information about task group sizes (e.g. ``do_populate_sysroot`` | ||
8556 | tasks have a total size). The ``sstate-task-sizes.txt`` file exists | ||
8557 | only when an extensible SDK is created. | ||
8558 | |||
8559 | - ``sstate-package-sizes.txt:`` A text file containing name-value pairs | ||
8560 | with information for the shared-state packages and sizes in the SDK. | ||
8561 | The ``sstate-package-sizes.txt`` file exists only when an extensible | ||
8562 | SDK is created. | ||
8563 | |||
8564 | - ``sdk-files:`` A folder that contains copies of the files mentioned | ||
8565 | in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output. | ||
8566 | Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is | ||
8567 | specific to the extensible SDK although you can set it differently if | ||
8568 | you would like to pull in specific files from the standard SDK. | ||
8569 | |||
8570 | The default files are ``conf/local.conf``, ``conf/bblayers.conf``, | ||
8571 | ``conf/auto.conf``, ``conf/locked-sigs.inc``, and | ||
8572 | ``conf/devtool.conf``. Thus, for an extensible SDK, these files get | ||
8573 | copied into the ``sdk-files`` directory. | ||
8574 | |||
8575 | - The following information appears under each of the ``host`` and | ||
8576 | ``target`` directories for the portions of the SDK that run on the | ||
8577 | host and on the target, respectively: | ||
8578 | |||
8579 | .. note:: | ||
8580 | |||
8581 | The following files for the most part are empty when producing an | ||
8582 | extensible SDK because this type of SDK is not constructed from | ||
8583 | packages as is the standard SDK. | ||
8584 | |||
8585 | - ``depends.dot:`` Dependency graph for the SDK that is compatible | ||
8586 | with ``graphviz``. | ||
8587 | |||
8588 | - ``installed-package-names.txt:`` A list of installed packages by | ||
8589 | name only. | ||
8590 | |||
8591 | - ``installed-package-sizes.txt:`` A list of installed packages | ||
8592 | ordered by size. | ||
8593 | |||
8594 | - ``installed-packages.txt:`` A list of installed packages with full | ||
8595 | package filenames. | ||
8596 | |||
8597 | Here is an example of ``sdk-info.txt``: | ||
8598 | :: | ||
8599 | |||
8600 | DISTRO = poky | ||
8601 | DISTRO_VERSION = 1.3+snapshot-20130327 | ||
8602 | SDK_NAME = poky-glibc-i686-arm | ||
8603 | SDK_VERSION = 1.3+snapshot | ||
8604 | SDKMACHINE = | ||
8605 | SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs | ||
8606 | BAD_RECOMMENDATIONS = | ||
8607 | SDKSIZE = 352712 | ||
8608 | |||
8609 | Other than ``SDKSIZE``, which is | ||
8610 | the total size of the files in the SDK in Kbytes, the name-value pairs | ||
8611 | are variables that might have influenced the content of the SDK. This | ||
8612 | information is often useful when you are trying to determine why a | ||
8613 | change in the package or file listings has occurred. | ||
8614 | |||
8615 | Examining Build History Information | ||
8616 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8617 | |||
8618 | You can examine build history output from the command line or from a web | ||
8619 | interface. | ||
8620 | |||
8621 | To see any changes that have occurred (assuming you have | ||
8622 | :term:`BUILDHISTORY_COMMIT` = "1"), | ||
8623 | you can simply use any Git command that allows you to view the history | ||
8624 | of a repository. Here is one method: | ||
8625 | :: | ||
8626 | |||
8627 | $ git log -p | ||
8628 | |||
8629 | You need to realize, | ||
8630 | however, that this method does show changes that are not significant | ||
8631 | (e.g. a package's size changing by a few bytes). | ||
8632 | |||
8633 | A command-line tool called ``buildhistory-diff`` does exist, though, | ||
8634 | that queries the Git repository and prints just the differences that | ||
8635 | might be significant in human-readable form. Here is an example: | ||
8636 | :: | ||
8637 | |||
8638 | $ ~/poky/poky/scripts/buildhistory-diff . HEAD^ | ||
8639 | Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt): | ||
8640 | /etc/anotherpkg.conf was added | ||
8641 | /sbin/anotherpkg was added | ||
8642 | * (installed-package-names.txt): | ||
8643 | * anotherpkg was added | ||
8644 | Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt): | ||
8645 | anotherpkg was added | ||
8646 | packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" | ||
8647 | * PR changed from "r0" to "r1" | ||
8648 | * PV changed from "0.1.10" to "0.1.12" | ||
8649 | packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%) | ||
8650 | * PR changed from "r0" to "r1" | ||
8651 | * PV changed from "0.1.10" to "0.1.12" | ||
8652 | |||
8653 | .. note:: | ||
8654 | |||
8655 | The | ||
8656 | buildhistory-diff | ||
8657 | tool requires the | ||
8658 | GitPython | ||
8659 | package. Be sure to install it using Pip3 as follows: | ||
8660 | :: | ||
8661 | |||
8662 | $ pip3 install GitPython --user | ||
8663 | |||
8664 | |||
8665 | Alternatively, you can install | ||
8666 | python3-git | ||
8667 | using the appropriate distribution package manager (e.g. | ||
8668 | apt-get | ||
8669 | , | ||
8670 | dnf | ||
8671 | , or | ||
8672 | zipper | ||
8673 | ). | ||
8674 | |||
8675 | To see changes to the build history using a web interface, follow the | ||
8676 | instruction in the ``README`` file here. | ||
8677 | http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/. | ||
8678 | |||
8679 | Here is a sample screenshot of the interface: | ||
8680 | |||
8681 | .. image:: figures/buildhistory-web.png | ||
8682 | :align: center | ||
8683 | |||
8684 | Performing Automated Runtime Testing | ||
8685 | ==================================== | ||
8686 | |||
8687 | The OpenEmbedded build system makes available a series of automated | ||
8688 | tests for images to verify runtime functionality. You can run these | ||
8689 | tests on either QEMU or actual target hardware. Tests are written in | ||
8690 | Python making use of the ``unittest`` module, and the majority of them | ||
8691 | run commands on the target system over SSH. This section describes how | ||
8692 | you set up the environment to use these tests, run available tests, and | ||
8693 | write and add your own tests. | ||
8694 | |||
8695 | For information on the test and QA infrastructure available within the | ||
8696 | Yocto Project, see the ":ref:`ref-manual/ref-release-process:testing and quality assurance`" | ||
8697 | section in the Yocto Project Reference Manual. | ||
8698 | |||
8699 | Enabling Tests | ||
8700 | -------------- | ||
8701 | |||
8702 | Depending on whether you are planning to run tests using QEMU or on the | ||
8703 | hardware, you have to take different steps to enable the tests. See the | ||
8704 | following subsections for information on how to enable both types of | ||
8705 | tests. | ||
8706 | |||
8707 | .. _qemu-image-enabling-tests: | ||
8708 | |||
8709 | Enabling Runtime Tests on QEMU | ||
8710 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8711 | |||
8712 | In order to run tests, you need to do the following: | ||
8713 | |||
8714 | - *Set up to avoid interaction with sudo for networking:* To | ||
8715 | accomplish this, you must do one of the following: | ||
8716 | |||
8717 | - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all | ||
8718 | commands or just for ``runqemu-ifup``. You must provide the full | ||
8719 | path as that can change if you are using multiple clones of the | ||
8720 | source repository. | ||
8721 | |||
8722 | .. note:: | ||
8723 | |||
8724 | On some distributions, you also need to comment out "Defaults | ||
8725 | requiretty" in | ||
8726 | /etc/sudoers | ||
8727 | . | ||
8728 | |||
8729 | - Manually configure a tap interface for your system. | ||
8730 | |||
8731 | - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which | ||
8732 | should generate a list of tap devices. This is the option | ||
8733 | typically chosen for Autobuilder-type environments. | ||
8734 | |||
8735 | .. note:: | ||
8736 | |||
8737 | - Be sure to use an absolute path when calling this script | ||
8738 | with sudo. | ||
8739 | |||
8740 | - The package recipe ``qemu-helper-native`` is required to run | ||
8741 | this script. Build the package using the following command: | ||
8742 | $ bitbake qemu-helper-native | ||
8743 | |||
8744 | - *Set the DISPLAY variable:* You need to set this variable so that | ||
8745 | you have an X server available (e.g. start ``vncserver`` for a | ||
8746 | headless machine). | ||
8747 | |||
8748 | - *Be sure your host's firewall accepts incoming connections from | ||
8749 | 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an | ||
8750 | HTTP server on a random high number port, which is used to serve | ||
8751 | files to the target. The DNF module serves | ||
8752 | ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands. | ||
8753 | That means your host's firewall must accept incoming connections from | ||
8754 | 192.168.7.0/24, which is the default IP range used for tap devices by | ||
8755 | ``runqemu``. | ||
8756 | |||
8757 | - *Be sure your host has the correct packages installed:* Depending | ||
8758 | your host's distribution, you need to have the following packages | ||
8759 | installed: | ||
8760 | |||
8761 | - Ubuntu and Debian: ``sysstat`` and ``iproute2`` | ||
8762 | |||
8763 | - OpenSUSE: ``sysstat`` and ``iproute2`` | ||
8764 | |||
8765 | - Fedora: ``sysstat`` and ``iproute`` | ||
8766 | |||
8767 | - CentOS: ``sysstat`` and ``iproute`` | ||
8768 | |||
8769 | Once you start running the tests, the following happens: | ||
8770 | |||
8771 | 1. A copy of the root filesystem is written to ``${WORKDIR}/testimage``. | ||
8772 | |||
8773 | 2. The image is booted under QEMU using the standard ``runqemu`` script. | ||
8774 | |||
8775 | 3. A default timeout of 500 seconds occurs to allow for the boot process | ||
8776 | to reach the login prompt. You can change the timeout period by | ||
8777 | setting | ||
8778 | :term:`TEST_QEMUBOOT_TIMEOUT` | ||
8779 | in the ``local.conf`` file. | ||
8780 | |||
8781 | 4. Once the boot process is reached and the login prompt appears, the | ||
8782 | tests run. The full boot log is written to | ||
8783 | ``${WORKDIR}/testimage/qemu_boot_log``. | ||
8784 | |||
8785 | 5. Each test module loads in the order found in ``TEST_SUITES``. You can | ||
8786 | find the full output of the commands run over SSH in | ||
8787 | ``${WORKDIR}/testimgage/ssh_target_log``. | ||
8788 | |||
8789 | 6. If no failures occur, the task running the tests ends successfully. | ||
8790 | You can find the output from the ``unittest`` in the task log at | ||
8791 | ``${WORKDIR}/temp/log.do_testimage``. | ||
8792 | |||
8793 | .. _hardware-image-enabling-tests: | ||
8794 | |||
8795 | Enabling Runtime Tests on Hardware | ||
8796 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8797 | |||
8798 | The OpenEmbedded build system can run tests on real hardware, and for | ||
8799 | certain devices it can also deploy the image to be tested onto the | ||
8800 | device beforehand. | ||
8801 | |||
8802 | For automated deployment, a "master image" is installed onto the | ||
8803 | hardware once as part of setup. Then, each time tests are to be run, the | ||
8804 | following occurs: | ||
8805 | |||
8806 | 1. The master image is booted into and used to write the image to be | ||
8807 | tested to a second partition. | ||
8808 | |||
8809 | 2. The device is then rebooted using an external script that you need to | ||
8810 | provide. | ||
8811 | |||
8812 | 3. The device boots into the image to be tested. | ||
8813 | |||
8814 | When running tests (independent of whether the image has been deployed | ||
8815 | automatically or not), the device is expected to be connected to a | ||
8816 | network on a pre-determined IP address. You can either use static IP | ||
8817 | addresses written into the image, or set the image to use DHCP and have | ||
8818 | your DHCP server on the test network assign a known IP address based on | ||
8819 | the MAC address of the device. | ||
8820 | |||
8821 | In order to run tests on hardware, you need to set ``TEST_TARGET`` to an | ||
8822 | appropriate value. For QEMU, you do not have to change anything, the | ||
8823 | default value is "qemu". For running tests on hardware, the following | ||
8824 | options exist: | ||
8825 | |||
8826 | - *"simpleremote":* Choose "simpleremote" if you are going to run tests | ||
8827 | on a target system that is already running the image to be tested and | ||
8828 | is available on the network. You can use "simpleremote" in | ||
8829 | conjunction with either real hardware or an image running within a | ||
8830 | separately started QEMU or any other virtual machine manager. | ||
8831 | |||
8832 | - *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is | ||
8833 | an EFI-based machine with ``systemd-boot`` as bootloader and | ||
8834 | ``core-image-testmaster`` (or something similar) is installed. Also, | ||
8835 | your hardware under test must be in a DHCP-enabled network that gives | ||
8836 | it the same IP address for each reboot. | ||
8837 | |||
8838 | If you choose "SystemdbootTarget", there are additional requirements | ||
8839 | and considerations. See the "`Selecting | ||
8840 | SystemdbootTarget <#selecting-systemdboottarget>`__" section, which | ||
8841 | follows, for more information. | ||
8842 | |||
8843 | - *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying | ||
8844 | images and running tests on the BeagleBone "Black" or original | ||
8845 | "White" hardware. For information on how to use these tests, see the | ||
8846 | comments at the top of the BeagleBoneTarget | ||
8847 | ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file. | ||
8848 | |||
8849 | - *"EdgeRouterTarget":* Choose "EdgeRouterTarget" is you are deploying | ||
8850 | images and running tests on the Ubiquiti Networks EdgeRouter Lite. | ||
8851 | For information on how to use these tests, see the comments at the | ||
8852 | top of the EdgeRouterTarget | ||
8853 | ``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file. | ||
8854 | |||
8855 | - *"GrubTarget":* Choose the "supports deploying images and running | ||
8856 | tests on any generic PC that boots using GRUB. For information on how | ||
8857 | to use these tests, see the comments at the top of the GrubTarget | ||
8858 | ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file. | ||
8859 | |||
8860 | - *"your-target":* Create your own custom target if you want to run | ||
8861 | tests when you are deploying images and running tests on a custom | ||
8862 | machine within your BSP layer. To do this, you need to add a Python | ||
8863 | unit that defines the target class under ``lib/oeqa/controllers/`` | ||
8864 | within your layer. You must also provide an empty ``__init__.py``. | ||
8865 | For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``. | ||
8866 | |||
8867 | Selecting SystemdbootTarget | ||
8868 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8869 | |||
8870 | If you did not set ``TEST_TARGET`` to "SystemdbootTarget", then you do | ||
8871 | not need any information in this section. You can skip down to the | ||
8872 | "`Running Tests <#qemu-image-running-tests>`__" section. | ||
8873 | |||
8874 | If you did set ``TEST_TARGET`` to "SystemdbootTarget", you also need to | ||
8875 | perform a one-time setup of your master image by doing the following: | ||
8876 | |||
8877 | 1. *Set EFI_PROVIDER:* Be sure that ``EFI_PROVIDER`` is as follows: | ||
8878 | :: | ||
8879 | |||
8880 | EFI_PROVIDER = "systemd-boot" | ||
8881 | |||
8882 | 2. *Build the master image:* Build the ``core-image-testmaster`` image. | ||
8883 | The ``core-image-testmaster`` recipe is provided as an example for a | ||
8884 | "master" image and you can customize the image recipe as you would | ||
8885 | any other recipe. | ||
8886 | |||
8887 | Here are the image recipe requirements: | ||
8888 | |||
8889 | - Inherits ``core-image`` so that kernel modules are installed. | ||
8890 | |||
8891 | - Installs normal linux utilities not busybox ones (e.g. ``bash``, | ||
8892 | ``coreutils``, ``tar``, ``gzip``, and ``kmod``). | ||
8893 | |||
8894 | - Uses a custom Initial RAM Disk (initramfs) image with a custom | ||
8895 | installer. A normal image that you can install usually creates a | ||
8896 | single rootfs partition. This image uses another installer that | ||
8897 | creates a specific partition layout. Not all Board Support | ||
8898 | Packages (BSPs) can use an installer. For such cases, you need to | ||
8899 | manually create the following partition layout on the target: | ||
8900 | |||
8901 | - First partition mounted under ``/boot``, labeled "boot". | ||
8902 | |||
8903 | - The main rootfs partition where this image gets installed, | ||
8904 | which is mounted under ``/``. | ||
8905 | |||
8906 | - Another partition labeled "testrootfs" where test images get | ||
8907 | deployed. | ||
8908 | |||
8909 | 3. *Install image:* Install the image that you just built on the target | ||
8910 | system. | ||
8911 | |||
8912 | The final thing you need to do when setting ``TEST_TARGET`` to | ||
8913 | "SystemdbootTarget" is to set up the test image: | ||
8914 | |||
8915 | 1. *Set up your local.conf file:* Make sure you have the following | ||
8916 | statements in your ``local.conf`` file: | ||
8917 | :: | ||
8918 | |||
8919 | IMAGE_FSTYPES += "tar.gz" | ||
8920 | INHERIT += "testimage" | ||
8921 | TEST_TARGET = "SystemdbootTarget" | ||
8922 | TEST_TARGET_IP = "192.168.2.3" | ||
8923 | |||
8924 | 2. *Build your test image:* Use BitBake to build the image: | ||
8925 | :: | ||
8926 | |||
8927 | $ bitbake core-image-sato | ||
8928 | |||
8929 | Power Control | ||
8930 | ~~~~~~~~~~~~~ | ||
8931 | |||
8932 | For most hardware targets other than "simpleremote", you can control | ||
8933 | power: | ||
8934 | |||
8935 | - You can use ``TEST_POWERCONTROL_CMD`` together with | ||
8936 | ``TEST_POWERCONTROL_EXTRA_ARGS`` as a command that runs on the host | ||
8937 | and does power cycling. The test code passes one argument to that | ||
8938 | command: off, on or cycle (off then on). Here is an example that | ||
8939 | could appear in your ``local.conf`` file: | ||
8940 | :: | ||
8941 | |||
8942 | TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" | ||
8943 | |||
8944 | In this example, the expect | ||
8945 | script does the following: | ||
8946 | :: | ||
8947 | |||
8948 | ssh test@10.11.12.1 "pyctl nuc1 arg" | ||
8949 | |||
8950 | It then runs a Python script that controls power for a label called | ||
8951 | ``nuc1``. | ||
8952 | |||
8953 | .. note:: | ||
8954 | |||
8955 | You need to customize | ||
8956 | TEST_POWERCONTROL_CMD | ||
8957 | and | ||
8958 | TEST_POWERCONTROL_EXTRA_ARGS | ||
8959 | for your own setup. The one requirement is that it accepts "on", | ||
8960 | "off", and "cycle" as the last argument. | ||
8961 | |||
8962 | - When no command is defined, it connects to the device over SSH and | ||
8963 | uses the classic reboot command to reboot the device. Classic reboot | ||
8964 | is fine as long as the machine actually reboots (i.e. the SSH test | ||
8965 | has not failed). It is useful for scenarios where you have a simple | ||
8966 | setup, typically with a single board, and where some manual | ||
8967 | interaction is okay from time to time. | ||
8968 | |||
8969 | If you have no hardware to automatically perform power control but still | ||
8970 | wish to experiment with automated hardware testing, you can use the | ||
8971 | dialog-power-control script that shows a dialog prompting you to perform | ||
8972 | the required power action. This script requires either KDialog or Zenity | ||
8973 | to be installed. To use this script, set the | ||
8974 | :term:`TEST_POWERCONTROL_CMD` | ||
8975 | variable as follows: | ||
8976 | :: | ||
8977 | |||
8978 | TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" | ||
8979 | |||
8980 | Serial Console Connection | ||
8981 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
8982 | |||
8983 | For test target classes requiring a serial console to interact with the | ||
8984 | bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget), | ||
8985 | you need to specify a command to use to connect to the serial console of | ||
8986 | the target machine by using the | ||
8987 | :term:`TEST_SERIALCONTROL_CMD` | ||
8988 | variable and optionally the | ||
8989 | :term:`TEST_SERIALCONTROL_EXTRA_ARGS` | ||
8990 | variable. | ||
8991 | |||
8992 | These cases could be a serial terminal program if the machine is | ||
8993 | connected to a local serial port, or a ``telnet`` or ``ssh`` command | ||
8994 | connecting to a remote console server. Regardless of the case, the | ||
8995 | command simply needs to connect to the serial console and forward that | ||
8996 | connection to standard input and output as any normal terminal program | ||
8997 | does. For example, to use the picocom terminal program on serial device | ||
8998 | ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows: | ||
8999 | :: | ||
9000 | |||
9001 | TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" | ||
9002 | |||
9003 | For local | ||
9004 | devices where the serial port device disappears when the device reboots, | ||
9005 | an additional "serdevtry" wrapper script is provided. To use this | ||
9006 | wrapper, simply prefix the terminal command with | ||
9007 | ``${COREBASE}/scripts/contrib/serdevtry``: | ||
9008 | :: | ||
9009 | |||
9010 | TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0" | ||
9011 | |||
9012 | .. _qemu-image-running-tests: | ||
9013 | |||
9014 | Running Tests | ||
9015 | ------------- | ||
9016 | |||
9017 | You can start the tests automatically or manually: | ||
9018 | |||
9019 | - *Automatically running tests:* To run the tests automatically after | ||
9020 | the OpenEmbedded build system successfully creates an image, first | ||
9021 | set the | ||
9022 | :term:`TESTIMAGE_AUTO` | ||
9023 | variable to "1" in your ``local.conf`` file in the | ||
9024 | :term:`Build Directory`: | ||
9025 | :: | ||
9026 | |||
9027 | TESTIMAGE_AUTO = "1" | ||
9028 | |||
9029 | Next, build your image. If the image successfully builds, the | ||
9030 | tests run: | ||
9031 | :: | ||
9032 | |||
9033 | bitbake core-image-sato | ||
9034 | |||
9035 | - *Manually running tests:* To manually run the tests, first globally | ||
9036 | inherit the | ||
9037 | :ref:`testimage <ref-classes-testimage*>` class | ||
9038 | by editing your ``local.conf`` file: | ||
9039 | :: | ||
9040 | |||
9041 | INHERIT += "testimage" | ||
9042 | |||
9043 | Next, use BitBake to run the tests: | ||
9044 | :: | ||
9045 | |||
9046 | bitbake -c testimage image | ||
9047 | |||
9048 | All test files reside in ``meta/lib/oeqa/runtime`` in the | ||
9049 | :term:`Source Directory`. A test name maps | ||
9050 | directly to a Python module. Each test module may contain a number of | ||
9051 | individual tests. Tests are usually grouped together by the area tested | ||
9052 | (e.g tests for systemd reside in ``meta/lib/oeqa/runtime/systemd.py``). | ||
9053 | |||
9054 | You can add tests to any layer provided you place them in the proper | ||
9055 | area and you extend :term:`BBPATH` in | ||
9056 | the ``local.conf`` file as normal. Be sure that tests reside in | ||
9057 | ``layer/lib/oeqa/runtime``. | ||
9058 | |||
9059 | .. note:: | ||
9060 | |||
9061 | Be sure that module names do not collide with module names used in | ||
9062 | the default set of test modules in | ||
9063 | meta/lib/oeqa/runtime | ||
9064 | . | ||
9065 | |||
9066 | You can change the set of tests run by appending or overriding | ||
9067 | :term:`TEST_SUITES` variable in | ||
9068 | ``local.conf``. Each name in ``TEST_SUITES`` represents a required test | ||
9069 | for the image. Test modules named within ``TEST_SUITES`` cannot be | ||
9070 | skipped even if a test is not suitable for an image (e.g. running the | ||
9071 | RPM tests on an image without ``rpm``). Appending "auto" to | ||
9072 | ``TEST_SUITES`` causes the build system to try to run all tests that are | ||
9073 | suitable for the image (i.e. each test module may elect to skip itself). | ||
9074 | |||
9075 | The order you list tests in ``TEST_SUITES`` is important and influences | ||
9076 | test dependencies. Consequently, tests that depend on other tests should | ||
9077 | be added after the test on which they depend. For example, since the | ||
9078 | ``ssh`` test depends on the ``ping`` test, "ssh" needs to come after | ||
9079 | "ping" in the list. The test class provides no re-ordering or dependency | ||
9080 | handling. | ||
9081 | |||
9082 | .. note:: | ||
9083 | |||
9084 | Each module can have multiple classes with multiple test methods. | ||
9085 | And, Python | ||
9086 | unittest | ||
9087 | rules apply. | ||
9088 | |||
9089 | Here are some things to keep in mind when running tests: | ||
9090 | |||
9091 | - The default tests for the image are defined as: | ||
9092 | :: | ||
9093 | |||
9094 | DEFAULT_TEST_SUITES_pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg" | ||
9095 | |||
9096 | - Add your own test to the list of the by using the following: | ||
9097 | :: | ||
9098 | |||
9099 | TEST_SUITES_append = " mytest" | ||
9100 | |||
9101 | - Run a specific list of tests as follows: TEST_SUITES = "test1 test2 | ||
9102 | test3" Remember, order is important. Be sure to place a test that is | ||
9103 | dependent on another test later in the order. | ||
9104 | |||
9105 | Exporting Tests | ||
9106 | --------------- | ||
9107 | |||
9108 | You can export tests so that they can run independently of the build | ||
9109 | system. Exporting tests is required if you want to be able to hand the | ||
9110 | test execution off to a scheduler. You can only export tests that are | ||
9111 | defined in :term:`TEST_SUITES`. | ||
9112 | |||
9113 | If your image is already built, make sure the following are set in your | ||
9114 | ``local.conf`` file: | ||
9115 | :: | ||
9116 | |||
9117 | INHERIT +="testexport" | ||
9118 | TEST_TARGET_IP = "IP-address-for-the-test-target" | ||
9119 | TEST_SERVER_IP = "IP-address-for-the-test-server" | ||
9120 | |||
9121 | You can then export the tests with the | ||
9122 | following BitBake command form: | ||
9123 | :: | ||
9124 | |||
9125 | $ bitbake image -c testexport | ||
9126 | |||
9127 | Exporting the tests places them in the | ||
9128 | :term:`Build Directory` in | ||
9129 | ``tmp/testexport/``\ image, which is controlled by the | ||
9130 | ``TEST_EXPORT_DIR`` variable. | ||
9131 | |||
9132 | You can now run the tests outside of the build environment: | ||
9133 | :: | ||
9134 | |||
9135 | $ cd tmp/testexport/image | ||
9136 | $ ./runexported.py testdata.json | ||
9137 | |||
9138 | Here is a complete example that shows IP addresses and uses the | ||
9139 | ``core-image-sato`` image: | ||
9140 | :: | ||
9141 | |||
9142 | INHERIT +="testexport" | ||
9143 | TEST_TARGET_IP = "192.168.7.2" | ||
9144 | TEST_SERVER_IP = "192.168.7.1" | ||
9145 | |||
9146 | Use BitBake to export the tests: | ||
9147 | :: | ||
9148 | |||
9149 | $ bitbake core-image-sato -c testexport | ||
9150 | |||
9151 | Run the tests outside of | ||
9152 | the build environment using the following: | ||
9153 | :: | ||
9154 | |||
9155 | $ cd tmp/testexport/core-image-sato | ||
9156 | $ ./runexported.py testdata.json | ||
9157 | |||
9158 | .. _qemu-image-writing-new-tests: | ||
9159 | |||
9160 | Writing New Tests | ||
9161 | ----------------- | ||
9162 | |||
9163 | As mentioned previously, all new test files need to be in the proper | ||
9164 | place for the build system to find them. New tests for additional | ||
9165 | functionality outside of the core should be added to the layer that adds | ||
9166 | the functionality, in ``layer/lib/oeqa/runtime`` (as long as | ||
9167 | :term:`BBPATH` is extended in the | ||
9168 | layer's ``layer.conf`` file as normal). Just remember the following: | ||
9169 | |||
9170 | - Filenames need to map directly to test (module) names. | ||
9171 | |||
9172 | - Do not use module names that collide with existing core tests. | ||
9173 | |||
9174 | - Minimally, an empty ``__init__.py`` file must exist in the runtime | ||
9175 | directory. | ||
9176 | |||
9177 | To create a new test, start by copying an existing module (e.g. | ||
9178 | ``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use | ||
9179 | code from ``meta/lib/oeqa/utils``, which are helper classes. | ||
9180 | |||
9181 | .. note:: | ||
9182 | |||
9183 | Structure shell commands such that you rely on them and they return a | ||
9184 | single code for success. Be aware that sometimes you will need to | ||
9185 | parse the output. See the | ||
9186 | df.py | ||
9187 | and | ||
9188 | date.py | ||
9189 | modules for examples. | ||
9190 | |||
9191 | You will notice that all test classes inherit ``oeRuntimeTest``, which | ||
9192 | is found in ``meta/lib/oetest.py``. This base class offers some helper | ||
9193 | attributes, which are described in the following sections: | ||
9194 | |||
9195 | .. _qemu-image-writing-tests-class-methods: | ||
9196 | |||
9197 | Class Methods | ||
9198 | ~~~~~~~~~~~~~ | ||
9199 | |||
9200 | Class methods are as follows: | ||
9201 | |||
9202 | - *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed | ||
9203 | package list of the image, which is based on the manifest file that | ||
9204 | is generated during the ``do_rootfs`` task. | ||
9205 | |||
9206 | - *hasFeature(feature):* Returns "True" if the feature is in | ||
9207 | :term:`IMAGE_FEATURES` or | ||
9208 | :term:`DISTRO_FEATURES`. | ||
9209 | |||
9210 | .. _qemu-image-writing-tests-class-attributes: | ||
9211 | |||
9212 | Class Attributes | ||
9213 | ~~~~~~~~~~~~~~~~ | ||
9214 | |||
9215 | Class attributes are as follows: | ||
9216 | |||
9217 | - *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image. | ||
9218 | Otherwise, ``pscmd`` equals "ps" (busybox). | ||
9219 | |||
9220 | - *tc:* The called test context, which gives access to the | ||
9221 | following attributes: | ||
9222 | |||
9223 | - *d:* The BitBake datastore, which allows you to use stuff such | ||
9224 | as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``. | ||
9225 | |||
9226 | - *testslist and testsrequired:* Used internally. The tests | ||
9227 | do not need these. | ||
9228 | |||
9229 | - *filesdir:* The absolute path to | ||
9230 | ``meta/lib/oeqa/runtime/files``, which contains helper files for | ||
9231 | tests meant for copying on the target such as small files written | ||
9232 | in C for compilation. | ||
9233 | |||
9234 | - *target:* The target controller object used to deploy and | ||
9235 | start an image on a particular target (e.g. Qemu, SimpleRemote, | ||
9236 | and SystemdbootTarget). Tests usually use the following: | ||
9237 | |||
9238 | - *ip:* The target's IP address. | ||
9239 | |||
9240 | - *server_ip:* The host's IP address, which is usually used | ||
9241 | by the DNF test suite. | ||
9242 | |||
9243 | - *run(cmd, timeout=None):* The single, most used method. | ||
9244 | This command is a wrapper for: ``ssh root@host "cmd"``. The | ||
9245 | command returns a tuple: (status, output), which are what their | ||
9246 | names imply - the return code of "cmd" and whatever output it | ||
9247 | produces. The optional timeout argument represents the number | ||
9248 | of seconds the test should wait for "cmd" to return. If the | ||
9249 | argument is "None", the test uses the default instance's | ||
9250 | timeout period, which is 300 seconds. If the argument is "0", | ||
9251 | the test runs until the command returns. | ||
9252 | |||
9253 | - *copy_to(localpath, remotepath):* | ||
9254 | ``scp localpath root@ip:remotepath``. | ||
9255 | |||
9256 | - *copy_from(remotepath, localpath):* | ||
9257 | ``scp root@host:remotepath localpath``. | ||
9258 | |||
9259 | .. _qemu-image-writing-tests-instance-attributes: | ||
9260 | |||
9261 | Instance Attributes | ||
9262 | ~~~~~~~~~~~~~~~~~~~ | ||
9263 | |||
9264 | A single instance attribute exists, which is ``target``. The ``target`` | ||
9265 | instance attribute is identical to the class attribute of the same name, | ||
9266 | which is described in the previous section. This attribute exists as | ||
9267 | both an instance and class attribute so tests can use | ||
9268 | ``self.target.run(cmd)`` in instance methods instead of | ||
9269 | ``oeRuntimeTest.tc.target.run(cmd)``. | ||
9270 | |||
9271 | Installing Packages in the DUT Without the Package Manager | ||
9272 | ---------------------------------------------------------- | ||
9273 | |||
9274 | When a test requires a package built by BitBake, it is possible to | ||
9275 | install that package. Installing the package does not require a package | ||
9276 | manager be installed in the device under test (DUT). It does, however, | ||
9277 | require an SSH connection and the target must be using the | ||
9278 | ``sshcontrol`` class. | ||
9279 | |||
9280 | .. note:: | ||
9281 | |||
9282 | This method uses | ||
9283 | scp | ||
9284 | to copy files from the host to the target, which causes permissions | ||
9285 | and special attributes to be lost. | ||
9286 | |||
9287 | A JSON file is used to define the packages needed by a test. This file | ||
9288 | must be in the same path as the file used to define the tests. | ||
9289 | Furthermore, the filename must map directly to the test module name with | ||
9290 | a ``.json`` extension. | ||
9291 | |||
9292 | The JSON file must include an object with the test name as keys of an | ||
9293 | object or an array. This object (or array of objects) uses the following | ||
9294 | data: | ||
9295 | |||
9296 | - "pkg" - A mandatory string that is the name of the package to be | ||
9297 | installed. | ||
9298 | |||
9299 | - "rm" - An optional boolean, which defaults to "false", that specifies | ||
9300 | to remove the package after the test. | ||
9301 | |||
9302 | - "extract" - An optional boolean, which defaults to "false", that | ||
9303 | specifies if the package must be extracted from the package format. | ||
9304 | When set to "true", the package is not automatically installed into | ||
9305 | the DUT. | ||
9306 | |||
9307 | Following is an example JSON file that handles test "foo" installing | ||
9308 | package "bar" and test "foobar" installing packages "foo" and "bar". | ||
9309 | Once the test is complete, the packages are removed from the DUT. | ||
9310 | :: | ||
9311 | |||
9312 | { | ||
9313 | "foo": { | ||
9314 | "pkg": "bar" | ||
9315 | }, | ||
9316 | "foobar": [ | ||
9317 | { | ||
9318 | "pkg": "foo", | ||
9319 | "rm": true | ||
9320 | }, | ||
9321 | { | ||
9322 | "pkg": "bar", | ||
9323 | "rm": true | ||
9324 | } | ||
9325 | ] | ||
9326 | } | ||
9327 | |||
9328 | .. _usingpoky-debugging-tools-and-techniques: | ||
9329 | |||
9330 | Debugging Tools and Techniques | ||
9331 | ============================== | ||
9332 | |||
9333 | The exact method for debugging build failures depends on the nature of | ||
9334 | the problem and on the system's area from which the bug originates. | ||
9335 | Standard debugging practices such as comparison against the last known | ||
9336 | working version with examination of the changes and the re-application | ||
9337 | of steps to identify the one causing the problem are valid for the Yocto | ||
9338 | Project just as they are for any other system. Even though it is | ||
9339 | impossible to detail every possible potential failure, this section | ||
9340 | provides some general tips to aid in debugging given a variety of | ||
9341 | situations. | ||
9342 | |||
9343 | .. note:: | ||
9344 | |||
9345 | A useful feature for debugging is the error reporting tool. | ||
9346 | Configuring the Yocto Project to use this tool causes the | ||
9347 | OpenEmbedded build system to produce error reporting commands as part | ||
9348 | of the console output. You can enter the commands after the build | ||
9349 | completes to log error information into a common database, that can | ||
9350 | help you figure out what might be going wrong. For information on how | ||
9351 | to enable and use this feature, see the " | ||
9352 | Using the Error Reporting Tool | ||
9353 | " section. | ||
9354 | |||
9355 | The following list shows the debugging topics in the remainder of this | ||
9356 | section: | ||
9357 | |||
9358 | - "`Viewing Logs from Failed | ||
9359 | Tasks <#dev-debugging-viewing-logs-from-failed-tasks>`__" describes | ||
9360 | how to find and view logs from tasks that failed during the build | ||
9361 | process. | ||
9362 | |||
9363 | - "`Viewing Variable | ||
9364 | Values <#dev-debugging-viewing-variable-values>`__" describes how to | ||
9365 | use the BitBake ``-e`` option to examine variable values after a | ||
9366 | recipe has been parsed. | ||
9367 | |||
9368 | - ":ref:`dev-manual/dev-manual-common-tasks:viewing package information with \`\`oe-pkgdata-util\`\``" | ||
9369 | describes how to use the ``oe-pkgdata-util`` utility to query | ||
9370 | :term:`PKGDATA_DIR` and | ||
9371 | display package-related information for built packages. | ||
9372 | |||
9373 | - "`Viewing Dependencies Between Recipes and | ||
9374 | Tasks <#dev-viewing-dependencies-between-recipes-and-tasks>`__" | ||
9375 | describes how to use the BitBake ``-g`` option to display recipe | ||
9376 | dependency information used during the build. | ||
9377 | |||
9378 | - "`Viewing Task Variable | ||
9379 | Dependencies <#dev-viewing-task-variable-dependencies>`__" describes | ||
9380 | how to use the ``bitbake-dumpsig`` command in conjunction with key | ||
9381 | subdirectories in the | ||
9382 | :term:`Build Directory` to determine | ||
9383 | variable dependencies. | ||
9384 | |||
9385 | - "`Running Specific Tasks <#dev-debugging-taskrunning>`__" describes | ||
9386 | how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``) | ||
9387 | to run specific tasks in the build chain. It can be useful to run | ||
9388 | tasks "out-of-order" when trying isolate build issues. | ||
9389 | |||
9390 | - "`General BitBake Problems <#dev-debugging-bitbake>`__" describes how | ||
9391 | to use BitBake's ``-D`` debug output option to reveal more about what | ||
9392 | BitBake is doing during the build. | ||
9393 | |||
9394 | - "`Building with No Dependencies <#dev-debugging-buildfile>`__" | ||
9395 | describes how to use the BitBake ``-b`` option to build a recipe | ||
9396 | while ignoring dependencies. | ||
9397 | |||
9398 | - "`Recipe Logging Mechanisms <#recipe-logging-mechanisms>`__" | ||
9399 | describes how to use the many recipe logging functions to produce | ||
9400 | debugging output and report errors and warnings. | ||
9401 | |||
9402 | - "`Debugging Parallel Make Races <#debugging-parallel-make-races>`__" | ||
9403 | describes how to debug situations where the build consists of several | ||
9404 | parts that are run simultaneously and when the output or result of | ||
9405 | one part is not ready for use with a different part of the build that | ||
9406 | depends on that output. | ||
9407 | |||
9408 | - "`Debugging With the GNU Project Debugger (GDB) | ||
9409 | Remotely <#platdev-gdb-remotedebug>`__" describes how to use GDB to | ||
9410 | allow you to examine running programs, which can help you fix | ||
9411 | problems. | ||
9412 | |||
9413 | - "`Debugging with the GNU Project Debugger (GDB) on the | ||
9414 | Target <#debugging-with-the-gnu-project-debugger-gdb-on-the-target>`__" | ||
9415 | describes how to use GDB directly on target hardware for debugging. | ||
9416 | |||
9417 | - "`Other Debugging Tips <#dev-other-debugging-others>`__" describes | ||
9418 | miscellaneous debugging tips that can be useful. | ||
9419 | |||
9420 | .. _dev-debugging-viewing-logs-from-failed-tasks: | ||
9421 | |||
9422 | Viewing Logs from Failed Tasks | ||
9423 | ------------------------------ | ||
9424 | |||
9425 | You can find the log for a task in the file | ||
9426 | ``${``\ :term:`WORKDIR`\ ``}/temp/log.do_``\ taskname. | ||
9427 | For example, the log for the | ||
9428 | :ref:`ref-tasks-compile` task of the | ||
9429 | QEMU minimal image for the x86 machine (``qemux86``) might be in | ||
9430 | ``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``. | ||
9431 | To see the commands :term:`BitBake` ran | ||
9432 | to generate a log, look at the corresponding ``run.do_``\ taskname file | ||
9433 | in the same directory. | ||
9434 | |||
9435 | ``log.do_``\ taskname and ``run.do_``\ taskname are actually symbolic | ||
9436 | links to ``log.do_``\ taskname\ ``.``\ pid and | ||
9437 | ``log.run_``\ taskname\ ``.``\ pid, where pid is the PID the task had | ||
9438 | when it ran. The symlinks always point to the files corresponding to the | ||
9439 | most recent run. | ||
9440 | |||
9441 | .. _dev-debugging-viewing-variable-values: | ||
9442 | |||
9443 | Viewing Variable Values | ||
9444 | ----------------------- | ||
9445 | |||
9446 | Sometimes you need to know the value of a variable as a result of | ||
9447 | BitBake's parsing step. This could be because some unexpected behavior | ||
9448 | occurred in your project. Perhaps an attempt to :ref:`modify a variable | ||
9449 | <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:modifying existing | ||
9450 | variables>` did not work out as expected. | ||
9451 | |||
9452 | BitBake's ``-e`` option is used to display variable values after | ||
9453 | parsing. The following command displays the variable values after the | ||
9454 | configuration files (i.e. ``local.conf``, ``bblayers.conf``, | ||
9455 | ``bitbake.conf`` and so forth) have been parsed: | ||
9456 | :: | ||
9457 | |||
9458 | $ bitbake -e | ||
9459 | |||
9460 | The following command displays variable values after a specific recipe has | ||
9461 | been parsed. The variables include those from the configuration as well: | ||
9462 | :: | ||
9463 | |||
9464 | $ bitbake -e recipename | ||
9465 | |||
9466 | .. note:: | ||
9467 | |||
9468 | Each recipe has its own private set of variables (datastore). | ||
9469 | Internally, after parsing the configuration, a copy of the resulting | ||
9470 | datastore is made prior to parsing each recipe. This copying implies | ||
9471 | that variables set in one recipe will not be visible to other | ||
9472 | recipes. | ||
9473 | |||
9474 | Likewise, each task within a recipe gets a private datastore based on | ||
9475 | the recipe datastore, which means that variables set within one task | ||
9476 | will not be visible to other tasks. | ||
9477 | |||
9478 | In the output of ``bitbake -e``, each variable is preceded by a | ||
9479 | description of how the variable got its value, including temporary | ||
9480 | values that were later overriden. This description also includes | ||
9481 | variable flags (varflags) set on the variable. The output can be very | ||
9482 | helpful during debugging. | ||
9483 | |||
9484 | Variables that are exported to the environment are preceded by | ||
9485 | ``export`` in the output of ``bitbake -e``. See the following example: | ||
9486 | :: | ||
9487 | |||
9488 | export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" | ||
9489 | |||
9490 | In addition to variable values, the output of the ``bitbake -e`` and | ||
9491 | ``bitbake -e`` recipe commands includes the following information: | ||
9492 | |||
9493 | - The output starts with a tree listing all configuration files and | ||
9494 | classes included globally, recursively listing the files they include | ||
9495 | or inherit in turn. Much of the behavior of the OpenEmbedded build | ||
9496 | system (including the behavior of the :ref:`ref-manual/ref-tasks:normal recipe build tasks`) is | ||
9497 | implemented in the | ||
9498 | :ref:`base <ref-classes-base>` class and the | ||
9499 | classes it inherits, rather than being built into BitBake itself. | ||
9500 | |||
9501 | - After the variable values, all functions appear in the output. For | ||
9502 | shell functions, variables referenced within the function body are | ||
9503 | expanded. If a function has been modified using overrides or using | ||
9504 | override-style operators like ``_append`` and ``_prepend``, then the | ||
9505 | final assembled function body appears in the output. | ||
9506 | |||
9507 | Viewing Package Information with ``oe-pkgdata-util`` | ||
9508 | ---------------------------------------------------- | ||
9509 | |||
9510 | You can use the ``oe-pkgdata-util`` command-line utility to query | ||
9511 | :term:`PKGDATA_DIR` and display | ||
9512 | various package-related information. When you use the utility, you must | ||
9513 | use it to view information on packages that have already been built. | ||
9514 | |||
9515 | Following are a few of the available ``oe-pkgdata-util`` subcommands. | ||
9516 | |||
9517 | .. note:: | ||
9518 | |||
9519 | You can use the standard \* and ? globbing wildcards as part of | ||
9520 | package names and paths. | ||
9521 | |||
9522 | - ``oe-pkgdata-util list-pkgs [pattern]``: Lists all packages | ||
9523 | that have been built, optionally limiting the match to packages that | ||
9524 | match pattern. | ||
9525 | |||
9526 | - ``oe-pkgdata-util list-pkg-files package ...``: Lists the | ||
9527 | files and directories contained in the given packages. | ||
9528 | |||
9529 | .. note:: | ||
9530 | |||
9531 | A different way to view the contents of a package is to look at | ||
9532 | the | ||
9533 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
9534 | directory of the recipe that generates the package. This directory | ||
9535 | is created by the | ||
9536 | :ref:`ref-tasks-package` task | ||
9537 | and has one subdirectory for each package the recipe generates, | ||
9538 | which contains the files stored in that package. | ||
9539 | |||
9540 | If you want to inspect the ``${WORKDIR}/packages-split`` | ||
9541 | directory, make sure that | ||
9542 | :ref:`rm_work <ref-classes-rm-work>` is not | ||
9543 | enabled when you build the recipe. | ||
9544 | |||
9545 | - ``oe-pkgdata-util find-path path ...``: Lists the names of | ||
9546 | the packages that contain the given paths. For example, the following | ||
9547 | tells us that ``/usr/share/man/man1/make.1`` is contained in the | ||
9548 | ``make-doc`` package: | ||
9549 | :: | ||
9550 | |||
9551 | $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 make-doc: /usr/share/man/man1/make.1 | ||
9552 | |||
9553 | - ``oe-pkgdata-util lookup-recipe package ...``: Lists the name | ||
9554 | of the recipes that produce the given packages. | ||
9555 | |||
9556 | For more information on the ``oe-pkgdata-util`` command, use the help | ||
9557 | facility: | ||
9558 | :: | ||
9559 | |||
9560 | $ oe-pkgdata-util DASHDASHhelp | ||
9561 | $ oe-pkgdata-util subcommand --help | ||
9562 | |||
9563 | .. _dev-viewing-dependencies-between-recipes-and-tasks: | ||
9564 | |||
9565 | Viewing Dependencies Between Recipes and Tasks | ||
9566 | ---------------------------------------------- | ||
9567 | |||
9568 | Sometimes it can be hard to see why BitBake wants to build other recipes | ||
9569 | before the one you have specified. Dependency information can help you | ||
9570 | understand why a recipe is built. | ||
9571 | |||
9572 | To generate dependency information for a recipe, run the following | ||
9573 | command: | ||
9574 | :: | ||
9575 | |||
9576 | $ bitbake -g recipename | ||
9577 | |||
9578 | This command writes the following files in the current directory: | ||
9579 | |||
9580 | - ``pn-buildlist``: A list of recipes/targets involved in building | ||
9581 | recipename. "Involved" here means that at least one task from the | ||
9582 | recipe needs to run when building recipename from scratch. Targets | ||
9583 | that are in | ||
9584 | :term:`ASSUME_PROVIDED` | ||
9585 | are not listed. | ||
9586 | |||
9587 | - ``task-depends.dot``: A graph showing dependencies between tasks. | ||
9588 | |||
9589 | The graphs are in | ||
9590 | `DOT <https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29>`__ | ||
9591 | format and can be converted to images (e.g. using the ``dot`` tool from | ||
9592 | `Graphviz <http://www.graphviz.org/>`__). | ||
9593 | |||
9594 | .. note:: | ||
9595 | |||
9596 | - DOT files use a plain text format. The graphs generated using the | ||
9597 | ``bitbake -g`` command are often so large as to be difficult to | ||
9598 | read without special pruning (e.g. with Bitbake's ``-I`` option) | ||
9599 | and processing. Despite the form and size of the graphs, the | ||
9600 | corresponding ``.dot`` files can still be possible to read and | ||
9601 | provide useful information. | ||
9602 | |||
9603 | As an example, the ``task-depends.dot`` file contains lines such | ||
9604 | as the following: | ||
9605 | :: | ||
9606 | |||
9607 | "libxslt.do_configure" -> "libxml2.do_populate_sysroot" | ||
9608 | |||
9609 | The above example line reveals that the | ||
9610 | :ref:`ref-tasks-configure` | ||
9611 | task in ``libxslt`` depends on the | ||
9612 | :ref:`ref-tasks-populate_sysroot` | ||
9613 | task in ``libxml2``, which is a normal | ||
9614 | :term:`DEPENDS` dependency | ||
9615 | between the two recipes. | ||
9616 | |||
9617 | - For an example of how ``.dot`` files can be processed, see the | ||
9618 | ``scripts/contrib/graph-tool`` Python script, which finds and | ||
9619 | displays paths between graph nodes. | ||
9620 | |||
9621 | You can use a different method to view dependency information by using | ||
9622 | the following command: | ||
9623 | :: | ||
9624 | |||
9625 | $ bitbake -g -u taskexp recipename | ||
9626 | |||
9627 | This command | ||
9628 | displays a GUI window from which you can view build-time and runtime | ||
9629 | dependencies for the recipes involved in building recipename. | ||
9630 | |||
9631 | .. _dev-viewing-task-variable-dependencies: | ||
9632 | |||
9633 | Viewing Task Variable Dependencies | ||
9634 | ---------------------------------- | ||
9635 | |||
9636 | As mentioned in the | ||
9637 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section of the BitBake | ||
9638 | User Manual, BitBake tries to automatically determine what variables a | ||
9639 | task depends on so that it can rerun the task if any values of the | ||
9640 | variables change. This determination is usually reliable. However, if | ||
9641 | you do things like construct variable names at runtime, then you might | ||
9642 | have to manually declare dependencies on those variables using | ||
9643 | ``vardeps`` as described in the | ||
9644 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section of the BitBake | ||
9645 | User Manual. | ||
9646 | |||
9647 | If you are unsure whether a variable dependency is being picked up | ||
9648 | automatically for a given task, you can list the variable dependencies | ||
9649 | BitBake has determined by doing the following: | ||
9650 | |||
9651 | 1. Build the recipe containing the task: | ||
9652 | :: | ||
9653 | |||
9654 | $ bitbake recipename | ||
9655 | |||
9656 | 2. Inside the :term:`STAMPS_DIR` | ||
9657 | directory, find the signature data (``sigdata``) file that | ||
9658 | corresponds to the task. The ``sigdata`` files contain a pickled | ||
9659 | Python database of all the metadata that went into creating the input | ||
9660 | checksum for the task. As an example, for the | ||
9661 | :ref:`ref-tasks-fetch` task of the | ||
9662 | ``db`` recipe, the ``sigdata`` file might be found in the following | ||
9663 | location: | ||
9664 | :: | ||
9665 | |||
9666 | ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
9667 | |||
9668 | For tasks that are accelerated through the shared state | ||
9669 | (:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>`) cache, an | ||
9670 | additional ``siginfo`` file is written into | ||
9671 | :term:`SSTATE_DIR` along with | ||
9672 | the cached task output. The ``siginfo`` files contain exactly the | ||
9673 | same information as ``sigdata`` files. | ||
9674 | |||
9675 | 3. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here | ||
9676 | is an example: | ||
9677 | :: | ||
9678 | |||
9679 | $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
9680 | |||
9681 | In the output of the above command, you will find a line like the | ||
9682 | following, which lists all the (inferred) variable dependencies for | ||
9683 | the task. This list also includes indirect dependencies from | ||
9684 | variables depending on other variables, recursively. | ||
9685 | :: | ||
9686 | |||
9687 | Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] | ||
9688 | |||
9689 | .. note:: | ||
9690 | |||
9691 | Functions (e.g. | ||
9692 | base_do_fetch | ||
9693 | ) also count as variable dependencies. These functions in turn | ||
9694 | depend on the variables they reference. | ||
9695 | |||
9696 | The output of ``bitbake-dumpsig`` also includes the value each | ||
9697 | variable had, a list of dependencies for each variable, and | ||
9698 | :term:`bitbake:BB_HASHBASE_WHITELIST` | ||
9699 | information. | ||
9700 | |||
9701 | There is also a ``bitbake-diffsigs`` command for comparing two | ||
9702 | ``siginfo`` or ``sigdata`` files. This command can be helpful when | ||
9703 | trying to figure out what changed between two versions of a task. If you | ||
9704 | call ``bitbake-diffsigs`` with just one file, the command behaves like | ||
9705 | ``bitbake-dumpsig``. | ||
9706 | |||
9707 | You can also use BitBake to dump out the signature construction | ||
9708 | information without executing tasks by using either of the following | ||
9709 | BitBake command-line options: | ||
9710 | :: | ||
9711 | |||
9712 | ‐‐dump-signatures=SIGNATURE_HANDLER | ||
9713 | -S SIGNATURE_HANDLER | ||
9714 | |||
9715 | |||
9716 | .. note:: | ||
9717 | |||
9718 | Two common values for | ||
9719 | SIGNATURE_HANDLER | ||
9720 | are "none" and "printdiff", which dump only the signature or compare | ||
9721 | the dumped signature with the cached one, respectively. | ||
9722 | |||
9723 | Using BitBake with either of these options causes BitBake to dump out | ||
9724 | ``sigdata`` files in the ``stamps`` directory for every task it would | ||
9725 | have executed instead of building the specified target package. | ||
9726 | |||
9727 | .. _dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task: | ||
9728 | |||
9729 | Viewing Metadata Used to Create the Input Signature of a Shared State Task | ||
9730 | -------------------------------------------------------------------------- | ||
9731 | |||
9732 | Seeing what metadata went into creating the input signature of a shared | ||
9733 | state (sstate) task can be a useful debugging aid. This information is | ||
9734 | available in signature information (``siginfo``) files in | ||
9735 | :term:`SSTATE_DIR`. For | ||
9736 | information on how to view and interpret information in ``siginfo`` | ||
9737 | files, see the "`Viewing Task Variable | ||
9738 | Dependencies <#dev-viewing-task-variable-dependencies>`__" section. | ||
9739 | |||
9740 | For conceptual information on shared state, see the | ||
9741 | ":ref:`overview-manual/overview-manual-concepts:shared state`" | ||
9742 | section in the Yocto Project Overview and Concepts Manual. | ||
9743 | |||
9744 | .. _dev-invalidating-shared-state-to-force-a-task-to-run: | ||
9745 | |||
9746 | Invalidating Shared State to Force a Task to Run | ||
9747 | ------------------------------------------------ | ||
9748 | |||
9749 | The OpenEmbedded build system uses | ||
9750 | :ref:`checksums <overview-checksums>` and | ||
9751 | :ref:`overview-manual/overview-manual-concepts:shared state` cache to avoid unnecessarily | ||
9752 | rebuilding tasks. Collectively, this scheme is known as "shared state | ||
9753 | code." | ||
9754 | |||
9755 | As with all schemes, this one has some drawbacks. It is possible that | ||
9756 | you could make implicit changes to your code that the checksum | ||
9757 | calculations do not take into account. These implicit changes affect a | ||
9758 | task's output but do not trigger the shared state code into rebuilding a | ||
9759 | recipe. Consider an example during which a tool changes its output. | ||
9760 | Assume that the output of ``rpmdeps`` changes. The result of the change | ||
9761 | should be that all the ``package`` and ``package_write_rpm`` shared | ||
9762 | state cache items become invalid. However, because the change to the | ||
9763 | output is external to the code and therefore implicit, the associated | ||
9764 | shared state cache items do not become invalidated. In this case, the | ||
9765 | build process uses the cached items rather than running the task again. | ||
9766 | Obviously, these types of implicit changes can cause problems. | ||
9767 | |||
9768 | To avoid these problems during the build, you need to understand the | ||
9769 | effects of any changes you make. Realize that changes you make directly | ||
9770 | to a function are automatically factored into the checksum calculation. | ||
9771 | Thus, these explicit changes invalidate the associated area of shared | ||
9772 | state cache. However, you need to be aware of any implicit changes that | ||
9773 | are not obvious changes to the code and could affect the output of a | ||
9774 | given task. | ||
9775 | |||
9776 | When you identify an implicit change, you can easily take steps to | ||
9777 | invalidate the cache and force the tasks to run. The steps you can take | ||
9778 | are as simple as changing a function's comments in the source code. For | ||
9779 | example, to invalidate package shared state files, change the comment | ||
9780 | statements of | ||
9781 | :ref:`ref-tasks-package` or the | ||
9782 | comments of one of the functions it calls. Even though the change is | ||
9783 | purely cosmetic, it causes the checksum to be recalculated and forces | ||
9784 | the build system to run the task again. | ||
9785 | |||
9786 | .. note:: | ||
9787 | |||
9788 | For an example of a commit that makes a cosmetic change to invalidate | ||
9789 | shared state, see this | ||
9790 | commit | ||
9791 | . | ||
9792 | |||
9793 | .. _dev-debugging-taskrunning: | ||
9794 | |||
9795 | Running Specific Tasks | ||
9796 | ---------------------- | ||
9797 | |||
9798 | Any given recipe consists of a set of tasks. The standard BitBake | ||
9799 | behavior in most cases is: ``do_fetch``, ``do_unpack``, ``do_patch``, | ||
9800 | ``do_configure``, ``do_compile``, ``do_install``, ``do_package``, | ||
9801 | ``do_package_write_*``, and ``do_build``. The default task is | ||
9802 | ``do_build`` and any tasks on which it depends build first. Some tasks, | ||
9803 | such as ``do_devshell``, are not part of the default build chain. If you | ||
9804 | wish to run a task that is not part of the default build chain, you can | ||
9805 | use the ``-c`` option in BitBake. Here is an example: | ||
9806 | :: | ||
9807 | |||
9808 | $ bitbake matchbox-desktop -c devshell | ||
9809 | |||
9810 | The ``-c`` option respects task dependencies, which means that all other | ||
9811 | tasks (including tasks from other recipes) that the specified task | ||
9812 | depends on will be run before the task. Even when you manually specify a | ||
9813 | task to run with ``-c``, BitBake will only run the task if it considers | ||
9814 | it "out of date". See the | ||
9815 | ":ref:`overview-manual/overview-manual-concepts:stamp files and the rerunning of tasks`" | ||
9816 | section in the Yocto Project Overview and Concepts Manual for how | ||
9817 | BitBake determines whether a task is "out of date". | ||
9818 | |||
9819 | If you want to force an up-to-date task to be rerun (e.g. because you | ||
9820 | made manual modifications to the recipe's | ||
9821 | :term:`WORKDIR` that you want to try | ||
9822 | out), then you can use the ``-f`` option. | ||
9823 | |||
9824 | .. note:: | ||
9825 | |||
9826 | The reason | ||
9827 | -f | ||
9828 | is never required when running the | ||
9829 | do_devshell | ||
9830 | task is because the | ||
9831 | [ | ||
9832 | nostamp | ||
9833 | ] | ||
9834 | variable flag is already set for the task. | ||
9835 | |||
9836 | The following example shows one way you can use the ``-f`` option: | ||
9837 | :: | ||
9838 | |||
9839 | $ bitbake matchbox-desktop | ||
9840 | . | ||
9841 | . | ||
9842 | make some changes to the source code in the work directory | ||
9843 | . | ||
9844 | . | ||
9845 | $ bitbake matchbox-desktop -c compile -f | ||
9846 | $ bitbake matchbox-desktop | ||
9847 | |||
9848 | This sequence first builds and then recompiles ``matchbox-desktop``. The | ||
9849 | last command reruns all tasks (basically the packaging tasks) after the | ||
9850 | compile. BitBake recognizes that the ``do_compile`` task was rerun and | ||
9851 | therefore understands that the other tasks also need to be run again. | ||
9852 | |||
9853 | Another, shorter way to rerun a task and all | ||
9854 | :ref:`ref-manual/ref-tasks:normal recipe build tasks` | ||
9855 | that depend on it is to use the ``-C`` option. | ||
9856 | |||
9857 | .. note:: | ||
9858 | |||
9859 | This option is upper-cased and is separate from the | ||
9860 | -c | ||
9861 | option, which is lower-cased. | ||
9862 | |||
9863 | Using this option invalidates the given task and then runs the | ||
9864 | :ref:`ref-tasks-build` task, which is | ||
9865 | the default task if no task is given, and the tasks on which it depends. | ||
9866 | You could replace the final two commands in the previous example with | ||
9867 | the following single command: | ||
9868 | :: | ||
9869 | |||
9870 | $ bitbake matchbox-desktop -C compile | ||
9871 | |||
9872 | Internally, the ``-f`` and ``-C`` options work by tainting (modifying) | ||
9873 | the input checksum of the specified task. This tainting indirectly | ||
9874 | causes the task and its dependent tasks to be rerun through the normal | ||
9875 | task dependency mechanisms. | ||
9876 | |||
9877 | .. note:: | ||
9878 | |||
9879 | BitBake explicitly keeps track of which tasks have been tainted in | ||
9880 | this fashion, and will print warnings such as the following for | ||
9881 | builds involving such tasks: | ||
9882 | :: | ||
9883 | |||
9884 | WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run | ||
9885 | |||
9886 | |||
9887 | The purpose of the warning is to let you know that the work directory | ||
9888 | and build output might not be in the clean state they would be in for | ||
9889 | a "normal" build, depending on what actions you took. To get rid of | ||
9890 | such warnings, you can remove the work directory and rebuild the | ||
9891 | recipe, as follows: | ||
9892 | :: | ||
9893 | |||
9894 | $ bitbake matchbox-desktop -c clean | ||
9895 | $ bitbake matchbox-desktop | ||
9896 | |||
9897 | |||
9898 | You can view a list of tasks in a given package by running the | ||
9899 | ``do_listtasks`` task as follows: | ||
9900 | :: | ||
9901 | |||
9902 | $ bitbake matchbox-desktop -c listtasks | ||
9903 | |||
9904 | The results appear as output to the console and are also in | ||
9905 | the file ``${WORKDIR}/temp/log.do_listtasks``. | ||
9906 | |||
9907 | .. _dev-debugging-bitbake: | ||
9908 | |||
9909 | General BitBake Problems | ||
9910 | ------------------------ | ||
9911 | |||
9912 | You can see debug output from BitBake by using the ``-D`` option. The | ||
9913 | debug output gives more information about what BitBake is doing and the | ||
9914 | reason behind it. Each ``-D`` option you use increases the logging | ||
9915 | level. The most common usage is ``-DDD``. | ||
9916 | |||
9917 | The output from ``bitbake -DDD -v`` targetname can reveal why BitBake | ||
9918 | chose a certain version of a package or why BitBake picked a certain | ||
9919 | provider. This command could also help you in a situation where you | ||
9920 | think BitBake did something unexpected. | ||
9921 | |||
9922 | .. _dev-debugging-buildfile: | ||
9923 | |||
9924 | Building with No Dependencies | ||
9925 | ----------------------------- | ||
9926 | |||
9927 | To build a specific recipe (``.bb`` file), you can use the following | ||
9928 | command form: | ||
9929 | :: | ||
9930 | |||
9931 | $ bitbake -b somepath/somerecipe.bb | ||
9932 | |||
9933 | This command form does | ||
9934 | not check for dependencies. Consequently, you should use it only when | ||
9935 | you know existing dependencies have been met. | ||
9936 | |||
9937 | .. note:: | ||
9938 | |||
9939 | You can also specify fragments of the filename. In this case, BitBake | ||
9940 | checks for a unique match. | ||
9941 | |||
9942 | Recipe Logging Mechanisms | ||
9943 | ------------------------- | ||
9944 | |||
9945 | The Yocto Project provides several logging functions for producing | ||
9946 | debugging output and reporting errors and warnings. For Python | ||
9947 | functions, the following logging functions exist. All of these functions | ||
9948 | log to ``${T}/log.do_``\ task, and can also log to standard output | ||
9949 | (stdout) with the right settings: | ||
9950 | |||
9951 | - ``bb.plain(msg)``: Writes msg as is to the log while also | ||
9952 | logging to stdout. | ||
9953 | |||
9954 | - ``bb.note(msg)``: Writes "NOTE: msg" to the log. Also logs to | ||
9955 | stdout if BitBake is called with "-v". | ||
9956 | |||
9957 | - ``bb.debug(level, msg)``: Writes "DEBUG: msg" to the | ||
9958 | log. Also logs to stdout if the log level is greater than or equal to | ||
9959 | level. See the ":ref:`-D <bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax>`" option | ||
9960 | in the BitBake User Manual for more information. | ||
9961 | |||
9962 | - ``bb.warn(msg)``: Writes "WARNING: msg" to the log while also | ||
9963 | logging to stdout. | ||
9964 | |||
9965 | - ``bb.error(msg)``: Writes "ERROR: msg" to the log while also | ||
9966 | logging to standard out (stdout). | ||
9967 | |||
9968 | .. note:: | ||
9969 | |||
9970 | Calling this function does not cause the task to fail. | ||
9971 | |||
9972 | - ``bb.fatal(``\ msg\ ``)``: This logging function is similar to | ||
9973 | ``bb.error(``\ msg\ ``)`` but also causes the calling task to fail. | ||
9974 | |||
9975 | .. note:: | ||
9976 | |||
9977 | bb.fatal() | ||
9978 | raises an exception, which means you do not need to put a "return" | ||
9979 | statement after the function. | ||
9980 | |||
9981 | The same logging functions are also available in shell functions, under | ||
9982 | the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``, | ||
9983 | and ``bbfatal``. The | ||
9984 | :ref:`logging <ref-classes-logging>` class | ||
9985 | implements these functions. See that class in the ``meta/classes`` | ||
9986 | folder of the :term:`Source Directory` for information. | ||
9987 | |||
9988 | Logging With Python | ||
9989 | ~~~~~~~~~~~~~~~~~~~ | ||
9990 | |||
9991 | When creating recipes using Python and inserting code that handles build | ||
9992 | logs, keep in mind the goal is to have informative logs while keeping | ||
9993 | the console as "silent" as possible. Also, if you want status messages | ||
9994 | in the log, use the "debug" loglevel. | ||
9995 | |||
9996 | Following is an example written in Python. The code handles logging for | ||
9997 | a function that determines the number of tasks needed to be run. See the | ||
9998 | ":ref:`ref-tasks-listtasks`" | ||
9999 | section for additional information: | ||
10000 | :: | ||
10001 | |||
10002 | python do_listtasks() { | ||
10003 | bb.debug(2, "Starting to figure out the task list") | ||
10004 | if noteworthy_condition: | ||
10005 | bb.note("There are 47 tasks to run") | ||
10006 | bb.debug(2, "Got to point xyz") | ||
10007 | if warning_trigger: | ||
10008 | bb.warn("Detected warning_trigger, this might be a problem later.") | ||
10009 | if recoverable_error: | ||
10010 | bb.error("Hit recoverable_error, you really need to fix this!") | ||
10011 | if fatal_error: | ||
10012 | bb.fatal("fatal_error detected, unable to print the task list") | ||
10013 | bb.plain("The tasks present are abc") | ||
10014 | bb.debug(2, "Finished figuring out the tasklist") | ||
10015 | } | ||
10016 | |||
10017 | Logging With Bash | ||
10018 | ~~~~~~~~~~~~~~~~~ | ||
10019 | |||
10020 | When creating recipes using Bash and inserting code that handles build | ||
10021 | logs, you have the same goals - informative with minimal console output. | ||
10022 | The syntax you use for recipes written in Bash is similar to that of | ||
10023 | recipes written in Python described in the previous section. | ||
10024 | |||
10025 | Following is an example written in Bash. The code logs the progress of | ||
10026 | the ``do_my_function`` function. | ||
10027 | :: | ||
10028 | |||
10029 | do_my_function() { | ||
10030 | bbdebug 2 "Running do_my_function" | ||
10031 | if [ exceptional_condition ]; then | ||
10032 | bbnote "Hit exceptional_condition" | ||
10033 | fi | ||
10034 | bbdebug 2 "Got to point xyz" | ||
10035 | if [ warning_trigger ]; then | ||
10036 | bbwarn "Detected warning_trigger, this might cause a problem later." | ||
10037 | fi | ||
10038 | if [ recoverable_error ]; then | ||
10039 | bberror "Hit recoverable_error, correcting" | ||
10040 | fi | ||
10041 | if [ fatal_error ]; then | ||
10042 | bbfatal "fatal_error detected" | ||
10043 | fi | ||
10044 | bbdebug 2 "Completed do_my_function" | ||
10045 | } | ||
10046 | |||
10047 | |||
10048 | Debugging Parallel Make Races | ||
10049 | ----------------------------- | ||
10050 | |||
10051 | A parallel ``make`` race occurs when the build consists of several parts | ||
10052 | that are run simultaneously and a situation occurs when the output or | ||
10053 | result of one part is not ready for use with a different part of the | ||
10054 | build that depends on that output. Parallel make races are annoying and | ||
10055 | can sometimes be difficult to reproduce and fix. However, some simple | ||
10056 | tips and tricks exist that can help you debug and fix them. This section | ||
10057 | presents a real-world example of an error encountered on the Yocto | ||
10058 | Project autobuilder and the process used to fix it. | ||
10059 | |||
10060 | .. note:: | ||
10061 | |||
10062 | If you cannot properly fix a | ||
10063 | make | ||
10064 | race condition, you can work around it by clearing either the | ||
10065 | PARALLEL_MAKE | ||
10066 | or | ||
10067 | PARALLEL_MAKEINST | ||
10068 | variables. | ||
10069 | |||
10070 | The Failure | ||
10071 | ~~~~~~~~~~~ | ||
10072 | |||
10073 | For this example, assume that you are building an image that depends on | ||
10074 | the "neard" package. And, during the build, BitBake runs into problems | ||
10075 | and creates the following output. | ||
10076 | |||
10077 | .. note:: | ||
10078 | |||
10079 | This example log file has longer lines artificially broken to make | ||
10080 | the listing easier to read. | ||
10081 | |||
10082 | If you examine the output or the log file, you see the failure during | ||
10083 | ``make``: | ||
10084 | :: | ||
10085 | |||
10086 | | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] | ||
10087 | | DEBUG: Executing shell function do_compile | ||
10088 | | NOTE: make -j 16 | ||
10089 | | make --no-print-directory all-am | ||
10090 | | /bin/mkdir -p include/near | ||
10091 | | /bin/mkdir -p include/near | ||
10092 | | /bin/mkdir -p include/near | ||
10093 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10094 | 0.14-r0/neard-0.14/include/types.h include/near/types.h | ||
10095 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10096 | 0.14-r0/neard-0.14/include/log.h include/near/log.h | ||
10097 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10098 | 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h | ||
10099 | | /bin/mkdir -p include/near | ||
10100 | | /bin/mkdir -p include/near | ||
10101 | | /bin/mkdir -p include/near | ||
10102 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10103 | 0.14-r0/neard-0.14/include/tag.h include/near/tag.h | ||
10104 | | /bin/mkdir -p include/near | ||
10105 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10106 | 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h | ||
10107 | | /bin/mkdir -p include/near | ||
10108 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10109 | 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h | ||
10110 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10111 | 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h | ||
10112 | | /bin/mkdir -p include/near | ||
10113 | | /bin/mkdir -p include/near | ||
10114 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10115 | 0.14-r0/neard-0.14/include/setting.h include/near/setting.h | ||
10116 | | /bin/mkdir -p include/near | ||
10117 | | /bin/mkdir -p include/near | ||
10118 | | /bin/mkdir -p include/near | ||
10119 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10120 | 0.14-r0/neard-0.14/include/device.h include/near/device.h | ||
10121 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10122 | 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h | ||
10123 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10124 | 0.14-r0/neard-0.14/include/snep.h include/near/snep.h | ||
10125 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10126 | 0.14-r0/neard-0.14/include/version.h include/near/version.h | ||
10127 | | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
10128 | 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h | ||
10129 | | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h | ||
10130 | | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/ | ||
10131 | build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ | ||
10132 | yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 | ||
10133 | -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ | ||
10134 | lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/ | ||
10135 | tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/ | ||
10136 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ | ||
10137 | yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 | ||
10138 | -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" | ||
10139 | -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c | ||
10140 | -o tools/snep-send.o tools/snep-send.c | ||
10141 | | In file included from tools/snep-send.c:16:0: | ||
10142 | | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
10143 | | #include <near/dbus.h> | ||
10144 | | ^ | ||
10145 | | compilation terminated. | ||
10146 | | make[1]: *** [tools/snep-send.o] Error 1 | ||
10147 | | make[1]: *** Waiting for unfinished jobs.... | ||
10148 | | make: *** [all] Error 2 | ||
10149 | | ERROR: oe_runmake failed | ||
10150 | |||
10151 | Reproducing the Error | ||
10152 | ~~~~~~~~~~~~~~~~~~~~~ | ||
10153 | |||
10154 | Because race conditions are intermittent, they do not manifest | ||
10155 | themselves every time you do the build. In fact, most times the build | ||
10156 | will complete without problems even though the potential race condition | ||
10157 | exists. Thus, once the error surfaces, you need a way to reproduce it. | ||
10158 | |||
10159 | In this example, compiling the "neard" package is causing the problem. | ||
10160 | So the first thing to do is build "neard" locally. Before you start the | ||
10161 | build, set the | ||
10162 | :term:`PARALLEL_MAKE` variable | ||
10163 | in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a | ||
10164 | high value for ``PARALLEL_MAKE`` increases the chances of the race | ||
10165 | condition showing up: | ||
10166 | :: | ||
10167 | |||
10168 | $ bitbake neard | ||
10169 | |||
10170 | Once the local build for "neard" completes, start a ``devshell`` build: | ||
10171 | :: | ||
10172 | |||
10173 | $ bitbake neard -c devshell | ||
10174 | |||
10175 | For information on how to use a | ||
10176 | ``devshell``, see the "`Using a Development | ||
10177 | Shell <#platdev-appdev-devshell>`__" section. | ||
10178 | |||
10179 | In the ``devshell``, do the following: | ||
10180 | :: | ||
10181 | |||
10182 | $ make clean | ||
10183 | $ make tools/snep-send.o | ||
10184 | |||
10185 | The ``devshell`` commands cause the failure to clearly | ||
10186 | be visible. In this case, a missing dependency exists for the "neard" | ||
10187 | Makefile target. Here is some abbreviated, sample output with the | ||
10188 | missing dependency clearly visible at the end: | ||
10189 | :: | ||
10190 | |||
10191 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... | ||
10192 | . | ||
10193 | . | ||
10194 | . | ||
10195 | tools/snep-send.c | ||
10196 | In file included from tools/snep-send.c:16:0: | ||
10197 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
10198 | #include <near/dbus.h> | ||
10199 | ^ | ||
10200 | compilation terminated. | ||
10201 | make: *** [tools/snep-send.o] Error 1 | ||
10202 | $ | ||
10203 | |||
10204 | |||
10205 | Creating a Patch for the Fix | ||
10206 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10207 | |||
10208 | Because there is a missing dependency for the Makefile target, you need | ||
10209 | to patch the ``Makefile.am`` file, which is generated from | ||
10210 | ``Makefile.in``. You can use Quilt to create the patch: | ||
10211 | :: | ||
10212 | |||
10213 | $ quilt new parallelmake.patch | ||
10214 | Patch patches/parallelmake.patch is now on top | ||
10215 | $ quilt add Makefile.am | ||
10216 | File Makefile.am added to patch patches/parallelmake.patch | ||
10217 | |||
10218 | For more information on using Quilt, see the | ||
10219 | "`Using Quilt in Your Workflow <#using-a-quilt-workflow>`__" section. | ||
10220 | |||
10221 | At this point you need to make the edits to ``Makefile.am`` to add the | ||
10222 | missing dependency. For our example, you have to add the following line | ||
10223 | to the file: | ||
10224 | :: | ||
10225 | |||
10226 | tools/snep-send.$(OBJEXT): include/near/dbus.h | ||
10227 | |||
10228 | Once you have edited the file, use the ``refresh`` command to create the | ||
10229 | patch: | ||
10230 | :: | ||
10231 | |||
10232 | $ quilt refresh | ||
10233 | Refreshed patch patches/parallelmake.patch | ||
10234 | |||
10235 | Once | ||
10236 | the patch file exists, you need to add it back to the originating recipe | ||
10237 | folder. Here is an example assuming a top-level | ||
10238 | :term:`Source Directory` named ``poky``: | ||
10239 | :: | ||
10240 | |||
10241 | $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard | ||
10242 | |||
10243 | The final thing you need to do to implement the fix in the build is to | ||
10244 | update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the | ||
10245 | :term:`SRC_URI` statement includes | ||
10246 | the patch file. The recipe file is in the folder above the patch. Here | ||
10247 | is what the edited ``SRC_URI`` statement would look like: | ||
10248 | :: | ||
10249 | |||
10250 | SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ | ||
10251 | file://neard.in \ | ||
10252 | file://neard.service.in \ | ||
10253 | file://parallelmake.patch \ | ||
10254 | " | ||
10255 | |||
10256 | With the patch complete and moved to the correct folder and the | ||
10257 | ``SRC_URI`` statement updated, you can exit the ``devshell``: | ||
10258 | :: | ||
10259 | |||
10260 | $ exit | ||
10261 | |||
10262 | Testing the Build | ||
10263 | ~~~~~~~~~~~~~~~~~ | ||
10264 | |||
10265 | With everything in place, you can get back to trying the build again | ||
10266 | locally: | ||
10267 | :: | ||
10268 | |||
10269 | $ bitbake neard This build should succeed. | ||
10270 | |||
10271 | Now you can open up a ``devshell`` again and repeat the clean and make | ||
10272 | operations as follows: | ||
10273 | :: | ||
10274 | |||
10275 | $ bitbake neard -c devshell | ||
10276 | $ make clean | ||
10277 | $ make tools/snep-send.o | ||
10278 | |||
10279 | The build should work without issue. | ||
10280 | |||
10281 | As with all solved problems, if they originated upstream, you need to | ||
10282 | submit the fix for the recipe in OE-Core and upstream so that the | ||
10283 | problem is taken care of at its source. See the "`Submitting a Change to | ||
10284 | the Yocto Project <#how-to-submit-a-change>`__" section for more | ||
10285 | information. | ||
10286 | |||
10287 | .. _platdev-gdb-remotedebug: | ||
10288 | |||
10289 | Debugging With the GNU Project Debugger (GDB) Remotely | ||
10290 | ------------------------------------------------------ | ||
10291 | |||
10292 | GDB allows you to examine running programs, which in turn helps you to | ||
10293 | understand and fix problems. It also allows you to perform post-mortem | ||
10294 | style analysis of program crashes. GDB is available as a package within | ||
10295 | the Yocto Project and is installed in SDK images by default. See the | ||
10296 | ":ref:`ref-manual/ref-images:Images`" chapter in the Yocto | ||
10297 | Project Reference Manual for a description of these images. You can find | ||
10298 | information on GDB at http://sourceware.org/gdb/. | ||
10299 | |||
10300 | .. note:: | ||
10301 | |||
10302 | For best results, install debug ( | ||
10303 | -dbg | ||
10304 | ) packages for the applications you are going to debug. Doing so | ||
10305 | makes extra debug symbols available that give you more meaningful | ||
10306 | output. | ||
10307 | |||
10308 | Sometimes, due to memory or disk space constraints, it is not possible | ||
10309 | to use GDB directly on the remote target to debug applications. These | ||
10310 | constraints arise because GDB needs to load the debugging information | ||
10311 | and the binaries of the process being debugged. Additionally, GDB needs | ||
10312 | to perform many computations to locate information such as function | ||
10313 | names, variable names and values, stack traces and so forth - even | ||
10314 | before starting the debugging process. These extra computations place | ||
10315 | more load on the target system and can alter the characteristics of the | ||
10316 | program being debugged. | ||
10317 | |||
10318 | To help get past the previously mentioned constraints, you can use | ||
10319 | gdbserver, which runs on the remote target and does not load any | ||
10320 | debugging information from the debugged process. Instead, a GDB instance | ||
10321 | processes the debugging information that is run on a remote computer - | ||
10322 | the host GDB. The host GDB then sends control commands to gdbserver to | ||
10323 | make it stop or start the debugged program, as well as read or write | ||
10324 | memory regions of that debugged program. All the debugging information | ||
10325 | loaded and processed as well as all the heavy debugging is done by the | ||
10326 | host GDB. Offloading these processes gives the gdbserver running on the | ||
10327 | target a chance to remain small and fast. | ||
10328 | |||
10329 | Because the host GDB is responsible for loading the debugging | ||
10330 | information and for doing the necessary processing to make actual | ||
10331 | debugging happen, you have to make sure the host can access the | ||
10332 | unstripped binaries complete with their debugging information and also | ||
10333 | be sure the target is compiled with no optimizations. The host GDB must | ||
10334 | also have local access to all the libraries used by the debugged | ||
10335 | program. Because gdbserver does not need any local debugging | ||
10336 | information, the binaries on the remote target can remain stripped. | ||
10337 | However, the binaries must also be compiled without optimization so they | ||
10338 | match the host's binaries. | ||
10339 | |||
10340 | To remain consistent with GDB documentation and terminology, the binary | ||
10341 | being debugged on the remote target machine is referred to as the | ||
10342 | "inferior" binary. For documentation on GDB see the `GDB | ||
10343 | site <http://sourceware.org/gdb/documentation/>`__. | ||
10344 | |||
10345 | The following steps show you how to debug using the GNU project | ||
10346 | debugger. | ||
10347 | |||
10348 | 1. *Configure your build system to construct the companion debug | ||
10349 | filesystem:* | ||
10350 | |||
10351 | In your ``local.conf`` file, set the following: | ||
10352 | :: | ||
10353 | |||
10354 | IMAGE_GEN_DEBUGFS = "1" | ||
10355 | IMAGE_FSTYPES_DEBUGFS = "tar.bz2" | ||
10356 | |||
10357 | These options cause the | ||
10358 | OpenEmbedded build system to generate a special companion filesystem | ||
10359 | fragment, which contains the matching source and debug symbols to | ||
10360 | your deployable filesystem. The build system does this by looking at | ||
10361 | what is in the deployed filesystem, and pulling the corresponding | ||
10362 | ``-dbg`` packages. | ||
10363 | |||
10364 | The companion debug filesystem is not a complete filesystem, but only | ||
10365 | contains the debug fragments. This filesystem must be combined with | ||
10366 | the full filesystem for debugging. Subsequent steps in this procedure | ||
10367 | show how to combine the partial filesystem with the full filesystem. | ||
10368 | |||
10369 | 2. *Configure the system to include gdbserver in the target filesystem:* | ||
10370 | |||
10371 | Make the following addition in either your ``local.conf`` file or in | ||
10372 | an image recipe: | ||
10373 | :: | ||
10374 | |||
10375 | IMAGE_INSTALL_append = " gdbserver" | ||
10376 | |||
10377 | The change makes | ||
10378 | sure the ``gdbserver`` package is included. | ||
10379 | |||
10380 | 3. *Build the environment:* | ||
10381 | |||
10382 | Use the following command to construct the image and the companion | ||
10383 | Debug Filesystem: | ||
10384 | :: | ||
10385 | |||
10386 | $ bitbake image | ||
10387 | |||
10388 | Build the cross GDB component and | ||
10389 | make it available for debugging. Build the SDK that matches the | ||
10390 | image. Building the SDK is best for a production build that can be | ||
10391 | used later for debugging, especially during long term maintenance: | ||
10392 | :: | ||
10393 | |||
10394 | $ bitbake -c populate_sdk image | ||
10395 | |||
10396 | Alternatively, you can build the minimal toolchain components that | ||
10397 | match the target. Doing so creates a smaller than typical SDK and | ||
10398 | only contains a minimal set of components with which to build simple | ||
10399 | test applications, as well as run the debugger: | ||
10400 | :: | ||
10401 | |||
10402 | $ bitbake meta-toolchain | ||
10403 | |||
10404 | A final method is to build Gdb itself within the build system: | ||
10405 | :: | ||
10406 | |||
10407 | $ bitbake gdb-cross-<architecture> | ||
10408 | |||
10409 | Doing so produces a temporary copy of | ||
10410 | ``cross-gdb`` you can use for debugging during development. While | ||
10411 | this is the quickest approach, the two previous methods in this step | ||
10412 | are better when considering long-term maintenance strategies. | ||
10413 | |||
10414 | .. note:: | ||
10415 | |||
10416 | If you run | ||
10417 | bitbake gdb-cross | ||
10418 | , the OpenEmbedded build system suggests the actual image (e.g. | ||
10419 | gdb-cross-i586 | ||
10420 | ). The suggestion is usually the actual name you want to use. | ||
10421 | |||
10422 | 4. *Set up the* ``debugfs`` | ||
10423 | |||
10424 | Run the following commands to set up the ``debugfs``: | ||
10425 | :: | ||
10426 | |||
10427 | $ mkdir debugfs | ||
10428 | $ cd debugfs | ||
10429 | $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2 | ||
10430 | $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2 | ||
10431 | |||
10432 | 5. *Set up GDB* | ||
10433 | |||
10434 | Install the SDK (if you built one) and then source the correct | ||
10435 | environment file. Sourcing the environment file puts the SDK in your | ||
10436 | ``PATH`` environment variable. | ||
10437 | |||
10438 | If you are using the build system, Gdb is located in | ||
10439 | build-dir/tmp/sysroots/host/usr/bin/architecture/architecture-gdb | ||
10440 | |||
10441 | 6. *Boot the target:* | ||
10442 | |||
10443 | For information on how to run QEMU, see the `QEMU | ||
10444 | Documentation <http://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__. | ||
10445 | |||
10446 | .. note:: | ||
10447 | |||
10448 | Be sure to verify that your host can access the target via TCP. | ||
10449 | |||
10450 | 7. *Debug a program:* | ||
10451 | |||
10452 | Debugging a program involves running gdbserver on the target and then | ||
10453 | running Gdb on the host. The example in this step debugs ``gzip``: | ||
10454 | :: | ||
10455 | |||
10456 | root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help | ||
10457 | |||
10458 | For | ||
10459 | additional gdbserver options, see the `GDB Server | ||
10460 | Documentation <https://www.gnu.org/software/gdb/documentation/>`__. | ||
10461 | |||
10462 | After running gdbserver on the target, you need to run Gdb on the | ||
10463 | host and configure it and connect to the target. Use these commands: | ||
10464 | :: | ||
10465 | |||
10466 | $ cd directory-holding-the-debugfs-directory | ||
10467 | $ arch-gdb | ||
10468 | (gdb) set sysroot debugfs | ||
10469 | (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug | ||
10470 | (gdb) target remote IP-of-target:1234 | ||
10471 | |||
10472 | At this | ||
10473 | point, everything should automatically load (i.e. matching binaries, | ||
10474 | symbols and headers). | ||
10475 | |||
10476 | .. note:: | ||
10477 | |||
10478 | The Gdb | ||
10479 | set | ||
10480 | commands in the previous example can be placed into the users | ||
10481 | ~/.gdbinit | ||
10482 | file. Upon starting, Gdb automatically runs whatever commands are | ||
10483 | in that file. | ||
10484 | |||
10485 | 8. *Deploying without a full image rebuild:* | ||
10486 | |||
10487 | In many cases, during development you want a quick method to deploy a | ||
10488 | new binary to the target and debug it, without waiting for a full | ||
10489 | image build. | ||
10490 | |||
10491 | One approach to solving this situation is to just build the component | ||
10492 | you want to debug. Once you have built the component, copy the | ||
10493 | executable directly to both the target and the host ``debugfs``. | ||
10494 | |||
10495 | If the binary is processed through the debug splitting in | ||
10496 | OpenEmbedded, you should also copy the debug items (i.e. ``.debug`` | ||
10497 | contents and corresponding ``/usr/src/debug`` files) from the work | ||
10498 | directory. Here is an example: | ||
10499 | :: | ||
10500 | |||
10501 | $ bitbake bash | ||
10502 | $ bitbake -c devshell bash | ||
10503 | $ cd .. | ||
10504 | $ scp packages-split/bash/bin/bash target:/bin/bash | ||
10505 | $ cp -a packages-split/bash-dbg/\* path/debugfs | ||
10506 | |||
10507 | Debugging with the GNU Project Debugger (GDB) on the Target | ||
10508 | ----------------------------------------------------------- | ||
10509 | |||
10510 | The previous section addressed using GDB remotely for debugging | ||
10511 | purposes, which is the most usual case due to the inherent hardware | ||
10512 | limitations on many embedded devices. However, debugging in the target | ||
10513 | hardware itself is also possible with more powerful devices. This | ||
10514 | section describes what you need to do in order to support using GDB to | ||
10515 | debug on the target hardware. | ||
10516 | |||
10517 | To support this kind of debugging, you need do the following: | ||
10518 | |||
10519 | - Ensure that GDB is on the target. You can do this by adding "gdb" to | ||
10520 | :term:`IMAGE_INSTALL`: | ||
10521 | IMAGE_INSTALL_append = " gdb" Alternatively, you can add | ||
10522 | "tools-debug" to | ||
10523 | :term:`IMAGE_FEATURES`: | ||
10524 | :: | ||
10525 | |||
10526 | IMAGE_FEATURES_append = " tools-debug" | ||
10527 | |||
10528 | - Ensure that debug symbols are present. You can make sure these | ||
10529 | symbols are present by installing ``-dbg``: | ||
10530 | :: | ||
10531 | |||
10532 | IMAGE_INSTALL_append = "packagename-dbg" | ||
10533 | |||
10534 | Alternatively, you can do the following to include | ||
10535 | all the debug symbols: | ||
10536 | :: | ||
10537 | |||
10538 | IMAGE_FEATURES_append = " dbg-pkgs" | ||
10539 | |||
10540 | .. note:: | ||
10541 | |||
10542 | To improve the debug information accuracy, you can reduce the level | ||
10543 | of optimization used by the compiler. For example, when adding the | ||
10544 | following line to your | ||
10545 | local.conf | ||
10546 | file, you will reduce optimization from | ||
10547 | FULL_OPTIMIZATION | ||
10548 | of "-O2" to | ||
10549 | DEBUG_OPTIMIZATION | ||
10550 | of "-O -fno-omit-frame-pointer": | ||
10551 | :: | ||
10552 | |||
10553 | DEBUG_BUILD = "1" | ||
10554 | |||
10555 | |||
10556 | Consider that this will reduce the application's performance and is | ||
10557 | recommended only for debugging purposes. | ||
10558 | |||
10559 | .. _dev-other-debugging-others: | ||
10560 | |||
10561 | Other Debugging Tips | ||
10562 | -------------------- | ||
10563 | |||
10564 | Here are some other tips that you might find useful: | ||
10565 | |||
10566 | - When adding new packages, it is worth watching for undesirable items | ||
10567 | making their way into compiler command lines. For example, you do not | ||
10568 | want references to local system files like ``/usr/lib/`` or | ||
10569 | ``/usr/include/``. | ||
10570 | |||
10571 | - If you want to remove the ``psplash`` boot splashscreen, add | ||
10572 | ``psplash=false`` to the kernel command line. Doing so prevents | ||
10573 | ``psplash`` from loading and thus allows you to see the console. It | ||
10574 | is also possible to switch out of the splashscreen by switching the | ||
10575 | virtual console (e.g. Fn+Left or Fn+Right on a Zaurus). | ||
10576 | |||
10577 | - Removing :term:`TMPDIR` (usually | ||
10578 | ``tmp/``, within the | ||
10579 | :term:`Build Directory`) can often fix | ||
10580 | temporary build issues. Removing ``TMPDIR`` is usually a relatively | ||
10581 | cheap operation, because task output will be cached in | ||
10582 | :term:`SSTATE_DIR` (usually | ||
10583 | ``sstate-cache/``, which is also in the Build Directory). | ||
10584 | |||
10585 | .. note:: | ||
10586 | |||
10587 | Removing | ||
10588 | TMPDIR | ||
10589 | might be a workaround rather than a fix. Consequently, trying to | ||
10590 | determine the underlying cause of an issue before removing the | ||
10591 | directory is a good idea. | ||
10592 | |||
10593 | - Understanding how a feature is used in practice within existing | ||
10594 | recipes can be very helpful. It is recommended that you configure | ||
10595 | some method that allows you to quickly search through files. | ||
10596 | |||
10597 | Using GNU Grep, you can use the following shell function to | ||
10598 | recursively search through common recipe-related files, skipping | ||
10599 | binary files, ``.git`` directories, and the Build Directory (assuming | ||
10600 | its name starts with "build"): | ||
10601 | :: | ||
10602 | |||
10603 | g() { | ||
10604 | grep -Ir \ | ||
10605 | --exclude-dir=.git \ | ||
10606 | --exclude-dir='build*' \ | ||
10607 | --include='*.bb*' \ | ||
10608 | --include='*.inc*' \ | ||
10609 | --include='*.conf*' \ | ||
10610 | --include='*.py*' \ | ||
10611 | "$@" | ||
10612 | } | ||
10613 | |||
10614 | Following are some usage examples: | ||
10615 | :: | ||
10616 | |||
10617 | $ g FOO # Search recursively for "FOO" | ||
10618 | $ g -i foo # Search recursively for "foo", ignoring case | ||
10619 | $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" | ||
10620 | |||
10621 | If figuring | ||
10622 | out how some feature works requires a lot of searching, it might | ||
10623 | indicate that the documentation should be extended or improved. In | ||
10624 | such cases, consider filing a documentation bug using the Yocto | ||
10625 | Project implementation of | ||
10626 | :yocto_bugs:`Bugzilla <>`. For information on | ||
10627 | how to submit a bug against the Yocto Project, see the Yocto Project | ||
10628 | Bugzilla :yocto_wiki:`wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>` | ||
10629 | and the "`Submitting a Defect Against the Yocto | ||
10630 | Project <#submitting-a-defect-against-the-yocto-project>`__" section. | ||
10631 | |||
10632 | .. note:: | ||
10633 | |||
10634 | The manuals might not be the right place to document variables | ||
10635 | that are purely internal and have a limited scope (e.g. internal | ||
10636 | variables used to implement a single | ||
10637 | .bbclass | ||
10638 | file). | ||
10639 | |||
10640 | Making Changes to the Yocto Project | ||
10641 | =================================== | ||
10642 | |||
10643 | Because the Yocto Project is an open-source, community-based project, | ||
10644 | you can effect changes to the project. This section presents procedures | ||
10645 | that show you how to submit a defect against the project and how to | ||
10646 | submit a change. | ||
10647 | |||
10648 | Submitting a Defect Against the Yocto Project | ||
10649 | --------------------------------------------- | ||
10650 | |||
10651 | Use the Yocto Project implementation of | ||
10652 | `Bugzilla <http://www.bugzilla.org/about/>`__ to submit a defect (bug) | ||
10653 | against the Yocto Project. For additional information on this | ||
10654 | implementation of Bugzilla see the :ref:"`Yocto Project | ||
10655 | Bugzilla <resources-bugtracker>`" section in the | ||
10656 | Yocto Project Reference Manual. For more detail on any of the following | ||
10657 | steps, see the Yocto Project | ||
10658 | :yocto_wiki:`Bugzilla wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`. | ||
10659 | |||
10660 | Use the following general steps to submit a bug" | ||
10661 | |||
10662 | 1. Open the Yocto Project implementation of :yocto_bugs:`Bugzilla <>`. | ||
10663 | |||
10664 | 2. Click "File a Bug" to enter a new bug. | ||
10665 | |||
10666 | 3. Choose the appropriate "Classification", "Product", and "Component" | ||
10667 | for which the bug was found. Bugs for the Yocto Project fall into | ||
10668 | one of several classifications, which in turn break down into | ||
10669 | several products and components. For example, for a bug against the | ||
10670 | ``meta-intel`` layer, you would choose "Build System, Metadata & | ||
10671 | Runtime", "BSPs", and "bsps-meta-intel", respectively. | ||
10672 | |||
10673 | 4. Choose the "Version" of the Yocto Project for which you found the | ||
10674 | bug (e.g. DISTRO). | ||
10675 | |||
10676 | 5. Determine and select the "Severity" of the bug. The severity | ||
10677 | indicates how the bug impacted your work. | ||
10678 | |||
10679 | 6. Choose the "Hardware" that the bug impacts. | ||
10680 | |||
10681 | 7. Choose the "Architecture" that the bug impacts. | ||
10682 | |||
10683 | 8. Choose a "Documentation change" item for the bug. Fixing a bug might | ||
10684 | or might not affect the Yocto Project documentation. If you are | ||
10685 | unsure of the impact to the documentation, select "Don't Know". | ||
10686 | |||
10687 | 9. Provide a brief "Summary" of the bug. Try to limit your summary to | ||
10688 | just a line or two and be sure to capture the essence of the bug. | ||
10689 | |||
10690 | 10. Provide a detailed "Description" of the bug. You should provide as | ||
10691 | much detail as you can about the context, behavior, output, and so | ||
10692 | forth that surrounds the bug. You can even attach supporting files | ||
10693 | for output from logs by using the "Add an attachment" button. | ||
10694 | |||
10695 | 11. Click the "Submit Bug" button submit the bug. A new Bugzilla number | ||
10696 | is assigned to the bug and the defect is logged in the bug tracking | ||
10697 | system. | ||
10698 | |||
10699 | Once you file a bug, the bug is processed by the Yocto Project Bug | ||
10700 | Triage Team and further details concerning the bug are assigned (e.g. | ||
10701 | priority and owner). You are the "Submitter" of the bug and any further | ||
10702 | categorization, progress, or comments on the bug result in Bugzilla | ||
10703 | sending you an automated email concerning the particular change or | ||
10704 | progress to the bug. | ||
10705 | |||
10706 | .. _how-to-submit-a-change: | ||
10707 | |||
10708 | Submitting a Change to the Yocto Project | ||
10709 | ---------------------------------------- | ||
10710 | |||
10711 | Contributions to the Yocto Project and OpenEmbedded are very welcome. | ||
10712 | Because the system is extremely configurable and flexible, we recognize | ||
10713 | that developers will want to extend, configure or optimize it for their | ||
10714 | specific uses. | ||
10715 | |||
10716 | The Yocto Project uses a mailing list and a patch-based workflow that is | ||
10717 | similar to the Linux kernel but contains important differences. In | ||
10718 | general, a mailing list exists through which you can submit patches. You | ||
10719 | should send patches to the appropriate mailing list so that they can be | ||
10720 | reviewed and merged by the appropriate maintainer. The specific mailing | ||
10721 | list you need to use depends on the location of the code you are | ||
10722 | changing. Each component (e.g. layer) should have a ``README`` file that | ||
10723 | indicates where to send the changes and which process to follow. | ||
10724 | |||
10725 | You can send the patch to the mailing list using whichever approach you | ||
10726 | feel comfortable with to generate the patch. Once sent, the patch is | ||
10727 | usually reviewed by the community at large. If somebody has concerns | ||
10728 | with the patch, they will usually voice their concern over the mailing | ||
10729 | list. If a patch does not receive any negative reviews, the maintainer | ||
10730 | of the affected layer typically takes the patch, tests it, and then | ||
10731 | based on successful testing, merges the patch. | ||
10732 | |||
10733 | The "poky" repository, which is the Yocto Project's reference build | ||
10734 | environment, is a hybrid repository that contains several individual | ||
10735 | pieces (e.g. BitBake, Metadata, documentation, and so forth) built using | ||
10736 | the combo-layer tool. The upstream location used for submitting changes | ||
10737 | varies by component: | ||
10738 | |||
10739 | - *Core Metadata:* Send your patch to the | ||
10740 | `openembedded-core <http://lists.openembedded.org/mailman/listinfo/openembedded-core>`__ | ||
10741 | mailing list. For example, a change to anything under the ``meta`` or | ||
10742 | ``scripts`` directories should be sent to this mailing list. | ||
10743 | |||
10744 | - *BitBake:* For changes to BitBake (i.e. anything under the | ||
10745 | ``bitbake`` directory), send your patch to the | ||
10746 | `bitbake-devel <http://lists.openembedded.org/mailman/listinfo/bitbake-devel>`__ | ||
10747 | mailing list. | ||
10748 | |||
10749 | - *"meta-\*" trees:* These trees contain Metadata. Use the | ||
10750 | `poky <https://lists.yoctoproject.org/g/poky>`__ mailing list. | ||
10751 | |||
10752 | - *Documentation*: For changes to the Yocto Project documentation, use the `docs | ||
10753 | <https://lists.yoctoproject.org/g/docs>`__ mailing list. | ||
10754 | |||
10755 | For changes to other layers hosted in the Yocto Project source | ||
10756 | repositories (i.e. ``yoctoproject.org``) and tools use the `Yocto Project | ||
10757 | <https://lists.yoctoproject.org/g/yocto/>`__ general mailing list. | ||
10758 | |||
10759 | .. note:: | ||
10760 | |||
10761 | Sometimes a layer's documentation specifies to use a particular | ||
10762 | mailing list. If so, use that list. | ||
10763 | |||
10764 | For additional recipes that do not fit into the core Metadata, you | ||
10765 | should determine which layer the recipe should go into and submit the | ||
10766 | change in the manner recommended by the documentation (e.g. the | ||
10767 | ``README`` file) supplied with the layer. If in doubt, please ask on the | ||
10768 | Yocto general mailing list or on the openembedded-devel mailing list. | ||
10769 | |||
10770 | You can also push a change upstream and request a maintainer to pull the | ||
10771 | change into the component's upstream repository. You do this by pushing | ||
10772 | to a contribution repository that is upstream. See the ":ref:`gs-git-workflows-and-the-yocto-project`" | ||
10773 | section in the Yocto Project Overview and Concepts Manual for additional | ||
10774 | concepts on working in the Yocto Project development environment. | ||
10775 | |||
10776 | Two commonly used testing repositories exist for OpenEmbedded-Core: | ||
10777 | |||
10778 | - *"ross/mut" branch:* The "mut" (master-under-test) tree exists in the | ||
10779 | ``poky-contrib`` repository in the | ||
10780 | :yocto_git:`Yocto Project source repositories <>`. | ||
10781 | |||
10782 | - *"master-next" branch:* This branch is part of the main "poky" | ||
10783 | repository in the Yocto Project source repositories. | ||
10784 | |||
10785 | Maintainers use these branches to test submissions prior to merging | ||
10786 | patches. Thus, you can get an idea of the status of a patch based on | ||
10787 | whether the patch has been merged into one of these branches. | ||
10788 | |||
10789 | .. note:: | ||
10790 | |||
10791 | This system is imperfect and changes can sometimes get lost in the | ||
10792 | flow. Asking about the status of a patch or change is reasonable if | ||
10793 | the change has been idle for a while with no feedback. The Yocto | ||
10794 | Project does have plans to use | ||
10795 | Patchwork | ||
10796 | to track the status of patches and also to automatically preview | ||
10797 | patches. | ||
10798 | |||
10799 | The following sections provide procedures for submitting a change. | ||
10800 | |||
10801 | .. _pushing-a-change-upstream: | ||
10802 | |||
10803 | Using Scripts to Push a Change Upstream and Request a Pull | ||
10804 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10805 | |||
10806 | Follow this procedure to push a change to an upstream "contrib" Git | ||
10807 | repository: | ||
10808 | |||
10809 | .. note:: | ||
10810 | |||
10811 | You can find general Git information on how to push a change upstream | ||
10812 | in the | ||
10813 | Git Community Book | ||
10814 | . | ||
10815 | |||
10816 | 1. *Make Your Changes Locally:* Make your changes in your local Git | ||
10817 | repository. You should make small, controlled, isolated changes. | ||
10818 | Keeping changes small and isolated aids review, makes | ||
10819 | merging/rebasing easier and keeps the change history clean should | ||
10820 | anyone need to refer to it in future. | ||
10821 | |||
10822 | 2. *Stage Your Changes:* Stage your changes by using the ``git add`` | ||
10823 | command on each file you changed. | ||
10824 | |||
10825 | 3. *Commit Your Changes:* Commit the change by using the ``git commit`` | ||
10826 | command. Make sure your commit information follows standards by | ||
10827 | following these accepted conventions: | ||
10828 | |||
10829 | - Be sure to include a "Signed-off-by:" line in the same style as | ||
10830 | required by the Linux kernel. Adding this line signifies that you, | ||
10831 | the submitter, have agreed to the Developer's Certificate of | ||
10832 | Origin 1.1 as follows: | ||
10833 | :: | ||
10834 | |||
10835 | Developer's Certificate of Origin 1.1 | ||
10836 | |||
10837 | By making a contribution to this project, I certify that: | ||
10838 | |||
10839 | (a) The contribution was created in whole or in part by me and I | ||
10840 | have the right to submit it under the open source license | ||
10841 | indicated in the file; or | ||
10842 | |||
10843 | (b) The contribution is based upon previous work that, to the best | ||
10844 | of my knowledge, is covered under an appropriate open source | ||
10845 | license and I have the right under that license to submit that | ||
10846 | work with modifications, whether created in whole or in part | ||
10847 | by me, under the same open source license (unless I am | ||
10848 | permitted to submit under a different license), as indicated | ||
10849 | in the file; or | ||
10850 | |||
10851 | (c) The contribution was provided directly to me by some other | ||
10852 | person who certified (a), (b) or (c) and I have not modified | ||
10853 | it. | ||
10854 | |||
10855 | (d) I understand and agree that this project and the contribution | ||
10856 | are public and that a record of the contribution (including all | ||
10857 | personal information I submit with it, including my sign-off) is | ||
10858 | maintained indefinitely and may be redistributed consistent with | ||
10859 | this project or the open source license(s) involved. | ||
10860 | |||
10861 | - Provide a single-line summary of the change. and, if more | ||
10862 | explanation is needed, provide more detail in the body of the | ||
10863 | commit. This summary is typically viewable in the "shortlist" of | ||
10864 | changes. Thus, providing something short and descriptive that | ||
10865 | gives the reader a summary of the change is useful when viewing a | ||
10866 | list of many commits. You should prefix this short description | ||
10867 | with the recipe name (if changing a recipe), or else with the | ||
10868 | short form path to the file being changed. | ||
10869 | |||
10870 | - For the body of the commit message, provide detailed information | ||
10871 | that describes what you changed, why you made the change, and the | ||
10872 | approach you used. It might also be helpful if you mention how you | ||
10873 | tested the change. Provide as much detail as you can in the body | ||
10874 | of the commit message. | ||
10875 | |||
10876 | .. note:: | ||
10877 | |||
10878 | You do not need to provide a more detailed explanation of a | ||
10879 | change if the change is minor to the point of the single line | ||
10880 | summary providing all the information. | ||
10881 | |||
10882 | - If the change addresses a specific bug or issue that is associated | ||
10883 | with a bug-tracking ID, include a reference to that ID in your | ||
10884 | detailed description. For example, the Yocto Project uses a | ||
10885 | specific convention for bug references - any commit that addresses | ||
10886 | a specific bug should use the following form for the detailed | ||
10887 | description. Be sure to use the actual bug-tracking ID from | ||
10888 | Bugzilla for bug-id: | ||
10889 | :: | ||
10890 | |||
10891 | Fixes [YOCTO #bug-id] | ||
10892 | |||
10893 | detailed description of change | ||
10894 | |||
10895 | 4. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for | ||
10896 | permissions to push to an upstream contrib repository, push the | ||
10897 | change to that repository: | ||
10898 | :: | ||
10899 | |||
10900 | $ git push upstream_remote_repo local_branch_name | ||
10901 | |||
10902 | For example, suppose you have permissions to push | ||
10903 | into the upstream ``meta-intel-contrib`` repository and you are | ||
10904 | working in a local branch named your_name\ ``/README``. The following | ||
10905 | command pushes your local commits to the ``meta-intel-contrib`` | ||
10906 | upstream repository and puts the commit in a branch named | ||
10907 | your_name\ ``/README``: | ||
10908 | :: | ||
10909 | |||
10910 | $ git push meta-intel-contrib your_name/README | ||
10911 | |||
10912 | 5. *Determine Who to Notify:* Determine the maintainer or the mailing | ||
10913 | list that you need to notify for the change. | ||
10914 | |||
10915 | Before submitting any change, you need to be sure who the maintainer | ||
10916 | is or what mailing list that you need to notify. Use either these | ||
10917 | methods to find out: | ||
10918 | |||
10919 | - *Maintenance File:* Examine the ``maintainers.inc`` file, which is | ||
10920 | located in the :term:`Source Directory` at | ||
10921 | ``meta/conf/distro/include``, to see who is responsible for code. | ||
10922 | |||
10923 | - *Search by File:* Using :ref:`overview-manual/overview-manual-development-environment:git`, you can | ||
10924 | enter the following command to bring up a short list of all | ||
10925 | commits against a specific file: | ||
10926 | :: | ||
10927 | |||
10928 | git shortlog -- filename | ||
10929 | |||
10930 | Just provide the name of the file for which you are interested. The | ||
10931 | information returned is not ordered by history but does include a | ||
10932 | list of everyone who has committed grouped by name. From the list, | ||
10933 | you can see who is responsible for the bulk of the changes against | ||
10934 | the file. | ||
10935 | |||
10936 | - *Examine the List of Mailing Lists:* For a list of the Yocto | ||
10937 | Project and related mailing lists, see the ":ref:`Mailing | ||
10938 | lists <resources-mailinglist>`" section in | ||
10939 | the Yocto Project Reference Manual. | ||
10940 | |||
10941 | 6. *Make a Pull Request:* Notify the maintainer or the mailing list that | ||
10942 | you have pushed a change by making a pull request. | ||
10943 | |||
10944 | The Yocto Project provides two scripts that conveniently let you | ||
10945 | generate and send pull requests to the Yocto Project. These scripts | ||
10946 | are ``create-pull-request`` and ``send-pull-request``. You can find | ||
10947 | these scripts in the ``scripts`` directory within the | ||
10948 | :term:`Source Directory` (e.g. | ||
10949 | ``~/poky/scripts``). | ||
10950 | |||
10951 | Using these scripts correctly formats the requests without | ||
10952 | introducing any whitespace or HTML formatting. The maintainer that | ||
10953 | receives your patches either directly or through the mailing list | ||
10954 | needs to be able to save and apply them directly from your emails. | ||
10955 | Using these scripts is the preferred method for sending patches. | ||
10956 | |||
10957 | First, create the pull request. For example, the following command | ||
10958 | runs the script, specifies the upstream repository in the contrib | ||
10959 | directory into which you pushed the change, and provides a subject | ||
10960 | line in the created patch files: | ||
10961 | :: | ||
10962 | |||
10963 | $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README" | ||
10964 | |||
10965 | Running this script forms ``*.patch`` files in a folder named | ||
10966 | ``pull-``\ PID in the current directory. One of the patch files is a | ||
10967 | cover letter. | ||
10968 | |||
10969 | Before running the ``send-pull-request`` script, you must edit the | ||
10970 | cover letter patch to insert information about your change. After | ||
10971 | editing the cover letter, send the pull request. For example, the | ||
10972 | following command runs the script and specifies the patch directory | ||
10973 | and email address. In this example, the email address is a mailing | ||
10974 | list: | ||
10975 | :: | ||
10976 | |||
10977 | $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org | ||
10978 | |||
10979 | You need to follow the prompts as the script is interactive. | ||
10980 | |||
10981 | .. note:: | ||
10982 | |||
10983 | For help on using these scripts, simply provide the | ||
10984 | -h | ||
10985 | argument as follows: | ||
10986 | :: | ||
10987 | |||
10988 | $ poky/scripts/create-pull-request -h | ||
10989 | $ poky/scripts/send-pull-request -h | ||
10990 | |||
10991 | |||
10992 | .. _submitting-a-patch: | ||
10993 | |||
10994 | Using Email to Submit a Patch | ||
10995 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
10996 | |||
10997 | You can submit patches without using the ``create-pull-request`` and | ||
10998 | ``send-pull-request`` scripts described in the previous section. | ||
10999 | However, keep in mind, the preferred method is to use the scripts. | ||
11000 | |||
11001 | Depending on the components changed, you need to submit the email to a | ||
11002 | specific mailing list. For some guidance on which mailing list to use, | ||
11003 | see the `list <#figuring-out-the-mailing-list-to-use>`__ at the | ||
11004 | beginning of this section. For a description of all the available | ||
11005 | mailing lists, see the ":ref:`Mailing Lists <resources-mailinglist>`" section in the | ||
11006 | Yocto Project Reference Manual. | ||
11007 | |||
11008 | Here is the general procedure on how to submit a patch through email | ||
11009 | without using the scripts: | ||
11010 | |||
11011 | 1. *Make Your Changes Locally:* Make your changes in your local Git | ||
11012 | repository. You should make small, controlled, isolated changes. | ||
11013 | Keeping changes small and isolated aids review, makes | ||
11014 | merging/rebasing easier and keeps the change history clean should | ||
11015 | anyone need to refer to it in future. | ||
11016 | |||
11017 | 2. *Stage Your Changes:* Stage your changes by using the ``git add`` | ||
11018 | command on each file you changed. | ||
11019 | |||
11020 | 3. *Commit Your Changes:* Commit the change by using the | ||
11021 | ``git commit --signoff`` command. Using the ``--signoff`` option | ||
11022 | identifies you as the person making the change and also satisfies the | ||
11023 | Developer's Certificate of Origin (DCO) shown earlier. | ||
11024 | |||
11025 | When you form a commit, you must follow certain standards established | ||
11026 | by the Yocto Project development team. See `Step | ||
11027 | 3 <#making-sure-you-have-correct-commit-information>`__ in the | ||
11028 | previous section for information on how to provide commit information | ||
11029 | that meets Yocto Project commit message standards. | ||
11030 | |||
11031 | 4. *Format the Commit:* Format the commit into an email message. To | ||
11032 | format commits, use the ``git format-patch`` command. When you | ||
11033 | provide the command, you must include a revision list or a number of | ||
11034 | patches as part of the command. For example, either of these two | ||
11035 | commands takes your most recent single commit and formats it as an | ||
11036 | email message in the current directory: | ||
11037 | :: | ||
11038 | |||
11039 | $ git format-patch -1 | ||
11040 | |||
11041 | or :: | ||
11042 | |||
11043 | $ git format-patch HEAD~ | ||
11044 | |||
11045 | After the command is run, the current directory contains a numbered | ||
11046 | ``.patch`` file for the commit. | ||
11047 | |||
11048 | If you provide several commits as part of the command, the | ||
11049 | ``git format-patch`` command produces a series of numbered files in | ||
11050 | the current directory – one for each commit. If you have more than | ||
11051 | one patch, you should also use the ``--cover`` option with the | ||
11052 | command, which generates a cover letter as the first "patch" in the | ||
11053 | series. You can then edit the cover letter to provide a description | ||
11054 | for the series of patches. For information on the | ||
11055 | ``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed | ||
11056 | using the ``man git-format-patch`` command. | ||
11057 | |||
11058 | .. note:: | ||
11059 | |||
11060 | If you are or will be a frequent contributor to the Yocto Project | ||
11061 | or to OpenEmbedded, you might consider requesting a contrib area | ||
11062 | and the necessary associated rights. | ||
11063 | |||
11064 | 5. *Import the Files Into Your Mail Client:* Import the files into your | ||
11065 | mail client by using the ``git send-email`` command. | ||
11066 | |||
11067 | .. note:: | ||
11068 | |||
11069 | In order to use | ||
11070 | git send-email | ||
11071 | , you must have the proper Git packages installed on your host. | ||
11072 | For Ubuntu, Debian, and Fedora the package is | ||
11073 | git-email | ||
11074 | . | ||
11075 | |||
11076 | The ``git send-email`` command sends email by using a local or remote | ||
11077 | Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or | ||
11078 | through a direct ``smtp`` configuration in your Git ``~/.gitconfig`` | ||
11079 | file. If you are submitting patches through email only, it is very | ||
11080 | important that you submit them without any whitespace or HTML | ||
11081 | formatting that either you or your mailer introduces. The maintainer | ||
11082 | that receives your patches needs to be able to save and apply them | ||
11083 | directly from your emails. A good way to verify that what you are | ||
11084 | sending will be applicable by the maintainer is to do a dry run and | ||
11085 | send them to yourself and then save and apply them as the maintainer | ||
11086 | would. | ||
11087 | |||
11088 | The ``git send-email`` command is the preferred method for sending | ||
11089 | your patches using email since there is no risk of compromising | ||
11090 | whitespace in the body of the message, which can occur when you use | ||
11091 | your own mail client. The command also has several options that let | ||
11092 | you specify recipients and perform further editing of the email | ||
11093 | message. For information on how to use the ``git send-email`` | ||
11094 | command, see ``GIT-SEND-EMAIL(1)`` displayed using the | ||
11095 | ``man git-send-email`` command. | ||
11096 | |||
11097 | Working With Licenses | ||
11098 | ===================== | ||
11099 | |||
11100 | As mentioned in the ":ref:`overview-manual/overview-manual-development-environment:licensing`" | ||
11101 | section in the Yocto Project Overview and Concepts Manual, open source | ||
11102 | projects are open to the public and they consequently have different | ||
11103 | licensing structures in place. This section describes the mechanism by | ||
11104 | which the :term:`OpenEmbedded Build System` | ||
11105 | tracks changes to | ||
11106 | licensing text and covers how to maintain open source license compliance | ||
11107 | during your project's lifecycle. The section also describes how to | ||
11108 | enable commercially licensed recipes, which by default are disabled. | ||
11109 | |||
11110 | .. _usingpoky-configuring-LIC_FILES_CHKSUM: | ||
11111 | |||
11112 | Tracking License Changes | ||
11113 | ------------------------ | ||
11114 | |||
11115 | The license of an upstream project might change in the future. In order | ||
11116 | to prevent these changes going unnoticed, the | ||
11117 | :term:`LIC_FILES_CHKSUM` | ||
11118 | variable tracks changes to the license text. The checksums are validated | ||
11119 | at the end of the configure step, and if the checksums do not match, the | ||
11120 | build will fail. | ||
11121 | |||
11122 | .. _usingpoky-specifying-LIC_FILES_CHKSUM: | ||
11123 | |||
11124 | Specifying the ``LIC_FILES_CHKSUM`` Variable | ||
11125 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11126 | |||
11127 | The ``LIC_FILES_CHKSUM`` variable contains checksums of the license text | ||
11128 | in the source code for the recipe. Following is an example of how to | ||
11129 | specify ``LIC_FILES_CHKSUM``: | ||
11130 | :: | ||
11131 | |||
11132 | LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ | ||
11133 | file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ | ||
11134 | file://licfile2.txt;endline=50;md5=zzzz \ | ||
11135 | ..." | ||
11136 | |||
11137 | .. note:: | ||
11138 | |||
11139 | - When using "beginline" and "endline", realize that line numbering | ||
11140 | begins with one and not zero. Also, the included lines are | ||
11141 | inclusive (i.e. lines five through and including 29 in the | ||
11142 | previous example for ``licfile1.txt``). | ||
11143 | |||
11144 | - When a license check fails, the selected license text is included | ||
11145 | as part of the QA message. Using this output, you can determine | ||
11146 | the exact start and finish for the needed license text. | ||
11147 | |||
11148 | The build system uses the :term:`S` | ||
11149 | variable as the default directory when searching files listed in | ||
11150 | ``LIC_FILES_CHKSUM``. The previous example employs the default | ||
11151 | directory. | ||
11152 | |||
11153 | Consider this next example: | ||
11154 | :: | ||
11155 | |||
11156 | LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ | ||
11157 | md5=bb14ed3c4cda583abc85401304b5cd4e" | ||
11158 | LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" | ||
11159 | |||
11160 | The first line locates a file in ``${S}/src/ls.c`` and isolates lines | ||
11161 | five through 16 as license text. The second line refers to a file in | ||
11162 | :term:`WORKDIR`. | ||
11163 | |||
11164 | Note that ``LIC_FILES_CHKSUM`` variable is mandatory for all recipes, | ||
11165 | unless the ``LICENSE`` variable is set to "CLOSED". | ||
11166 | |||
11167 | .. _usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax: | ||
11168 | |||
11169 | Explanation of Syntax | ||
11170 | ~~~~~~~~~~~~~~~~~~~~~ | ||
11171 | |||
11172 | As mentioned in the previous section, the ``LIC_FILES_CHKSUM`` variable | ||
11173 | lists all the important files that contain the license text for the | ||
11174 | source code. It is possible to specify a checksum for an entire file, or | ||
11175 | a specific section of a file (specified by beginning and ending line | ||
11176 | numbers with the "beginline" and "endline" parameters, respectively). | ||
11177 | The latter is useful for source files with a license notice header, | ||
11178 | README documents, and so forth. If you do not use the "beginline" | ||
11179 | parameter, then it is assumed that the text begins on the first line of | ||
11180 | the file. Similarly, if you do not use the "endline" parameter, it is | ||
11181 | assumed that the license text ends with the last line of the file. | ||
11182 | |||
11183 | The "md5" parameter stores the md5 checksum of the license text. If the | ||
11184 | license text changes in any way as compared to this parameter then a | ||
11185 | mismatch occurs. This mismatch triggers a build failure and notifies the | ||
11186 | developer. Notification allows the developer to review and address the | ||
11187 | license text changes. Also note that if a mismatch occurs during the | ||
11188 | build, the correct md5 checksum is placed in the build log and can be | ||
11189 | easily copied to the recipe. | ||
11190 | |||
11191 | There is no limit to how many files you can specify using the | ||
11192 | ``LIC_FILES_CHKSUM`` variable. Generally, however, every project | ||
11193 | requires a few specifications for license tracking. Many projects have a | ||
11194 | "COPYING" file that stores the license information for all the source | ||
11195 | code files. This practice allows you to just track the "COPYING" file as | ||
11196 | long as it is kept up to date. | ||
11197 | |||
11198 | .. note:: | ||
11199 | |||
11200 | - If you specify an empty or invalid "md5" parameter, | ||
11201 | :term:`BitBake` returns an md5 | ||
11202 | mis-match error and displays the correct "md5" parameter value | ||
11203 | during the build. The correct parameter is also captured in the | ||
11204 | build log. | ||
11205 | |||
11206 | - If the whole file contains only license text, you do not need to | ||
11207 | use the "beginline" and "endline" parameters. | ||
11208 | |||
11209 | Enabling Commercially Licensed Recipes | ||
11210 | -------------------------------------- | ||
11211 | |||
11212 | By default, the OpenEmbedded build system disables components that have | ||
11213 | commercial or other special licensing requirements. Such requirements | ||
11214 | are defined on a recipe-by-recipe basis through the | ||
11215 | :term:`LICENSE_FLAGS` variable | ||
11216 | definition in the affected recipe. For instance, the | ||
11217 | ``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe | ||
11218 | contains the following statement: | ||
11219 | :: | ||
11220 | |||
11221 | LICENSE_FLAGS = "commercial" | ||
11222 | |||
11223 | Here is a | ||
11224 | slightly more complicated example that contains both an explicit recipe | ||
11225 | name and version (after variable expansion): | ||
11226 | :: | ||
11227 | |||
11228 | LICENSE_FLAGS = "license_${PN}_${PV}" | ||
11229 | |||
11230 | In order for a component restricted by a | ||
11231 | ``LICENSE_FLAGS`` definition to be enabled and included in an image, it | ||
11232 | needs to have a matching entry in the global | ||
11233 | :term:`LICENSE_FLAGS_WHITELIST` | ||
11234 | variable, which is a variable typically defined in your ``local.conf`` | ||
11235 | file. For example, to enable the | ||
11236 | ``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you | ||
11237 | could add either the string "commercial_gst-plugins-ugly" or the more | ||
11238 | general string "commercial" to ``LICENSE_FLAGS_WHITELIST``. See the | ||
11239 | "`License Flag Matching <#license-flag-matching>`__" section for a full | ||
11240 | explanation of how ``LICENSE_FLAGS`` matching works. Here is the | ||
11241 | example: | ||
11242 | :: | ||
11243 | |||
11244 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly" | ||
11245 | |||
11246 | Likewise, to additionally enable the package built from the recipe | ||
11247 | containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that | ||
11248 | the actual recipe name was ``emgd_1.10.bb``, the following string would | ||
11249 | enable that package as well as the original ``gst-plugins-ugly`` | ||
11250 | package: | ||
11251 | :: | ||
11252 | |||
11253 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10" | ||
11254 | |||
11255 | As a convenience, you do not need to specify the | ||
11256 | complete license string in the whitelist for every package. You can use | ||
11257 | an abbreviated form, which consists of just the first portion or | ||
11258 | portions of the license string before the initial underscore character | ||
11259 | or characters. A partial string will match any license that contains the | ||
11260 | given string as the first portion of its license. For example, the | ||
11261 | following whitelist string will also match both of the packages | ||
11262 | previously mentioned as well as any other packages that have licenses | ||
11263 | starting with "commercial" or "license". | ||
11264 | :: | ||
11265 | |||
11266 | LICENSE_FLAGS_WHITELIST = "commercial license" | ||
11267 | |||
11268 | License Flag Matching | ||
11269 | ~~~~~~~~~~~~~~~~~~~~~ | ||
11270 | |||
11271 | License flag matching allows you to control what recipes the | ||
11272 | OpenEmbedded build system includes in the build. Fundamentally, the | ||
11273 | build system attempts to match ``LICENSE_FLAGS`` strings found in | ||
11274 | recipes against ``LICENSE_FLAGS_WHITELIST`` strings found in the | ||
11275 | whitelist. A match causes the build system to include a recipe in the | ||
11276 | build, while failure to find a match causes the build system to exclude | ||
11277 | a recipe. | ||
11278 | |||
11279 | In general, license flag matching is simple. However, understanding some | ||
11280 | concepts will help you correctly and effectively use matching. | ||
11281 | |||
11282 | Before a flag defined by a particular recipe is tested against the | ||
11283 | contents of the whitelist, the expanded string ``_${PN}`` is appended to | ||
11284 | the flag. This expansion makes each ``LICENSE_FLAGS`` value | ||
11285 | recipe-specific. After expansion, the string is then matched against the | ||
11286 | whitelist. Thus, specifying ``LICENSE_FLAGS = "commercial"`` in recipe | ||
11287 | "foo", for example, results in the string ``"commercial_foo"``. And, to | ||
11288 | create a match, that string must appear in the whitelist. | ||
11289 | |||
11290 | Judicious use of the ``LICENSE_FLAGS`` strings and the contents of the | ||
11291 | ``LICENSE_FLAGS_WHITELIST`` variable allows you a lot of flexibility for | ||
11292 | including or excluding recipes based on licensing. For example, you can | ||
11293 | broaden the matching capabilities by using license flags string subsets | ||
11294 | in the whitelist. | ||
11295 | |||
11296 | .. note:: | ||
11297 | |||
11298 | When using a string subset, be sure to use the part of the expanded | ||
11299 | string that precedes the appended underscore character (e.g. | ||
11300 | usethispart_1.3 | ||
11301 | , | ||
11302 | usethispart_1.4 | ||
11303 | , and so forth). | ||
11304 | |||
11305 | For example, simply specifying the string "commercial" in the whitelist | ||
11306 | matches any expanded ``LICENSE_FLAGS`` definition that starts with the | ||
11307 | string "commercial" such as "commercial_foo" and "commercial_bar", which | ||
11308 | are the strings the build system automatically generates for | ||
11309 | hypothetical recipes named "foo" and "bar" assuming those recipes simply | ||
11310 | specify the following: | ||
11311 | :: | ||
11312 | |||
11313 | LICENSE_FLAGS = "commercial" | ||
11314 | |||
11315 | Thus, you can choose | ||
11316 | to exhaustively enumerate each license flag in the whitelist and allow | ||
11317 | only specific recipes into the image, or you can use a string subset | ||
11318 | that causes a broader range of matches to allow a range of recipes into | ||
11319 | the image. | ||
11320 | |||
11321 | This scheme works even if the ``LICENSE_FLAGS`` string already has | ||
11322 | ``_${PN}`` appended. For example, the build system turns the license | ||
11323 | flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match | ||
11324 | both the general "commercial" and the specific "commercial_1.2_foo" | ||
11325 | strings found in the whitelist, as expected. | ||
11326 | |||
11327 | Here are some other scenarios: | ||
11328 | |||
11329 | - You can specify a versioned string in the recipe such as | ||
11330 | "commercial_foo_1.2" in a "foo" recipe. The build system expands this | ||
11331 | string to "commercial_foo_1.2_foo". Combine this license flag with a | ||
11332 | whitelist that has the string "commercial" and you match the flag | ||
11333 | along with any other flag that starts with the string "commercial". | ||
11334 | |||
11335 | - Under the same circumstances, you can use "commercial_foo" in the | ||
11336 | whitelist and the build system not only matches "commercial_foo_1.2" | ||
11337 | but also matches any license flag with the string "commercial_foo", | ||
11338 | regardless of the version. | ||
11339 | |||
11340 | - You can be very specific and use both the package and version parts | ||
11341 | in the whitelist (e.g. "commercial_foo_1.2") to specifically match a | ||
11342 | versioned recipe. | ||
11343 | |||
11344 | Other Variables Related to Commercial Licenses | ||
11345 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11346 | |||
11347 | Other helpful variables related to commercial license handling exist and | ||
11348 | are defined in the | ||
11349 | ``poky/meta/conf/distro/include/default-distrovars.inc`` file: | ||
11350 | :: | ||
11351 | |||
11352 | COMMERCIAL_AUDIO_PLUGINS ?= "" | ||
11353 | COMMERCIAL_VIDEO_PLUGINS ?= "" | ||
11354 | |||
11355 | If you | ||
11356 | want to enable these components, you can do so by making sure you have | ||
11357 | statements similar to the following in your ``local.conf`` configuration | ||
11358 | file: | ||
11359 | :: | ||
11360 | |||
11361 | COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \ | ||
11362 | gst-plugins-ugly-mpegaudioparse" | ||
11363 | COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \ | ||
11364 | gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse" | ||
11365 | LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp" | ||
11366 | |||
11367 | |||
11368 | Of course, you could also create a matching whitelist for those | ||
11369 | components using the more general "commercial" in the whitelist, but | ||
11370 | that would also enable all the other packages with ``LICENSE_FLAGS`` | ||
11371 | containing "commercial", which you may or may not want: | ||
11372 | :: | ||
11373 | |||
11374 | LICENSE_FLAGS_WHITELIST = "commercial" | ||
11375 | |||
11376 | Specifying audio and video plugins as part of the | ||
11377 | ``COMMERCIAL_AUDIO_PLUGINS`` and ``COMMERCIAL_VIDEO_PLUGINS`` statements | ||
11378 | (along with the enabling ``LICENSE_FLAGS_WHITELIST``) includes the | ||
11379 | plugins or components into built images, thus adding support for media | ||
11380 | formats or components. | ||
11381 | |||
11382 | Maintaining Open Source License Compliance During Your Product's Lifecycle | ||
11383 | -------------------------------------------------------------------------- | ||
11384 | |||
11385 | One of the concerns for a development organization using open source | ||
11386 | software is how to maintain compliance with various open source | ||
11387 | licensing during the lifecycle of the product. While this section does | ||
11388 | not provide legal advice or comprehensively cover all scenarios, it does | ||
11389 | present methods that you can use to assist you in meeting the compliance | ||
11390 | requirements during a software release. | ||
11391 | |||
11392 | With hundreds of different open source licenses that the Yocto Project | ||
11393 | tracks, it is difficult to know the requirements of each and every | ||
11394 | license. However, the requirements of the major FLOSS licenses can begin | ||
11395 | to be covered by assuming that three main areas of concern exist: | ||
11396 | |||
11397 | - Source code must be provided. | ||
11398 | |||
11399 | - License text for the software must be provided. | ||
11400 | |||
11401 | - Compilation scripts and modifications to the source code must be | ||
11402 | provided. | ||
11403 | |||
11404 | There are other requirements beyond the scope of these three and the | ||
11405 | methods described in this section (e.g. the mechanism through which | ||
11406 | source code is distributed). | ||
11407 | |||
11408 | As different organizations have different methods of complying with open | ||
11409 | source licensing, this section is not meant to imply that there is only | ||
11410 | one single way to meet your compliance obligations, but rather to | ||
11411 | describe one method of achieving compliance. The remainder of this | ||
11412 | section describes methods supported to meet the previously mentioned | ||
11413 | three requirements. Once you take steps to meet these requirements, and | ||
11414 | prior to releasing images, sources, and the build system, you should | ||
11415 | audit all artifacts to ensure completeness. | ||
11416 | |||
11417 | .. note:: | ||
11418 | |||
11419 | The Yocto Project generates a license manifest during image creation | ||
11420 | that is located in | ||
11421 | ${DEPLOY_DIR}/licenses/ | ||
11422 | image_name-datestamp | ||
11423 | to assist with any audits. | ||
11424 | |||
11425 | Providing the Source Code | ||
11426 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11427 | |||
11428 | Compliance activities should begin before you generate the final image. | ||
11429 | The first thing you should look at is the requirement that tops the list | ||
11430 | for most compliance groups - providing the source. The Yocto Project has | ||
11431 | a few ways of meeting this requirement. | ||
11432 | |||
11433 | One of the easiest ways to meet this requirement is to provide the | ||
11434 | entire :term:`DL_DIR` used by the | ||
11435 | build. This method, however, has a few issues. The most obvious is the | ||
11436 | size of the directory since it includes all sources used in the build | ||
11437 | and not just the source used in the released image. It will include | ||
11438 | toolchain source, and other artifacts, which you would not generally | ||
11439 | release. However, the more serious issue for most companies is | ||
11440 | accidental release of proprietary software. The Yocto Project provides | ||
11441 | an :ref:`archiver <ref-classes-archiver>` class to | ||
11442 | help avoid some of these concerns. | ||
11443 | |||
11444 | Before you employ ``DL_DIR`` or the ``archiver`` class, you need to | ||
11445 | decide how you choose to provide source. The source ``archiver`` class | ||
11446 | can generate tarballs and SRPMs and can create them with various levels | ||
11447 | of compliance in mind. | ||
11448 | |||
11449 | One way of doing this (but certainly not the only way) is to release | ||
11450 | just the source as a tarball. You can do this by adding the following to | ||
11451 | the ``local.conf`` file found in the | ||
11452 | :term:`Build Directory`: | ||
11453 | :: | ||
11454 | |||
11455 | INHERIT += "archiver" | ||
11456 | ARCHIVER_MODE[src] = "original" | ||
11457 | |||
11458 | During the creation of your | ||
11459 | image, the source from all recipes that deploy packages to the image is | ||
11460 | placed within subdirectories of ``DEPLOY_DIR/sources`` based on the | ||
11461 | :term:`LICENSE` for each recipe. | ||
11462 | Releasing the entire directory enables you to comply with requirements | ||
11463 | concerning providing the unmodified source. It is important to note that | ||
11464 | the size of the directory can get large. | ||
11465 | |||
11466 | A way to help mitigate the size issue is to only release tarballs for | ||
11467 | licenses that require the release of source. Let us assume you are only | ||
11468 | concerned with GPL code as identified by running the following script: | ||
11469 | :: | ||
11470 | |||
11471 | # Script to archive a subset of packages matching specific license(s) | ||
11472 | # Source and license files are copied into sub folders of package folder | ||
11473 | # Must be run from build folder | ||
11474 | #!/bin/bash | ||
11475 | src_release_dir="source-release" | ||
11476 | mkdir -p $src_release_dir | ||
11477 | for a in tmp/deploy/sources/*; do | ||
11478 | for d in $a/*; do | ||
11479 | # Get package name from path | ||
11480 | p=`basename $d` | ||
11481 | p=${p%-*} | ||
11482 | p=${p%-*} | ||
11483 | # Only archive GPL packages (update *GPL* regex for your license check) | ||
11484 | numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l` | ||
11485 | if [ $numfiles -gt 1 ]; then | ||
11486 | echo Archiving $p | ||
11487 | mkdir -p $src_release_dir/$p/source | ||
11488 | cp $d/* $src_release_dir/$p/source 2> /dev/null | ||
11489 | mkdir -p $src_release_dir/$p/license | ||
11490 | cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null | ||
11491 | fi | ||
11492 | done | ||
11493 | done | ||
11494 | |||
11495 | At this point, you | ||
11496 | could create a tarball from the ``gpl_source_release`` directory and | ||
11497 | provide that to the end user. This method would be a step toward | ||
11498 | achieving compliance with section 3a of GPLv2 and with section 6 of | ||
11499 | GPLv3. | ||
11500 | |||
11501 | Providing License Text | ||
11502 | ~~~~~~~~~~~~~~~~~~~~~~ | ||
11503 | |||
11504 | One requirement that is often overlooked is inclusion of license text. | ||
11505 | This requirement also needs to be dealt with prior to generating the | ||
11506 | final image. Some licenses require the license text to accompany the | ||
11507 | binary. You can achieve this by adding the following to your | ||
11508 | ``local.conf`` file: | ||
11509 | :: | ||
11510 | |||
11511 | COPY_LIC_MANIFEST = "1" | ||
11512 | COPY_LIC_DIRS = "1" | ||
11513 | LICENSE_CREATE_PACKAGE = "1" | ||
11514 | |||
11515 | Adding these statements to the | ||
11516 | configuration file ensures that the licenses collected during package | ||
11517 | generation are included on your image. | ||
11518 | |||
11519 | .. note:: | ||
11520 | |||
11521 | Setting all three variables to "1" results in the image having two | ||
11522 | copies of the same license file. One copy resides in | ||
11523 | ``/usr/share/common-licenses`` and the other resides in | ||
11524 | ``/usr/share/license``. | ||
11525 | |||
11526 | The reason for this behavior is because | ||
11527 | :term:`COPY_LIC_DIRS` and | ||
11528 | :term:`COPY_LIC_MANIFEST` | ||
11529 | add a copy of the license when the image is built but do not offer a | ||
11530 | path for adding licenses for newly installed packages to an image. | ||
11531 | :term:`LICENSE_CREATE_PACKAGE` | ||
11532 | adds a separate package and an upgrade path for adding licenses to an | ||
11533 | image. | ||
11534 | |||
11535 | As the source ``archiver`` class has already archived the original | ||
11536 | unmodified source that contains the license files, you would have | ||
11537 | already met the requirements for inclusion of the license information | ||
11538 | with source as defined by the GPL and other open source licenses. | ||
11539 | |||
11540 | Providing Compilation Scripts and Source Code Modifications | ||
11541 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11542 | |||
11543 | At this point, we have addressed all we need to prior to generating the | ||
11544 | image. The next two requirements are addressed during the final | ||
11545 | packaging of the release. | ||
11546 | |||
11547 | By releasing the version of the OpenEmbedded build system and the layers | ||
11548 | used during the build, you will be providing both compilation scripts | ||
11549 | and the source code modifications in one step. | ||
11550 | |||
11551 | If the deployment team has a :ref:`overview-manual/overview-manual-concepts:bsp layer` | ||
11552 | and a distro layer, and those | ||
11553 | those layers are used to patch, compile, package, or modify (in any way) | ||
11554 | any open source software included in your released images, you might be | ||
11555 | required to release those layers under section 3 of GPLv2 or section 1 | ||
11556 | of GPLv3. One way of doing that is with a clean checkout of the version | ||
11557 | of the Yocto Project and layers used during your build. Here is an | ||
11558 | example: | ||
11559 | :: | ||
11560 | |||
11561 | # We built using the dunfell branch of the poky repo | ||
11562 | $ git clone -b dunfell git://git.yoctoproject.org/poky | ||
11563 | $ cd poky | ||
11564 | # We built using the release_branch for our layers | ||
11565 | $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer | ||
11566 | $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer | ||
11567 | # clean up the .git repos | ||
11568 | $ find . -name ".git" -type d -exec rm -rf {} \; | ||
11569 | |||
11570 | One | ||
11571 | thing a development organization might want to consider for end-user | ||
11572 | convenience is to modify ``meta-poky/conf/bblayers.conf.sample`` to | ||
11573 | ensure that when the end user utilizes the released build system to | ||
11574 | build an image, the development organization's layers are included in | ||
11575 | the ``bblayers.conf`` file automatically: | ||
11576 | :: | ||
11577 | |||
11578 | # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf | ||
11579 | # changes incompatibly | ||
11580 | POKY_BBLAYERS_CONF_VERSION = "2" | ||
11581 | |||
11582 | BBPATH = "${TOPDIR}" | ||
11583 | BBFILES ?= "" | ||
11584 | |||
11585 | BBLAYERS ?= " \ | ||
11586 | ##OEROOT##/meta \ | ||
11587 | ##OEROOT##/meta-poky \ | ||
11588 | ##OEROOT##/meta-yocto-bsp \ | ||
11589 | ##OEROOT##/meta-mylayer \ | ||
11590 | " | ||
11591 | |||
11592 | Creating and | ||
11593 | providing an archive of the :term:`Metadata` | ||
11594 | layers (recipes, configuration files, and so forth) enables you to meet | ||
11595 | your requirements to include the scripts to control compilation as well | ||
11596 | as any modifications to the original source. | ||
11597 | |||
11598 | Copying Licenses that Do Not Exist | ||
11599 | ---------------------------------- | ||
11600 | |||
11601 | Some packages, such as the linux-firmware package, have many licenses | ||
11602 | that are not in any way common. You can avoid adding a lot of these | ||
11603 | types of common license files, which are only applicable to a specific | ||
11604 | package, by using the | ||
11605 | :term:`NO_GENERIC_LICENSE` | ||
11606 | variable. Using this variable also avoids QA errors when you use a | ||
11607 | non-common, non-CLOSED license in a recipe. | ||
11608 | |||
11609 | The following is an example that uses the ``LICENSE.Abilis.txt`` file as | ||
11610 | the license from the fetched source: | ||
11611 | :: | ||
11612 | |||
11613 | NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt" | ||
11614 | |||
11615 | Using the Error Reporting Tool | ||
11616 | ============================== | ||
11617 | |||
11618 | The error reporting tool allows you to submit errors encountered during | ||
11619 | builds to a central database. Outside of the build environment, you can | ||
11620 | use a web interface to browse errors, view statistics, and query for | ||
11621 | errors. The tool works using a client-server system where the client | ||
11622 | portion is integrated with the installed Yocto Project | ||
11623 | :term:`Source Directory` (e.g. ``poky``). | ||
11624 | The server receives the information collected and saves it in a | ||
11625 | database. | ||
11626 | |||
11627 | A live instance of the error reporting server exists at | ||
11628 | http://errors.yoctoproject.org. This server exists so that when | ||
11629 | you want to get help with build failures, you can submit all of the | ||
11630 | information on the failure easily and then point to the URL in your bug | ||
11631 | report or send an email to the mailing list. | ||
11632 | |||
11633 | .. note:: | ||
11634 | |||
11635 | If you send error reports to this server, the reports become publicly | ||
11636 | visible. | ||
11637 | |||
11638 | Enabling and Using the Tool | ||
11639 | --------------------------- | ||
11640 | |||
11641 | By default, the error reporting tool is disabled. You can enable it by | ||
11642 | inheriting the | ||
11643 | :ref:`report-error <ref-classes-report-error>` | ||
11644 | class by adding the following statement to the end of your | ||
11645 | ``local.conf`` file in your | ||
11646 | :term:`Build Directory`. | ||
11647 | :: | ||
11648 | |||
11649 | INHERIT += "report-error" | ||
11650 | |||
11651 | By default, the error reporting feature stores information in | ||
11652 | ``${``\ :term:`LOG_DIR`\ ``}/error-report``. | ||
11653 | However, you can specify a directory to use by adding the following to | ||
11654 | your ``local.conf`` file: | ||
11655 | :: | ||
11656 | |||
11657 | ERR_REPORT_DIR = "path" | ||
11658 | |||
11659 | Enabling error | ||
11660 | reporting causes the build process to collect the errors and store them | ||
11661 | in a file as previously described. When the build system encounters an | ||
11662 | error, it includes a command as part of the console output. You can run | ||
11663 | the command to send the error file to the server. For example, the | ||
11664 | following command sends the errors to an upstream server: | ||
11665 | :: | ||
11666 | |||
11667 | $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt | ||
11668 | |||
11669 | In the previous example, the errors are sent to a public database | ||
11670 | available at http://errors.yoctoproject.org, which is used by the | ||
11671 | entire community. If you specify a particular server, you can send the | ||
11672 | errors to a different database. Use the following command for more | ||
11673 | information on available options: | ||
11674 | :: | ||
11675 | |||
11676 | $ send-error-report --help | ||
11677 | |||
11678 | When sending the error file, you are prompted to review the data being | ||
11679 | sent as well as to provide a name and optional email address. Once you | ||
11680 | satisfy these prompts, the command returns a link from the server that | ||
11681 | corresponds to your entry in the database. For example, here is a | ||
11682 | typical link: http://errors.yoctoproject.org/Errors/Details/9522/ | ||
11683 | |||
11684 | Following the link takes you to a web interface where you can browse, | ||
11685 | query the errors, and view statistics. | ||
11686 | |||
11687 | Disabling the Tool | ||
11688 | ------------------ | ||
11689 | |||
11690 | To disable the error reporting feature, simply remove or comment out the | ||
11691 | following statement from the end of your ``local.conf`` file in your | ||
11692 | :term:`Build Directory`. | ||
11693 | :: | ||
11694 | |||
11695 | INHERIT += "report-error" | ||
11696 | |||
11697 | Setting Up Your Own Error Reporting Server | ||
11698 | ------------------------------------------ | ||
11699 | |||
11700 | If you want to set up your own error reporting server, you can obtain | ||
11701 | the code from the Git repository at | ||
11702 | http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/. | ||
11703 | Instructions on how to set it up are in the README document. | ||
11704 | |||
11705 | .. _dev-using-wayland-and-weston: | ||
11706 | |||
11707 | Using Wayland and Weston | ||
11708 | ======================== | ||
11709 | |||
11710 | `Wayland <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)>`__ | ||
11711 | is a computer display server protocol that provides a method for | ||
11712 | compositing window managers to communicate directly with applications | ||
11713 | and video hardware and expects them to communicate with input hardware | ||
11714 | using other libraries. Using Wayland with supporting targets can result | ||
11715 | in better control over graphics frame rendering than an application | ||
11716 | might otherwise achieve. | ||
11717 | |||
11718 | The Yocto Project provides the Wayland protocol libraries and the | ||
11719 | reference | ||
11720 | `Weston <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston>`__ | ||
11721 | compositor as part of its release. You can find the integrated packages | ||
11722 | in the ``meta`` layer of the :term:`Source Directory`. | ||
11723 | Specifically, you | ||
11724 | can find the recipes that build both Wayland and Weston at | ||
11725 | ``meta/recipes-graphics/wayland``. | ||
11726 | |||
11727 | You can build both the Wayland and Weston packages for use only with | ||
11728 | targets that accept the `Mesa 3D and Direct Rendering | ||
11729 | Infrastructure <https://en.wikipedia.org/wiki/Mesa_(computer_graphics)>`__, | ||
11730 | which is also known as Mesa DRI. This implies that you cannot build and | ||
11731 | use the packages if your target uses, for example, the Intel Embedded | ||
11732 | Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI. | ||
11733 | |||
11734 | .. note:: | ||
11735 | |||
11736 | Due to lack of EGL support, Weston 1.0.3 will not run directly on the | ||
11737 | emulated QEMU hardware. However, this version of Weston will run | ||
11738 | under X emulation without issues. | ||
11739 | |||
11740 | This section describes what you need to do to implement Wayland and use | ||
11741 | the Weston compositor when building an image for a supporting target. | ||
11742 | |||
11743 | Enabling Wayland in an Image | ||
11744 | ---------------------------- | ||
11745 | |||
11746 | To enable Wayland, you need to enable it to be built and enable it to be | ||
11747 | included (installed) in the image. | ||
11748 | |||
11749 | .. _enable-building: | ||
11750 | |||
11751 | Building Wayland | ||
11752 | ~~~~~~~~~~~~~~~~ | ||
11753 | |||
11754 | To cause Mesa to build the ``wayland-egl`` platform and Weston to build | ||
11755 | Wayland with Kernel Mode Setting | ||
11756 | (`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__) | ||
11757 | support, include the "wayland" flag in the | ||
11758 | :term:`DISTRO_FEATURES` | ||
11759 | statement in your ``local.conf`` file: | ||
11760 | :: | ||
11761 | |||
11762 | DISTRO_FEATURES_append = " wayland" | ||
11763 | |||
11764 | .. note:: | ||
11765 | |||
11766 | If X11 has been enabled elsewhere, Weston will build Wayland with X11 | ||
11767 | support | ||
11768 | |||
11769 | .. _enable-installation-in-an-image: | ||
11770 | |||
11771 | Installing Wayland and Weston | ||
11772 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11773 | |||
11774 | To install the Wayland feature into an image, you must include the | ||
11775 | following | ||
11776 | :term:`CORE_IMAGE_EXTRA_INSTALL` | ||
11777 | statement in your ``local.conf`` file: | ||
11778 | :: | ||
11779 | |||
11780 | CORE_IMAGE_EXTRA_INSTALL += "wayland weston" | ||
11781 | |||
11782 | Running Weston | ||
11783 | -------------- | ||
11784 | |||
11785 | To run Weston inside X11, enabling it as described earlier and building | ||
11786 | a Sato image is sufficient. If you are running your image under Sato, a | ||
11787 | Weston Launcher appears in the "Utility" category. | ||
11788 | |||
11789 | Alternatively, you can run Weston through the command-line interpretor | ||
11790 | (CLI), which is better suited for development work. To run Weston under | ||
11791 | the CLI, you need to do the following after your image is built: | ||
11792 | |||
11793 | 1. Run these commands to export ``XDG_RUNTIME_DIR``: | ||
11794 | :: | ||
11795 | |||
11796 | mkdir -p /tmp/$USER-weston | ||
11797 | chmod 0700 /tmp/$USER-weston | ||
11798 | export XDG_RUNTIME_DIR=/tmp/$USER-weston | ||
11799 | |||
11800 | 2. Launch Weston in the shell: | ||
11801 | :: | ||
11802 | |||
11803 | weston | ||