summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual/dev-manual-common-tasks.rst
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.rst')
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.rst10227
1 files changed, 10227 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.rst b/documentation/dev-manual/dev-manual-common-tasks.rst
new file mode 100644
index 0000000000..b36c97a6a3
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-common-tasks.rst
@@ -0,0 +1,10227 @@
1************
2Common Tasks
3************
4
5This chapter describes fundamental procedures such as creating layers,
6adding new software packages, extending or customizing images, porting
7work to new hardware (adding a new machine), and so forth. You will find
8that the procedures documented here occur often in the development cycle
9using the Yocto Project.
10
11Understanding and Creating Layers
12=================================
13
14The OpenEmbedded build system supports organizing
15`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ into multiple layers.
16Layers allow you to isolate different types of customizations from each
17other. For introductory information on the Yocto Project Layer Model,
18see the "`The Yocto Project Layer
19Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__" section in
20the Yocto Project Overview and Concepts Manual.
21
22Creating Your Own Layer
23-----------------------
24
25It is very easy to create your own layers to use with the OpenEmbedded
26build system. The Yocto Project ships with tools that speed up creating
27layers. This section describes the steps you perform by hand to create
28layers so that you can better understand them. For information about the
29layer-creation tools, see the "`Creating a New BSP Layer Using the
30``bitbake-layers``
31Script <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
32section in the Yocto Project Board Support Package (BSP) Developer's
33Guide and the "`Creating a General Layer Using the ``bitbake-layers``
34Script <#creating-a-general-layer-using-the-bitbake-layers-script>`__"
35section further down in this manual.
36
37Follow these general steps to create your layer without using tools:
38
391. *Check Existing Layers:* Before creating a new layer, you should be
40 sure someone has not already created a layer containing the Metadata
41 you need. You can see the `OpenEmbedded Metadata
42 Index <http://layers.openembedded.org/layerindex/layers/>`__ for a
43 list of layers from the OpenEmbedded community that can be used in
44 the Yocto Project. You could find a layer that is identical or close
45 to what you need.
46
472. *Create a Directory:* Create the directory for your layer. When you
48 create the layer, be sure to create the directory in an area not
49 associated with the Yocto Project `Source
50 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. the cloned
51 ``poky`` repository).
52
53 While not strictly required, prepend the name of the directory with
54 the string "meta-". For example: meta-mylayer meta-GUI_xyz
55 meta-mymachine With rare exceptions, a layer's name follows this
56 form: meta-root_name Following this layer naming convention can save
57 you trouble later when tools, components, or variables "assume" your
58 layer name begins with "meta-". A notable example is in configuration
59 files as shown in the following step where layer names without the
60 "meta-" string are appended to several variables used in the
61 configuration.
62
633. *Create a Layer Configuration File:* Inside your new layer folder,
64 you need to create a ``conf/layer.conf`` file. It is easiest to take
65 an existing layer configuration file and copy that to your layer's
66 ``conf`` directory and then modify the file as needed.
67
68 The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project
69 `Source
70 Repositories <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf>`__
71 demonstrates the required syntax. For your layer, you need to replace
72 "yoctobsp" with a unique identifier for your layer (e.g. "machinexyz"
73 for a layer named "meta-machinexyz"): # We have a conf and classes
74 directory, add to BBPATH BBPATH .= ":${LAYERDIR}" # We have
75 recipes-\* directories, add to BBFILES BBFILES +=
76 "${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
77 BBFILE_COLLECTIONS += "yoctobsp" BBFILE_PATTERN_yoctobsp =
78 "^${LAYERDIR}/" BBFILE_PRIORITY_yoctobsp = "5" LAYERVERSION_yoctobsp
79 = "4" LAYERSERIES_COMPAT_yoctobsp = "DISTRO_NAME_NO_CAP" Following is
80 an explanation of the layer configuration file:
81
82 - ```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__: Adds the layer's
83 root directory to BitBake's search path. Through the use of the
84 ``BBPATH`` variable, BitBake locates class files (``.bbclass``),
85 configuration files, and files that are included with ``include``
86 and ``require`` statements. For these cases, BitBake uses the
87 first file that matches the name found in ``BBPATH``. This is
88 similar to the way the ``PATH`` variable is used for binaries. It
89 is recommended, therefore, that you use unique class and
90 configuration filenames in your custom layer.
91
92 - ```BBFILES`` <&YOCTO_DOCS_REF_URL;#var-BBFILES>`__: Defines the
93 location for all recipes in the layer.
94
95 - ```BBFILE_COLLECTIONS`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS>`__:
96 Establishes the current layer through a unique identifier that is
97 used throughout the OpenEmbedded build system to refer to the
98 layer. In this example, the identifier "yoctobsp" is the
99 representation for the container layer named "meta-yocto-bsp".
100
101 - ```BBFILE_PATTERN`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN>`__:
102 Expands immediately during parsing to provide the directory of the
103 layer.
104
105 - ```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__:
106 Establishes a priority to use for recipes in the layer when the
107 OpenEmbedded build finds recipes of the same name in different
108 layers.
109
110 - ```LAYERVERSION`` <&YOCTO_DOCS_REF_URL;#var-LAYERVERSION>`__:
111 Establishes a version number for the layer. You can use this
112 version number to specify this exact version of the layer as a
113 dependency when using the
114 ```LAYERDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS>`__
115 variable.
116
117 - ```LAYERDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS>`__:
118 Lists all layers on which this layer depends (if any).
119
120 - ```LAYERSERIES_COMPAT`` <&YOCTO_DOCS_REF_URL;#var-LAYERSERIES_COMPAT>`__:
121 Lists the `Yocto Project <&YOCTO_WIKI_URL;/wiki/Releases>`__
122 releases for which the current version is compatible. This
123 variable is a good way to indicate if your particular layer is
124 current.
125
1264. *Add Content:* Depending on the type of layer, add the content. If
127 the layer adds support for a machine, add the machine configuration
128 in a ``conf/machine/`` file within the layer. If the layer adds
129 distro policy, add the distro configuration in a ``conf/distro/``
130 file within the layer. If the layer introduces new recipes, put the
131 recipes you need in ``recipes-*`` subdirectories within the layer.
132
133 .. note::
134
135 For an explanation of layer hierarchy that is compliant with the
136 Yocto Project, see the "
137 Example Filesystem Layout
138 " section in the Yocto Project Board Support Package (BSP)
139 Developer's Guide.
140
1415. *Optionally Test for Compatibility:* If you want permission to use
142 the Yocto Project Compatibility logo with your layer or application
143 that uses your layer, perform the steps to apply for compatibility.
144 See the "`Making Sure Your Layer is Compatible With Yocto
145 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
146 section for more information.
147
148.. _best-practices-to-follow-when-creating-layers:
149
150Following Best Practices When Creating Layers
151---------------------------------------------
152
153To create layers that are easier to maintain and that will not impact
154builds for other machines, you should consider the information in the
155following list:
156
157- *Avoid "Overlaying" Entire Recipes from Other Layers in Your
158 Configuration:* In other words, do not copy an entire recipe into
159 your layer and then modify it. Rather, use an append file
160 (``.bbappend``) to override only those parts of the original recipe
161 you need to modify.
162
163- *Avoid Duplicating Include Files:* Use append files (``.bbappend``)
164 for each recipe that uses an include file. Or, if you are introducing
165 a new recipe that requires the included file, use the path relative
166 to the original layer directory to refer to the file. For example,
167 use ``require recipes-core/``\ package\ ``/``\ file\ ``.inc`` instead
168 of ``require``\ file\ ``.inc``. If you're finding you have to overlay
169 the include file, it could indicate a deficiency in the include file
170 in the layer to which it originally belongs. If this is the case, you
171 should try to address that deficiency instead of overlaying the
172 include file. For example, you could address this by getting the
173 maintainer of the include file to add a variable or variables to make
174 it easy to override the parts needing to be overridden.
175
176- *Structure Your Layers:* Proper use of overrides within append files
177 and placement of machine-specific files within your layer can ensure
178 that a build is not using the wrong Metadata and negatively impacting
179 a build for a different machine. Following are some examples:
180
181 - *Modify Variables to Support a Different Machine:* Suppose you
182 have a layer named ``meta-one`` that adds support for building
183 machine "one". To do so, you use an append file named
184 ``base-files.bbappend`` and create a dependency on "foo" by
185 altering the ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__
186 variable: DEPENDS = "foo" The dependency is created during any
187 build that includes the layer ``meta-one``. However, you might not
188 want this dependency for all machines. For example, suppose you
189 are building for machine "two" but your ``bblayers.conf`` file has
190 the ``meta-one`` layer included. During the build, the
191 ``base-files`` for machine "two" will also have the dependency on
192 ``foo``.
193
194 To make sure your changes apply only when building machine "one",
195 use a machine override with the ``DEPENDS`` statement: DEPENDS_one
196 = "foo" You should follow the same strategy when using ``_append``
197 and ``_prepend`` operations: DEPENDS_append_one = " foo"
198 DEPENDS_prepend_one = "foo " As an actual example, here's a
199 snippet from the generic kernel include file ``linux-yocto.inc``,
200 wherein the kernel compile and link options are adjusted in the
201 case of a subset of the supported architectures:
202 DEPENDS_append_aarch64 = " libgcc" KERNEL_CC_append_aarch64 = "
203 ${TOOLCHAIN_OPTIONS}" KERNEL_LD_append_aarch64 = "
204 ${TOOLCHAIN_OPTIONS}" DEPENDS_append_nios2 = " libgcc"
205 KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
206 KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
207 DEPENDS_append_arc = " libgcc" KERNEL_CC_append_arc = "
208 ${TOOLCHAIN_OPTIONS}" KERNEL_LD_append_arc = "
209 ${TOOLCHAIN_OPTIONS}" KERNEL_FEATURES_append_qemuall="
210 features/debug/printk.scc"
211
212 .. note::
213
214 Avoiding "+=" and "=+" and using machine-specific
215 \_append
216 and
217 \_prepend
218 operations is recommended as well.
219
220 - *Place Machine-Specific Files in Machine-Specific Locations:* When
221 you have a base recipe, such as ``base-files.bb``, that contains a
222 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement to a
223 file, you can use an append file to cause the build to use your
224 own version of the file. For example, an append file in your layer
225 at ``meta-one/recipes-core/base-files/base-files.bbappend`` could
226 extend ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
227 using
228 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
229 as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" The
230 build for machine "one" will pick up your machine-specific file as
231 long as you have the file in
232 ``meta-one/recipes-core/base-files/base-files/``. However, if you
233 are building for a different machine and the ``bblayers.conf``
234 file includes the ``meta-one`` layer and the location of your
235 machine-specific file is the first location where that file is
236 found according to ``FILESPATH``, builds for all machines will
237 also use that machine-specific file.
238
239 You can make sure that a machine-specific file is used for a
240 particular machine by putting the file in a subdirectory specific
241 to the machine. For example, rather than placing the file in
242 ``meta-one/recipes-core/base-files/base-files/`` as shown above,
243 put it in ``meta-one/recipes-core/base-files/base-files/one/``.
244 Not only does this make sure the file is used only when building
245 for machine "one", but the build process locates the file more
246 quickly.
247
248 In summary, you need to place all files referenced from
249 ``SRC_URI`` in a machine-specific subdirectory within the layer in
250 order to restrict those files to machine-specific builds.
251
252- *Perform Steps to Apply for Yocto Project Compatibility:* If you want
253 permission to use the Yocto Project Compatibility logo with your
254 layer or application that uses your layer, perform the steps to apply
255 for compatibility. See the "`Making Sure Your Layer is Compatible
256 With Yocto
257 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
258 section for more information.
259
260- *Follow the Layer Naming Convention:* Store custom layers in a Git
261 repository that use the ``meta-layer_name`` format.
262
263- *Group Your Layers Locally:* Clone your repository alongside other
264 cloned ``meta`` directories from the `Source
265 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
266
267Making Sure Your Layer is Compatible With Yocto Project
268-------------------------------------------------------
269
270When you create a layer used with the Yocto Project, it is advantageous
271to make sure that the layer interacts well with existing Yocto Project
272layers (i.e. the layer is compatible with the Yocto Project). Ensuring
273compatibility makes the layer easy to be consumed by others in the Yocto
274Project community and could allow you permission to use the Yocto
275Project Compatible Logo.
276
277.. note::
278
279 Only Yocto Project member organizations are permitted to use the
280 Yocto Project Compatible Logo. The logo is not available for general
281 use. For information on how to become a Yocto Project member
282 organization, see the
283 Yocto Project Website
284 .
285
286The Yocto Project Compatibility Program consists of a layer application
287process that requests permission to use the Yocto Project Compatibility
288Logo for your layer and application. The process consists of two parts:
289
2901. Successfully passing a script (``yocto-check-layer``) that when run
291 against your layer, tests it against constraints based on experiences
292 of how layers have worked in the real world and where pitfalls have
293 been found. Getting a "PASS" result from the script is required for
294 successful compatibility registration.
295
2962. Completion of an application acceptance form, which you can find at
297 ` <https://www.yoctoproject.org/webform/yocto-project-compatible-registration>`__.
298
299To be granted permission to use the logo, you need to satisfy the
300following:
301
302- Be able to check the box indicating that you got a "PASS" when
303 running the script against your layer.
304
305- Answer "Yes" to the questions on the form or have an acceptable
306 explanation for any questions answered "No".
307
308- Be a Yocto Project Member Organization.
309
310The remainder of this section presents information on the registration
311form and on the ``yocto-check-layer`` script.
312
313Yocto Project Compatible Program Application
314~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
315
316Use the form to apply for your layer's approval. Upon successful
317application, you can use the Yocto Project Compatibility Logo with your
318layer and the application that uses your layer.
319
320To access the form, use this link:
321` <https://www.yoctoproject.org/webform/yocto-project-compatible-registration>`__.
322Follow the instructions on the form to complete your application.
323
324The application consists of the following sections:
325
326- *Contact Information:* Provide your contact information as the fields
327 require. Along with your information, provide the released versions
328 of the Yocto Project for which your layer is compatible.
329
330- *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the
331 items in the checklist. Space exists at the bottom of the form for
332 any explanations for items for which you answered "No".
333
334- *Recommendations:* Provide answers for the questions regarding Linux
335 kernel use and build success.
336
337``yocto-check-layer`` Script
338~~~~~~~~~~~~~~~~~~~~~~~~~~~~
339
340The ``yocto-check-layer`` script provides you a way to assess how
341compatible your layer is with the Yocto Project. You should run this
342script prior to using the form to apply for compatibility as described
343in the previous section. You need to achieve a "PASS" result in order to
344have your application form successfully processed.
345
346The script divides tests into three areas: COMMON, BSP, and DISTRO. For
347example, given a distribution layer (DISTRO), the layer must pass both
348the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP
349layer, the layer must pass the COMMON and BSP set of tests.
350
351To execute the script, enter the following commands from your build
352directory: $ source oe-init-build-env $ yocto-check-layer
353your_layer_directory Be sure to provide the actual directory for your
354layer as part of the command.
355
356Entering the command causes the script to determine the type of layer
357and then to execute a set of specific tests against the layer. The
358following list overviews the test:
359
360- ``common.test_readme``: Tests if a ``README`` file exists in the
361 layer and the file is not empty.
362
363- ``common.test_parse``: Tests to make sure that BitBake can parse the
364 files without error (i.e. ``bitbake -p``).
365
366- ``common.test_show_environment``: Tests that the global or per-recipe
367 environment is in order without errors (i.e. ``bitbake -e``).
368
369- ``common.test_world``: Verifies that ``bitbake world`` works.
370
371- ``common.test_signatures``: Tests to be sure that BSP and DISTRO
372 layers do not come with recipes that change signatures.
373
374- ``common.test_layerseries_compat``: Verifies layer compatibility is
375 set properly.
376
377- ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine
378 configurations.
379
380- ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not
381 set the machine when the layer is added.
382
383- ``bsp.test_machine_world``: Verifies that ``bitbake world`` works
384 regardless of which machine is selected.
385
386- ``bsp.test_machine_signatures``: Verifies that building for a
387 particular machine affects only the signature of tasks specific to
388 that machine.
389
390- ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has
391 distro configurations.
392
393- ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer
394 does not set the distribution when the layer is added.
395
396Enabling Your Layer
397-------------------
398
399Before the OpenEmbedded build system can use your new layer, you need to
400enable it. To enable your layer, simply add your layer's path to the
401``BBLAYERS`` variable in your ``conf/bblayers.conf`` file, which is
402found in the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
403The following example shows how to enable a layer named
404``meta-mylayer``: # POKY_BBLAYERS_CONF_VERSION is increased each time
405build/conf/bblayers.conf # changes incompatibly
406POKY_BBLAYERS_CONF_VERSION = "2" BBPATH = "${TOPDIR}" BBFILES ?= ""
407BBLAYERS ?= " \\ /home/user/poky/meta \\ /home/user/poky/meta-poky \\
408/home/user/poky/meta-yocto-bsp \\ /home/user/poky/meta-mylayer \\ "
409
410BitBake parses each ``conf/layer.conf`` file from the top down as
411specified in the ``BBLAYERS`` variable within the ``conf/bblayers.conf``
412file. During the processing of each ``conf/layer.conf`` file, BitBake
413adds the recipes, classes and configurations contained within the
414particular layer to the source directory.
415
416.. _using-bbappend-files:
417
418Using .bbappend Files in Your Layer
419-----------------------------------
420
421A recipe that appends Metadata to another recipe is called a BitBake
422append file. A BitBake append file uses the ``.bbappend`` file type
423suffix, while the corresponding recipe to which Metadata is being
424appended uses the ``.bb`` file type suffix.
425
426You can use a ``.bbappend`` file in your layer to make additions or
427changes to the content of another layer's recipe without having to copy
428the other layer's recipe into your layer. Your ``.bbappend`` file
429resides in your layer, while the main ``.bb`` recipe file to which you
430are appending Metadata resides in a different layer.
431
432Being able to append information to an existing recipe not only avoids
433duplication, but also automatically applies recipe changes from a
434different layer into your layer. If you were copying recipes, you would
435have to manually merge changes as they occur.
436
437When you create an append file, you must use the same root name as the
438corresponding recipe file. For example, the append file
439``someapp_DISTRO.bbappend`` must apply to ``someapp_DISTRO.bb``. This
440means the original recipe and append file names are version
441number-specific. If the corresponding recipe is renamed to update to a
442newer version, you must also rename and possibly update the
443corresponding ``.bbappend`` as well. During the build process, BitBake
444displays an error on starting if it detects a ``.bbappend`` file that
445does not have a corresponding recipe with a matching name. See the
446```BB_DANGLINGAPPENDS_WARNONLY`` <&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY>`__
447variable for information on how to handle this error.
448
449As an example, consider the main formfactor recipe and a corresponding
450formfactor append file both from the `Source
451Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Here is the main
452formfactor recipe, which is named ``formfactor_0.0.bb`` and located in
453the "meta" layer at ``meta/recipes-bsp/formfactor``: SUMMARY = "Device
454formfactor information" SECTION = "base" LICENSE = "MIT"
455LIC_FILES_CHKSUM =
456"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
457PR = "r45" SRC_URI = "file://config file://machconfig" S = "${WORKDIR}"
458PACKAGE_ARCH = "${MACHINE_ARCH}" INHIBIT_DEFAULT_DEPS = "1" do_install()
459{ # Install file only if it has contents install -d
460${D}${sysconfdir}/formfactor/ install -m 0644 ${S}/config
461${D}${sysconfdir}/formfactor/ if [ -s "${S}/machconfig" ]; then install
462-m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ fi } In the main
463recipe, note the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
464variable, which tells the OpenEmbedded build system where to find files
465during the build.
466
467Following is the append file, which is named ``formfactor_0.0.bbappend``
468and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The
469file is in the layer at ``recipes-bsp/formfactor``:
470FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
471
472By default, the build system uses the
473```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ variable to
474locate files. This append file extends the locations by setting the
475```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
476variable. Setting this variable in the ``.bbappend`` file is the most
477reliable and recommended method for adding directories to the search
478path used by the build system to find files.
479
480The statement in this example extends the directories to include
481``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``,
482which resolves to a directory named ``formfactor`` in the same directory
483in which the append file resides (i.e.
484``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must
485have the supporting directory structure set up that will contain any
486files or patches you will be including from the layer.
487
488Using the immediate expansion assignment operator ``:=`` is important
489because of the reference to ``THISDIR``. The trailing colon character is
490important as it ensures that items in the list remain colon-separated.
491
492.. note::
493
494 BitBake automatically defines the ``THISDIR`` variable. You should
495 never set this variable yourself. Using "_prepend" as part of the
496 ``FILESEXTRAPATHS`` ensures your path will be searched prior to other
497 paths in the final list.
498
499 Also, not all append files add extra files. Many append files simply
500 exist to add build options (e.g. ``systemd``). For these cases, your
501 append file would not even use the ``FILESEXTRAPATHS`` statement.
502
503Prioritizing Your Layer
504-----------------------
505
506Each layer is assigned a priority value. Priority values control which
507layer takes precedence if there are recipe files with the same name in
508multiple layers. For these cases, the recipe file from the layer with a
509higher priority number takes precedence. Priority values also affect the
510order in which multiple ``.bbappend`` files for the same recipe are
511applied. You can either specify the priority manually, or allow the
512build system to calculate it based on the layer's dependencies.
513
514To specify the layer's priority manually, use the
515```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__
516variable and append the layer's root name: BBFILE_PRIORITY_mylayer = "1"
517
518.. note::
519
520 It is possible for a recipe with a lower version number
521 ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ in a layer that has a higher
522 priority to take precedence.
523
524 Also, the layer priority does not currently affect the precedence
525 order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake
526 might address this.
527
528Managing Layers
529---------------
530
531You can use the BitBake layer management tool ``bitbake-layers`` to
532provide a view into the structure of recipes across a multi-layer
533project. Being able to generate output that reports on configured layers
534with their paths and priorities and on ``.bbappend`` files and their
535applicable recipes can help to reveal potential problems.
536
537For help on the BitBake layer management tool, use the following
538command: $ bitbake-layers --help NOTE: Starting bitbake server... usage:
539bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ...
540BitBake layers utility optional arguments: -d, --debug Enable debug
541output -q, --quiet Print only errors -F, --force Force add without
542recipe parse verification --color COLOR Colorize output (where COLOR is
543auto, always, never) -h, --help show this help message and exit
544subcommands: <subcommand> show-layers show current configured layers.
545show-overlayed list overlayed recipes (where the same recipe exists in
546another layer) show-recipes list available recipes, showing the layer
547they are provided by show-appends list bbappend files and recipe files
548they apply to show-cross-depends Show dependencies between recipes that
549cross layer boundaries. add-layer Add one or more layers to
550bblayers.conf. remove-layer Remove one or more layers from
551bblayers.conf. flatten flatten layer configuration into a separate
552output directory. layerindex-fetch Fetches a layer from a layer index
553along with its dependent layers, and adds them to conf/bblayers.conf.
554layerindex-show-depends Find layer dependencies from layer index.
555create-layer Create a basic layer Use bitbake-layers <subcommand> --help
556to get help on a specific command
557
558The following list describes the available commands:
559
560- *``help:``* Displays general help or help on a specified command.
561
562- *``show-layers:``* Shows the current configured layers.
563
564- *``show-overlayed:``* Lists overlayed recipes. A recipe is overlayed
565 when a recipe with the same name exists in another layer that has a
566 higher layer priority.
567
568- *``show-recipes:``* Lists available recipes and the layers that
569 provide them.
570
571- *``show-appends:``* Lists ``.bbappend`` files and the recipe files to
572 which they apply.
573
574- *``show-cross-depends:``* Lists dependency relationships between
575 recipes that cross layer boundaries.
576
577- *``add-layer:``* Adds a layer to ``bblayers.conf``.
578
579- *``remove-layer:``* Removes a layer from ``bblayers.conf``
580
581- *``flatten:``* Flattens the layer configuration into a separate
582 output directory. Flattening your layer configuration builds a
583 "flattened" directory that contains the contents of all layers, with
584 any overlayed recipes removed and any ``.bbappend`` files appended to
585 the corresponding recipes. You might have to perform some manual
586 cleanup of the flattened layer as follows:
587
588 - Non-recipe files (such as patches) are overwritten. The flatten
589 command shows a warning for these files.
590
591 - Anything beyond the normal layer setup has been added to the
592 ``layer.conf`` file. Only the lowest priority layer's
593 ``layer.conf`` is used.
594
595 - Overridden and appended items from ``.bbappend`` files need to be
596 cleaned up. The contents of each ``.bbappend`` end up in the
597 flattened recipe. However, if there are appended or changed
598 variable values, you need to tidy these up yourself. Consider the
599 following example. Here, the ``bitbake-layers`` command adds the
600 line ``#### bbappended ...`` so that you know where the following
601 lines originate: ... DESCRIPTION = "A useful utility" ...
602 EXTRA_OECONF = "--enable-something" ... #### bbappended from
603 meta-anotherlayer #### DESCRIPTION = "Customized utility"
604 EXTRA_OECONF += "--enable-somethingelse" Ideally, you would tidy
605 up these utilities as follows: ... DESCRIPTION = "Customized
606 utility" ... EXTRA_OECONF = "--enable-something
607 --enable-somethingelse" ...
608
609- *``layerindex-fetch``:* Fetches a layer from a layer index, along
610 with its dependent layers, and adds the layers to the
611 ``conf/bblayers.conf`` file.
612
613- *``layerindex-show-depends``:* Finds layer dependencies from the
614 layer index.
615
616- *``create-layer``:* Creates a basic layer.
617
618Creating a General Layer Using the ``bitbake-layers`` Script
619------------------------------------------------------------
620
621The ``bitbake-layers`` script with the ``create-layer`` subcommand
622simplifies creating a new general layer.
623
624.. note::
625
626 - For information on BSP layers, see the "`BSP
627 Layers <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__" section in the Yocto
628 Project Board Specific (BSP) Developer's Guide.
629
630 - In order to use a layer with the OpenEmbedded build system, you
631 need to add the layer to your ``bblayers.conf`` configuration
632 file. See the "`Adding a Layer Using the ``bitbake-layers``
633 Script <#adding-a-layer-using-the-bitbake-layers-script>`__"
634 section for more information.
635
636The default mode of the script's operation with this subcommand is to
637create a layer with the following:
638
639- A layer priority of 6.
640
641- A ``conf`` subdirectory that contains a ``layer.conf`` file.
642
643- A ``recipes-example`` subdirectory that contains a further
644 subdirectory named ``example``, which contains an ``example.bb``
645 recipe file.
646
647- A ``COPYING.MIT``, which is the license statement for the layer. The
648 script assumes you want to use the MIT license, which is typical for
649 most layers, for the contents of the layer itself.
650
651- A ``README`` file, which is a file describing the contents of your
652 new layer.
653
654In its simplest form, you can use the following command form to create a
655layer. The command creates a layer whose name corresponds to
656your_layer_name in the current directory: $ bitbake-layers create-layer
657your_layer_name As an example, the following command creates a layer
658named ``meta-scottrif`` in your home directory: $ cd /usr/home $
659bitbake-layers create-layer meta-scottrif NOTE: Starting bitbake
660server... Add your new layer with 'bitbake-layers add-layer
661meta-scottrif'
662
663If you want to set the priority of the layer to other than the default
664value of "6", you can either use the ``DASHDASHpriority`` option or you
665can edit the
666```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__ value
667in the ``conf/layer.conf`` after the script creates it. Furthermore, if
668you want to give the example recipe file some name other than the
669default, you can use the ``DASHDASHexample-recipe-name`` option.
670
671The easiest way to see how the ``bitbake-layers create-layer`` command
672works is to experiment with the script. You can also read the usage
673information by entering the following: $ bitbake-layers create-layer
674--help NOTE: Starting bitbake server... usage: bitbake-layers
675create-layer [-h] [--priority PRIORITY] [--example-recipe-name
676EXAMPLERECIPE] layerdir Create a basic layer positional arguments:
677layerdir Layer directory to create optional arguments: -h, --help show
678this help message and exit --priority PRIORITY, -p PRIORITY Layer
679directory to create --example-recipe-name EXAMPLERECIPE, -e
680EXAMPLERECIPE Filename of the example recipe
681
682Adding a Layer Using the ``bitbake-layers`` Script
683--------------------------------------------------
684
685Once you create your general layer, you must add it to your
686``bblayers.conf`` file. Adding the layer to this configuration file
687makes the OpenEmbedded build system aware of your layer so that it can
688search it for metadata.
689
690Add your layer by using the ``bitbake-layers add-layer`` command: $
691bitbake-layers add-layer your_layer_name Here is an example that adds a
692layer named ``meta-scottrif`` to the configuration file. Following the
693command that adds the layer is another ``bitbake-layers`` command that
694shows the layers that are in your ``bblayers.conf`` file: $
695bitbake-layers add-layer meta-scottrif NOTE: Starting bitbake server...
696Parsing recipes: 100%
697\|##########################################################\| Time:
6980:00:49 Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055
699targets, 56 skipped, 0 masked, 0 errors. $ bitbake-layers show-layers
700NOTE: Starting bitbake server... layer path priority
701==========================================================================
702meta /home/scottrif/poky/meta 5 meta-poky /home/scottrif/poky/meta-poky
7035 meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5 workspace
704/home/scottrif/poky/build/workspace 99 meta-scottrif
705/home/scottrif/poky/build/meta-scottrif 6 Adding the layer to this file
706enables the build system to locate the layer during the build.
707
708.. note::
709
710 During a build, the OpenEmbedded build system looks in the layers
711 from the top of the list down to the bottom in that order.
712
713.. _usingpoky-extend-customimage:
714
715Customizing Images
716==================
717
718You can customize images to satisfy particular requirements. This
719section describes several methods and provides guidelines for each.
720
721.. _usingpoky-extend-customimage-localconf:
722
723Customizing Images Using ``local.conf``
724---------------------------------------
725
726Probably the easiest way to customize an image is to add a package by
727way of the ``local.conf`` configuration file. Because it is limited to
728local use, this method generally only allows you to add packages and is
729not as flexible as creating your own customized image. When you add
730packages using local variables this way, you need to realize that these
731variable changes are in effect for every build and consequently affect
732all images, which might not be what you require.
733
734To add a package to your image using the local configuration file, use
735the ``IMAGE_INSTALL`` variable with the ``_append`` operator:
736IMAGE_INSTALL_append = " strace" Use of the syntax is important -
737specifically, the space between the quote and the package name, which is
738``strace`` in this example. This space is required since the ``_append``
739operator does not add the space.
740
741Furthermore, you must use ``_append`` instead of the ``+=`` operator if
742you want to avoid ordering issues. The reason for this is because doing
743so unconditionally appends to the variable and avoids ordering problems
744due to the variable being set in image recipes and ``.bbclass`` files
745with operators like ``?=``. Using ``_append`` ensures the operation
746takes affect.
747
748As shown in its simplest use, ``IMAGE_INSTALL_append`` affects all
749images. It is possible to extend the syntax so that the variable applies
750to a specific image only. Here is an example:
751IMAGE_INSTALL_append_pn-core-image-minimal = " strace" This example adds
752``strace`` to the ``core-image-minimal`` image only.
753
754You can add packages using a similar approach through the
755``CORE_IMAGE_EXTRA_INSTALL`` variable. If you use this variable, only
756``core-image-*`` images are affected.
757
758.. _usingpoky-extend-customimage-imagefeatures:
759
760Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES``
761-------------------------------------------------------------------------------
762
763Another method for customizing your image is to enable or disable
764high-level image features by using the
765```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ and
766```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
767variables. Although the functions for both variables are nearly
768equivalent, best practices dictate using ``IMAGE_FEATURES`` from within
769a recipe and using ``EXTRA_IMAGE_FEATURES`` from within your
770``local.conf`` file, which is found in the `Build
771Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
772
773To understand how these features work, the best reference is
774``meta/classes/core-image.bbclass``. This class lists out the available
775``IMAGE_FEATURES`` of which most map to package groups while some, such
776as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general
777configuration settings.
778
779In summary, the file looks at the contents of the ``IMAGE_FEATURES``
780variable and then maps or configures the feature accordingly. Based on
781this information, the build system automatically adds the appropriate
782packages or configurations to the
783```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__ variable.
784Effectively, you are enabling extra features by extending the class or
785creating a custom class for use with specialized image ``.bb`` files.
786
787Use the ``EXTRA_IMAGE_FEATURES`` variable from within your local
788configuration file. Using a separate area from which to enable features
789with this variable helps you avoid overwriting the features in the image
790recipe that are enabled with ``IMAGE_FEATURES``. The value of
791``EXTRA_IMAGE_FEATURES`` is added to ``IMAGE_FEATURES`` within
792``meta/conf/bitbake.conf``.
793
794To illustrate how you can use these variables to modify your image,
795consider an example that selects the SSH server. The Yocto Project ships
796with two SSH servers you can use with your images: Dropbear and OpenSSH.
797Dropbear is a minimal SSH server appropriate for resource-constrained
798environments, while OpenSSH is a well-known standard SSH server
799implementation. By default, the ``core-image-sato`` image is configured
800to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb``
801images both include OpenSSH. The ``core-image-minimal`` image does not
802contain an SSH server.
803
804You can customize your image and change these defaults. Edit the
805``IMAGE_FEATURES`` variable in your recipe or use the
806``EXTRA_IMAGE_FEATURES`` in your ``local.conf`` file so that it
807configures the image you are working with to include
808``ssh-server-dropbear`` or ``ssh-server-openssh``.
809
810.. note::
811
812 See the "
813 Images
814 " section in the Yocto Project Reference Manual for a complete list
815 of image features that ship with the Yocto Project.
816
817.. _usingpoky-extend-customimage-custombb:
818
819Customizing Images Using Custom .bb Files
820-----------------------------------------
821
822You can also customize an image by creating a custom recipe that defines
823additional software as part of the image. The following example shows
824the form for the two lines you need: IMAGE_INSTALL =
825"packagegroup-core-x11-base package1 package2" inherit core-image
826
827Defining the software using a custom recipe gives you total control over
828the contents of the image. It is important to use the correct names of
829packages in the ``IMAGE_INSTALL`` variable. You must use the
830OpenEmbedded notation and not the Debian notation for the names (e.g.
831``glibc-dev`` instead of ``libc6-dev``).
832
833The other method for creating a custom image is to base it on an
834existing image. For example, if you want to create an image based on
835``core-image-sato`` but add the additional package ``strace`` to the
836image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new
837``.bb`` and add the following line to the end of the copy: IMAGE_INSTALL
838+= "strace"
839
840.. _usingpoky-extend-customimage-customtasks:
841
842Customizing Images Using Custom Package Groups
843----------------------------------------------
844
845For complex custom images, the best approach for customizing an image is
846to create a custom package group recipe that is used to build the image
847or images. A good example of a package group recipe is
848``meta/recipes-core/packagegroups/packagegroup-base.bb``.
849
850If you examine that recipe, you see that the ``PACKAGES`` variable lists
851the package group packages to produce. The ``inherit packagegroup``
852statement sets appropriate default values and automatically adds
853``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each
854package specified in the ``PACKAGES`` statement.
855
856.. note::
857
858 The
859 inherit packagegroup
860 line should be located near the top of the recipe, certainly before
861 the
862 PACKAGES
863 statement.
864
865For each package you specify in ``PACKAGES``, you can use ``RDEPENDS``
866and ``RRECOMMENDS`` entries to provide a list of packages the parent
867task package should contain. You can see examples of these further down
868in the ``packagegroup-base.bb`` recipe.
869
870Here is a short, fabricated example showing the same basic pieces for a
871hypothetical packagegroup defined in ``packagegroup-custom.bb``, where
872the variable ``PN`` is the standard way to abbreviate the reference to
873the full packagegroup name ``packagegroup-custom``: DESCRIPTION = "My
874Custom Package Groups" inherit packagegroup PACKAGES = "\\ ${PN}-apps \\
875${PN}-tools \\ " RDEPENDS_${PN}-apps = "\\ dropbear \\ portmap \\
876psplash" RDEPENDS_${PN}-tools = "\\ oprofile \\ oprofileui-server \\
877lttng-tools" RRECOMMENDS_${PN}-tools = "\\ kernel-module-oprofile"
878
879In the previous example, two package group packages are created with
880their dependencies and their recommended package dependencies listed:
881``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To
882build an image using these package group packages, you need to add
883``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to
884``IMAGE_INSTALL``. For other forms of image dependencies see the other
885areas of this section.
886
887.. _usingpoky-extend-customimage-image-name:
888
889Customizing an Image Hostname
890-----------------------------
891
892By default, the configured hostname (i.e. ``/etc/hostname``) in an image
893is the same as the machine name. For example, if
894```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ equals "qemux86", the
895configured hostname written to ``/etc/hostname`` is "qemux86".
896
897You can customize this name by altering the value of the "hostname"
898variable in the ``base-files`` recipe using either an append file or a
899configuration file. Use the following in an append file:
900hostname="myhostname" Use the following in a configuration file:
901hostname_pn-base-files = "myhostname"
902
903Changing the default value of the variable "hostname" can be useful in
904certain situations. For example, suppose you need to do extensive
905testing on an image and you would like to easily identify the image
906under test from existing images with typical default hostnames. In this
907situation, you could change the default hostname to "testme", which
908results in all the images using the name "testme". Once testing is
909complete and you do not need to rebuild the image for test any longer,
910you can easily reset the default hostname.
911
912Another point of interest is that if you unset the variable, the image
913will have no default hostname in the filesystem. Here is an example that
914unsets the variable in a configuration file: hostname_pn-base-files = ""
915Having no default hostname in the filesystem is suitable for
916environments that use dynamic hostnames such as virtual machines.
917
918.. _new-recipe-writing-a-new-recipe:
919
920Writing a New Recipe
921====================
922
923Recipes (``.bb`` files) are fundamental components in the Yocto Project
924environment. Each software component built by the OpenEmbedded build
925system requires a recipe to define the component. This section describes
926how to create, write, and test a new recipe.
927
928.. note::
929
930 For information on variables that are useful for recipes and for
931 information about recipe naming issues, see the "
932 Required
933 " section of the Yocto Project Reference Manual.
934
935.. _new-recipe-overview:
936
937Overview
938--------
939
940The following figure shows the basic process for creating a new recipe.
941The remainder of the section provides details for the steps.
942
943.. _new-recipe-locate-or-automatically-create-a-base-recipe:
944
945Locate or Automatically Create a Base Recipe
946--------------------------------------------
947
948You can always write a recipe from scratch. However, three choices exist
949that can help you quickly get a start on a new recipe:
950
951- *``devtool add``:* A command that assists in creating a recipe and an
952 environment conducive to development.
953
954- *``recipetool create``:* A command provided by the Yocto Project that
955 automates creation of a base recipe based on the source files.
956
957- *Existing Recipes:* Location and modification of an existing recipe
958 that is similar in function to the recipe you need.
959
960.. note::
961
962 For information on recipe syntax, see the "
963 Recipe Syntax
964 " section.
965
966.. _new-recipe-creating-the-base-recipe-using-devtool:
967
968Creating the Base Recipe Using ``devtool add``
969~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
970
971The ``devtool add`` command uses the same logic for auto-creating the
972recipe as ``recipetool create``, which is listed below. Additionally,
973however, ``devtool add`` sets up an environment that makes it easy for
974you to patch the source and to make changes to the recipe as is often
975necessary when adding a recipe to build a new piece of software to be
976included in a build.
977
978You can find a complete description of the ``devtool add`` command in
979the "`A Closer Look at ``devtool``
980add <&YOCTO_DOCS_SDK_URL;#sdk-a-closer-look-at-devtool-add>`__" section
981in the Yocto Project Application Development and the Extensible Software
982Development Kit (eSDK) manual.
983
984.. _new-recipe-creating-the-base-recipe-using-recipetool:
985
986Creating the Base Recipe Using ``recipetool create``
987~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
988
989``recipetool create`` automates creation of a base recipe given a set of
990source code files. As long as you can extract or point to the source
991files, the tool will construct a recipe and automatically configure all
992pre-build information into the recipe. For example, suppose you have an
993application that builds using Autotools. Creating the base recipe using
994``recipetool`` results in a recipe that has the pre-build dependencies,
995license requirements, and checksums configured.
996
997To run the tool, you just need to be in your `Build
998Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ and have sourced the
999build environment setup script (i.e.
1000```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__).
1001To get help on the tool, use the following command: $ recipetool -h
1002NOTE: Starting bitbake server... usage: recipetool [-d] [-q] [--color
1003COLOR] [-h] <subcommand> ... OpenEmbedded recipe tool options: -d,
1004--debug Enable debug output -q, --quiet Print only errors --color COLOR
1005Colorize output (where COLOR is auto, always, never) -h, --help show
1006this help message and exit subcommands: create Create a new recipe
1007newappend Create a bbappend for the specified target in the specified
1008layer setvar Set a variable within a recipe appendfile Create/update a
1009bbappend to replace a target file appendsrcfiles Create/update a
1010bbappend to add or replace source files appendsrcfile Create/update a
1011bbappend to add or replace a source file Use recipetool <subcommand>
1012--help to get help on a specific command
1013
1014Running ``recipetool create -o`` OUTFILE creates the base recipe and
1015locates it properly in the layer that contains your source files.
1016Following are some syntax examples:
1017
1018Use this syntax to generate a recipe based on source. Once generated,
1019the recipe resides in the existing source code layer: recipetool create
1020-o OUTFILE source Use this syntax to generate a recipe using code that
1021you extract from source. The extracted code is placed in its own layer
1022defined by EXTERNALSRC. recipetool create -o OUTFILE -x EXTERNALSRC
1023source Use this syntax to generate a recipe based on source. The options
1024direct ``recipetool`` to generate debugging information. Once generated,
1025the recipe resides in the existing source code layer: recipetool create
1026-d -o OUTFILE source
1027
1028.. _new-recipe-locating-and-using-a-similar-recipe:
1029
1030Locating and Using a Similar Recipe
1031~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1032
1033Before writing a recipe from scratch, it is often useful to discover
1034whether someone else has already written one that meets (or comes close
1035to meeting) your needs. The Yocto Project and OpenEmbedded communities
1036maintain many recipes that might be candidates for what you are doing.
1037You can find a good central index of these recipes in the `OpenEmbedded
1038Layer Index <http://layers.openembedded.org>`__.
1039
1040Working from an existing recipe or a skeleton recipe is the best way to
1041get started. Here are some points on both methods:
1042
1043- *Locate and modify a recipe that is close to what you want to do:*
1044 This method works when you are familiar with the current recipe
1045 space. The method does not work so well for those new to the Yocto
1046 Project or writing recipes.
1047
1048 Some risks associated with this method are using a recipe that has
1049 areas totally unrelated to what you are trying to accomplish with
1050 your recipe, not recognizing areas of the recipe that you might have
1051 to add from scratch, and so forth. All these risks stem from
1052 unfamiliarity with the existing recipe space.
1053
1054- *Use and modify the following skeleton recipe:* If for some reason
1055 you do not want to use ``recipetool`` and you cannot find an existing
1056 recipe that is close to meeting your needs, you can use the following
1057 structure to provide the fundamental areas of a new recipe.
1058 DESCRIPTION = "" HOMEPAGE = "" LICENSE = "" SECTION = "" DEPENDS = ""
1059 LIC_FILES_CHKSUM = "" SRC_URI = ""
1060
1061.. _new-recipe-storing-and-naming-the-recipe:
1062
1063Storing and Naming the Recipe
1064-----------------------------
1065
1066Once you have your base recipe, you should put it in your own layer and
1067name it appropriately. Locating it correctly ensures that the
1068OpenEmbedded build system can find it when you use BitBake to process
1069the recipe.
1070
1071- *Storing Your Recipe:* The OpenEmbedded build system locates your
1072 recipe through the layer's ``conf/layer.conf`` file and the
1073 ```BBFILES`` <&YOCTO_DOCS_REF_URL;#var-BBFILES>`__ variable. This
1074 variable sets up a path from which the build system can locate
1075 recipes. Here is the typical use: BBFILES +=
1076 "${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
1077 Consequently, you need to be sure you locate your new recipe inside
1078 your layer such that it can be found.
1079
1080 You can find more information on how layers are structured in the
1081 "`Understanding and Creating
1082 Layers <#understanding-and-creating-layers>`__" section.
1083
1084- *Naming Your Recipe:* When you name your recipe, you need to follow
1085 this naming convention: basename_version.bb Use lower-cased
1086 characters and do not include the reserved suffixes ``-native``,
1087 ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use them
1088 as part of your recipe name unless the string applies). Here are some
1089 examples: cups_1.7.0.bb gawk_4.0.2.bb irssi_0.8.16-rc1.bb
1090
1091.. _new-recipe-running-a-build-on-the-recipe:
1092
1093Running a Build on the Recipe
1094-----------------------------
1095
1096Creating a new recipe is usually an iterative process that requires
1097using BitBake to process the recipe multiple times in order to
1098progressively discover and add information to the recipe file.
1099
1100Assuming you have sourced the build environment setup script (i.e.
1101````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) and you are in
1102the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, use
1103BitBake to process your recipe. All you need to provide is the
1104``basename`` of the recipe as described in the previous section: $
1105bitbake basename
1106
1107During the build, the OpenEmbedded build system creates a temporary work
1108directory for each recipe
1109(``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``)
1110where it keeps extracted source files, log files, intermediate
1111compilation and packaging files, and so forth.
1112
1113The path to the per-recipe temporary work directory depends on the
1114context in which it is being built. The quickest way to find this path
1115is to have BitBake return it by running the following: $ bitbake -e
1116basename \| grep ^WORKDIR= As an example, assume a Source Directory
1117top-level folder named ``poky``, a default Build Directory at
1118``poky/build``, and a ``qemux86-poky-linux`` machine target system.
1119Furthermore, suppose your recipe is named ``foo_1.3.0.bb``. In this
1120case, the work directory the build system uses to build the package
1121would be as follows: poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
1122Inside this directory you can find sub-directories such as ``image``,
1123``packages-split``, and ``temp``. After the build, you can examine these
1124to determine how well the build went.
1125
1126.. note::
1127
1128 You can find log files for each task in the recipe's
1129 temp
1130 directory (e.g.
1131 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp
1132 ). Log files are named
1133 log.
1134 taskname
1135 (e.g.
1136 log.do_configure
1137 ,
1138 log.do_fetch
1139 , and
1140 log.do_compile
1141 ).
1142
1143You can find more information about the build process in "`The Yocto
1144Project Development
1145Environment <&YOCTO_DOCS_OM_URL;#overview-development-environment>`__"
1146chapter of the Yocto Project Overview and Concepts Manual.
1147
1148.. _new-recipe-fetching-code:
1149
1150Fetching Code
1151-------------
1152
1153The first thing your recipe must do is specify how to fetch the source
1154files. Fetching is controlled mainly through the
1155```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. Your recipe
1156must have a ``SRC_URI`` variable that points to where the source is
1157located. For a graphical representation of source locations, see the
1158"`Sources <&YOCTO_DOCS_OM_URL;#sources-dev-environment>`__" section in
1159the Yocto Project Overview and Concepts Manual.
1160
1161The ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ task uses
1162the prefix of each entry in the ``SRC_URI`` variable value to determine
1163which `fetcher <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__ to use to get your
1164source files. It is the ``SRC_URI`` variable that triggers the fetcher.
1165The ```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task uses
1166the variable after source is fetched to apply patches. The OpenEmbedded
1167build system uses
1168```FILESOVERRIDES`` <&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES>`__ for
1169scanning directory locations for local files in ``SRC_URI``.
1170
1171The ``SRC_URI`` variable in your recipe must define each unique location
1172for your source files. It is good practice to not hard-code version
1173numbers in a URL used in ``SRC_URI``. Rather than hard-code these
1174values, use ``${``\ ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__\ ``}``,
1175which causes the fetch process to use the version specified in the
1176recipe filename. Specifying the version in this manner means that
1177upgrading the recipe to a future version is as simple as renaming the
1178recipe to match the new version.
1179
1180Here is a simple example from the
1181``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source
1182comes from a single tarball. Notice the use of the
1183```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ variable: SRC_URI =
1184"https://strace.io/files/${PV}/strace-${PV}.tar.xz \\
1185
1186Files mentioned in ``SRC_URI`` whose names end in a typical archive
1187extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so
1188forth), are automatically extracted during the
1189```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ task. For
1190another example that specifies these types of files, see the
1191"`Autotooled Package <#new-recipe-autotooled-package>`__" section.
1192
1193Another way of specifying source is from an SCM. For Git repositories,
1194you must specify ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ and
1195you should specify ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ to include
1196the revision with ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__. Here
1197is an example from the recipe
1198``meta/recipes-kernel/blktrace/blktrace_git.bb``: SRCREV =
1199"d6918c8832793b4205ed3bfede78c2f915c23385" PR = "r6" PV =
1200"1.0.5+git${SRCPV}" SRC_URI = "git://git.kernel.dk/blktrace.git \\
1201file://ldflags.patch"
1202
1203If your ``SRC_URI`` statement includes URLs pointing to individual files
1204fetched from a remote server other than a version control system,
1205BitBake attempts to verify the files against checksums defined in your
1206recipe to ensure they have not been tampered with or otherwise modified
1207since the recipe was written. Two checksums are used:
1208``SRC_URI[md5sum]`` and ``SRC_URI[sha256sum]``.
1209
1210If your ``SRC_URI`` variable points to more than a single URL (excluding
1211SCM URLs), you need to provide the ``md5`` and ``sha256`` checksums for
1212each URL. For these cases, you provide a name for each URL as part of
1213the ``SRC_URI`` and then reference that name in the subsequent checksum
1214statements. Here is an example combining lines from the files
1215``git.inc`` and ``git_2.24.1.bb``: SRC_URI =
1216"${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \\
1217${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages"
1218SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b"
1219SRC_URI[tarball.sha256sum] =
1220"ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02"
1221SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d"
1222SRC_URI[manpages.sha256sum] =
1223"9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230"
1224
1225Proper values for ``md5`` and ``sha256`` checksums might be available
1226with other signatures on the download page for the upstream source (e.g.
1227``md5``, ``sha1``, ``sha256``, ``GPG``, and so forth). Because the
1228OpenEmbedded build system only deals with ``sha256sum`` and ``md5sum``,
1229you should verify all the signatures you find by hand.
1230
1231If no ``SRC_URI`` checksums are specified when you attempt to build the
1232recipe, or you provide an incorrect checksum, the build will produce an
1233error for each missing or incorrect checksum. As part of the error
1234message, the build system provides the checksum string corresponding to
1235the fetched file. Once you have the correct checksums, you can copy and
1236paste them into your recipe and then run the build again to continue.
1237
1238.. note::
1239
1240 As mentioned, if the upstream source provides signatures for
1241 verifying the downloaded source code, you should verify those
1242 manually before setting the checksum values in the recipe and
1243 continuing with the build.
1244
1245This final example is a bit more complicated and is from the
1246``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The
1247example's ``SRC_URI`` statement identifies multiple files as the source
1248files for the recipe: a tarball, a patch file, a desktop file, and an
1249icon. SRC_URI =
1250"http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \\
1251file://xwc.patch \\ file://rxvt.desktop \\ file://rxvt.png"
1252
1253When you specify local files using the ``file://`` URI protocol, the
1254build system fetches files from the local machine. The path is relative
1255to the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ variable
1256and searches specific directories in a certain order:
1257``${``\ ```BP`` <&YOCTO_DOCS_REF_URL;#var-BP>`__\ ``}``,
1258``${``\ ```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__\ ``}``, and
1259``files``. The directories are assumed to be subdirectories of the
1260directory in which the recipe or append file resides. For another
1261example that specifies these types of files, see the "`Single .c File
1262Package (Hello
1263World!) <#new-recipe-single-c-file-package-hello-world>`__" section.
1264
1265The previous example also specifies a patch file. Patch files are files
1266whose names usually end in ``.patch`` or ``.diff`` but can end with
1267compressed suffixes such as ``diff.gz`` and ``patch.bz2``, for example.
1268The build system automatically applies patches as described in the
1269"`Patching Code <#new-recipe-patching-code>`__" section.
1270
1271.. _new-recipe-unpacking-code:
1272
1273Unpacking Code
1274--------------
1275
1276During the build, the
1277```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ task unpacks
1278the source with ``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``
1279pointing to where it is unpacked.
1280
1281If you are fetching your source files from an upstream source archived
1282tarball and the tarball's internal structure matches the common
1283convention of a top-level subdirectory named
1284``${``\ ```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__\ ``}-${``\ ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__\ ``}``,
1285then you do not need to set ``S``. However, if ``SRC_URI`` specifies to
1286fetch source from an archive that does not use this convention, or from
1287an SCM like Git or Subversion, your recipe needs to define ``S``.
1288
1289If processing your recipe using BitBake successfully unpacks the source
1290files, you need to be sure that the directory pointed to by ``${S}``
1291matches the structure of the source.
1292
1293.. _new-recipe-patching-code:
1294
1295Patching Code
1296-------------
1297
1298Sometimes it is necessary to patch code after it has been fetched. Any
1299files mentioned in ``SRC_URI`` whose names end in ``.patch`` or
1300``.diff`` or compressed versions of these suffixes (e.g. ``diff.gz`` are
1301treated as patches. The
1302```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task
1303automatically applies these patches.
1304
1305The build system should be able to apply patches with the "-p1" option
1306(i.e. one directory level in the path will be stripped off). If your
1307patch needs to have more directory levels stripped off, specify the
1308number of levels using the "striplevel" option in the ``SRC_URI`` entry
1309for the patch. Alternatively, if your patch needs to be applied in a
1310specific subdirectory that is not specified in the patch file, use the
1311"patchdir" option in the entry.
1312
1313As with all local files referenced in
1314```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ using ``file://``,
1315you should place patch files in a directory next to the recipe either
1316named the same as the base name of the recipe
1317(```BP`` <&YOCTO_DOCS_REF_URL;#var-BP>`__ and
1318```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__) or "files".
1319
1320.. _new-recipe-licensing:
1321
1322Licensing
1323---------
1324
1325Your recipe needs to have both the
1326```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ and
1327```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
1328variables:
1329
1330- *``LICENSE``:* This variable specifies the license for the software.
1331 If you do not know the license under which the software you are
1332 building is distributed, you should go to the source code and look
1333 for that information. Typical files containing this information
1334 include ``COPYING``, ``LICENSE``, and ``README`` files. You could
1335 also find the information near the top of a source file. For example,
1336 given a piece of software licensed under the GNU General Public
1337 License version 2, you would set ``LICENSE`` as follows: LICENSE =
1338 "GPLv2"
1339
1340 The licenses you specify within ``LICENSE`` can have any name as long
1341 as you do not use spaces, since spaces are used as separators between
1342 license names. For standard licenses, use the names of the files in
1343 ``meta/files/common-licenses/`` or the ``SPDXLICENSEMAP`` flag names
1344 defined in ``meta/conf/licenses.conf``.
1345
1346- *``LIC_FILES_CHKSUM``:* The OpenEmbedded build system uses this
1347 variable to make sure the license text has not changed. If it has,
1348 the build produces an error and it affords you the chance to figure
1349 it out and correct the problem.
1350
1351 You need to specify all applicable licensing files for the software.
1352 At the end of the configuration step, the build process will compare
1353 the checksums of the files to be sure the text has not changed. Any
1354 differences result in an error with the message containing the
1355 current checksum. For more explanation and examples of how to set the
1356 ``LIC_FILES_CHKSUM`` variable, see the "`Tracking License
1357 Changes <#>`__" section.
1358
1359 To determine the correct checksum string, you can list the
1360 appropriate files in the ``LIC_FILES_CHKSUM`` variable with incorrect
1361 md5 strings, attempt to build the software, and then note the
1362 resulting error messages that will report the correct md5 strings.
1363 See the "`Fetching Code <#new-recipe-fetching-code>`__" section for
1364 additional information.
1365
1366 Here is an example that assumes the software has a ``COPYING`` file:
1367 LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" When you try to build the
1368 software, the build system will produce an error and give you the
1369 correct string that you can substitute into the recipe file for a
1370 subsequent build.
1371
1372.. _new-dependencies:
1373
1374Dependencies
1375------------
1376
1377Most software packages have a short list of other packages that they
1378require, which are called dependencies. These dependencies fall into two
1379main categories: build-time dependencies, which are required when the
1380software is built; and runtime dependencies, which are required to be
1381installed on the target in order for the software to run.
1382
1383Within a recipe, you specify build-time dependencies using the
1384```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable. Although
1385nuances exist, items specified in ``DEPENDS`` should be names of other
1386recipes. It is important that you specify all build-time dependencies
1387explicitly. If you do not, due to the parallel nature of BitBake's
1388execution, you can end up with a race condition where the dependency is
1389present for one task of a recipe (e.g.
1390```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__) and
1391then gone when the next task runs (e.g.
1392```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__).
1393
1394Another consideration is that configure scripts might automatically
1395check for optional dependencies and enable corresponding functionality
1396if those dependencies are found. This behavior means that to ensure
1397deterministic results and thus avoid more race conditions, you need to
1398either explicitly specify these dependencies as well, or tell the
1399configure script explicitly to disable the functionality. If you wish to
1400make a recipe that is more generally useful (e.g. publish the recipe in
1401a layer for others to use), instead of hard-disabling the functionality,
1402you can use the
1403```PACKAGECONFIG`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG>`__ variable
1404to allow functionality and the corresponding dependencies to be enabled
1405and disabled easily by other users of the recipe.
1406
1407Similar to build-time dependencies, you specify runtime dependencies
1408through a variable -
1409```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__, which is
1410package-specific. All variables that are package-specific need to have
1411the name of the package added to the end as an override. Since the main
1412package for a recipe has the same name as the recipe, and the recipe's
1413name can be found through the
1414``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` variable, then
1415you specify the dependencies for the main package by setting
1416``RDEPENDS_${PN}``. If the package were named ``${PN}-tools``, then you
1417would set ``RDEPENDS_${PN}-tools``, and so forth.
1418
1419Some runtime dependencies will be set automatically at packaging time.
1420These dependencies include any shared library dependencies (i.e. if a
1421package "example" contains "libexample" and another package "mypackage"
1422contains a binary that links to "libexample" then the OpenEmbedded build
1423system will automatically add a runtime dependency to "mypackage" on
1424"example"). See the "`Automatically Added Runtime
1425Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
1426section in the Yocto Project Overview and Concepts Manual for further
1427details.
1428
1429.. _new-recipe-configuring-the-recipe:
1430
1431Configuring the Recipe
1432----------------------
1433
1434Most software provides some means of setting build-time configuration
1435options before compilation. Typically, setting these options is
1436accomplished by running a configure script with options, or by modifying
1437a build configuration file.
1438
1439.. note::
1440
1441 As of Yocto Project Release 1.7, some of the core recipes that
1442 package binary configuration scripts now disable the scripts due to
1443 the scripts previously requiring error-prone path substitution. The
1444 OpenEmbedded build system uses
1445 pkg-config
1446 now, which is much more robust. You can find a list of the
1447 \*-config
1448 scripts that are disabled list in the "
1449 Binary Configuration Scripts Disabled
1450 " section in the Yocto Project Reference Manual.
1451
1452A major part of build-time configuration is about checking for
1453build-time dependencies and possibly enabling optional functionality as
1454a result. You need to specify any build-time dependencies for the
1455software you are building in your recipe's
1456```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ value, in terms of
1457other recipes that satisfy those dependencies. You can often find
1458build-time or runtime dependencies described in the software's
1459documentation.
1460
1461The following list provides configuration items of note based on how
1462your software is built:
1463
1464- *Autotools:* If your source files have a ``configure.ac`` file, then
1465 your software is built using Autotools. If this is the case, you just
1466 need to worry about modifying the configuration.
1467
1468 When using Autotools, your recipe needs to inherit the
1469 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class
1470 and your recipe does not have to contain a
1471 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task.
1472 However, you might still want to make some adjustments. For example,
1473 you can set
1474 ```EXTRA_OECONF`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF>`__ or
1475 ```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
1476 to pass any needed configure options that are specific to the recipe.
1477
1478- *CMake:* If your source files have a ``CMakeLists.txt`` file, then
1479 your software is built using CMake. If this is the case, you just
1480 need to worry about modifying the configuration.
1481
1482 When you use CMake, your recipe needs to inherit the
1483 ```cmake`` <&YOCTO_DOCS_REF_URL;#ref-classes-cmake>`__ class and your
1484 recipe does not have to contain a
1485 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task.
1486 You can make some adjustments by setting
1487 ```EXTRA_OECMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE>`__ to
1488 pass any needed configure options that are specific to the recipe.
1489
1490 .. note::
1491
1492 If you need to install one or more custom CMake toolchain files
1493 that are supplied by the application you are building, install the
1494 files to
1495 ${D}${datadir}/cmake/
1496 Modules during
1497 do_install
1498 .
1499
1500- *Other:* If your source files do not have a ``configure.ac`` or
1501 ``CMakeLists.txt`` file, then your software is built using some
1502 method other than Autotools or CMake. If this is the case, you
1503 normally need to provide a
1504 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task
1505 in your recipe unless, of course, there is nothing to configure.
1506
1507 Even if your software is not being built by Autotools or CMake, you
1508 still might not need to deal with any configuration issues. You need
1509 to determine if configuration is even a required step. You might need
1510 to modify a Makefile or some configuration file used for the build to
1511 specify necessary build options. Or, perhaps you might need to run a
1512 provided, custom configure script with the appropriate options.
1513
1514 For the case involving a custom configure script, you would run
1515 ``./configure --help`` and look for the options you need to set.
1516
1517Once configuration succeeds, it is always good practice to look at the
1518``log.do_configure`` file to ensure that the appropriate options have
1519been enabled and no additional build-time dependencies need to be added
1520to ``DEPENDS``. For example, if the configure script reports that it
1521found something not mentioned in ``DEPENDS``, or that it did not find
1522something that it needed for some desired optional functionality, then
1523you would need to add those to ``DEPENDS``. Looking at the log might
1524also reveal items being checked for, enabled, or both that you do not
1525want, or items not being found that are in ``DEPENDS``, in which case
1526you would need to look at passing extra options to the configure script
1527as needed. For reference information on configure options specific to
1528the software you are building, you can consult the output of the
1529``./configure --help`` command within ``${S}`` or consult the software's
1530upstream documentation.
1531
1532.. _new-recipe-using-headers-to-interface-with-devices:
1533
1534Using Headers to Interface with Devices
1535---------------------------------------
1536
1537If your recipe builds an application that needs to communicate with some
1538device or needs an API into a custom kernel, you will need to provide
1539appropriate header files. Under no circumstances should you ever modify
1540the existing
1541``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc`` file.
1542These headers are used to build ``libc`` and must not be compromised
1543with custom or machine-specific header information. If you customize
1544``libc`` through modified headers all other applications that use
1545``libc`` thus become affected.
1546
1547.. note::
1548
1549 Never copy and customize the
1550 libc
1551 header file (i.e.
1552 meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
1553 ).
1554
1555The correct way to interface to a device or custom kernel is to use a
1556separate package that provides the additional headers for the driver or
1557other unique interfaces. When doing so, your application also becomes
1558responsible for creating a dependency on that specific provider.
1559
1560Consider the following:
1561
1562- Never modify ``linux-libc-headers.inc``. Consider that file to be
1563 part of the ``libc`` system, and not something you use to access the
1564 kernel directly. You should access ``libc`` through specific ``libc``
1565 calls.
1566
1567- Applications that must talk directly to devices should either provide
1568 necessary headers themselves, or establish a dependency on a special
1569 headers package that is specific to that driver.
1570
1571For example, suppose you want to modify an existing header that adds I/O
1572control or network support. If the modifications are used by a small
1573number programs, providing a unique version of a header is easy and has
1574little impact. When doing so, bear in mind the guidelines in the
1575previous list.
1576
1577.. note::
1578
1579 If for some reason your changes need to modify the behavior of the
1580 libc
1581 , and subsequently all other applications on the system, use a
1582 .bbappend
1583 to modify the
1584 linux-kernel-headers.inc
1585 file. However, take care to not make the changes machine specific.
1586
1587Consider a case where your kernel is older and you need an older
1588``libc`` ABI. The headers installed by your recipe should still be a
1589standard mainline kernel, not your own custom one.
1590
1591When you use custom kernel headers you need to get them from
1592```STAGING_KERNEL_DIR`` <&YOCTO_DOCS_REF_URL;#var-STAGING_KERNEL_DIR>`__,
1593which is the directory with kernel headers that are required to build
1594out-of-tree modules. Your recipe will also need the following:
1595do_configure[depends] += "virtual/kernel:do_shared_workdir"
1596
1597.. _new-recipe-compilation:
1598
1599Compilation
1600-----------
1601
1602During a build, the ``do_compile`` task happens after source is fetched,
1603unpacked, and configured. If the recipe passes through ``do_compile``
1604successfully, nothing needs to be done.
1605
1606However, if the compile step fails, you need to diagnose the failure.
1607Here are some common issues that cause failures.
1608
1609.. note::
1610
1611 For cases where improper paths are detected for configuration files
1612 or for when libraries/headers cannot be found, be sure you are using
1613 the more robust
1614 pkg-config
1615 . See the note in section "
1616 Configuring the Recipe
1617 " for additional information.
1618
1619- *Parallel build failures:* These failures manifest themselves as
1620 intermittent errors, or errors reporting that a file or directory
1621 that should be created by some other part of the build process could
1622 not be found. This type of failure can occur even if, upon
1623 inspection, the file or directory does exist after the build has
1624 failed, because that part of the build process happened in the wrong
1625 order.
1626
1627 To fix the problem, you need to either satisfy the missing dependency
1628 in the Makefile or whatever script produced the Makefile, or (as a
1629 workaround) set
1630 ```PARALLEL_MAKE`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ to an
1631 empty string: PARALLEL_MAKE = ""
1632
1633 For information on parallel Makefile issues, see the "`Debugging
1634 Parallel Make Races <#debugging-parallel-make-races>`__" section.
1635
1636- *Improper host path usage:* This failure applies to recipes building
1637 for the target or ``nativesdk`` only. The failure occurs when the
1638 compilation process uses improper headers, libraries, or other files
1639 from the host system when cross-compiling for the target.
1640
1641 To fix the problem, examine the ``log.do_compile`` file to identify
1642 the host paths being used (e.g. ``/usr/include``, ``/usr/lib``, and
1643 so forth) and then either add configure options, apply a patch, or do
1644 both.
1645
1646- *Failure to find required libraries/headers:* If a build-time
1647 dependency is missing because it has not been declared in
1648 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__, or because the
1649 dependency exists but the path used by the build process to find the
1650 file is incorrect and the configure step did not detect it, the
1651 compilation process could fail. For either of these failures, the
1652 compilation process notes that files could not be found. In these
1653 cases, you need to go back and add additional options to the
1654 configure script as well as possibly add additional build-time
1655 dependencies to ``DEPENDS``.
1656
1657 Occasionally, it is necessary to apply a patch to the source to
1658 ensure the correct paths are used. If you need to specify paths to
1659 find files staged into the sysroot from other recipes, use the
1660 variables that the OpenEmbedded build system provides (e.g.
1661 ``STAGING_BINDIR``, ``STAGING_INCDIR``, ``STAGING_DATADIR``, and so
1662 forth).
1663
1664.. _new-recipe-installing:
1665
1666Installing
1667----------
1668
1669During ``do_install``, the task copies the built files along with their
1670hierarchy to locations that would mirror their locations on the target
1671device. The installation process copies files from the
1672``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``,
1673``${``\ ```B`` <&YOCTO_DOCS_REF_URL;#var-B>`__\ ``}``, and
1674``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``
1675directories to the ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}``
1676directory to create the structure as it should appear on the target
1677system.
1678
1679How your software is built affects what you must do to be sure your
1680software is installed correctly. The following list describes what you
1681must do for installation depending on the type of build system used by
1682the software being built:
1683
1684- *Autotools and CMake:* If the software your recipe is building uses
1685 Autotools or CMake, the OpenEmbedded build system understands how to
1686 install the software. Consequently, you do not have to have a
1687 ``do_install`` task as part of your recipe. You just need to make
1688 sure the install portion of the build completes with no issues.
1689 However, if you wish to install additional files not already being
1690 installed by ``make install``, you should do this using a
1691 ``do_install_append`` function using the install command as described
1692 in the "Manual" bulleted item later in this list.
1693
1694- *Other (using ``make install``):* You need to define a ``do_install``
1695 function in your recipe. The function should call
1696 ``oe_runmake install`` and will likely need to pass in the
1697 destination directory as well. How you pass that path is dependent on
1698 how the ``Makefile`` being run is written (e.g. ``DESTDIR=${D}``,
1699 ``PREFIX=${D}``, ``INSTALLROOT=${D}``, and so forth).
1700
1701 For an example recipe using ``make install``, see the
1702 "`Makefile-Based Package <#new-recipe-makefile-based-package>`__"
1703 section.
1704
1705- *Manual:* You need to define a ``do_install`` function in your
1706 recipe. The function must first use ``install -d`` to create the
1707 directories under
1708 ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}``. Once the
1709 directories exist, your function can use ``install`` to manually
1710 install the built software into the directories.
1711
1712 You can find more information on ``install`` at
1713 ` <http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html>`__.
1714
1715For the scenarios that do not use Autotools or CMake, you need to track
1716the installation and diagnose and fix any issues until everything
1717installs correctly. You need to look in the default location of
1718``${D}``, which is ``${WORKDIR}/image``, to be sure your files have been
1719installed correctly.
1720
1721.. note::
1722
1723 - During the installation process, you might need to modify some of
1724 the installed files to suit the target layout. For example, you
1725 might need to replace hard-coded paths in an initscript with
1726 values of variables provided by the build system, such as
1727 replacing ``/usr/bin/`` with ``${bindir}``. If you do perform such
1728 modifications during ``do_install``, be sure to modify the
1729 destination file after copying rather than before copying.
1730 Modifying after copying ensures that the build system can
1731 re-execute ``do_install`` if needed.
1732
1733 - ``oe_runmake install``, which can be run directly or can be run
1734 indirectly by the
1735 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ and
1736 ```cmake`` <&YOCTO_DOCS_REF_URL;#ref-classes-cmake>`__ classes,
1737 runs ``make install`` in parallel. Sometimes, a Makefile can have
1738 missing dependencies between targets that can result in race
1739 conditions. If you experience intermittent failures during
1740 ``do_install``, you might be able to work around them by disabling
1741 parallel Makefile installs by adding the following to the recipe:
1742 PARALLEL_MAKEINST = "" See
1743 ```PARALLEL_MAKEINST`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST>`__
1744 for additional information.
1745
1746 - If you need to install one or more custom CMake toolchain files
1747 that are supplied by the application you are building, install the
1748 files to ``${D}${datadir}/cmake/`` Modules during
1749 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__.
1750
1751.. _new-recipe-enabling-system-services:
1752
1753Enabling System Services
1754------------------------
1755
1756If you want to install a service, which is a process that usually starts
1757on boot and runs in the background, then you must include some
1758additional definitions in your recipe.
1759
1760If you are adding services and the service initialization script or the
1761service file itself is not installed, you must provide for that
1762installation in your recipe using a ``do_install_append`` function. If
1763your recipe already has a ``do_install`` function, update the function
1764near its end rather than adding an additional ``do_install_append``
1765function.
1766
1767When you create the installation for your services, you need to
1768accomplish what is normally done by ``make install``. In other words,
1769make sure your installation arranges the output similar to how it is
1770arranged on the target system.
1771
1772The OpenEmbedded build system provides support for starting services two
1773different ways:
1774
1775- *SysVinit:* SysVinit is a system and service manager that manages the
1776 init system used to control the very basic functions of your system.
1777 The init program is the first program started by the Linux kernel
1778 when the system boots. Init then controls the startup, running and
1779 shutdown of all other programs.
1780
1781 To enable a service using SysVinit, your recipe needs to inherit the
1782 ```update-rc.d`` <&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d>`__
1783 class. The class helps facilitate safely installing the package on
1784 the target.
1785
1786 You will need to set the
1787 ```INITSCRIPT_PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES>`__,
1788 ```INITSCRIPT_NAME`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME>`__,
1789 and
1790 ```INITSCRIPT_PARAMS`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS>`__
1791 variables within your recipe.
1792
1793- *systemd:* System Management Daemon (systemd) was designed to replace
1794 SysVinit and to provide enhanced management of services. For more
1795 information on systemd, see the systemd homepage at
1796 ` <http://freedesktop.org/wiki/Software/systemd/>`__.
1797
1798 To enable a service using systemd, your recipe needs to inherit the
1799 ```systemd`` <&YOCTO_DOCS_REF_URL;#ref-classes-systemd>`__ class. See
1800 the ``systemd.bbclass`` file located in your `Source
1801 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. section for
1802 more information.
1803
1804.. _new-recipe-packaging:
1805
1806Packaging
1807---------
1808
1809Successful packaging is a combination of automated processes performed
1810by the OpenEmbedded build system and some specific steps you need to
1811take. The following list describes the process:
1812
1813- *Splitting Files*: The ``do_package`` task splits the files produced
1814 by the recipe into logical components. Even software that produces a
1815 single binary might still have debug symbols, documentation, and
1816 other logical components that should be split out. The ``do_package``
1817 task ensures that files are split up and packaged correctly.
1818
1819- *Running QA Checks*: The
1820 ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__ class adds a
1821 step to the package generation process so that output quality
1822 assurance checks are generated by the OpenEmbedded build system. This
1823 step performs a range of checks to be sure the build's output is free
1824 of common problems that show up during runtime. For information on
1825 these checks, see the
1826 ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__ class and
1827 the "`QA Error and Warning
1828 Messages <&YOCTO_DOCS_REF_URL;#ref-qa-checks>`__" chapter in the
1829 Yocto Project Reference Manual.
1830
1831- *Hand-Checking Your Packages*: After you build your software, you
1832 need to be sure your packages are correct. Examine the
1833 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/packages-split``
1834 directory and make sure files are where you expect them to be. If you
1835 discover problems, you can set
1836 ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__,
1837 ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__,
1838 ``do_install(_append)``, and so forth as needed.
1839
1840- *Splitting an Application into Multiple Packages*: If you need to
1841 split an application into several packages, see the "`Splitting an
1842 Application into Multiple
1843 Packages <#splitting-an-application-into-multiple-packages>`__"
1844 section for an example.
1845
1846- *Installing a Post-Installation Script*: For an example showing how
1847 to install a post-installation script, see the "`Post-Installation
1848 Scripts <#new-recipe-post-installation-scripts>`__" section.
1849
1850- *Marking Package Architecture*: Depending on what your recipe is
1851 building and how it is configured, it might be important to mark the
1852 packages produced as being specific to a particular machine, or to
1853 mark them as not being specific to a particular machine or
1854 architecture at all.
1855
1856 By default, packages apply to any machine with the same architecture
1857 as the target machine. When a recipe produces packages that are
1858 machine-specific (e.g. the
1859 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ value is passed
1860 into the configure script or a patch is applied only for a particular
1861 machine), you should mark them as such by adding the following to the
1862 recipe: PACKAGE_ARCH = "${MACHINE_ARCH}"
1863
1864 On the other hand, if the recipe produces packages that do not
1865 contain anything specific to the target machine or architecture at
1866 all (e.g. recipes that simply package script files or configuration
1867 files), you should use the
1868 ```allarch`` <&YOCTO_DOCS_REF_URL;#ref-classes-allarch>`__ class to
1869 do this for you by adding this to your recipe: inherit allarch
1870 Ensuring that the package architecture is correct is not critical
1871 while you are doing the first few builds of your recipe. However, it
1872 is important in order to ensure that your recipe rebuilds (or does
1873 not rebuild) appropriately in response to changes in configuration,
1874 and to ensure that you get the appropriate packages installed on the
1875 target machine, particularly if you run separate builds for more than
1876 one target machine.
1877
1878.. _new-sharing-files-between-recipes:
1879
1880Sharing Files Between Recipes
1881-----------------------------
1882
1883Recipes often need to use files provided by other recipes on the build
1884host. For example, an application linking to a common library needs
1885access to the library itself and its associated headers. The way this
1886access is accomplished is by populating a sysroot with files. Each
1887recipe has two sysroots in its work directory, one for target files
1888(``recipe-sysroot``) and one for files that are native to the build host
1889(``recipe-sysroot-native``).
1890
1891.. note::
1892
1893 You could find the term "staging" used within the Yocto project
1894 regarding files populating sysroots (e.g. the
1895 STAGING_DIR
1896 variable).
1897
1898Recipes should never populate the sysroot directly (i.e. write files
1899into sysroot). Instead, files should be installed into standard
1900locations during the
1901```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task within
1902the ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}`` directory. The
1903reason for this limitation is that almost all files that populate the
1904sysroot are cataloged in manifests in order to ensure the files can be
1905removed later when a recipe is either modified or removed. Thus, the
1906sysroot is able to remain free from stale files.
1907
1908A subset of the files installed by the
1909```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task are
1910used by the
1911```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
1912task as defined by the the
1913```SYSROOT_DIRS`` <&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS>`__ variable to
1914automatically populate the sysroot. It is possible to modify the list of
1915directories that populate the sysroot. The following example shows how
1916you could add the ``/opt`` directory to the list of directories within a
1917recipe: SYSROOT_DIRS += "/opt"
1918
1919For a more complete description of the
1920```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
1921task and its associated functions, see the
1922```staging`` <&YOCTO_DOCS_REF_URL;#ref-classes-staging>`__ class.
1923
1924.. _metadata-virtual-providers:
1925
1926Using Virtual Providers
1927-----------------------
1928
1929Prior to a build, if you know that several different recipes provide the
1930same functionality, you can use a virtual provider (i.e. ``virtual/*``)
1931as a placeholder for the actual provider. The actual provider is
1932determined at build-time.
1933
1934A common scenario where a virtual provider is used would be for the
1935kernel recipe. Suppose you have three kernel recipes whose
1936```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ values map to ``kernel-big``,
1937``kernel-mid``, and ``kernel-small``. Furthermore, each of these recipes
1938in some way uses a ```PROVIDES`` <&YOCTO_DOCS_REF_URL;#var-PROVIDES>`__
1939statement that essentially identifies itself as being able to provide
1940``virtual/kernel``. Here is one way through the
1941```kernel`` <&YOCTO_DOCS_REF_URL;#ref-classes-kernel>`__ class: PROVIDES
1942+= "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") ==
1943"kernel") else "" }" Any recipe that inherits the ``kernel`` class is
1944going to utilize a ``PROVIDES`` statement that identifies that recipe as
1945being able to provide the ``virtual/kernel`` item.
1946
1947Now comes the time to actually build an image and you need a kernel
1948recipe, but which one? You can configure your build to call out the
1949kernel recipe you want by using the
1950```PREFERRED_PROVIDER`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__
1951variable. As an example, consider the
1952```x86-base.inc`` <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc>`__
1953include file, which is a machine (i.e.
1954```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__) configuration file.
1955This include file is the reason all x86-based machines use the
1956``linux-yocto`` kernel. Here are the relevant lines from the include
1957file: PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto"
1958PREFERRED_VERSION_linux-yocto ??= "4.15%"
1959
1960When you use a virtual provider, you do not have to "hard code" a recipe
1961name as a build dependency. You can use the
1962```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable to state the
1963build is dependent on ``virtual/kernel`` for example: DEPENDS =
1964"virtual/kernel" During the build, the OpenEmbedded build system picks
1965the correct recipe needed for the ``virtual/kernel`` dependency based on
1966the ``PREFERRED_PROVIDER`` variable. If you want to use the small kernel
1967mentioned at the beginning of this section, configure your build as
1968follows: PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small"
1969
1970.. note::
1971
1972 Any recipe that
1973 PROVIDES
1974 a
1975 virtual/\*
1976 item that is ultimately not selected through
1977 PREFERRED_PROVIDER
1978 does not get built. Preventing these recipes from building is usually
1979 the desired behavior since this mechanism's purpose is to select
1980 between mutually exclusive alternative providers.
1981
1982The following lists specific examples of virtual providers:
1983
1984- ``virtual/kernel``: Provides the name of the kernel recipe to use
1985 when building a kernel image.
1986
1987- ``virtual/bootloader``: Provides the name of the bootloader to use
1988 when building an image.
1989
1990- ``virtual/libgbm``: Provides ``gbm.pc``.
1991
1992- ``virtual/egl``: Provides ``egl.pc`` and possibly ``wayland-egl.pc``.
1993
1994- ``virtual/libgl``: Provides ``gl.pc`` (i.e. libGL).
1995
1996- ``virtual/libgles1``: Provides ``glesv1_cm.pc`` (i.e. libGLESv1_CM).
1997
1998- ``virtual/libgles2``: Provides ``glesv2.pc`` (i.e. libGLESv2).
1999
2000Properly Versioning Pre-Release Recipes
2001---------------------------------------
2002
2003Sometimes the name of a recipe can lead to versioning problems when the
2004recipe is upgraded to a final release. For example, consider the
2005``irssi_0.8.16-rc1.bb`` recipe file in the list of example recipes in
2006the "`Storing and Naming the
2007Recipe <#new-recipe-storing-and-naming-the-recipe>`__" section. This
2008recipe is at a release candidate stage (i.e. "rc1"). When the recipe is
2009released, the recipe filename becomes ``irssi_0.8.16.bb``. The version
2010change from ``0.8.16-rc1`` to ``0.8.16`` is seen as a decrease by the
2011build system and package managers, so the resulting packages will not
2012correctly trigger an upgrade.
2013
2014In order to ensure the versions compare properly, the recommended
2015convention is to set ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ within the
2016recipe to "previous_version+current_version". You can use an additional
2017variable so that you can use the current version elsewhere. Here is an
2018example: REALPV = "0.8.16-rc1" PV = "0.8.15+${REALPV}"
2019
2020.. _new-recipe-post-installation-scripts:
2021
2022Post-Installation Scripts
2023-------------------------
2024
2025Post-installation scripts run immediately after installing a package on
2026the target or during image creation when a package is included in an
2027image. To add a post-installation script to a package, add a
2028``pkg_postinst_``\ PACKAGENAME\ ``()`` function to the recipe file
2029(``.bb``) and replace PACKAGENAME with the name of the package you want
2030to attach to the ``postinst`` script. To apply the post-installation
2031script to the main package for the recipe, which is usually what is
2032required, specify
2033``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` in place of
2034PACKAGENAME.
2035
2036A post-installation function has the following structure:
2037pkg_postinst_PACKAGENAME() { # Commands to carry out }
2038
2039The script defined in the post-installation function is called when the
2040root filesystem is created. If the script succeeds, the package is
2041marked as installed.
2042
2043.. note::
2044
2045 Any RPM post-installation script that runs on the target should
2046 return a 0 exit code. RPM does not allow non-zero exit codes for
2047 these scripts, and the RPM package manager will cause the package to
2048 fail installation on the target.
2049
2050Sometimes it is necessary for the execution of a post-installation
2051script to be delayed until the first boot. For example, the script might
2052need to be executed on the device itself. To delay script execution
2053until boot time, you must explicitly mark post installs to defer to the
2054target. You can use ``pkg_postinst_ontarget()`` or call
2055``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``. Any
2056failure of a ``pkg_postinst()`` script (including exit 1) triggers an
2057error during the
2058```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__ task.
2059
2060If you have recipes that use ``pkg_postinst`` function and they require
2061the use of non-standard native tools that have dependencies during
2062rootfs construction, you need to use the
2063```PACKAGE_WRITE_DEPS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_WRITE_DEPS>`__
2064variable in your recipe to list these tools. If you do not use this
2065variable, the tools might be missing and execution of the
2066post-installation script is deferred until first boot. Deferring the
2067script to first boot is undesirable and for read-only rootfs impossible.
2068
2069.. note::
2070
2071 Equivalent support for pre-install, pre-uninstall, and post-uninstall
2072 scripts exist by way of
2073 pkg_preinst
2074 ,
2075 pkg_prerm
2076 , and
2077 pkg_postrm
2078 , respectively. These scrips work in exactly the same way as does
2079 pkg_postinst
2080 with the exception that they run at different times. Also, because of
2081 when they run, they are not applicable to being run at image creation
2082 time like
2083 pkg_postinst
2084 .
2085
2086.. _new-recipe-testing:
2087
2088Testing
2089-------
2090
2091The final step for completing your recipe is to be sure that the
2092software you built runs correctly. To accomplish runtime testing, add
2093the build's output packages to your image and test them on the target.
2094
2095For information on how to customize your image by adding specific
2096packages, see the "`Customizing
2097Images <#usingpoky-extend-customimage>`__" section.
2098
2099.. _new-recipe-testing-examples:
2100
2101Examples
2102--------
2103
2104To help summarize how to write a recipe, this section provides some
2105examples given various scenarios:
2106
2107- Recipes that use local files
2108
2109- Using an Autotooled package
2110
2111- Using a Makefile-based package
2112
2113- Splitting an application into multiple packages
2114
2115- Adding binaries to an image
2116
2117.. _new-recipe-single-c-file-package-hello-world:
2118
2119Single .c File Package (Hello World!)
2120~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2121
2122Building an application from a single file that is stored locally (e.g.
2123under ``files``) requires a recipe that has the file listed in the
2124``SRC_URI`` variable. Additionally, you need to manually write the
2125``do_compile`` and ``do_install`` tasks. The ``S`` variable defines the
2126directory containing the source code, which is set to
2127```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__ in this case - the
2128directory BitBake uses for the build. SUMMARY = "Simple helloworld
2129application" SECTION = "examples" LICENSE = "MIT" LIC_FILES_CHKSUM =
2130"file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
2131SRC_URI = "file://helloworld.c" S = "${WORKDIR}" do_compile() { ${CC}
2132helloworld.c -o helloworld } do_install() { install -d ${D}${bindir}
2133install -m 0755 helloworld ${D}${bindir} }
2134
2135By default, the ``helloworld``, ``helloworld-dbg``, and
2136``helloworld-dev`` packages are built. For information on how to
2137customize the packaging process, see the "`Splitting an Application into
2138Multiple Packages <#splitting-an-application-into-multiple-packages>`__"
2139section.
2140
2141.. _new-recipe-autotooled-package:
2142
2143Autotooled Package
2144~~~~~~~~~~~~~~~~~~
2145
2146Applications that use Autotools such as ``autoconf`` and ``automake``
2147require a recipe that has a source archive listed in ``SRC_URI`` and
2148also inherit the
2149```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class,
2150which contains the definitions of all the steps needed to build an
2151Autotool-based application. The result of the build is automatically
2152packaged. And, if the application uses NLS for localization, packages
2153with local information are generated (one package per language).
2154Following is one example: (``hello_2.3.bb``) SUMMARY = "GNU Helloworld
2155application" SECTION = "examples" LICENSE = "GPLv2+" LIC_FILES_CHKSUM =
2156"file://COPYING;md5=751419260aa954499f7abaabaa882bbe" SRC_URI =
2157"${GNU_MIRROR}/hello/hello-${PV}.tar.gz" inherit autotools gettext
2158
2159The variable ``LIC_FILES_CHKSUM`` is used to track source license
2160changes as described in the "`Tracking License
2161Changes <#usingpoky-configuring-LIC_FILES_CHKSUM>`__" section in the
2162Yocto Project Overview and Concepts Manual. You can quickly create
2163Autotool-based recipes in a manner similar to the previous example.
2164
2165.. _new-recipe-makefile-based-package:
2166
2167Makefile-Based Package
2168~~~~~~~~~~~~~~~~~~~~~~
2169
2170Applications that use GNU ``make`` also require a recipe that has the
2171source archive listed in ``SRC_URI``. You do not need to add a
2172``do_compile`` step since by default BitBake starts the ``make`` command
2173to compile the application. If you need additional ``make`` options, you
2174should store them in the
2175```EXTRA_OEMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE>`__ or
2176```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
2177variables. BitBake passes these options into the GNU ``make``
2178invocation. Note that a ``do_install`` task is still required.
2179Otherwise, BitBake runs an empty ``do_install`` task by default.
2180
2181Some applications might require extra parameters to be passed to the
2182compiler. For example, the application might need an additional header
2183path. You can accomplish this by adding to the ``CFLAGS`` variable. The
2184following example shows this: CFLAGS_prepend = "-I ${S}/include "
2185
2186In the following example, ``mtd-utils`` is a makefile-based package:
2187SUMMARY = "Tools for managing memory technology devices" SECTION =
2188"base" DEPENDS = "zlib lzo e2fsprogs util-linux" HOMEPAGE =
2189"http://www.linux-mtd.infradead.org/" LICENSE = "GPLv2+"
2190LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3
2191\\
2192file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
2193# Use the latest version at 26 Oct, 2013 SRCREV =
2194"9f107132a6a073cce37434ca9cda6917dd8d866b" SRC_URI =
2195"git://git.infradead.org/mtd-utils.git \\
2196file://add-exclusion-to-mkfs-jffs2-git-2.patch \\ " PV =
2197"1.5.1+git${SRCPV}" S = "${WORKDIR}/git" EXTRA_OEMAKE = "'CC=${CC}'
2198'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include
2199-DWITHOUT_XATTR' 'BUILDDIR=${S}'" do_install () { oe_runmake install
2200DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir}
2201INCLUDEDIR=${includedir} } PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs
2202mtd-utils-misc" FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2
2203${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool"
2204FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*"
2205FILES_mtd-utils-misc = "${sbindir}/nftl\* ${sbindir}/ftl\*
2206${sbindir}/rfd\* ${sbindir}/doc\* ${sbindir}/serve_image
2207${sbindir}/recv_image" PARALLEL_MAKE = "" BBCLASSEXTEND = "native"
2208
2209Splitting an Application into Multiple Packages
2210~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2211
2212You can use the variables ``PACKAGES`` and ``FILES`` to split an
2213application into multiple packages.
2214
2215Following is an example that uses the ``libxpm`` recipe. By default,
2216this recipe generates a single package that contains the library along
2217with a few binaries. You can modify the recipe to split the binaries
2218into separate packages: require xorg-lib-common.inc SUMMARY = "Xpm: X
2219Pixmap extension library" LICENSE = "BSD" LIC_FILES_CHKSUM =
2220"file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" DEPENDS +=
2221"libxext libsm libxt" PE = "1" XORG_PN = "libXpm" PACKAGES =+ "sxpm
2222cxpm" FILES_cxpm = "${bindir}/cxpm" FILES_sxpm = "${bindir}/sxpm"
2223
2224In the previous example, we want to ship the ``sxpm`` and ``cxpm``
2225binaries in separate packages. Since ``bindir`` would be packaged into
2226the main ``PN`` package by default, we prepend the ``PACKAGES`` variable
2227so additional package names are added to the start of list. This results
2228in the extra ``FILES_*`` variables then containing information that
2229define which files and directories go into which packages. Files
2230included by earlier packages are skipped by latter packages. Thus, the
2231main ``PN`` package does not include the above listed files.
2232
2233Packaging Externally Produced Binaries
2234~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2235
2236Sometimes, you need to add pre-compiled binaries to an image. For
2237example, suppose that binaries for proprietary code exist, which are
2238created by a particular division of a company. Your part of the company
2239needs to use those binaries as part of an image that you are building
2240using the OpenEmbedded build system. Since you only have the binaries
2241and not the source code, you cannot use a typical recipe that expects to
2242fetch the source specified in
2243```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ and then compile it.
2244
2245One method is to package the binaries and then install them as part of
2246the image. Generally, it is not a good idea to package binaries since,
2247among other things, it can hinder the ability to reproduce builds and
2248could lead to compatibility problems with ABI in the future. However,
2249sometimes you have no choice.
2250
2251The easiest solution is to create a recipe that uses the
2252```bin_package`` <&YOCTO_DOCS_REF_URL;#ref-classes-bin-package>`__ class
2253and to be sure that you are using default locations for build artifacts.
2254In most cases, the ``bin_package`` class handles "skipping" the
2255configure and compile steps as well as sets things up to grab packages
2256from the appropriate area. In particular, this class sets ``noexec`` on
2257both the ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__
2258and ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ tasks,
2259sets ``FILES_${PN}`` to "/" so that it picks up all files, and sets up a
2260```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task, which
2261effectively copies all files from ``${S}`` to ``${D}``. The
2262``bin_package`` class works well when the files extracted into ``${S}``
2263are already laid out in the way they should be laid out on the target.
2264For more information on these variables, see the
2265```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__,
2266```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__,
2267```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__, and
2268```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__ variables in the Yocto Project
2269Reference Manual's variable glossary.
2270
2271.. note::
2272
2273 - Using ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ is a good
2274 idea even for components distributed in binary form, and is often
2275 necessary for shared libraries. For a shared library, listing the
2276 library dependencies in ``DEPENDS`` makes sure that the libraries
2277 are available in the staging sysroot when other recipes link
2278 against the library, which might be necessary for successful
2279 linking.
2280
2281 - Using ``DEPENDS`` also allows runtime dependencies between
2282 packages to be added automatically. See the "`Automatically Added
2283 Runtime
2284 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
2285 section in the Yocto Project Overview and Concepts Manual for more
2286 information.
2287
2288If you cannot use the ``bin_package`` class, you need to be sure you are
2289doing the following:
2290
2291- Create a recipe where the
2292 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ and
2293 ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ tasks do
2294 nothing: It is usually sufficient to just not define these tasks in
2295 the recipe, because the default implementations do nothing unless a
2296 Makefile is found in
2297 ``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``.
2298
2299 If ``${S}`` might contain a Makefile, or if you inherit some class
2300 that replaces ``do_configure`` and ``do_compile`` with custom
2301 versions, then you can use the
2302 ``[``\ ```noexec`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__\ ``]``
2303 flag to turn the tasks into no-ops, as follows: do_configure[noexec]
2304 = "1" do_compile[noexec] = "1" Unlike
2305 ```deleting the tasks`` <&YOCTO_DOCS_BB_URL;#deleting-a-task>`__,
2306 using the flag preserves the dependency chain from the
2307 ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__,
2308 ```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__, and
2309 ```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ tasks to the
2310 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task.
2311
2312- Make sure your ``do_install`` task installs the binaries
2313 appropriately.
2314
2315- Ensure that you set up ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__
2316 (usually
2317 ``FILES_${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``) to
2318 point to the files you have installed, which of course depends on
2319 where you have installed them and whether those files are in
2320 different locations than the defaults.
2321
2322Following Recipe Style Guidelines
2323---------------------------------
2324
2325When writing recipes, it is good to conform to existing style
2326guidelines. The `OpenEmbedded
2327Styleguide <http://www.openembedded.org/wiki/Styleguide>`__ wiki page
2328provides rough guidelines for preferred recipe style.
2329
2330It is common for existing recipes to deviate a bit from this style.
2331However, aiming for at least a consistent style is a good idea. Some
2332practices, such as omitting spaces around ``=`` operators in assignments
2333or ordering recipe components in an erratic way, are widely seen as poor
2334style.
2335
2336Recipe Syntax
2337-------------
2338
2339Understanding recipe file syntax is important for writing recipes. The
2340following list overviews the basic items that make up a BitBake recipe
2341file. For more complete BitBake syntax descriptions, see the "`Syntax
2342and Operators <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata>`__"
2343chapter of the BitBake User Manual.
2344
2345- *Variable Assignments and Manipulations:* Variable assignments allow
2346 a value to be assigned to a variable. The assignment can be static
2347 text or might include the contents of other variables. In addition to
2348 the assignment, appending and prepending operations are also
2349 supported.
2350
2351 The following example shows some of the ways you can use variables in
2352 recipes: S = "${WORKDIR}/postfix-${PV}" CFLAGS += "-DNO_ASM"
2353 SRC_URI_append = " file://fixup.patch"
2354
2355- *Functions:* Functions provide a series of actions to be performed.
2356 You usually use functions to override the default implementation of a
2357 task function or to complement a default function (i.e. append or
2358 prepend to an existing function). Standard functions use ``sh`` shell
2359 syntax, although access to OpenEmbedded variables and internal
2360 methods are also available.
2361
2362 The following is an example function from the ``sed`` recipe:
2363 do_install () { autotools_do_install install -d ${D}${base_bindir} mv
2364 ${D}${bindir}/sed ${D}${base_bindir}/sed rmdir ${D}${bindir}/ } It is
2365 also possible to implement new functions that are called between
2366 existing tasks as long as the new functions are not replacing or
2367 complementing the default functions. You can implement functions in
2368 Python instead of shell. Both of these options are not seen in the
2369 majority of recipes.
2370
2371- *Keywords:* BitBake recipes use only a few keywords. You use keywords
2372 to include common functions (``inherit``), load parts of a recipe
2373 from other files (``include`` and ``require``) and export variables
2374 to the environment (``export``).
2375
2376 The following example shows the use of some of these keywords: export
2377 POSTCONF = "${STAGING_BINDIR}/postconf" inherit autoconf require
2378 otherfile.inc
2379
2380- *Comments (#):* Any lines that begin with the hash character (``#``)
2381 are treated as comment lines and are ignored: # This is a comment
2382
2383This next list summarizes the most important and most commonly used
2384parts of the recipe syntax. For more information on these parts of the
2385syntax, you can reference the `Syntax and
2386Operators <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata>`__ chapter
2387in the BitBake User Manual.
2388
2389- *Line Continuation (\):* Use the backward slash (``\``) character to
2390 split a statement over multiple lines. Place the slash character at
2391 the end of the line that is to be continued on the next line: VAR =
2392 "A really long \\ line"
2393
2394 .. note::
2395
2396 You cannot have any characters including spaces or tabs after the
2397 slash character.
2398
2399- *Using Variables (${VARNAME}):* Use the ``${VARNAME}`` syntax to
2400 access the contents of a variable: SRC_URI =
2401 "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
2402
2403 .. note::
2404
2405 It is important to understand that the value of a variable
2406 expressed in this form does not get substituted automatically. The
2407 expansion of these expressions happens on-demand later (e.g.
2408 usually when a function that makes reference to the variable
2409 executes). This behavior ensures that the values are most
2410 appropriate for the context in which they are finally used. On the
2411 rare occasion that you do need the variable expression to be
2412 expanded immediately, you can use the
2413 :=
2414 operator instead of
2415 =
2416 when you make the assignment, but this is not generally needed.
2417
2418- *Quote All Assignments ("value"):* Use double quotes around values in
2419 all variable assignments (e.g. ``"value"``). Following is an example:
2420 VAR1 = "${OTHERVAR}" VAR2 = "The version is ${PV}"
2421
2422- *Conditional Assignment (?=):* Conditional assignment is used to
2423 assign a value to a variable, but only when the variable is currently
2424 unset. Use the question mark followed by the equal sign (``?=``) to
2425 make a "soft" assignment used for conditional assignment. Typically,
2426 "soft" assignments are used in the ``local.conf`` file for variables
2427 that are allowed to come through from the external environment.
2428
2429 Here is an example where ``VAR1`` is set to "New value" if it is
2430 currently empty. However, if ``VAR1`` has already been set, it
2431 remains unchanged: VAR1 ?= "New value" In this next example, ``VAR1``
2432 is left with the value "Original value": VAR1 = "Original value" VAR1
2433 ?= "New value"
2434
2435- *Appending (+=):* Use the plus character followed by the equals sign
2436 (``+=``) to append values to existing variables.
2437
2438 .. note::
2439
2440 This operator adds a space between the existing content of the
2441 variable and the new content.
2442
2443 Here is an example: SRC_URI += "file://fix-makefile.patch"
2444
2445- *Prepending (=+):* Use the equals sign followed by the plus character
2446 (``=+``) to prepend values to existing variables.
2447
2448 .. note::
2449
2450 This operator adds a space between the new content and the
2451 existing content of the variable.
2452
2453 Here is an example: VAR =+ "Starts"
2454
2455- *Appending (_append):* Use the ``_append`` operator to append values
2456 to existing variables. This operator does not add any additional
2457 space. Also, the operator is applied after all the ``+=``, and ``=+``
2458 operators have been applied and after all ``=`` assignments have
2459 occurred.
2460
2461 The following example shows the space being explicitly added to the
2462 start to ensure the appended value is not merged with the existing
2463 value: SRC_URI_append = " file://fix-makefile.patch" You can also use
2464 the ``_append`` operator with overrides, which results in the actions
2465 only being performed for the specified target or machine:
2466 SRC_URI_append_sh4 = " file://fix-makefile.patch"
2467
2468- *Prepending (_prepend):* Use the ``_prepend`` operator to prepend
2469 values to existing variables. This operator does not add any
2470 additional space. Also, the operator is applied after all the ``+=``,
2471 and ``=+`` operators have been applied and after all ``=``
2472 assignments have occurred.
2473
2474 The following example shows the space being explicitly added to the
2475 end to ensure the prepended value is not merged with the existing
2476 value: CFLAGS_prepend = "-I${S}/myincludes " You can also use the
2477 ``_prepend`` operator with overrides, which results in the actions
2478 only being performed for the specified target or machine:
2479 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
2480
2481- *Overrides:* You can use overrides to set a value conditionally,
2482 typically based on how the recipe is being built. For example, to set
2483 the ```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable's
2484 value to "standard/base" for any target
2485 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__, except for
2486 qemuarm where it should be set to "standard/arm-versatile-926ejs",
2487 you would do the following: KBRANCH = "standard/base" KBRANCH_qemuarm
2488 = "standard/arm-versatile-926ejs" Overrides are also used to separate
2489 alternate values of a variable in other situations. For example, when
2490 setting variables such as
2491 ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ and
2492 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ that are
2493 specific to individual packages produced by a recipe, you should
2494 always use an override that specifies the name of the package.
2495
2496- *Indentation:* Use spaces for indentation rather than than tabs. For
2497 shell functions, both currently work. However, it is a policy
2498 decision of the Yocto Project to use tabs in shell functions. Realize
2499 that some layers have a policy to use spaces for all indentation.
2500
2501- *Using Python for Complex Operations:* For more advanced processing,
2502 it is possible to use Python code during variable assignments (e.g.
2503 search and replacement on a variable).
2504
2505 You indicate Python code using the ``${@python_code}`` syntax for the
2506 variable assignment: SRC_URI =
2507 "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.',
2508 '')}.tgz
2509
2510- *Shell Function Syntax:* Write shell functions as if you were writing
2511 a shell script when you describe a list of actions to take. You
2512 should ensure that your script works with a generic ``sh`` and that
2513 it does not require any ``bash`` or other shell-specific
2514 functionality. The same considerations apply to various system
2515 utilities (e.g. ``sed``, ``grep``, ``awk``, and so forth) that you
2516 might wish to use. If in doubt, you should check with multiple
2517 implementations - including those from BusyBox.
2518
2519.. _platdev-newmachine:
2520
2521Adding a New Machine
2522====================
2523
2524Adding a new machine to the Yocto Project is a straightforward process.
2525This section describes how to add machines that are similar to those
2526that the Yocto Project already supports.
2527
2528.. note::
2529
2530 Although well within the capabilities of the Yocto Project, adding a
2531 totally new architecture might require changes to
2532 gcc/glibc
2533 and to the site information, which is beyond the scope of this
2534 manual.
2535
2536For a complete example that shows how to add a new machine, see the
2537"`Creating a New BSP Layer Using the ``bitbake-layers``
2538Script <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
2539section in the Yocto Project Board Support Package (BSP) Developer's
2540Guide.
2541
2542.. _platdev-newmachine-conffile:
2543
2544Adding the Machine Configuration File
2545-------------------------------------
2546
2547To add a new machine, you need to add a new machine configuration file
2548to the layer's ``conf/machine`` directory. This configuration file
2549provides details about the device you are adding.
2550
2551The OpenEmbedded build system uses the root name of the machine
2552configuration file to reference the new machine. For example, given a
2553machine configuration file named ``crownbay.conf``, the build system
2554recognizes the machine as "crownbay".
2555
2556The most important variables you must set in your machine configuration
2557file or include from a lower-level configuration file are as follows:
2558
2559- ``TARGET_ARCH`` (e.g. "arm")
2560
2561- ``PREFERRED_PROVIDER_virtual/kernel``
2562
2563- ``MACHINE_FEATURES`` (e.g. "apm screen wifi")
2564
2565You might also need these variables:
2566
2567- ``SERIAL_CONSOLES`` (e.g. "115200;ttyS0 115200;ttyS1")
2568
2569- ``KERNEL_IMAGETYPE`` (e.g. "zImage")
2570
2571- ``IMAGE_FSTYPES`` (e.g. "tar.gz jffs2")
2572
2573You can find full details on these variables in the reference section.
2574You can leverage existing machine ``.conf`` files from
2575``meta-yocto-bsp/conf/machine/``.
2576
2577.. _platdev-newmachine-kernel:
2578
2579Adding a Kernel for the Machine
2580-------------------------------
2581
2582The OpenEmbedded build system needs to be able to build a kernel for the
2583machine. You need to either create a new kernel recipe for this machine,
2584or extend an existing kernel recipe. You can find several kernel recipe
2585examples in the Source Directory at ``meta/recipes-kernel/linux`` that
2586you can use as references.
2587
2588If you are creating a new kernel recipe, normal recipe-writing rules
2589apply for setting up a ``SRC_URI``. Thus, you need to specify any
2590necessary patches and set ``S`` to point at the source code. You need to
2591create a ``do_configure`` task that configures the unpacked kernel with
2592a ``defconfig`` file. You can do this by using a ``make defconfig``
2593command or, more commonly, by copying in a suitable ``defconfig`` file
2594and then running ``make oldconfig``. By making use of ``inherit kernel``
2595and potentially some of the ``linux-*.inc`` files, most other
2596functionality is centralized and the defaults of the class normally work
2597well.
2598
2599If you are extending an existing kernel recipe, it is usually a matter
2600of adding a suitable ``defconfig`` file. The file needs to be added into
2601a location similar to ``defconfig`` files used for other machines in a
2602given kernel recipe. A possible way to do this is by listing the file in
2603the ``SRC_URI`` and adding the machine to the expression in
2604``COMPATIBLE_MACHINE``: COMPATIBLE_MACHINE = '(qemux86|qemumips)' For
2605more information on ``defconfig`` files, see the "`Changing the
2606Configuration <&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration>`__"
2607section in the Yocto Project Linux Kernel Development Manual.
2608
2609.. _platdev-newmachine-formfactor:
2610
2611Adding a Formfactor Configuration File
2612--------------------------------------
2613
2614A formfactor configuration file provides information about the target
2615hardware for which the image is being built and information that the
2616build system cannot obtain from other sources such as the kernel. Some
2617examples of information contained in a formfactor configuration file
2618include framebuffer orientation, whether or not the system has a
2619keyboard, the positioning of the keyboard in relation to the screen, and
2620the screen resolution.
2621
2622The build system uses reasonable defaults in most cases. However, if
2623customization is necessary, you need to create a ``machconfig`` file in
2624the ``meta/recipes-bsp/formfactor/files`` directory. This directory
2625contains directories for specific machines such as ``qemuarm`` and
2626``qemux86``. For information about the settings available and the
2627defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file
2628found in the same area.
2629
2630Following is an example for "qemuarm" machine: HAVE_TOUCHSCREEN=1
2631HAVE_KEYBOARD=1 DISPLAY_CAN_ROTATE=0 DISPLAY_ORIENTATION=0
2632#DISPLAY_WIDTH_PIXELS=640 #DISPLAY_HEIGHT_PIXELS=480 #DISPLAY_BPP=16
2633DISPLAY_DPI=150 DISPLAY_SUBPIXEL_ORDER=vrgb
2634
2635.. _gs-upgrading-recipes:
2636
2637Upgrading Recipes
2638=================
2639
2640Over time, upstream developers publish new versions for software built
2641by layer recipes. It is recommended to keep recipes up-to-date with
2642upstream version releases.
2643
2644While several methods exist that allow you upgrade a recipe, you might
2645consider checking on the upgrade status of a recipe first. You can do so
2646using the ``devtool check-upgrade-status`` command. See the "`Checking
2647on the Upgrade Status of a
2648Recipe <&YOCTO_DOCS_REF_URL;#devtool-checking-on-the-upgrade-status-of-a-recipe>`__"
2649section in the Yocto Project Reference Manual for more information.
2650
2651The remainder of this section describes three ways you can upgrade a
2652recipe. You can use the Automated Upgrade Helper (AUH) to set up
2653automatic version upgrades. Alternatively, you can use
2654``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
2655you can manually upgrade a recipe by editing the recipe itself.
2656
2657.. _gs-using-the-auto-upgrade-helper:
2658
2659Using the Auto Upgrade Helper (AUH)
2660-----------------------------------
2661
2662The AUH utility works in conjunction with the OpenEmbedded build system
2663in order to automatically generate upgrades for recipes based on new
2664versions being published upstream. Use AUH when you want to create a
2665service that performs the upgrades automatically and optionally sends
2666you an email with the results.
2667
2668AUH allows you to update several recipes with a single use. You can also
2669optionally perform build and integration tests using images with the
2670results saved to your hard drive and emails of results optionally sent
2671to recipe maintainers. Finally, AUH creates Git commits with appropriate
2672commit messages in the layer's tree for the changes made to recipes.
2673
2674.. note::
2675
2676 Conditions do exist when you should not use AUH to upgrade recipes
2677 and you should instead use either
2678 devtool upgrade
2679 or upgrade your recipes manually:
2680
2681 - When AUH cannot complete the upgrade sequence. This situation
2682 usually results because custom patches carried by the recipe
2683 cannot be automatically rebased to the new version. In this case,
2684 ``devtool upgrade`` allows you to manually resolve conflicts.
2685
2686 - When for any reason you want fuller control over the upgrade
2687 process. For example, when you want special arrangements for
2688 testing.
2689
2690The following steps describe how to set up the AUH utility:
2691
26921. *Be Sure the Development Host is Set Up:* You need to be sure that
2693 your development host is set up to use the Yocto Project. For
2694 information on how to set up your host, see the "`Preparing the Build
2695 Host <#dev-preparing-the-build-host>`__" section.
2696
26972. *Make Sure Git is Configured:* The AUH utility requires Git to be
2698 configured because AUH uses Git to save upgrades. Thus, you must have
2699 Git user and email configured. The following command shows your
2700 configurations: $ git config --list If you do not have the user and
2701 email configured, you can use the following commands to do so: $ git
2702 config --global user.name some_name $ git config --global user.email
2703 username@domain.com
2704
27053. *Clone the AUH Repository:* To use AUH, you must clone the repository
2706 onto your development host. The following command uses Git to create
2707 a local copy of the repository on your system: $ git clone
2708 git://git.yoctoproject.org/auto-upgrade-helper Cloning into
2709 'auto-upgrade-helper'... remote: Counting objects: 768, done. remote:
2710 Compressing objects: 100% (300/300), done. remote: Total 768 (delta
2711 499), reused 703 (delta 434) Receiving objects: 100% (768/768),
2712 191.47 KiB \| 98.00 KiB/s, done. Resolving deltas: 100% (499/499),
2713 done. Checking connectivity... done. AUH is not part of the
2714 `OpenEmbedded-Core (OE-Core) <&YOCTO_DOCS_REF_URL;#oe-core>`__ or
2715 `Poky <&YOCTO_DOCS_REF_URL;#poky>`__ repositories.
2716
27174. *Create a Dedicated Build Directory:* Run the
2718 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__
2719 script to create a fresh build directory that you use exclusively for
2720 running the AUH utility: $ cd ~/poky $ source oe-init-build-env
2721 your_AUH_build_directory Re-using an existing build directory and its
2722 configurations is not recommended as existing settings could cause
2723 AUH to fail or behave undesirably.
2724
27255. *Make Configurations in Your Local Configuration File:* Several
2726 settings need to exist in the ``local.conf`` file in the build
2727 directory you just created for AUH. Make these following
2728 configurations:
2729
2730 - If you want to enable `Build
2731 History <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__,
2732 which is optional, you need the following lines in the
2733 ``conf/local.conf`` file: INHERIT =+ "buildhistory"
2734 BUILDHISTORY_COMMIT = "1" With this configuration and a successful
2735 upgrade, a build history "diff" file appears in the
2736 ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
2737 your build directory.
2738
2739 - If you want to enable testing through the
2740 ```testimage`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__
2741 class, which is optional, you need to have the following set in
2742 your ``conf/local.conf`` file: INHERIT += "testimage"
2743
2744 .. note::
2745
2746 If your distro does not enable by default ptest, which Poky
2747 does, you need the following in your
2748 local.conf
2749 file:
2750 ::
2751
2752 DISTRO_FEATURES_append = " ptest"
2753
2754
27556. *Optionally Start a vncserver:* If you are running in a server
2756 without an X11 session, you need to start a vncserver: $ vncserver :1
2757 $ export DISPLAY=:1
2758
27597. *Create and Edit an AUH Configuration File:* You need to have the
2760 ``upgrade-helper/upgrade-helper.conf`` configuration file in your
2761 build directory. You can find a sample configuration file in the `AUH
2762 source
2763 repository <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/>`__.
2764
2765 Read through the sample file and make configurations as needed. For
2766 example, if you enabled build history in your ``local.conf`` as
2767 described earlier, you must enable it in ``upgrade-helper.conf``.
2768
2769 Also, if you are using the default ``maintainers.inc`` file supplied
2770 with Poky and located in ``meta-yocto`` and you do not set a
2771 "maintainers_whitelist" or "global_maintainer_override" in the
2772 ``upgrade-helper.conf`` configuration, and you specify "-e all" on
2773 the AUH command-line, the utility automatically sends out emails to
2774 all the default maintainers. Please avoid this.
2775
2776This next set of examples describes how to use the AUH:
2777
2778- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
2779 following form: $ upgrade-helper.py recipe_name For example, this
2780 command upgrades the ``xmodmap`` recipe: $ upgrade-helper.py xmodmap
2781
2782- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
2783 specific recipe to a particular version, use the following form: $
2784 upgrade-helper.py recipe_name -t version For example, this command
2785 upgrades the ``xmodmap`` recipe to version 1.2.3: $ upgrade-helper.py
2786 xmodmap -t 1.2.3
2787
2788- *Upgrading all Recipes to the Latest Versions and Suppressing Email
2789 Notifications:* To upgrade all recipes to their most recent versions
2790 and suppress the email notifications, use the following command: $
2791 upgrade-helper.py all
2792
2793- *Upgrading all Recipes to the Latest Versions and Send Email
2794 Notifications:* To upgrade all recipes to their most recent versions
2795 and send email messages to maintainers for each attempted recipe as
2796 well as a status email, use the following command: $
2797 upgrade-helper.py -e all
2798
2799Once you have run the AUH utility, you can find the results in the AUH
2800build directory: ${BUILDDIR}/upgrade-helper/timestamp The AUH utility
2801also creates recipe update commits from successful upgrade attempts in
2802the layer tree.
2803
2804You can easily set up to run the AUH utility on a regular basis by using
2805a cron job. See the
2806```weeklyjob.sh`` <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh>`__
2807file distributed with the utility for an example.
2808
2809.. _gs-using-devtool-upgrade:
2810
2811Using ``devtool upgrade``
2812-------------------------
2813
2814As mentioned earlier, an alternative method for upgrading recipes to
2815newer versions is to use
2816```devtool upgrade`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__.
2817You can read about ``devtool upgrade`` in general in the "`Use
2818``devtool upgrade`` to Create a Version of the Recipe that Supports a
2819Newer Version of the
2820Software <&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software>`__"
2821section in the Yocto Project Application Development and the Extensible
2822Software Development Kit (eSDK) Manual.
2823
2824To see all the command-line options available with ``devtool upgrade``,
2825use the following help command: $ devtool upgrade -h
2826
2827If you want to find out what version a recipe is currently at upstream
2828without any attempt to upgrade your local version of the recipe, you can
2829use the following command: $ devtool latest-version recipe_name
2830
2831As mentioned in the previous section describing AUH, ``devtool upgrade``
2832works in a less-automated manner than AUH. Specifically,
2833``devtool upgrade`` only works on a single recipe that you name on the
2834command line, cannot perform build and integration testing using images,
2835and does not automatically generate commits for changes in the source
2836tree. Despite all these "limitations", ``devtool upgrade`` updates the
2837recipe file to the new upstream version and attempts to rebase custom
2838patches contained by the recipe as needed.
2839
2840.. note::
2841
2842 AUH uses much of
2843 devtool upgrade
2844 behind the scenes making AUH somewhat of a "wrapper" application for
2845 devtool upgrade
2846 .
2847
2848A typical scenario involves having used Git to clone an upstream
2849repository that you use during build operations. Because you are (or
2850have) built the recipe in the past, the layer is likely added to your
2851configuration already. If for some reason, the layer is not added, you
2852could add it easily using the
2853```bitbake-layers`` <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__
2854script. For example, suppose you use the ``nano.bb`` recipe from the
2855``meta-oe`` layer in the ``meta-openembedded`` repository. For this
2856example, assume that the layer has been cloned into following area:
2857/home/scottrif/meta-openembedded The following command from your `Build
2858Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ adds the layer to
2859your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``): $
2860bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe NOTE:
2861Starting bitbake server... Parsing recipes: 100%
2862\|##########################################\| Time: 0:00:55 Parsing of
28631431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56
2864skipped, 0 masked, 0 errors. Removing 12 recipes from the x86_64
2865sysroot: 100% \|##############\| Time: 0:00:00 Removing 1 recipes from
2866the x86_64_i586 sysroot: 100% \|##########\| Time: 0:00:00 Removing 5
2867recipes from the i586 sysroot: 100% \|#################\| Time: 0:00:00
2868Removing 5 recipes from the qemux86 sysroot: 100% \|##############\|
2869Time: 0:00:00 For this example, assume that the ``nano.bb`` recipe that
2870is upstream has a 2.9.3 version number. However, the version in the
2871local repository is 2.7.4. The following command from your build
2872directory automatically upgrades the recipe for you:
2873
2874.. note::
2875
2876 Using the
2877 -V
2878 option is not necessary. Omitting the version number causes
2879 devtool upgrade
2880 to upgrade the recipe to the most recent version.
2881
2882$ devtool upgrade nano -V 2.9.3 NOTE: Starting bitbake server... NOTE:
2883Creating workspace layer in /home/scottrif/poky/build/workspace Parsing
2884recipes: 100% \|##########################################\| Time:
28850:00:46 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040
2886targets, 56 skipped, 0 masked, 0 errors. NOTE: Extracting current
2887version source... NOTE: Resolving any missing task queue dependencies .
2888. . NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE:
2889Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun
2890and all succeeded. Adding changed files: 100%
2891\|#####################################\| Time: 0:00:00 NOTE: Upgraded
2892source extracted to /home/scottrif/poky/build/workspace/sources/nano
2893NOTE: New recipe is
2894/home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
2895Continuing with this example, you can use ``devtool build`` to build the
2896newly upgraded recipe: $ devtool build nano NOTE: Starting bitbake
2897server... Loading cache: 100%
2898\|################################################################################################\|
2899Time: 0:00:01 Loaded 2040 entries from dependency cache. Parsing
2900recipes: 100%
2901\|##############################################################################################\|
2902Time: 0:00:00 Parsing of 1432 .bb files complete (1431 cached, 1
2903parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving
2904any missing task queue dependencies . . . NOTE: Executing SetScene Tasks
2905NOTE: Executing RunQueue Tasks NOTE: nano: compiling from external
2906source tree /home/scottrif/poky/build/workspace/sources/nano NOTE: Tasks
2907Summary: Attempted 520 tasks of which 304 didn't need to be rerun and
2908all succeeded. Within the ``devtool upgrade`` workflow, opportunity
2909exists to deploy and test your rebuilt software. For this example,
2910however, running ``devtool finish`` cleans up the workspace once the
2911source in your workspace is clean. This usually means using Git to stage
2912and submit commits for the changes generated by the upgrade process.
2913
2914Once the tree is clean, you can clean things up in this example with the
2915following command from the ``${BUILDDIR}/workspace/sources/nano``
2916directory: $ devtool finish nano meta-oe NOTE: Starting bitbake
2917server... Loading cache: 100%
2918\|################################################################################################\|
2919Time: 0:00:00 Loaded 2040 entries from dependency cache. Parsing
2920recipes: 100%
2921\|##############################################################################################\|
2922Time: 0:00:01 Parsing of 1432 .bb files complete (1431 cached, 1
2923parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. NOTE: Adding new
2924patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch NOTE:
2925Updating recipe nano_2.9.3.bb NOTE: Removing file
2926/home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
2927NOTE: Moving recipe file to
2928/home/scottrif/meta-openembedded/meta-oe/recipes-support/nano NOTE:
2929Leaving source tree /home/scottrif/poky/build/workspace/sources/nano
2930as-is; if you no longer need it then please delete it manually Using the
2931``devtool finish`` command cleans up the workspace and creates a patch
2932file based on your commits. The tool puts all patch files back into the
2933source directory in a sub-directory named ``nano`` in this case.
2934
2935.. _dev-manually-upgrading-a-recipe:
2936
2937Manually Upgrading a Recipe
2938---------------------------
2939
2940If for some reason you choose not to upgrade recipes using the `Auto
2941Upgrade Helper (AUH) <#gs-using-the-auto-upgrade-helper>`__ or by using
2942```devtool upgrade`` <#gs-using-devtool-upgrade>`__, you can manually
2943edit the recipe files to upgrade the versions.
2944
2945.. note::
2946
2947 Manually updating multiple recipes scales poorly and involves many
2948 steps. The recommendation to upgrade recipe versions is through AUH
2949 or
2950 devtool upgrade
2951 , both of which automate some steps and provide guidance for others
2952 needed for the manual process.
2953
2954To manually upgrade recipe versions, follow these general steps:
2955
29561. *Change the Version:* Rename the recipe such that the version (i.e.
2957 the ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ part of the recipe name)
2958 changes appropriately. If the version is not part of the recipe name,
2959 change the value as it is set for ``PV`` within the recipe itself.
2960
29612. *Update ``SRCREV`` if Needed:* If the source code your recipe builds
2962 is fetched from Git or some other version control system, update
2963 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ to point to the
2964 commit hash that matches the new version.
2965
29663. *Build the Software:* Try to build the recipe using BitBake. Typical
2967 build failures include the following:
2968
2969 - License statements were updated for the new version. For this
2970 case, you need to review any changes to the license and update the
2971 values of ```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ and
2972 ```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
2973 as needed.
2974
2975 .. note::
2976
2977 License changes are often inconsequential. For example, the
2978 license text's copyright year might have changed.
2979
2980 - Custom patches carried by the older version of the recipe might
2981 fail to apply to the new version. For these cases, you need to
2982 review the failures. Patches might not be necessary for the new
2983 version of the software if the upgraded version has fixed those
2984 issues. If a patch is necessary and failing, you need to rebase it
2985 into the new version.
2986
29874. *Optionally Attempt to Build for Several Architectures:* Once you
2988 successfully build the new software for a given architecture, you
2989 could test the build for other architectures by changing the
2990 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable and
2991 rebuilding the software. This optional step is especially important
2992 if the recipe is to be released publicly.
2993
29945. *Check the Upstream Change Log or Release Notes:* Checking both these
2995 reveals if new features exist that could break
2996 backwards-compatibility. If so, you need to take steps to mitigate or
2997 eliminate that situation.
2998
29996. *Optionally Create a Bootable Image and Test:* If you want, you can
3000 test the new software by booting it onto actual hardware.
3001
30027. *Create a Commit with the Change in the Layer Repository:* After all
3003 builds work and any testing is successful, you can create commits for
3004 any changes in the layer holding your upgraded recipe.
3005
3006.. _finding-the-temporary-source-code:
3007
3008Finding Temporary Source Code
3009=============================
3010
3011You might find it helpful during development to modify the temporary
3012source code used by recipes to build packages. For example, suppose you
3013are developing a patch and you need to experiment a bit to figure out
3014your solution. After you have initially built the package, you can
3015iteratively tweak the source code, which is located in the `Build
3016Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, and then you can
3017force a re-compile and quickly test your altered code. Once you settle
3018on a solution, you can then preserve your changes in the form of
3019patches.
3020
3021During a build, the unpacked temporary source code used by recipes to
3022build packages is available in the Build Directory as defined by the
3023```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ variable. Below is the default
3024value for the ``S`` variable as defined in the
3025``meta/conf/bitbake.conf`` configuration file in the `Source
3026Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__: S =
3027"${WORKDIR}/${BP}" You should be aware that many recipes override the
3028``S`` variable. For example, recipes that fetch their source from Git
3029usually set ``S`` to ``${WORKDIR}/git``.
3030
3031.. note::
3032
3033 The
3034 BP
3035 represents the base recipe name, which consists of the name and
3036 version:
3037 ::
3038
3039 BP = "${BPN}-${PV}"
3040
3041
3042The path to the work directory for the recipe
3043(```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__) is defined as
3044follows:
3045${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} The
3046actual directory depends on several things:
3047
3048- ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__: The top-level build
3049 output directory.
3050
3051- ```MULTIMACH_TARGET_SYS`` <&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS>`__:
3052 The target system identifier.
3053
3054- ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__: The recipe name.
3055
3056- ```EXTENDPE`` <&YOCTO_DOCS_REF_URL;#var-EXTENDPE>`__: The epoch - (if
3057 ```PE`` <&YOCTO_DOCS_REF_URL;#var-PE>`__ is not specified, which is
3058 usually the case for most recipes, then ``EXTENDPE`` is blank).
3059
3060- ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The recipe version.
3061
3062- ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__: The recipe revision.
3063
3064As an example, assume a Source Directory top-level folder named
3065``poky``, a default Build Directory at ``poky/build``, and a
3066``qemux86-poky-linux`` machine target system. Furthermore, suppose your
3067recipe is named ``foo_1.3.0.bb``. In this case, the work directory the
3068build system uses to build the package would be as follows:
3069poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
3070
3071.. _using-a-quilt-workflow:
3072
3073Using Quilt in Your Workflow
3074============================
3075
3076`Quilt <http://savannah.nongnu.org/projects/quilt>`__ is a powerful tool
3077that allows you to capture source code changes without having a clean
3078source tree. This section outlines the typical workflow you can use to
3079modify source code, test changes, and then preserve the changes in the
3080form of a patch all using Quilt.
3081
3082.. note::
3083
3084 With regard to preserving changes to source files, if you clean a
3085 recipe or have
3086 rm_work
3087 enabled, the
3088 devtool
3089 workflow
3090 as described in the Yocto Project Application Development and the
3091 Extensible Software Development Kit (eSDK) manual is a safer
3092 development flow than the flow that uses Quilt.
3093
3094Follow these general steps:
3095
30961. *Find the Source Code:* Temporary source code used by the
3097 OpenEmbedded build system is kept in the `Build
3098 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. See the
3099 "`Finding Temporary Source
3100 Code <#finding-the-temporary-source-code>`__" section to learn how to
3101 locate the directory that has the temporary source code for a
3102 particular package.
3103
31042. *Change Your Working Directory:* You need to be in the directory that
3105 has the temporary source code. That directory is defined by the
3106 ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ variable.
3107
31083. *Create a New Patch:* Before modifying source code, you need to
3109 create a new patch. To create a new patch file, use ``quilt new`` as
3110 below: $ quilt new my_changes.patch
3111
31124. *Notify Quilt and Add Files:* After creating the patch, you need to
3113 notify Quilt about the files you plan to edit. You notify Quilt by
3114 adding the files to the patch you just created: $ quilt add file1.c
3115 file2.c file3.c
3116
31175. *Edit the Files:* Make your changes in the source code to the files
3118 you added to the patch.
3119
31206. *Test Your Changes:* Once you have modified the source code, the
3121 easiest way to test your changes is by calling the ``do_compile``
3122 task as shown in the following example: $ bitbake -c compile -f
3123 package The ``-f`` or ``--force`` option forces the specified task to
3124 execute. If you find problems with your code, you can just keep
3125 editing and re-testing iteratively until things work as expected.
3126
3127 .. note::
3128
3129 All the modifications you make to the temporary source code
3130 disappear once you run the
3131 do_clean
3132 or
3133 do_cleanall
3134 tasks using BitBake (i.e.
3135 bitbake -c clean
3136 package
3137 and
3138 bitbake -c cleanall
3139 package
3140 ). Modifications will also disappear if you use the
3141 rm_work
3142 feature as described in the "
3143 Conserving Disk Space During Builds
3144 " section.
3145
31467. *Generate the Patch:* Once your changes work as expected, you need to
3147 use Quilt to generate the final patch that contains all your
3148 modifications. $ quilt refresh At this point, the
3149 ``my_changes.patch`` file has all your edits made to the ``file1.c``,
3150 ``file2.c``, and ``file3.c`` files.
3151
3152 You can find the resulting patch file in the ``patches/``
3153 subdirectory of the source (``S``) directory.
3154
31558. *Copy the Patch File:* For simplicity, copy the patch file into a
3156 directory named ``files``, which you can create in the same directory
3157 that holds the recipe (``.bb``) file or the append (``.bbappend``)
3158 file. Placing the patch here guarantees that the OpenEmbedded build
3159 system will find the patch. Next, add the patch into the ``SRC_URI``
3160 of the recipe. Here is an example: SRC_URI +=
3161 "file://my_changes.patch"
3162
3163.. _platdev-appdev-devshell:
3164
3165Using a Development Shell
3166=========================
3167
3168When debugging certain commands or even when just editing packages,
3169``devshell`` can be a useful tool. When you invoke ``devshell``, all
3170tasks up to and including
3171```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ are run for the
3172specified target. Then, a new terminal is opened and you are placed in
3173``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``, the source
3174directory. In the new terminal, all the OpenEmbedded build-related
3175environment variables are still defined so you can use commands such as
3176``configure`` and ``make``. The commands execute just as if the
3177OpenEmbedded build system were executing them. Consequently, working
3178this way can be helpful when debugging a build or preparing software to
3179be used with the OpenEmbedded build system.
3180
3181Following is an example that uses ``devshell`` on a target named
3182``matchbox-desktop``: $ bitbake matchbox-desktop -c devshell
3183
3184This command spawns a terminal with a shell prompt within the
3185OpenEmbedded build environment. The
3186```OE_TERMINAL`` <&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL>`__ variable
3187controls what type of shell is opened.
3188
3189For spawned terminals, the following occurs:
3190
3191- The ``PATH`` variable includes the cross-toolchain.
3192
3193- The ``pkgconfig`` variables find the correct ``.pc`` files.
3194
3195- The ``configure`` command finds the Yocto Project site files as well
3196 as any other necessary files.
3197
3198Within this environment, you can run configure or compile commands as if
3199they were being run by the OpenEmbedded build system itself. As noted
3200earlier, the working directory also automatically changes to the Source
3201Directory (```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__).
3202
3203To manually run a specific task using ``devshell``, run the
3204corresponding ``run.*`` script in the
3205``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/temp``
3206directory (e.g., ``run.do_configure.``\ pid). If a task's script does
3207not exist, which would be the case if the task was skipped by way of the
3208sstate cache, you can create the task by first running it outside of the
3209``devshell``: $ bitbake -c task
3210
3211.. note::
3212
3213 - Execution of a task's ``run.*`` script and BitBake's execution of
3214 a task are identical. In other words, running the script re-runs
3215 the task just as it would be run using the ``bitbake -c`` command.
3216
3217 - Any ``run.*`` file that does not have a ``.pid`` extension is a
3218 symbolic link (symlink) to the most recent version of that file.
3219
3220Remember, that the ``devshell`` is a mechanism that allows you to get
3221into the BitBake task execution environment. And as such, all commands
3222must be called just as BitBake would call them. That means you need to
3223provide the appropriate options for cross-compilation and so forth as
3224applicable.
3225
3226When you are finished using ``devshell``, exit the shell or close the
3227terminal window.
3228
3229.. note::
3230
3231 - It is worth remembering that when using ``devshell`` you need to
3232 use the full compiler name such as ``arm-poky-linux-gnueabi-gcc``
3233 instead of just using ``gcc``. The same applies to other
3234 applications such as ``binutils``, ``libtool`` and so forth.
3235 BitBake sets up environment variables such as ``CC`` to assist
3236 applications, such as ``make`` to find the correct tools.
3237
3238 - It is also worth noting that ``devshell`` still works over X11
3239 forwarding and similar situations.
3240
3241.. _platdev-appdev-devpyshell:
3242
3243Using a Development Python Shell
3244================================
3245
3246Similar to working within a development shell as described in the
3247previous section, you can also spawn and work within an interactive
3248Python development shell. When debugging certain commands or even when
3249just editing packages, ``devpyshell`` can be a useful tool. When you
3250invoke ``devpyshell``, all tasks up to and including
3251```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ are run for the
3252specified target. Then a new terminal is opened. Additionally, key
3253Python objects and code are available in the same way they are to
3254BitBake tasks, in particular, the data store 'd'. So, commands such as
3255the following are useful when exploring the data store and running
3256functions: pydevshell> d.getVar("STAGING_DIR")
3257'/media/build1/poky/build/tmp/sysroots' pydevshell>
3258d.getVar("STAGING_DIR") '${TMPDIR}/sysroots' pydevshell> d.setVar("FOO",
3259"bar") pydevshell> d.getVar("FOO") 'bar' pydevshell> d.delVar("FOO")
3260pydevshell> d.getVar("FOO") pydevshell> bb.build.exec_func("do_unpack",
3261d) pydevshell> The commands execute just as if the OpenEmbedded build
3262system were executing them. Consequently, working this way can be
3263helpful when debugging a build or preparing software to be used with the
3264OpenEmbedded build system.
3265
3266Following is an example that uses ``devpyshell`` on a target named
3267``matchbox-desktop``: $ bitbake matchbox-desktop -c devpyshell
3268
3269This command spawns a terminal and places you in an interactive Python
3270interpreter within the OpenEmbedded build environment. The
3271```OE_TERMINAL`` <&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL>`__ variable
3272controls what type of shell is opened.
3273
3274When you are finished using ``devpyshell``, you can exit the shell
3275either by using Ctrl+d or closing the terminal window.
3276
3277.. _dev-building:
3278
3279Building
3280========
3281
3282This section describes various build procedures. For example, the steps
3283needed for a simple build, a target that uses multiple configurations,
3284building an image for more than one machine, and so forth.
3285
3286.. _dev-building-a-simple-image:
3287
3288Building a Simple Image
3289-----------------------
3290
3291In the development environment, you need to build an image whenever you
3292change hardware support, add or change system libraries, or add or
3293change services that have dependencies. Several methods exist that allow
3294you to build an image within the Yocto Project. This section presents
3295the basic steps you need to build a simple image using BitBake from a
3296build host running Linux.
3297
3298.. note::
3299
3300 - For information on how to build an image using
3301 `Toaster <&YOCTO_DOCS_REF_URL;#toaster-term>`__, see the `Toaster
3302 User Manual <&YOCTO_DOCS_TOAST_URL;>`__.
3303
3304 - For information on how to use ``devtool`` to build images, see the
3305 "`Using ``devtool`` in Your SDK
3306 Workflow <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__"
3307 section in the Yocto Project Application Development and the
3308 Extensible Software Development Kit (eSDK) manual.
3309
3310 - For a quick example on how to build an image using the
3311 OpenEmbedded build system, see the `Yocto Project Quick
3312 Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
3313
3314The build process creates an entire Linux distribution from source and
3315places it in your `Build
3316Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ under
3317``tmp/deploy/images``. For detailed information on the build process
3318using BitBake, see the
3319"`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__" section in the
3320Yocto Project Overview and Concepts Manual.
3321
3322The following figure and list overviews the build process:
3323
33241. *Set up Your Host Development System to Support Development Using the
3325 Yocto Project*: See the "`Setting Up to Use the Yocto
3326 Project <#dev-manual-start>`__" section for options on how to get a
3327 build host ready to use the Yocto Project.
3328
33292. *Initialize the Build Environment:* Initialize the build environment
3330 by sourcing the build environment script (i.e.
3331 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__): $ source
3332 OE_INIT_FILE [build_dir]
3333
3334 When you use the initialization script, the OpenEmbedded build system
3335 uses ``build`` as the default Build Directory in your current work
3336 directory. You can use a build_dir argument with the script to
3337 specify a different build directory.
3338
3339 .. note::
3340
3341 A common practice is to use a different Build Directory for
3342 different targets. For example,
3343 ~/build/x86
3344 for a
3345 qemux86
3346 target, and
3347 ~/build/arm
3348 for a
3349 qemuarm
3350 target.
3351
33523. *Make Sure Your ``local.conf`` File is Correct:* Ensure the
3353 ``conf/local.conf`` configuration file, which is found in the Build
3354 Directory, is set up how you want it. This file defines many aspects
3355 of the build environment including the target machine architecture
3356 through the ``MACHINE`` variable, the packaging format used during
3357 the build
3358 (```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__),
3359 and a centralized tarball download directory through the
3360 ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ variable.
3361
33624. *Build the Image:* Build the image using the ``bitbake`` command: $
3363 bitbake target
3364
3365 .. note::
3366
3367 For information on BitBake, see the
3368 BitBake User Manual
3369 .
3370
3371 The target is the name of the recipe you want to build. Common
3372 targets are the images in ``meta/recipes-core/images``,
3373 ``meta/recipes-sato/images``, and so forth all found in the `Source
3374 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Or, the target
3375 can be the name of a recipe for a specific piece of software such as
3376 BusyBox. For more details about the images the OpenEmbedded build
3377 system supports, see the
3378 "`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
3379 Project Reference Manual.
3380
3381 As an example, the following command builds the
3382 ``core-image-minimal`` image: $ bitbake core-image-minimal Once an
3383 image has been built, it often needs to be installed. The images and
3384 kernels built by the OpenEmbedded build system are placed in the
3385 Build Directory in ``tmp/deploy/images``. For information on how to
3386 run pre-built images such as ``qemux86`` and ``qemuarm``, see the
3387 `Yocto Project Application Development and the Extensible Software
3388 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual. For
3389 information about how to install these images, see the documentation
3390 for your particular board or machine.
3391
3392.. _dev-building-images-for-multiple-targets-using-multiple-configurations:
3393
3394Building Images for Multiple Targets Using Multiple Configurations
3395------------------------------------------------------------------
3396
3397You can use a single ``bitbake`` command to build multiple images or
3398packages for different targets where each image or package requires a
3399different configuration (multiple configuration builds). The builds, in
3400this scenario, are sometimes referred to as "multiconfigs", and this
3401section uses that term throughout.
3402
3403This section describes how to set up for multiple configuration builds
3404and how to account for cross-build dependencies between the
3405multiconfigs.
3406
3407.. _dev-setting-up-and-running-a-multiple-configuration-build:
3408
3409Setting Up and Running a Multiple Configuration Build
3410~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3411
3412To accomplish a multiple configuration build, you must define each
3413target's configuration separately using a parallel configuration file in
3414the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, and you
3415must follow a required file hierarchy. Additionally, you must enable the
3416multiple configuration builds in your ``local.conf`` file.
3417
3418Follow these steps to set up and execute multiple configuration builds:
3419
3420- *Create Separate Configuration Files*: You need to create a single
3421 configuration file for each build target (each multiconfig).
3422 Minimally, each configuration file must define the machine and the
3423 temporary directory BitBake uses for the build. Suggested practice
3424 dictates that you do not overlap the temporary directories used
3425 during the builds. However, it is possible that you can share the
3426 temporary directory
3427 (```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__). For example,
3428 consider a scenario with two different multiconfigs for the same
3429 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__: "qemux86" built
3430 for two distributions such as "poky" and "poky-lsb". In this case,
3431 you might want to use the same ``TMPDIR``.
3432
3433 Here is an example showing the minimal statements needed in a
3434 configuration file for a "qemux86" target whose temporary build
3435 directory is ``tmpmultix86``: MACHINE="qemux86"
3436 TMPDIR="${TOPDIR}/tmpmultix86"
3437
3438 The location for these multiconfig configuration files is specific.
3439 They must reside in the current build directory in a sub-directory of
3440 ``conf`` named ``multiconfig``. Following is an example that defines
3441 two configuration files for the "x86" and "arm" multiconfigs:
3442
3443 The reason for this required file hierarchy is because the ``BBPATH``
3444 variable is not constructed until the layers are parsed.
3445 Consequently, using the configuration file as a pre-configuration
3446 file is not possible unless it is located in the current working
3447 directory.
3448
3449- *Add the BitBake Multi-configuration Variable to the Local
3450 Configuration File*: Use the
3451 ```BBMULTICONFIG`` <&YOCTO_DOCS_REF_URL;#var-BBMULTICONFIG>`__
3452 variable in your ``conf/local.conf`` configuration file to specify
3453 each multiconfig. Continuing with the example from the previous
3454 figure, the ``BBMULTICONFIG`` variable needs to enable two
3455 multiconfigs: "x86" and "arm" by specifying each configuration file:
3456 BBMULTICONFIG = "x86 arm"
3457
3458 .. note::
3459
3460 A "default" configuration already exists by definition. This
3461 configuration is named: "" (i.e. empty string) and is defined by
3462 the variables coming from your
3463 local.conf
3464 file. Consequently, the previous example actually adds two
3465 additional configurations to your build: "arm" and "x86" along
3466 with "".
3467
3468- *Launch BitBake*: Use the following BitBake command form to launch
3469 the multiple configuration build: $ bitbake
3470 [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] For
3471 the example in this section, the following command applies: $ bitbake
3472 mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
3473 The previous BitBake command builds a ``core-image-minimal`` image
3474 that is configured through the ``x86.conf`` configuration file, a
3475 ``core-image-sato`` image that is configured through the ``arm.conf``
3476 configuration file and a ``core-image-base`` that is configured
3477 through your ``local.conf`` configuration file.
3478
3479.. note::
3480
3481 Support for multiple configuration builds in the Yocto Project DISTRO
3482 (DISTRO_NAME) Release does not include Shared State (sstate)
3483 optimizations. Consequently, if a build uses the same object twice
3484 in, for example, two different
3485 TMPDIR
3486 directories, the build either loads from an existing sstate cache for
3487 that build at the start or builds the object fresh.
3488
3489.. _dev-enabling-multiple-configuration-build-dependencies:
3490
3491Enabling Multiple Configuration Build Dependencies
3492~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3493
3494Sometimes dependencies can exist between targets (multiconfigs) in a
3495multiple configuration build. For example, suppose that in order to
3496build a ``core-image-sato`` image for an "x86" multiconfig, the root
3497filesystem of an "arm" multiconfig must exist. This dependency is
3498essentially that the
3499```do_image`` <&YOCTO_DOCS_REF_URL;#ref-tasks-image>`__ task in the
3500``core-image-sato`` recipe depends on the completion of the
3501```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__ task of the
3502``core-image-minimal`` recipe.
3503
3504To enable dependencies in a multiple configuration build, you must
3505declare the dependencies in the recipe using the following statement
3506form: task_or_package[mcdepends] =
3507"mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
3508To better show how to use this statement, consider the example scenario
3509from the first paragraph of this section. The following statement needs
3510to be added to the recipe that builds the ``core-image-sato`` image:
3511do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" In this
3512example, the from_multiconfig is "x86". The to_multiconfig is "arm". The
3513task on which the ``do_image`` task in the recipe depends is the
3514``do_rootfs`` task from the ``core-image-minimal`` recipe associated
3515with the "arm" multiconfig.
3516
3517Once you set up this dependency, you can build the "x86" multiconfig
3518using a BitBake command as follows: $ bitbake mc:x86:core-image-sato
3519This command executes all the tasks needed to create the
3520``core-image-sato`` image for the "x86" multiconfig. Because of the
3521dependency, BitBake also executes through the ``do_rootfs`` task for the
3522"arm" multiconfig build.
3523
3524Having a recipe depend on the root filesystem of another build might not
3525seem that useful. Consider this change to the statement in the
3526``core-image-sato`` recipe: do_image[mcdepends] =
3527"mc:x86:arm:core-image-minimal:do_image" In this case, BitBake must
3528create the ``core-image-minimal`` image for the "arm" build since the
3529"x86" build depends on it.
3530
3531Because "x86" and "arm" are enabled for multiple configuration builds
3532and have separate configuration files, BitBake places the artifacts for
3533each build in the respective temporary build directories (i.e.
3534```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__).
3535
3536.. _building-an-initramfs-image:
3537
3538Building an Initial RAM Filesystem (initramfs) Image
3539----------------------------------------------------
3540
3541An initial RAM filesystem (initramfs) image provides a temporary root
3542filesystem used for early system initialization (e.g. loading of modules
3543needed to locate and mount the "real" root filesystem).
3544
3545.. note::
3546
3547 The initramfs image is the successor of initial RAM disk (initrd). It
3548 is a "copy in and out" (cpio) archive of the initial filesystem that
3549 gets loaded into memory during the Linux startup process. Because
3550 Linux uses the contents of the archive during initialization, the
3551 initramfs image needs to contain all of the device drivers and tools
3552 needed to mount the final root filesystem.
3553
3554Follow these steps to create an initramfs image:
3555
35561. *Create the initramfs Image Recipe:* You can reference the
3557 ``core-image-minimal-initramfs.bb`` recipe found in the
3558 ``meta/recipes-core`` directory of the `Source
3559 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ as an example
3560 from which to work.
3561
35622. *Decide if You Need to Bundle the initramfs Image Into the Kernel
3563 Image:* If you want the initramfs image that is built to be bundled
3564 in with the kernel image, set the
3565 ```INITRAMFS_IMAGE_BUNDLE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE>`__
3566 variable to "1" in your ``local.conf`` configuration file and set the
3567 ```INITRAMFS_IMAGE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE>`__
3568 variable in the recipe that builds the kernel image.
3569
3570 .. note::
3571
3572 It is recommended that you do bundle the initramfs image with the
3573 kernel image to avoid circular dependencies between the kernel
3574 recipe and the initramfs recipe should the initramfs image include
3575 kernel modules.
3576
3577 Setting the ``INITRAMFS_IMAGE_BUNDLE`` flag causes the initramfs
3578 image to be unpacked into the ``${B}/usr/`` directory. The unpacked
3579 initramfs image is then passed to the kernel's ``Makefile`` using the
3580 ```CONFIG_INITRAMFS_SOURCE`` <&YOCTO_DOCS_REF_URL;#var-CONFIG_INITRAMFS_SOURCE>`__
3581 variable, allowing the initramfs image to be built into the kernel
3582 normally.
3583
3584 .. note::
3585
3586 If you choose to not bundle the initramfs image with the kernel
3587 image, you are essentially using an
3588 Initial RAM Disk (initrd)
3589 . Creating an initrd is handled primarily through the
3590 INITRD_IMAGE
3591 ,
3592 INITRD_LIVE
3593 , and
3594 INITRD_IMAGE_LIVE
3595 variables. For more information, see the
3596 image-live.bbclass
3597 file.
3598
35993. *Optionally Add Items to the initramfs Image Through the initramfs
3600 Image Recipe:* If you add items to the initramfs image by way of its
3601 recipe, you should use
3602 ```PACKAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL>`__
3603 rather than
3604 ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__.
3605 ``PACKAGE_INSTALL`` gives more direct control of what is added to the
3606 image as compared to the defaults you might not necessarily want that
3607 are set by the ```image`` <&YOCTO_DOCS_REF_URL;#ref-classes-image>`__
3608 or ```core-image`` <&YOCTO_DOCS_REF_URL;#ref-classes-core-image>`__
3609 classes.
3610
36114. *Build the Kernel Image and the initramfs Image:* Build your kernel
3612 image using BitBake. Because the initramfs image recipe is a
3613 dependency of the kernel image, the initramfs image is built as well
3614 and bundled with the kernel image if you used the
3615 ```INITRAMFS_IMAGE_BUNDLE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE>`__
3616 variable described earlier.
3617
3618Building a Tiny System
3619----------------------
3620
3621Very small distributions have some significant advantages such as
3622requiring less on-die or in-package memory (cheaper), better performance
3623through efficient cache usage, lower power requirements due to less
3624memory, faster boot times, and reduced development overhead. Some
3625real-world examples where a very small distribution gives you distinct
3626advantages are digital cameras, medical devices, and small headless
3627systems.
3628
3629This section presents information that shows you how you can trim your
3630distribution to even smaller sizes than the ``poky-tiny`` distribution,
3631which is around 5 Mbytes, that can be built out-of-the-box using the
3632Yocto Project.
3633
3634.. _tiny-system-overview:
3635
3636Overview
3637~~~~~~~~
3638
3639The following list presents the overall steps you need to consider and
3640perform to create distributions with smaller root filesystems, achieve
3641faster boot times, maintain your critical functionality, and avoid
3642initial RAM disks:
3643
3644- `Determine your goals and guiding
3645 principles. <#goals-and-guiding-principles>`__
3646
3647- `Understand what contributes to your image
3648 size. <#understand-what-gives-your-image-size>`__
3649
3650- `Reduce the size of the root
3651 filesystem. <#trim-the-root-filesystem>`__
3652
3653- `Reduce the size of the kernel. <#trim-the-kernel>`__
3654
3655- `Eliminate packaging
3656 requirements. <#remove-package-management-requirements>`__
3657
3658- `Look for other ways to minimize
3659 size. <#look-for-other-ways-to-minimize-size>`__
3660
3661- `Iterate on the process. <#iterate-on-the-process>`__
3662
3663Goals and Guiding Principles
3664~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3665
3666Before you can reach your destination, you need to know where you are
3667going. Here is an example list that you can use as a guide when creating
3668very small distributions:
3669
3670- Determine how much space you need (e.g. a kernel that is 1 Mbyte or
3671 less and a root filesystem that is 3 Mbytes or less).
3672
3673- Find the areas that are currently taking 90% of the space and
3674 concentrate on reducing those areas.
3675
3676- Do not create any difficult "hacks" to achieve your goals.
3677
3678- Leverage the device-specific options.
3679
3680- Work in a separate layer so that you keep changes isolated. For
3681 information on how to create layers, see the "`Understanding and
3682 Creating Layers <#understanding-and-creating-layers>`__" section.
3683
3684.. _understand-what-gives-your-image-size:
3685
3686Understand What Contributes to Your Image Size
3687~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3688
3689It is easiest to have something to start with when creating your own
3690distribution. You can use the Yocto Project out-of-the-box to create the
3691``poky-tiny`` distribution. Ultimately, you will want to make changes in
3692your own distribution that are likely modeled after ``poky-tiny``.
3693
3694.. note::
3695
3696 To use
3697 poky-tiny
3698 in your build, set the
3699 DISTRO
3700 variable in your
3701 local.conf
3702 file to "poky-tiny" as described in the "
3703 Creating Your Own Distribution
3704 " section.
3705
3706Understanding some memory concepts will help you reduce the system size.
3707Memory consists of static, dynamic, and temporary memory. Static memory
3708is the TEXT (code), DATA (initialized data in the code), and BSS
3709(uninitialized data) sections. Dynamic memory represents memory that is
3710allocated at runtime: stacks, hash tables, and so forth. Temporary
3711memory is recovered after the boot process. This memory consists of
3712memory used for decompressing the kernel and for the ``__init__``
3713functions.
3714
3715To help you see where you currently are with kernel and root filesystem
3716sizes, you can use two tools found in the `Source
3717Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ in the
3718``scripts/tiny/`` directory:
3719
3720- ``ksize.py``: Reports component sizes for the kernel build objects.
3721
3722- ``dirsize.py``: Reports component sizes for the root filesystem.
3723
3724This next tool and command help you organize configuration fragments and
3725view file dependencies in a human-readable form:
3726
3727- ``merge_config.sh``: Helps you manage configuration files and
3728 fragments within the kernel. With this tool, you can merge individual
3729 configuration fragments together. The tool allows you to make
3730 overrides and warns you of any missing configuration options. The
3731 tool is ideal for allowing you to iterate on configurations, create
3732 minimal configurations, and create configuration files for different
3733 machines without having to duplicate your process.
3734
3735 The ``merge_config.sh`` script is part of the Linux Yocto kernel Git
3736 repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``,
3737 ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig``
3738 directory.
3739
3740 For more information on configuration fragments, see the "`Creating
3741 Configuration
3742 Fragments <&YOCTO_DOCS_KERNEL_DEV_URL;#creating-config-fragments>`__"
3743 section in the Yocto Project Linux Kernel Development Manual.
3744
3745- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command
3746 with these options brings up a Dependency Explorer from which you can
3747 view file dependencies. Understanding these dependencies allows you
3748 to make informed decisions when cutting out various pieces of the
3749 kernel and root filesystem.
3750
3751Trim the Root Filesystem
3752~~~~~~~~~~~~~~~~~~~~~~~~
3753
3754The root filesystem is made up of packages for booting, libraries, and
3755applications. To change things, you can configure how the packaging
3756happens, which changes the way you build them. You can also modify the
3757filesystem itself or select a different filesystem.
3758
3759First, find out what is hogging your root filesystem by running the
3760``dirsize.py`` script from your root directory: $ cd
3761root-directory-of-image $ dirsize.py 100000 > dirsize-100k.log $ cat
3762dirsize-100k.log You can apply a filter to the script to ignore files
3763under a certain size. The previous example filters out any files below
3764100 Kbytes. The sizes reported by the tool are uncompressed, and thus
3765will be smaller by a relatively constant factor in a compressed root
3766filesystem. When you examine your log file, you can focus on areas of
3767the root filesystem that take up large amounts of memory.
3768
3769You need to be sure that what you eliminate does not cripple the
3770functionality you need. One way to see how packages relate to each other
3771is by using the Dependency Explorer UI with the BitBake command: $ cd
3772image-directory $ bitbake -u taskexp -g image Use the interface to
3773select potential packages you wish to eliminate and see their dependency
3774relationships.
3775
3776When deciding how to reduce the size, get rid of packages that result in
3777minimal impact on the feature set. For example, you might not need a VGA
3778display. Or, you might be able to get by with ``devtmpfs`` and ``mdev``
3779instead of ``udev``.
3780
3781Use your ``local.conf`` file to make changes. For example, to eliminate
3782``udev`` and ``glib``, set the following in the local configuration
3783file: VIRTUAL-RUNTIME_dev_manager = ""
3784
3785Finally, you should consider exactly the type of root filesystem you
3786need to meet your needs while also reducing its size. For example,
3787consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an
3788``initramfs`` using ``initramfs``. Be aware that ``ext3`` requires a 1
3789Mbyte journal. If you are okay with running read-only, you do not need
3790this journal.
3791
3792.. note::
3793
3794 After each round of elimination, you need to rebuild your system and
3795 then use the tools to see the effects of your reductions.
3796
3797Trim the Kernel
3798~~~~~~~~~~~~~~~
3799
3800The kernel is built by including policies for hardware-independent
3801aspects. What subsystems do you enable? For what architecture are you
3802building? Which drivers do you build by default?
3803
3804.. note::
3805
3806 You can modify the kernel source if you want to help with boot time.
3807
3808Run the ``ksize.py`` script from the top-level Linux build directory to
3809get an idea of what is making up the kernel: $ cd
3810top-level-linux-build-directory $ ksize.py > ksize.log $ cat ksize.log
3811When you examine the log, you will see how much space is taken up with
3812the built-in ``.o`` files for drivers, networking, core kernel files,
3813filesystem, sound, and so forth. The sizes reported by the tool are
3814uncompressed, and thus will be smaller by a relatively constant factor
3815in a compressed kernel image. Look to reduce the areas that are large
3816and taking up around the "90% rule."
3817
3818To examine, or drill down, into any particular area, use the ``-d``
3819option with the script: $ ksize.py -d > ksize.log Using this option
3820breaks out the individual file information for each area of the kernel
3821(e.g. drivers, networking, and so forth).
3822
3823Use your log file to see what you can eliminate from the kernel based on
3824features you can let go. For example, if you are not going to need
3825sound, you do not need any drivers that support sound.
3826
3827After figuring out what to eliminate, you need to reconfigure the kernel
3828to reflect those changes during the next build. You could run
3829``menuconfig`` and make all your changes at once. However, that makes it
3830difficult to see the effects of your individual eliminations and also
3831makes it difficult to replicate the changes for perhaps another target
3832device. A better method is to start with no configurations using
3833``allnoconfig``, create configuration fragments for individual changes,
3834and then manage the fragments into a single configuration file using
3835``merge_config.sh``. The tool makes it easy for you to iterate using the
3836configuration change and build cycle.
3837
3838Each time you make configuration changes, you need to rebuild the kernel
3839and check to see what impact your changes had on the overall size.
3840
3841Remove Package Management Requirements
3842~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3843
3844Packaging requirements add size to the image. One way to reduce the size
3845of the image is to remove all the packaging requirements from the image.
3846This reduction includes both removing the package manager and its unique
3847dependencies as well as removing the package management data itself.
3848
3849To eliminate all the packaging requirements for an image, be sure that
3850"package-management" is not part of your
3851```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__
3852statement for the image. When you remove this feature, you are removing
3853the package manager as well as its dependencies from the root
3854filesystem.
3855
3856Look for Other Ways to Minimize Size
3857~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3858
3859Depending on your particular circumstances, other areas that you can
3860trim likely exist. The key to finding these areas is through tools and
3861methods described here combined with experimentation and iteration. Here
3862are a couple of areas to experiment with:
3863
3864- ``glibc``: In general, follow this process:
3865
3866 1. Remove ``glibc`` features from
3867 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
3868 that you think you do not need.
3869
3870 2. Build your distribution.
3871
3872 3. If the build fails due to missing symbols in a package, determine
3873 if you can reconfigure the package to not need those features. For
3874 example, change the configuration to not support wide character
3875 support as is done for ``ncurses``. Or, if support for those
3876 characters is needed, determine what ``glibc`` features provide
3877 the support and restore the configuration.
3878
3879 4. Rebuild and repeat the process.
3880
3881- ``busybox``: For BusyBox, use a process similar as described for
3882 ``glibc``. A difference is you will need to boot the resulting system
3883 to see if you are able to do everything you expect from the running
3884 system. You need to be sure to integrate configuration fragments into
3885 Busybox because BusyBox handles its own core features and then allows
3886 you to add configuration fragments on top.
3887
3888Iterate on the Process
3889~~~~~~~~~~~~~~~~~~~~~~
3890
3891If you have not reached your goals on system size, you need to iterate
3892on the process. The process is the same. Use the tools and see just what
3893is taking up 90% of the root filesystem and the kernel. Decide what you
3894can eliminate without limiting your device beyond what you need.
3895
3896Depending on your system, a good place to look might be Busybox, which
3897provides a stripped down version of Unix tools in a single, executable
3898file. You might be able to drop virtual terminal services or perhaps
3899ipv6.
3900
3901Building Images for More than One Machine
3902-----------------------------------------
3903
3904A common scenario developers face is creating images for several
3905different machines that use the same software environment. In this
3906situation, it is tempting to set the tunings and optimization flags for
3907each build specifically for the targeted hardware (i.e. "maxing out" the
3908tunings). Doing so can considerably add to build times and package feed
3909maintenance collectively for the machines. For example, selecting tunes
3910that are extremely specific to a CPU core used in a system might enable
3911some micro optimizations in GCC for that particular system but would
3912otherwise not gain you much of a performance difference across the other
3913systems as compared to using a more general tuning across all the builds
3914(e.g. setting ```DEFAULTTUNE`` <&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE>`__
3915specifically for each machine's build). Rather than "max out" each
3916build's tunings, you can take steps that cause the OpenEmbedded build
3917system to reuse software across the various machines where it makes
3918sense.
3919
3920If build speed and package feed maintenance are considerations, you
3921should consider the points in this section that can help you optimize
3922your tunings to best consider build times and package feed maintenance.
3923
3924- *Share the Build Directory:* If at all possible, share the
3925 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ across builds. The
3926 Yocto Project supports switching between different
3927 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ values in the same
3928 ``TMPDIR``. This practice is well supported and regularly used by
3929 developers when building for multiple machines. When you use the same
3930 ``TMPDIR`` for multiple machine builds, the OpenEmbedded build system
3931 can reuse the existing native and often cross-recipes for multiple
3932 machines. Thus, build time decreases.
3933
3934 .. note::
3935
3936 If
3937 DISTRO
3938 settings change or fundamental configuration settings such as the
3939 filesystem layout, you need to work with a clean
3940 TMPDIR
3941 . Sharing
3942 TMPDIR
3943 under these circumstances might work but since it is not
3944 guaranteed, you should use a clean
3945 TMPDIR
3946 .
3947
3948- *Enable the Appropriate Package Architecture:* By default, the
3949 OpenEmbedded build system enables three levels of package
3950 architectures: "all", "tune" or "package", and "machine". Any given
3951 recipe usually selects one of these package architectures (types) for
3952 its output. Depending for what a given recipe creates packages,
3953 making sure you enable the appropriate package architecture can
3954 directly impact the build time.
3955
3956 A recipe that just generates scripts can enable "all" architecture
3957 because there are no binaries to build. To specifically enable "all"
3958 architecture, be sure your recipe inherits the
3959 ```allarch`` <&YOCTO_DOCS_REF_URL;#ref-classes-allarch>`__ class.
3960 This class is useful for "all" architectures because it configures
3961 many variables so packages can be used across multiple architectures.
3962
3963 If your recipe needs to generate packages that are machine-specific
3964 or when one of the build or runtime dependencies is already
3965 machine-architecture dependent, which makes your recipe also
3966 machine-architecture dependent, make sure your recipe enables the
3967 "machine" package architecture through the
3968 ```MACHINE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH>`__
3969 variable: PACKAGE_ARCH = "${MACHINE_ARCH}" When you do not
3970 specifically enable a package architecture through the
3971 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__, The
3972 OpenEmbedded build system defaults to the
3973 ```TUNE_PKGARCH`` <&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH>`__ setting:
3974 PACKAGE_ARCH = "${TUNE_PKGARCH}"
3975
3976- *Choose a Generic Tuning File if Possible:* Some tunes are more
3977 generic and can run on multiple targets (e.g. an ``armv5`` set of
3978 packages could run on ``armv6`` and ``armv7`` processors in most
3979 cases). Similarly, ``i486`` binaries could work on ``i586`` and
3980 higher processors. You should realize, however, that advances on
3981 newer processor versions would not be used.
3982
3983 If you select the same tune for several different machines, the
3984 OpenEmbedded build system reuses software previously built, thus
3985 speeding up the overall build time. Realize that even though a new
3986 sysroot for each machine is generated, the software is not recompiled
3987 and only one package feed exists.
3988
3989- *Manage Granular Level Packaging:* Sometimes cases exist where
3990 injecting another level of package architecture beyond the three
3991 higher levels noted earlier can be useful. For example, consider how
3992 NXP (formerly Freescale) allows for the easy reuse of binary packages
3993 in their layer
3994 ```meta-freescale`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/>`__.
3995 In this example, the
3996 ```fsl-dynamic-packagearch`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>`__
3997 class shares GPU packages for i.MX53 boards because all boards share
3998 the AMD GPU. The i.MX6-based boards can do the same because all
3999 boards share the Vivante GPU. This class inspects the BitBake
4000 datastore to identify if the package provides or depends on one of
4001 the sub-architecture values. If so, the class sets the
4002 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__ value
4003 based on the ``MACHINE_SUBARCH`` value. If the package does not
4004 provide or depend on one of the sub-architecture values but it
4005 matches a value in the machine-specific filter, it sets
4006 ```MACHINE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH>`__. This
4007 behavior reduces the number of packages built and saves build time by
4008 reusing binaries.
4009
4010- *Use Tools to Debug Issues:* Sometimes you can run into situations
4011 where software is being rebuilt when you think it should not be. For
4012 example, the OpenEmbedded build system might not be using shared
4013 state between machines when you think it should be. These types of
4014 situations are usually due to references to machine-specific
4015 variables such as ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__,
4016 ```SERIAL_CONSOLES`` <&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES>`__,
4017 ```XSERVER`` <&YOCTO_DOCS_REF_URL;#var-XSERVER>`__,
4018 ```MACHINE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES>`__,
4019 and so forth in code that is supposed to only be tune-specific or
4020 when the recipe depends
4021 (```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__,
4022 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__,
4023 ```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__,
4024 ```RSUGGESTS`` <&YOCTO_DOCS_REF_URL;#var-RSUGGESTS>`__, and so forth)
4025 on some other recipe that already has
4026 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__ defined
4027 as "${MACHINE_ARCH}".
4028
4029 .. note::
4030
4031 Patches to fix any issues identified are most welcome as these
4032 issues occasionally do occur.
4033
4034 For such cases, you can use some tools to help you sort out the
4035 situation:
4036
4037 - *``sstate-diff-machines.sh``:* You can find this tool in the
4038 ``scripts`` directory of the Source Repositories. See the comments
4039 in the script for information on how to use the tool.
4040
4041 - *BitBake's "-S printdiff" Option:* Using this option causes
4042 BitBake to try to establish the closest signature match it can
4043 (e.g. in the shared state cache) and then run ``bitbake-diffsigs``
4044 over the matches to determine the stamps and delta where these two
4045 stamp trees diverge.
4046
4047Building Software from an External Source
4048-----------------------------------------
4049
4050By default, the OpenEmbedded build system uses the `Build
4051Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ when building source
4052code. The build process involves fetching the source files, unpacking
4053them, and then patching them if necessary before the build takes place.
4054
4055Situations exist where you might want to build software from source
4056files that are external to and thus outside of the OpenEmbedded build
4057system. For example, suppose you have a project that includes a new BSP
4058with a heavily customized kernel. And, you want to minimize exposing the
4059build system to the development team so that they can focus on their
4060project and maintain everyone's workflow as much as possible. In this
4061case, you want a kernel source directory on the development machine
4062where the development occurs. You want the recipe's
4063```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable to point to
4064the external directory and use it as is, not copy it.
4065
4066To build from software that comes from an external source, all you need
4067to do is inherit the
4068```externalsrc`` <&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc>`__ class
4069and then set the
4070```EXTERNALSRC`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC>`__ variable to
4071point to your external source code. Here are the statements to put in
4072your ``local.conf`` file: INHERIT += "externalsrc"
4073EXTERNALSRC_pn-myrecipe = "path-to-your-source-tree"
4074
4075This next example shows how to accomplish the same thing by setting
4076``EXTERNALSRC`` in the recipe itself or in the recipe's append file:
4077EXTERNALSRC = "path" EXTERNALSRC_BUILD = "path"
4078
4079.. note::
4080
4081 In order for these settings to take effect, you must globally or
4082 locally inherit the
4083 externalsrc
4084 class.
4085
4086By default, ``externalsrc.bbclass`` builds the source code in a
4087directory separate from the external source directory as specified by
4088```EXTERNALSRC`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC>`__. If you need
4089to have the source built in the same directory in which it resides, or
4090some other nominated directory, you can set
4091```EXTERNALSRC_BUILD`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD>`__
4092to point to that directory: EXTERNALSRC_BUILD_pn-myrecipe =
4093"path-to-your-source-tree"
4094
4095Replicating a Build Offline
4096---------------------------
4097
4098It can be useful to take a "snapshot" of upstream sources used in a
4099build and then use that "snapshot" later to replicate the build offline.
4100To do so, you need to first prepare and populate your downloads
4101directory your "snapshot" of files. Once your downloads directory is
4102ready, you can use it at any time and from any machine to replicate your
4103build.
4104
4105Follow these steps to populate your Downloads directory:
4106
41071. *Create a Clean Downloads Directory:* Start with an empty downloads
4108 directory (```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__). You
4109 start with an empty downloads directory by either removing the files
4110 in the existing directory or by setting ``DL_DIR`` to point to either
4111 an empty location or one that does not yet exist.
4112
41132. *Generate Tarballs of the Source Git Repositories:* Edit your
4114 ``local.conf`` configuration file as follows: DL_DIR =
4115 "/home/your-download-dir/" BB_GENERATE_MIRROR_TARBALLS = "1" During
4116 the fetch process in the next step, BitBake gathers the source files
4117 and creates tarballs in the directory pointed to by ``DL_DIR``. See
4118 the
4119 ```BB_GENERATE_MIRROR_TARBALLS`` <&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS>`__
4120 variable for more information.
4121
41223. *Populate Your Downloads Directory Without Building:* Use BitBake to
4123 fetch your sources but inhibit the build: $ bitbake target
4124 --runonly=fetch The downloads directory (i.e. ``${DL_DIR}``) now has
4125 a "snapshot" of the source files in the form of tarballs, which can
4126 be used for the build.
4127
41284. *Optionally Remove Any Git or other SCM Subdirectories From the
4129 Downloads Directory:* If you want, you can clean up your downloads
4130 directory by removing any Git or other Source Control Management
4131 (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs
4132 already contain these subdirectories.
4133
4134Once your downloads directory has everything it needs regarding source
4135files, you can create your "own-mirror" and build your target.
4136Understand that you can use the files to build the target offline from
4137any machine and at any time.
4138
4139Follow these steps to build your target using the files in the downloads
4140directory:
4141
41421. *Using Local Files Only:* Inside your ``local.conf`` file, add the
4143 ```SOURCE_MIRROR_URL`` <&YOCTO_DOCS_REF_URL;#var-SOURCE_MIRROR_URL>`__
4144 variable, inherit the
4145 ```own-mirrors`` <&YOCTO_DOCS_REF_URL;#ref-classes-own-mirrors>`__
4146 class, and use the
4147 ```BB_NO_NETWORK`` <&YOCTO_DOCS_BB_URL;#var-bb-BB_NO_NETWORK>`__
4148 variable to your ``local.conf``. SOURCE_MIRROR_URL ?=
4149 "file:///home/your-download-dir/" INHERIT += "own-mirrors"
4150 BB_NO_NETWORK = "1" The ``SOURCE_MIRROR_URL`` and ``own-mirror``
4151 class set up the system to use the downloads directory as your "own
4152 mirror". Using the ``BB_NO_NETWORK`` variable makes sure that
4153 BitBake's fetching process in step 3 stays local, which means files
4154 from your "own-mirror" are used.
4155
41562. *Start With a Clean Build:* You can start with a clean build by
4157 removing the
4158 ``${``\ ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__\ ``}``
4159 directory or using a new `Build
4160 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
4161
41623. *Build Your Target:* Use BitBake to build your target: $ bitbake
4163 target The build completes using the known local "snapshot" of source
4164 files from your mirror. The resulting tarballs for your "snapshot" of
4165 source files are in the downloads directory.
4166
4167 .. note::
4168
4169 The offline build does not work if recipes attempt to find the
4170 latest version of software by setting
4171 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ to
4172 ``${``\ ```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__\ ``}``:
4173 SRCREV = "${AUTOREV}" When a recipe sets ``SRCREV`` to
4174 ``${AUTOREV}``, the build system accesses the network in an
4175 attempt to determine the latest version of software from the SCM.
4176 Typically, recipes that use ``AUTOREV`` are custom or modified
4177 recipes. Recipes that reside in public repositories usually do not
4178 use ``AUTOREV``.
4179
4180 If you do have recipes that use ``AUTOREV``, you can take steps to
4181 still use the recipes in an offline build. Do the following:
4182
4183 1. Use a configuration generated by enabling `build
4184 history <#maintaining-build-output-quality>`__.
4185
4186 2. Use the ``buildhistory-collect-srcrevs`` command to collect the
4187 stored ``SRCREV`` values from the build's history. For more
4188 information on collecting these values, see the "`Build History
4189 Package Information <#build-history-package-information>`__"
4190 section.
4191
4192 3. Once you have the correct source revisions, you can modify
4193 those recipes to to set ``SRCREV`` to specific versions of the
4194 software.
4195
4196Speeding Up a Build
4197===================
4198
4199Build time can be an issue. By default, the build system uses simple
4200controls to try and maximize build efficiency. In general, the default
4201settings for all the following variables result in the most efficient
4202build times when dealing with single socket systems (i.e. a single CPU).
4203If you have multiple CPUs, you might try increasing the default values
4204to gain more speed. See the descriptions in the glossary for each
4205variable for more information:
4206
4207- ```BB_NUMBER_THREADS``: <&YOCTO_DOCS_REF_URL;#var-BB_NUMBER_THREADS>`__
4208 The maximum number of threads BitBake simultaneously executes.
4209
4210- ```BB_NUMBER_PARSE_THREADS``: <&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS>`__
4211 The number of threads BitBake uses during parsing.
4212
4213- ```PARALLEL_MAKE``: <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ Extra
4214 options passed to the ``make`` command during the
4215 ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ task in
4216 order to specify parallel compilation on the local build host.
4217
4218- ```PARALLEL_MAKEINST``: <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST>`__
4219 Extra options passed to the ``make`` command during the
4220 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task in
4221 order to specify parallel installation on the local build host.
4222
4223As mentioned, these variables all scale to the number of processor cores
4224available on the build system. For single socket systems, this
4225auto-scaling ensures that the build system fundamentally takes advantage
4226of potential parallel operations during the build based on the build
4227machine's capabilities.
4228
4229Following are additional factors that can affect build speed:
4230
4231- File system type: The file system type that the build is being
4232 performed on can also influence performance. Using ``ext4`` is
4233 recommended as compared to ``ext2`` and ``ext3`` due to ``ext4``
4234 improved features such as extents.
4235
4236- Disabling the updating of access time using ``noatime``: The
4237 ``noatime`` mount option prevents the build system from updating file
4238 and directory access times.
4239
4240- Setting a longer commit: Using the "commit=" mount option increases
4241 the interval in seconds between disk cache writes. Changing this
4242 interval from the five second default to something longer increases
4243 the risk of data loss but decreases the need to write to the disk,
4244 thus increasing the build performance.
4245
4246- Choosing the packaging backend: Of the available packaging backends,
4247 IPK is the fastest. Additionally, selecting a singular packaging
4248 backend also helps.
4249
4250- Using ``tmpfs`` for ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__
4251 as a temporary file system: While this can help speed up the build,
4252 the benefits are limited due to the compiler using ``-pipe``. The
4253 build system goes to some lengths to avoid ``sync()`` calls into the
4254 file system on the principle that if there was a significant failure,
4255 the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__
4256 contents could easily be rebuilt.
4257
4258- Inheriting the
4259 ```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ class:
4260 Inheriting this class has shown to speed up builds due to
4261 significantly lower amounts of data stored in the data cache as well
4262 as on disk. Inheriting this class also makes cleanup of
4263 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ faster, at the
4264 expense of being easily able to dive into the source code. File
4265 system maintainers have recommended that the fastest way to clean up
4266 large numbers of files is to reformat partitions rather than delete
4267 files due to the linear nature of partitions. This, of course,
4268 assumes you structure the disk partitions and file systems in a way
4269 that this is practical.
4270
4271Aside from the previous list, you should keep some trade offs in mind
4272that can help you speed up the build:
4273
4274- Remove items from
4275 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
4276 that you might not need.
4277
4278- Exclude debug symbols and other debug information: If you do not need
4279 these symbols and other debug information, disabling the ``*-dbg``
4280 package generation can speed up the build. You can disable this
4281 generation by setting the
4282 ```INHIBIT_PACKAGE_DEBUG_SPLIT`` <&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_DEBUG_SPLIT>`__
4283 variable to "1".
4284
4285- Disable static library generation for recipes derived from
4286 ``autoconf`` or ``libtool``: Following is an example showing how to
4287 disable static libraries and still provide an override to handle
4288 exceptions: STATICLIBCONF = "--disable-static"
4289 STATICLIBCONF_sqlite3-native = "" EXTRA_OECONF += "${STATICLIBCONF}"
4290
4291 .. note::
4292
4293 - Some recipes need static libraries in order to work correctly
4294 (e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides,
4295 as in the previous example, account for these kinds of
4296 exceptions.
4297
4298 - Some packages have packaging code that assumes the presence of
4299 the static libraries. If so, you might need to exclude them as
4300 well.
4301
4302.. _platdev-working-with-libraries:
4303
4304Working With Libraries
4305======================
4306
4307Libraries are an integral part of your system. This section describes
4308some common practices you might find helpful when working with libraries
4309to build your system:
4310
4311- `How to include static library
4312 files <#including-static-library-files>`__
4313
4314- `How to use the Multilib feature to combine multiple versions of
4315 library files into a single
4316 image <#combining-multiple-versions-library-files-into-one-image>`__
4317
4318- `How to install multiple versions of the same library in parallel on
4319 the same
4320 system <#installing-multiple-versions-of-the-same-library>`__
4321
4322Including Static Library Files
4323------------------------------
4324
4325If you are building a library and the library offers static linking, you
4326can control which static library files (``*.a`` files) get included in
4327the built library.
4328
4329The ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ and
4330```FILES_*`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ variables in the
4331``meta/conf/bitbake.conf`` configuration file define how files installed
4332by the ``do_install`` task are packaged. By default, the ``PACKAGES``
4333variable includes ``${PN}-staticdev``, which represents all static
4334library files.
4335
4336.. note::
4337
4338 Some previously released versions of the Yocto Project defined the
4339 static library files through
4340 ${PN}-dev
4341 .
4342
4343Following is part of the BitBake configuration file, where you can see
4344how the static library files are defined: PACKAGE_BEFORE_PN ?= ""
4345PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
4346${PACKAGE_BEFORE_PN} ${PN}" PACKAGES_DYNAMIC = "^${PN}-locale-.*" FILES
4347= "" FILES_${PN} = "${bindir}/\* ${sbindir}/\* ${libexecdir}/\*
4348${libdir}/lib*${SOLIBS} \\ ${sysconfdir} ${sharedstatedir}
4349${localstatedir} \\ ${base_bindir}/\* ${base_sbindir}/\* \\
4350${base_libdir}/*${SOLIBS} \\ ${base_prefix}/lib/udev/rules.d
4351${prefix}/lib/udev/rules.d \\ ${datadir}/${BPN} ${libdir}/${BPN}/\* \\
4352${datadir}/pixmaps ${datadir}/applications \\ ${datadir}/idl
4353${datadir}/omf ${datadir}/sounds \\ ${libdir}/bonobo/servers"
4354FILES_${PN}-bin = "${bindir}/\* ${sbindir}/*" FILES_${PN}-doc =
4355"${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \\
4356${datadir}/gnome/help" SECTION_${PN}-doc = "doc" FILES_SOLIBSDEV ?=
4357"${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
4358FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \\
4359${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \\
4360${datadir}/aclocal ${base_libdir}/*.o \\ ${libdir}/${BPN}/*.la
4361${base_libdir}/*.la" SECTION_${PN}-dev = "devel" ALLOW_EMPTY_${PN}-dev =
4362"1" RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" FILES_${PN}-staticdev
4363= "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
4364SECTION_${PN}-staticdev = "devel" RDEPENDS_${PN}-staticdev = "${PN}-dev
4365(= ${EXTENDPKGV})"
4366
4367.. _combining-multiple-versions-library-files-into-one-image:
4368
4369Combining Multiple Versions of Library Files into One Image
4370-----------------------------------------------------------
4371
4372The build system offers the ability to build libraries with different
4373target optimizations or architecture formats and combine these together
4374into one system image. You can link different binaries in the image
4375against the different libraries as needed for specific use cases. This
4376feature is called "Multilib."
4377
4378An example would be where you have most of a system compiled in 32-bit
4379mode using 32-bit libraries, but you have something large, like a
4380database engine, that needs to be a 64-bit application and uses 64-bit
4381libraries. Multilib allows you to get the best of both 32-bit and 64-bit
4382libraries.
4383
4384While the Multilib feature is most commonly used for 32 and 64-bit
4385differences, the approach the build system uses facilitates different
4386target optimizations. You could compile some binaries to use one set of
4387libraries and other binaries to use a different set of libraries. The
4388libraries could differ in architecture, compiler options, or other
4389optimizations.
4390
4391Several examples exist in the ``meta-skeleton`` layer found in the
4392`Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__:
4393
4394- ``conf/multilib-example.conf`` configuration file
4395
4396- ``conf/multilib-example2.conf`` configuration file
4397
4398- ``recipes-multilib/images/core-image-multilib-example.bb`` recipe
4399
4400Preparing to Use Multilib
4401~~~~~~~~~~~~~~~~~~~~~~~~~
4402
4403User-specific requirements drive the Multilib feature. Consequently,
4404there is no one "out-of-the-box" configuration that likely exists to
4405meet your needs.
4406
4407In order to enable Multilib, you first need to ensure your recipe is
4408extended to support multiple libraries. Many standard recipes are
4409already extended and support multiple libraries. You can check in the
4410``meta/conf/multilib.conf`` configuration file in the `Source
4411Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ to see how this is
4412done using the
4413```BBCLASSEXTEND`` <&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND>`__ variable.
4414Eventually, all recipes will be covered and this list will not be
4415needed.
4416
4417For the most part, the Multilib class extension works automatically to
4418extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where
4419``MLPREFIX`` is the particular multilib (e.g. "lib32-" or "lib64-").
4420Standard variables such as
4421```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__,
4422```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__,
4423```RPROVIDES`` <&YOCTO_DOCS_REF_URL;#var-RPROVIDES>`__,
4424```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__,
4425```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__, and
4426```PACKAGES_DYNAMIC`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC>`__ are
4427automatically extended by the system. If you are extending any manual
4428code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure
4429those names are extended correctly. This automatic extension code
4430resides in ``multilib.bbclass``.
4431
4432Using Multilib
4433~~~~~~~~~~~~~~
4434
4435After you have set up the recipes, you need to define the actual
4436combination of multiple libraries you want to build. You accomplish this
4437through your ``local.conf`` configuration file in the `Build
4438Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. An example
4439configuration would be as follows: MACHINE = "qemux86-64" require
4440conf/multilib.conf MULTILIBS = "multilib:lib32"
4441DEFAULTTUNE_virtclass-multilib-lib32 = "x86" IMAGE_INSTALL_append = "
4442lib32-glib-2.0" This example enables an additional library named
4443``lib32`` alongside the normal target packages. When combining these
4444"lib32" alternatives, the example uses "x86" for tuning. For information
4445on this particular tuning, see
4446``meta/conf/machine/include/ia32/arch-ia32.inc``.
4447
4448The example then includes ``lib32-glib-2.0`` in all the images, which
4449illustrates one method of including a multiple library dependency. You
4450can use a normal image build to include this dependency, for example: $
4451bitbake core-image-sato You can also build Multilib packages
4452specifically with a command like this: $ bitbake lib32-glib-2.0
4453
4454Additional Implementation Details
4455~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4456
4457Generic implementation details as well as details that are specific to
4458package management systems exist. Following are implementation details
4459that exist regardless of the package management system:
4460
4461- The typical convention used for the class extension code as used by
4462 Multilib assumes that all package names specified in
4463 ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ that contain
4464 ``${PN}`` have ``${PN}`` at the start of the name. When that
4465 convention is not followed and ``${PN}`` appears at the middle or the
4466 end of a name, problems occur.
4467
4468- The ```TARGET_VENDOR`` <&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR>`__
4469 value under Multilib will be extended to "-vendormlmultilib" (e.g.
4470 "-pokymllib32" for a "lib32" Multilib with Poky). The reason for this
4471 slightly unwieldy contraction is that any "-" characters in the
4472 vendor string presently break Autoconf's ``config.sub``, and other
4473 separators are problematic for different reasons.
4474
4475For the RPM Package Management System, the following implementation
4476details exist:
4477
4478- A unique architecture is defined for the Multilib packages, along
4479 with creating a unique deploy folder under ``tmp/deploy/rpm`` in the
4480 `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. For
4481 example, consider ``lib32`` in a ``qemux86-64`` image. The possible
4482 architectures in the system are "all", "qemux86_64",
4483 "lib32_qemux86_64", and "lib32_x86".
4484
4485- The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM
4486 packaging. The naming for a normal RPM package and a Multilib RPM
4487 package in a ``qemux86-64`` system resolves to something similar to
4488 ``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``,
4489 respectively.
4490
4491- When installing a Multilib image, the RPM backend first installs the
4492 base image and then installs the Multilib libraries.
4493
4494- The build system relies on RPM to resolve the identical files in the
4495 two (or more) Multilib packages.
4496
4497For the IPK Package Management System, the following implementation
4498details exist:
4499
4500- The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK
4501 packaging. The naming for a normal RPM package and a Multilib IPK
4502 package in a ``qemux86-64`` system resolves to something like
4503 ``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw_x86.ipk``,
4504 respectively.
4505
4506- The IPK deploy folder is not modified with ``${MLPREFIX}`` because
4507 packages with and without the Multilib feature can exist in the same
4508 folder due to the ``${PN}`` differences.
4509
4510- IPK defines a sanity check for Multilib installation using certain
4511 rules for file comparison, overridden, etc.
4512
4513Installing Multiple Versions of the Same Library
4514------------------------------------------------
4515
4516Situations can exist where you need to install and use multiple versions
4517of the same library on the same system at the same time. These
4518situations almost always exist when a library API changes and you have
4519multiple pieces of software that depend on the separate versions of the
4520library. To accommodate these situations, you can install multiple
4521versions of the same library in parallel on the same system.
4522
4523The process is straightforward as long as the libraries use proper
4524versioning. With properly versioned libraries, all you need to do to
4525individually specify the libraries is create separate, appropriately
4526named recipes where the ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ part of
4527the name includes a portion that differentiates each library version
4528(e.g.the major part of the version number). Thus, instead of having a
4529single recipe that loads one version of a library (e.g. ``clutter``),
4530you provide multiple recipes that result in different versions of the
4531libraries you want. As an example, the following two recipes would allow
4532the two separate versions of the ``clutter`` library to co-exist on the
4533same system: clutter-1.6_1.6.20.bb clutter-1.8_1.8.4.bb Additionally, if
4534you have other recipes that depend on a given library, you need to use
4535the ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable to
4536create the dependency. Continuing with the same example, if you want to
4537have a recipe depend on the 1.8 version of the ``clutter`` library, use
4538the following in your recipe: DEPENDS = "clutter-1.8"
4539
4540Using x32 psABI
4541===============
4542
4543x32 processor-specific Application Binary Interface (`x32
4544psABI <https://software.intel.com/en-us/node/628948>`__) is a native
454532-bit processor-specific ABI for Intel 64 (x86-64) architectures. An
4546ABI defines the calling conventions between functions in a processing
4547environment. The interface determines what registers are used and what
4548the sizes are for various C data types.
4549
4550Some processing environments prefer using 32-bit applications even when
4551running on Intel 64-bit platforms. Consider the i386 psABI, which is a
4552very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not
4553provide efficient use and access of the Intel 64-bit processor
4554resources, leaving the system underutilized. Now consider the x86_64
4555psABI. This ABI is newer and uses 64-bits for data sizes and program
4556pointers. The extra bits increase the footprint size of the programs,
4557libraries, and also increases the memory and file system size
4558requirements. Executing under the x32 psABI enables user programs to
4559utilize CPU and system resources more efficiently while keeping the
4560memory footprint of the applications low. Extra bits are used for
4561registers but not for addressing mechanisms.
4562
4563The Yocto Project supports the final specifications of x32 psABI as
4564follows:
4565
4566- You can create packages and images in x32 psABI format on x86_64
4567 architecture targets.
4568
4569- You can successfully build recipes with the x32 toolchain.
4570
4571- You can create and boot ``core-image-minimal`` and
4572 ``core-image-sato`` images.
4573
4574- RPM Package Manager (RPM) support exists for x32 binaries.
4575
4576- Support for large images exists.
4577
4578To use the x32 psABI, you need to edit your ``conf/local.conf``
4579configuration file as follows: MACHINE = "qemux86-64" DEFAULTTUNE =
4580"x86-64-x32" baselib = "${@d.getVar('BASE_LIB_tune-' +
4581(d.getVar('DEFAULTTUNE') \\ or 'INVALID')) or 'lib'}" Once you have set
4582up your configuration file, use BitBake to build an image that supports
4583the x32 psABI. Here is an example: $ bitbake core-image-sato
4584
4585Enabling GObject Introspection Support
4586======================================
4587
4588`GObject
4589introspection <https://wiki.gnome.org/Projects/GObjectIntrospection>`__
4590is the standard mechanism for accessing GObject-based software from
4591runtime environments. GObject is a feature of the GLib library that
4592provides an object framework for the GNOME desktop and related software.
4593GObject Introspection adds information to GObject that allows objects
4594created within it to be represented across different programming
4595languages. If you want to construct GStreamer pipelines using Python, or
4596control UPnP infrastructure using Javascript and GUPnP, GObject
4597introspection is the only way to do it.
4598
4599This section describes the Yocto Project support for generating and
4600packaging GObject introspection data. GObject introspection data is a
4601description of the API provided by libraries built on top of GLib
4602framework, and, in particular, that framework's GObject mechanism.
4603GObject Introspection Repository (GIR) files go to ``-dev`` packages,
4604``typelib`` files go to main packages as they are packaged together with
4605libraries that are introspected.
4606
4607The data is generated when building such a library, by linking the
4608library with a small executable binary that asks the library to describe
4609itself, and then executing the binary and processing its output.
4610
4611Generating this data in a cross-compilation environment is difficult
4612because the library is produced for the target architecture, but its
4613code needs to be executed on the build host. This problem is solved with
4614the OpenEmbedded build system by running the code through QEMU, which
4615allows precisely that. Unfortunately, QEMU does not always work
4616perfectly as mentioned in the "`Known Issues <#known-issues>`__"
4617section.
4618
4619Enabling the Generation of Introspection Data
4620---------------------------------------------
4621
4622Enabling the generation of introspection data (GIR files) in your
4623library package involves the following:
4624
46251. Inherit the
4626 ```gobject-introspection`` <&YOCTO_DOCS_REF_URL;#ref-classes-gobject-introspection>`__
4627 class.
4628
46292. Make sure introspection is not disabled anywhere in the recipe or
4630 from anything the recipe includes. Also, make sure that
4631 "gobject-introspection-data" is not in
4632 ```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__
4633 and that "qemu-usermode" is not in
4634 ```MACHINE_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED>`__.
4635 If either of these conditions exist, nothing will happen.
4636
46373. Try to build the recipe. If you encounter build errors that look like
4638 something is unable to find ``.so`` libraries, check where these
4639 libraries are located in the source tree and add the following to the
4640 recipe: GIR_EXTRA_LIBS_PATH = "${B}/something/.libs"
4641
4642 .. note::
4643
4644 See recipes in the
4645 oe-core
4646 repository that use that
4647 GIR_EXTRA_LIBS_PATH
4648 variable as an example.
4649
46504. Look for any other errors, which probably mean that introspection
4651 support in a package is not entirely standard, and thus breaks down
4652 in a cross-compilation environment. For such cases, custom-made fixes
4653 are needed. A good place to ask and receive help in these cases is
4654 the `Yocto Project mailing
4655 lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__.
4656
4657.. note::
4658
4659 Using a library that no longer builds against the latest Yocto
4660 Project release and prints introspection related errors is a good
4661 candidate for the previous procedure.
4662
4663Disabling the Generation of Introspection Data
4664----------------------------------------------
4665
4666You might find that you do not want to generate introspection data. Or,
4667perhaps QEMU does not work on your build host and target architecture
4668combination. If so, you can use either of the following methods to
4669disable GIR file generations:
4670
4671- Add the following to your distro configuration:
4672 DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
4673 Adding this statement disables generating introspection data using
4674 QEMU but will still enable building introspection tools and libraries
4675 (i.e. building them does not require the use of QEMU).
4676
4677- Add the following to your machine configuration:
4678 MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode" Adding this
4679 statement disables the use of QEMU when building packages for your
4680 machine. Currently, this feature is used only by introspection
4681 recipes and has the same effect as the previously described option.
4682
4683 .. note::
4684
4685 Future releases of the Yocto Project might have other features
4686 affected by this option.
4687
4688If you disable introspection data, you can still obtain it through other
4689means such as copying the data from a suitable sysroot, or by generating
4690it on the target hardware. The OpenEmbedded build system does not
4691currently provide specific support for these techniques.
4692
4693Testing that Introspection Works in an Image
4694--------------------------------------------
4695
4696Use the following procedure to test if generating introspection data is
4697working in an image:
4698
46991. Make sure that "gobject-introspection-data" is not in
4700 ```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__
4701 and that "qemu-usermode" is not in
4702 ```MACHINE_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED>`__.
4703
47042. Build ``core-image-sato``.
4705
47063. Launch a Terminal and then start Python in the terminal.
4707
47084. Enter the following in the terminal: >>> from gi.repository import
4709 GLib >>> GLib.get_host_name()
4710
47115. For something a little more advanced, enter the following:
4712 http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html
4713
4714Known Issues
4715------------
4716
4717The following know issues exist for GObject Introspection Support:
4718
4719- ``qemu-ppc64`` immediately crashes. Consequently, you cannot build
4720 introspection data on that architecture.
4721
4722- x32 is not supported by QEMU. Consequently, introspection data is
4723 disabled.
4724
4725- musl causes transient GLib binaries to crash on assertion failures.
4726 Consequently, generating introspection data is disabled.
4727
4728- Because QEMU is not able to run the binaries correctly, introspection
4729 is disabled for some specific packages under specific architectures
4730 (e.g. ``gcr``, ``libsecret``, and ``webkit``).
4731
4732- QEMU usermode might not work properly when running 64-bit binaries
4733 under 32-bit host machines. In particular, "qemumips64" is known to
4734 not work under i686.
4735
4736.. _dev-optionally-using-an-external-toolchain:
4737
4738Optionally Using an External Toolchain
4739======================================
4740
4741You might want to use an external toolchain as part of your development.
4742If this is the case, the fundamental steps you need to accomplish are as
4743follows:
4744
4745- Understand where the installed toolchain resides. For cases where you
4746 need to build the external toolchain, you would need to take separate
4747 steps to build and install the toolchain.
4748
4749- Make sure you add the layer that contains the toolchain to your
4750 ``bblayers.conf`` file through the
4751 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable.
4752
4753- Set the ``EXTERNAL_TOOLCHAIN`` variable in your ``local.conf`` file
4754 to the location in which you installed the toolchain.
4755
4756A good example of an external toolchain used with the Yocto Project is
4757Mentor Graphics Sourcery G++ Toolchain. You can see information on how
4758to use that particular layer in the ``README`` file at
4759` <http://github.com/MentorEmbedded/meta-sourcery/>`__. You can find
4760further information by reading about the
4761```TCMODE`` <&YOCTO_DOCS_REF_URL;#var-TCMODE>`__ variable in the Yocto
4762Project Reference Manual's variable glossary.
4763
4764Creating Partitioned Images Using Wic
4765=====================================
4766
4767Creating an image for a particular hardware target using the
4768OpenEmbedded build system does not necessarily mean you can boot that
4769image as is on your device. Physical devices accept and boot images in
4770various ways depending on the specifics of the device. Usually,
4771information about the hardware can tell you what image format the device
4772requires. Should your device require multiple partitions on an SD card,
4773flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to
4774create the properly partitioned image.
4775
4776The ``wic`` command generates partitioned images from existing
4777OpenEmbedded build artifacts. Image generation is driven by partitioning
4778commands contained in an Openembedded kickstart file (``.wks``)
4779specified either directly on the command line or as one of a selection
4780of canned kickstart files as shown with the ``wic list images`` command
4781in the "`Using an Existing Kickstart
4782File <#using-a-provided-kickstart-file>`__" section. When you apply the
4783command to a given set of build artifacts, the result is an image or set
4784of images that can be directly written onto media and used on a
4785particular system.
4786
4787.. note::
4788
4789 For a kickstart file reference, see the "
4790 OpenEmbedded Kickstart (
4791 .wks
4792 ) Reference
4793 " Chapter in the Yocto Project Reference Manual.
4794
4795The ``wic`` command and the infrastructure it is based on is by
4796definition incomplete. The purpose of the command is to allow the
4797generation of customized images, and as such, was designed to be
4798completely extensible through a plugin interface. See the "`Using the
4799Wic PlugIn Interface <#wic-using-the-wic-plugin-interface>`__" section
4800for information on these plugins.
4801
4802This section provides some background information on Wic, describes what
4803you need to have in place to run the tool, provides instruction on how
4804to use the Wic utility, provides information on using the Wic plugins
4805interface, and provides several examples that show how to use Wic.
4806
4807.. _wic-background:
4808
4809Background
4810----------
4811
4812This section provides some background on the Wic utility. While none of
4813this information is required to use Wic, you might find it interesting.
4814
4815- The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The
4816 "oe" diphthong in "oeic" was promoted to the letter "w", because
4817 "oeic" is both difficult to remember and to pronounce.
4818
4819- Wic is loosely based on the Meego Image Creator (``mic``) framework.
4820 The Wic implementation has been heavily modified to make direct use
4821 of OpenEmbedded build artifacts instead of package installation and
4822 configuration, which are already incorporated within the OpenEmbedded
4823 artifacts.
4824
4825- Wic is a completely independent standalone utility that initially
4826 provides easier-to-use and more flexible replacements for an existing
4827 functionality in OE-Core's
4828 ```image-live`` <&YOCTO_DOCS_REF_URL;#ref-classes-image-live>`__
4829 class. The difference between Wic and those examples is that with Wic
4830 the functionality of those scripts is implemented by a
4831 general-purpose partitioning language, which is based on Redhat
4832 kickstart syntax.
4833
4834.. _wic-requirements:
4835
4836Requirements
4837------------
4838
4839In order to use the Wic utility with the OpenEmbedded Build system, your
4840system needs to meet the following requirements:
4841
4842- The Linux distribution on your development host must support the
4843 Yocto Project. See the "`Supported Linux
4844 Distributions <&YOCTO_DOCS_REF_URL;#detailed-supported-distros>`__"
4845 section in the Yocto Project Reference Manual for the list of
4846 distributions that support the Yocto Project.
4847
4848- The standard system utilities, such as ``cp``, must be installed on
4849 your development host system.
4850
4851- You must have sourced the build environment setup script (i.e.
4852 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) found in the
4853 `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
4854
4855- You need to have the build artifacts already available, which
4856 typically means that you must have already created an image using the
4857 Openembedded build system (e.g. ``core-image-minimal``). While it
4858 might seem redundant to generate an image in order to create an image
4859 using Wic, the current version of Wic requires the artifacts in the
4860 form generated by the OpenEmbedded build system.
4861
4862- You must build several native tools, which are built to run on the
4863 build system: $ bitbake parted-native dosfstools-native mtools-native
4864
4865- Include "wic" as part of the
4866 ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__
4867 variable.
4868
4869- Include the name of the `wic kickstart
4870 file <&YOCTO_DOCS_REF_URL;#openembedded-kickstart-wks-reference>`__
4871 as part of the ```WKS_FILE`` <&YOCTO_DOCS_REF_URL;#var-WKS_FILE>`__
4872 variable
4873
4874.. _wic-getting-help:
4875
4876Getting Help
4877------------
4878
4879You can get general help for the ``wic`` command by entering the ``wic``
4880command by itself or by entering the command with a help argument as
4881follows: $ wic -h $ wic --help $ wic help
4882
4883Currently, Wic supports seven commands: ``cp``, ``create``, ``help``,
4884``list``, ``ls``, ``rm``, and ``write``. You can get help for all these
4885commands except "help" by using the following form: $ wic help command
4886For example, the following command returns help for the ``write``
4887command: $ wic help write
4888
4889Wic supports help for three topics: ``overview``, ``plugins``, and
4890``kickstart``. You can get help for any topic using the following form:
4891$ wic help topic For example, the following returns overview help for
4892Wic: $ wic help overview
4893
4894One additional level of help exists for Wic. You can get help on
4895individual images through the ``list`` command. You can use the ``list``
4896command to return the available Wic images as follows: $ wic list images
4897genericx86 Create an EFI disk image for genericx86\* beaglebone-yocto
4898Create SD card image for Beaglebone edgerouter Create SD card image for
4899Edgerouter qemux86-directdisk Create a qemu machine 'pcbios' direct disk
4900image directdisk-gpt Create a 'pcbios' direct disk image mkefidisk
4901Create an EFI disk image directdisk Create a 'pcbios' direct disk image
4902systemd-bootdisk Create an EFI disk image with systemd-boot mkhybridiso
4903Create a hybrid ISO image sdimage-bootpart Create SD card image with a
4904boot partition directdisk-multi-rootfs Create multi rootfs image using
4905rootfs plugin directdisk-bootloader-config Create a 'pcbios' direct disk
4906image with custom bootloader config Once you know the list of available
4907Wic images, you can use ``help`` with the command to get help on a
4908particular image. For example, the following command returns help on the
4909"beaglebone-yocto" image: $ wic list beaglebone-yocto help Creates a
4910partitioned SD card image for Beaglebone. Boot files are located in the
4911first vfat partition.
4912
4913Operational Modes
4914-----------------
4915
4916You can use Wic in two different modes, depending on how much control
4917you need for specifying the Openembedded build artifacts that are used
4918for creating the image: Raw and Cooked:
4919
4920- *Raw Mode:* You explicitly specify build artifacts through Wic
4921 command-line arguments.
4922
4923- *Cooked Mode:* The current
4924 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ setting and image
4925 name are used to automatically locate and provide the build
4926 artifacts. You just supply a kickstart file and the name of the image
4927 from which to use artifacts.
4928
4929Regardless of the mode you use, you need to have the build artifacts
4930ready and available.
4931
4932Raw Mode
4933~~~~~~~~
4934
4935Running Wic in raw mode allows you to specify all the partitions through
4936the ``wic`` command line. The primary use for raw mode is if you have
4937built your kernel outside of the Yocto Project `Build
4938Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. In other words, you
4939can point to arbitrary kernel, root filesystem locations, and so forth.
4940Contrast this behavior with cooked mode where Wic looks in the Build
4941Directory (e.g. ``tmp/deploy/images/``\ machine).
4942
4943The general form of the ``wic`` command in raw mode is: $ wic create
4944wks_file options ... Where: wks_file: An OpenEmbedded kickstart file.
4945You can provide your own custom file or use a file from a set of
4946existing files as described by further options. optional arguments: -h,
4947--help show this help message and exit -o OUTDIR, --outdir OUTDIR name
4948of directory to create image in -e IMAGE_NAME, --image-name IMAGE_NAME
4949name of the image to use the artifacts from e.g. core- image-sato -r
4950ROOTFS_DIR, --rootfs-dir ROOTFS_DIR path to the /rootfs dir to use as
4951the .wks rootfs source -b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR path to
4952the dir containing the boot artifacts (e.g. /EFI or /syslinux dirs) to
4953use as the .wks bootimg source -k KERNEL_DIR, --kernel-dir KERNEL_DIR
4954path to the dir containing the kernel to use in the .wks bootimg -n
4955NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT path to the native
4956sysroot containing the tools to use to build the image -s,
4957--skip-build-check skip the build check -f, --build-rootfs build rootfs
4958-c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz} compress image with
4959specified compressor -m, --bmap generate .bmap --no-fstab-update Do not
4960change fstab file. -v VARS_DIR, --vars VARS_DIR directory with
4961<image>.env files that store bitbake variables -D, --debug output debug
4962information
4963
4964.. note::
4965
4966 You do not need root privileges to run Wic. In fact, you should not
4967 run as root when using the utility.
4968
4969Cooked Mode
4970~~~~~~~~~~~
4971
4972Running Wic in cooked mode leverages off artifacts in the Build
4973Directory. In other words, you do not have to specify kernel or root
4974filesystem locations as part of the command. All you need to provide is
4975a kickstart file and the name of the image from which to use artifacts
4976by using the "-e" option. Wic looks in the Build Directory (e.g.
4977``tmp/deploy/images/``\ machine) for artifacts.
4978
4979The general form of the ``wic`` command using Cooked Mode is as follows:
4980$ wic create wks_file -e IMAGE_NAME Where: wks_file: An OpenEmbedded
4981kickstart file. You can provide your own custom file or use a file from
4982a set of existing files provided with the Yocto Project release.
4983required argument: -e IMAGE_NAME, --image-name IMAGE_NAME name of the
4984image to use the artifacts from e.g. core- image-sato
4985
4986.. _using-a-provided-kickstart-file:
4987
4988Using an Existing Kickstart File
4989--------------------------------
4990
4991If you do not want to create your own kickstart file, you can use an
4992existing file provided by the Wic installation. As shipped, kickstart
4993files can be found in the Yocto Project `Source
4994Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ in the
4995following two locations: poky/meta-yocto-bsp/wic
4996poky/scripts/lib/wic/canned-wks Use the following command to list the
4997available kickstart files: $ wic list images genericx86 Create an EFI
4998disk image for genericx86\* beaglebone-yocto Create SD card image for
4999Beaglebone edgerouter Create SD card image for Edgerouter
5000qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
5001directdisk-gpt Create a 'pcbios' direct disk image mkefidisk Create an
5002EFI disk image directdisk Create a 'pcbios' direct disk image
5003systemd-bootdisk Create an EFI disk image with systemd-boot mkhybridiso
5004Create a hybrid ISO image sdimage-bootpart Create SD card image with a
5005boot partition directdisk-multi-rootfs Create multi rootfs image using
5006rootfs plugin directdisk-bootloader-config Create a 'pcbios' direct disk
5007image with custom bootloader config When you use an existing file, you
5008do not have to use the ``.wks`` extension. Here is an example in Raw
5009Mode that uses the ``directdisk`` file: $ wic create directdisk -r
5010rootfs_dir -b bootimg_dir \\ -k kernel_dir -n native_sysroot
5011
5012Here are the actual partition language commands used in the
5013``genericx86.wks`` file to generate an image: # short-description:
5014Create an EFI disk image for genericx86\* # long-description: Creates a
5015partitioned EFI disk image for genericx86\* machines part /boot --source
5016bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos
5017--active --align 1024 part / --source rootfs --ondisk sda --fstype=ext4
5018--label platform --align 1024 --use-uuid part swap --ondisk sda --size
501944 --label swap1 --fstype=swap bootloader --ptable gpt --timeout=5
5020--append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
5021
5022.. _wic-using-the-wic-plugin-interface:
5023
5024Using the Wic Plugin Interface
5025------------------------------
5026
5027You can extend and specialize Wic functionality by using Wic plugins.
5028This section explains the Wic plugin interface.
5029
5030.. note::
5031
5032 Wic plugins consist of "source" and "imager" plugins. Imager plugins
5033 are beyond the scope of this section.
5034
5035Source plugins provide a mechanism to customize partition content during
5036the Wic image generation process. You can use source plugins to map
5037values that you specify using ``--source`` commands in kickstart files
5038(i.e. ``*.wks``) to a plugin implementation used to populate a given
5039partition.
5040
5041.. note::
5042
5043 If you use plugins that have build-time dependencies (e.g. native
5044 tools, bootloaders, and so forth) when building a Wic image, you need
5045 to specify those dependencies using the
5046 WKS_FILE_DEPENDS
5047 variable.
5048
5049Source plugins are subclasses defined in plugin files. As shipped, the
5050Yocto Project provides several plugin files. You can see the source
5051plugin files that ship with the Yocto Project
5052`here <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source>`__.
5053Each of these plugin files contains source plugins that are designed to
5054populate a specific Wic image partition.
5055
5056Source plugins are subclasses of the ``SourcePlugin`` class, which is
5057defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example,
5058the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py``
5059file is a subclass of the ``SourcePlugin`` class, which is found in the
5060``pluginbase.py`` file.
5061
5062You can also implement source plugins in a layer outside of the Source
5063Repositories (external layer). To do so, be sure that your plugin files
5064are located in a directory whose path is
5065``scripts/lib/wic/plugins/source/`` within your external layer. When the
5066plugin files are located there, the source plugins they contain are made
5067available to Wic.
5068
5069When the Wic implementation needs to invoke a partition-specific
5070implementation, it looks for the plugin with the same name as the
5071``--source`` parameter used in the kickstart file given to that
5072partition. For example, if the partition is set up using the following
5073command in a kickstart file: part /boot --source bootimg-pcbios --ondisk
5074sda --label boot --active --align 1024 The methods defined as class
5075members of the matching source plugin (i.e. ``bootimg-pcbios``) in the
5076``bootimg-pcbios.py`` plugin file are used.
5077
5078To be more concrete, here is the corresponding plugin definition from
5079the ``bootimg-pcbios.py`` file for the previous command along with an
5080example method called by the Wic implementation when it needs to prepare
5081a partition using an implementation-specific function: . . . class
5082BootimgPcbiosPlugin(SourcePlugin): """ Create MBR boot partition and
5083install syslinux on it. """ name = 'bootimg-pcbios' . . . @classmethod
5084def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
5085oe_builddir, bootimg_dir, kernel_dir, rootfs_dir, native_sysroot): """
5086Called to do the actual content population for a partition i.e. it
5087'prepares' the partition to be incorporated into the image. In this
5088case, prepare content for legacy bios boot partition. """ . . . If a
5089subclass (plugin) itself does not implement a particular function, Wic
5090locates and uses the default version in the superclass. It is for this
5091reason that all source plugins are derived from the ``SourcePlugin``
5092class.
5093
5094The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines
5095a set of methods that source plugins can implement or override. Any
5096plugins (subclass of ``SourcePlugin``) that do not implement a
5097particular method inherit the implementation of the method from the
5098``SourcePlugin`` class. For more information, see the ``SourcePlugin``
5099class in the ``pluginbase.py`` file for details:
5100
5101The following list describes the methods implemented in the
5102``SourcePlugin`` class:
5103
5104- *``do_prepare_partition()``:* Called to populate a partition with
5105 actual content. In other words, the method prepares the final
5106 partition image that is incorporated into the disk image.
5107
5108- *``do_configure_partition()``:* Called before
5109 ``do_prepare_partition()`` to create custom configuration files for a
5110 partition (e.g. syslinux or grub configuration files).
5111
5112- *``do_install_disk()``:* Called after all partitions have been
5113 prepared and assembled into a disk image. This method provides a hook
5114 to allow finalization of a disk image (e.g. writing an MBR).
5115
5116- *``do_stage_partition()``:* Special content-staging hook called
5117 before ``do_prepare_partition()``. This method is normally empty.
5118
5119 Typically, a partition just uses the passed-in parameters (e.g. the
5120 unmodified value of ``bootimg_dir``). However, in some cases, things
5121 might need to be more tailored. As an example, certain files might
5122 additionally need to be taken from ``bootimg_dir + /boot``. This hook
5123 allows those files to be staged in a customized fashion.
5124
5125 .. note::
5126
5127 get_bitbake_var()
5128 allows you to access non-standard variables that you might want to
5129 use for this behavior.
5130
5131You can extend the source plugin mechanism. To add more hooks, create
5132more source plugin methods within ``SourcePlugin`` and the corresponding
5133derived subclasses. The code that calls the plugin methods uses the
5134``plugin.get_source_plugin_methods()`` function to find the method or
5135methods needed by the call. Retrieval of those methods is accomplished
5136by filling up a dict with keys that contain the method names of
5137interest. On success, these will be filled in with the actual methods.
5138See the Wic implementation for examples and details.
5139
5140.. _wic-usage-examples:
5141
5142Examples
5143--------
5144
5145This section provides several examples that show how to use the Wic
5146utility. All the examples assume the list of requirements in the
5147"`Requirements <#wic-requirements>`__" section have been met. The
5148examples assume the previously generated image is
5149``core-image-minimal``.
5150
5151.. _generate-an-image-using-a-provided-kickstart-file:
5152
5153Generate an Image using an Existing Kickstart File
5154~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5155
5156This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart
5157file: $ wic create mkefidisk -e core-image-minimal INFO: Building
5158wic-tools... . . . INFO: The new image(s) can be found here:
5159./mkefidisk-201804191017-sda.direct The following build artifacts were
5160used to create the image(s): ROOTFS_DIR:
5161/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5162BOOTIMG_DIR:
5163/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5164KERNEL_DIR:
5165/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5166NATIVE_SYSROOT:
5167/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5168INFO: The image(s) were created using OE kickstart file:
5169/home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
5170The previous example shows the easiest way to create an image by running
5171in cooked mode and supplying a kickstart file and the "-e" option to
5172point to the existing build artifacts. Your ``local.conf`` file needs to
5173have the ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable set
5174to the machine you are using, which is "qemux86" in this example.
5175
5176Once the image builds, the output provides image location, artifact use,
5177and kickstart file information.
5178
5179.. note::
5180
5181 You should always verify the details provided in the output to make
5182 sure that the image was indeed created exactly as expected.
5183
5184Continuing with the example, you can now write the image from the Build
5185Directory onto a USB stick, or whatever media for which you built your
5186image, and boot from the media. You can write the image by using
5187``bmaptool`` or ``dd``: $ oe-run-native bmaptool copy
5188mkefidisk-201804191017-sda.direct /dev/sdX or $ sudo dd
5189if=mkefidisk-201804191017-sda.direct of=/dev/sdX
5190
5191.. note::
5192
5193 For more information on how to use the
5194 bmaptool
5195 to flash a device with an image, see the "
5196 Flashing Images Using
5197 bmaptool
5198 " section.
5199
5200Using a Modified Kickstart File
5201~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5202
5203Because partitioned image creation is driven by the kickstart file, it
5204is easy to affect image creation by changing the parameters in the file.
5205This next example demonstrates that through modification of the
5206``directdisk-gpt`` kickstart file.
5207
5208As mentioned earlier, you can use the command ``wic list images`` to
5209show the list of existing kickstart files. The directory in which the
5210``directdisk-gpt.wks`` file resides is
5211``scripts/lib/image/canned-wks/``, which is located in the `Source
5212Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
5213Because available files reside in this directory, you can create and add
5214your own custom files to the directory. Subsequent use of the
5215``wic list images`` command would then include your kickstart files.
5216
5217In this example, the existing ``directdisk-gpt`` file already does most
5218of what is needed. However, for the hardware in this example, the image
5219will need to boot from ``sdb`` instead of ``sda``, which is what the
5220``directdisk-gpt`` kickstart file uses.
5221
5222The example begins by making a copy of the ``directdisk-gpt.wks`` file
5223in the ``scripts/lib/image/canned-wks`` directory and then by changing
5224the lines that specify the target disk from which to boot. $ cp
5225/home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \\
5226/home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5227Next, the example modifies the ``directdisksdb-gpt.wks`` file and
5228changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The
5229example changes the following two lines and leaves the remaining lines
5230untouched: part /boot --source bootimg-pcbios --ondisk sdb --label boot
5231--active --align 1024 part / --source rootfs --ondisk sdb --fstype=ext4
5232--label platform --align 1024 --use-uuid Once the lines are changed, the
5233example generates the ``directdisksdb-gpt`` image. The command points
5234the process at the ``core-image-minimal`` artifacts for the Next Unit of
5235Computing (nuc) ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ the
5236``local.conf``. $ wic create directdisksdb-gpt -e core-image-minimal
5237INFO: Building wic-tools... . . . Initialising tasks: 100%
5238\|#######################################\| Time: 0:00:01 NOTE:
5239Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Tasks
5240Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and
5241all succeeded. INFO: Creating image(s)... INFO: The new image(s) can be
5242found here: ./directdisksdb-gpt-201710090938-sdb.direct The following
5243build artifacts were used to create the image(s): ROOTFS_DIR:
5244/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5245BOOTIMG_DIR:
5246/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5247KERNEL_DIR:
5248/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5249NATIVE_SYSROOT:
5250/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5251INFO: The image(s) were created using OE kickstart file:
5252/home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5253Continuing with the example, you can now directly ``dd`` the image to a
5254USB stick, or whatever media for which you built your image, and boot
5255the resulting media: $ sudo dd
5256if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb 140966+0
5257records in 140966+0 records out 72174592 bytes (72 MB, 69 MiB) copied,
525878.0282 s, 925 kB/s $ sudo eject /dev/sdb
5259
5260Using a Modified Kickstart File and Running in Raw Mode
5261~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5262
5263This next example manually specifies each build artifact (runs in Raw
5264Mode) and uses a modified kickstart file. The example also uses the
5265``-o`` option to cause Wic to create the output somewhere other than the
5266default output directory, which is the current directory: $ wic create
5267/home/stephano/my_yocto/test.wks -o /home/stephano/testwic \\
5268--rootfs-dir
5269/home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs
5270\\ --bootimg-dir
5271/home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5272\\ --kernel-dir
5273/home/stephano/build/master/build/tmp/deploy/images/qemux86 \\
5274--native-sysroot
5275/home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
5276INFO: Creating image(s)... INFO: The new image(s) can be found here:
5277/home/stephano/testwic/test-201710091445-sdb.direct The following build
5278artifacts were used to create the image(s): ROOTFS_DIR:
5279/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5280BOOTIMG_DIR:
5281/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5282KERNEL_DIR:
5283/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5284NATIVE_SYSROOT:
5285/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5286INFO: The image(s) were created using OE kickstart file:
5287/home/stephano/my_yocto/test.wks For this example,
5288```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ did not have to be
5289specified in the ``local.conf`` file since the artifact is manually
5290specified.
5291
5292Using Wic to Manipulate an Image
5293~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5294
5295Wic image manipulation allows you to shorten turnaround time during
5296image development. For example, you can use Wic to delete the kernel
5297partition of a Wic image and then insert a newly built kernel. This
5298saves you time from having to rebuild the entire image each time you
5299modify the kernel.
5300
5301.. note::
5302
5303 In order to use Wic to manipulate a Wic image as in this example,
5304 your development machine must have the
5305 mtools
5306 package installed.
5307
5308The following example examines the contents of the Wic image, deletes
5309the existing kernel, and then inserts a new kernel:
5310
53111. *List the Partitions:* Use the ``wic ls`` command to list all the
5312 partitions in the Wic image: $ wic ls
5313 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic Num Start
5314 End Size Fstype 1 1048576 25041919 23993344 fat16 2 25165824 72157183
5315 46991360 ext4 The previous output shows two partitions in the
5316 ``core-image-minimal-qemux86.wic`` image.
5317
53182. *Examine a Particular Partition:* Use the ``wic ls`` command again
5319 but in a different form to examine a particular partition.
5320
5321 .. note::
5322
5323 You can get command usage on any Wic command using the following
5324 form:
5325 ::
5326
5327 $ wic help command
5328
5329
5330 For example, the following command shows you the various ways to
5331 use the
5332 wic ls
5333 command:
5334 ::
5335
5336 $ wic help ls
5337
5338
5339 The following command shows what is in Partition one: $ wic ls
5340 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1 Volume in
5341 drive : is boot Volume Serial Number is E894-1809 Directory for ::/
5342 libcom32 c32 186500 2017-10-09 16:06 libutil c32 24148 2017-10-09
5343 16:06 syslinux cfg 220 2017-10-09 16:06 vesamenu c32 27104 2017-10-09
5344 16:06 vmlinuz 6904608 2017-10-09 16:06 5 files 7 142 580 bytes 16 582
5345 656 bytes free The previous output shows five files, with the
5346 ``vmlinuz`` being the kernel.
5347
5348 .. note::
5349
5350 If you see the following error, you need to update or create a
5351 ~/.mtoolsrc
5352 file and be sure to have the line “mtools_skip_check=1“ in the
5353 file. Then, run the Wic command again:
5354 ::
5355
5356 ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
5357 output: Total number of sectors (47824) not a multiple of sectors per track (32)!
5358 Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
5359
5360
53613. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the
5362 ``vmlinuz`` file (kernel): $ wic rm
5363 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
5364
53654. *Add In the New Kernel:* Use the ``wic cp`` command to add the
5366 updated kernel to the Wic image. Depending on how you built your
5367 kernel, it could be in different places. If you used ``devtool`` and
5368 an SDK to build your kernel, it resides in the ``tmp/work`` directory
5369 of the extensible SDK. If you used ``make`` to build the kernel, the
5370 kernel will be in the ``workspace/sources`` area.
5371
5372 The following example assumes ``devtool`` was used to build the
5373 kernel: cp
5374 ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage
5375 \\
5376 ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
5377 Once the new kernel is added back into the image, you can use the
5378 ``dd`` command or ```bmaptool`` <#flashing-images-using-bmaptool>`__
5379 to flash your wic image onto an SD card or USB stick and test your
5380 target.
5381
5382 .. note::
5383
5384 Using
5385 bmaptool
5386 is generally 10 to 20 times faster than using
5387 dd
5388 .
5389
5390Flashing Images Using ``bmaptool``
5391==================================
5392
5393A fast and easy way to flash an image to a bootable device is to use
5394Bmaptool, which is integrated into the OpenEmbedded build system.
5395Bmaptool is a generic tool that creates a file's block map (bmap) and
5396then uses that map to copy the file. As compared to traditional tools
5397such as dd or cp, Bmaptool can copy (or flash) large files like raw
5398system image files much faster.
5399
5400.. note::
5401
5402 - If you are using Ubuntu or Debian distributions, you can install
5403 the ``bmap-tools`` package using the following command and then
5404 use the tool without specifying ``PATH`` even from the root
5405 account: $ sudo apt-get install bmap-tools
5406
5407 - If you are unable to install the ``bmap-tools`` package, you will
5408 need to build Bmaptool before using it. Use the following command:
5409 $ bitbake bmap-tools-native
5410
5411Following, is an example that shows how to flash a Wic image. Realize
5412that while this example uses a Wic image, you can use Bmaptool to flash
5413any type of image. Use these steps to flash an image using Bmaptool:
5414
54151. *Update your ``local.conf`` File:* You need to have the following set
5416 in your ``local.conf`` file before building your image: IMAGE_FSTYPES
5417 += "wic wic.bmap"
5418
54192. *Get Your Image:* Either have your image ready (pre-built with the
5420 ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__
5421 setting previously mentioned) or take the step to build the image: $
5422 bitbake image
5423
54243. *Flash the Device:* Flash the device with the image by using Bmaptool
5425 depending on your particular setup. The following commands assume the
5426 image resides in the Build Directory's ``deploy/images/`` area:
5427
5428 - If you have write access to the media, use this command form: $
5429 oe-run-native bmap-tools-native bmaptool copy
5430 build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
5431
5432 - If you do not have write access to the media, set your permissions
5433 first and then use the same command form: $ sudo chmod 666
5434 /dev/sdX $ oe-run-native bmap-tools-native bmaptool copy
5435 build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
5436
5437For help on the ``bmaptool`` command, use the following command: $
5438bmaptool --help
5439
5440Making Images More Secure
5441=========================
5442
5443Security is of increasing concern for embedded devices. Consider the
5444issues and problems discussed in just this sampling of work found across
5445the Internet:
5446
5447- *"*\ `Security Risks of Embedded
5448 Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"*
5449 by Bruce Schneier
5450
5451- *"*\ `Internet Census
5452 2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna
5453 Botnet
5454
5455- *"*\ `Security Issues for Embedded
5456 Devices <http://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"*
5457 by Jake Edge
5458
5459When securing your image is of concern, there are steps, tools, and
5460variables that you can consider to help you reach the security goals you
5461need for your particular device. Not all situations are identical when
5462it comes to making an image secure. Consequently, this section provides
5463some guidance and suggestions for consideration when you want to make
5464your image more secure.
5465
5466.. note::
5467
5468 Because the security requirements and risks are different for every
5469 type of device, this section cannot provide a complete reference on
5470 securing your custom OS. It is strongly recommended that you also
5471 consult other sources of information on embedded Linux system
5472 hardening and on security.
5473
5474General Considerations
5475----------------------
5476
5477General considerations exist that help you create more secure images.
5478You should consider the following suggestions to help make your device
5479more secure:
5480
5481- Scan additional code you are adding to the system (e.g. application
5482 code) by using static analysis tools. Look for buffer overflows and
5483 other potential security problems.
5484
5485- Pay particular attention to the security for any web-based
5486 administration interface.
5487
5488 Web interfaces typically need to perform administrative functions and
5489 tend to need to run with elevated privileges. Thus, the consequences
5490 resulting from the interface's security becoming compromised can be
5491 serious. Look for common web vulnerabilities such as
5492 cross-site-scripting (XSS), unvalidated inputs, and so forth.
5493
5494 As with system passwords, the default credentials for accessing a
5495 web-based interface should not be the same across all devices. This
5496 is particularly true if the interface is enabled by default as it can
5497 be assumed that many end-users will not change the credentials.
5498
5499- Ensure you can update the software on the device to mitigate
5500 vulnerabilities discovered in the future. This consideration
5501 especially applies when your device is network-enabled.
5502
5503- Ensure you remove or disable debugging functionality before producing
5504 the final image. For information on how to do this, see the
5505 "`Considerations Specific to the OpenEmbedded Build
5506 System <#considerations-specific-to-the-openembedded-build-system>`__"
5507 section.
5508
5509- Ensure you have no network services listening that are not needed.
5510
5511- Remove any software from the image that is not needed.
5512
5513- Enable hardware support for secure boot functionality when your
5514 device supports this functionality.
5515
5516Security Flags
5517--------------
5518
5519The Yocto Project has security flags that you can enable that help make
5520your build output more secure. The security flags are in the
5521``meta/conf/distro/include/security_flags.inc`` file in your `Source
5522Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
5523
5524.. note::
5525
5526 Depending on the recipe, certain security flags are enabled and
5527 disabled by default.
5528
5529Use the following line in your ``local.conf`` file or in your custom
5530distribution configuration file to enable the security compiler and
5531linker flags for your build: require
5532conf/distro/include/security_flags.inc
5533
5534Considerations Specific to the OpenEmbedded Build System
5535--------------------------------------------------------
5536
5537You can take some steps that are specific to the OpenEmbedded build
5538system to make your images more secure:
5539
5540- Ensure "debug-tweaks" is not one of your selected
5541 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__.
5542 When creating a new project, the default is to provide you with an
5543 initial ``local.conf`` file that enables this feature using the
5544 ```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
5545 variable with the line: EXTRA_IMAGE_FEATURES = "debug-tweaks" To
5546 disable that feature, simply comment out that line in your
5547 ``local.conf`` file, or make sure ``IMAGE_FEATURES`` does not contain
5548 "debug-tweaks" before producing your final image. Among other things,
5549 leaving this in place sets the root password as blank, which makes
5550 logging in for debugging or inspection easy during development but
5551 also means anyone can easily log in during production.
5552
5553- It is possible to set a root password for the image and also to set
5554 passwords for any extra users you might add (e.g. administrative or
5555 service type users). When you set up passwords for multiple images or
5556 users, you should not duplicate passwords.
5557
5558 To set up passwords, use the
5559 ```extrausers`` <&YOCTO_DOCS_REF_URL;#ref-classes-extrausers>`__
5560 class, which is the preferred method. For an example on how to set up
5561 both root and user passwords, see the
5562 "```extrausers.bbclass`` <&YOCTO_DOCS_REF_URL;#ref-classes-extrausers>`__"
5563 section.
5564
5565 .. note::
5566
5567 When adding extra user accounts or setting a root password, be
5568 cautious about setting the same password on every device. If you
5569 do this, and the password you have set is exposed, then every
5570 device is now potentially compromised. If you need this access but
5571 want to ensure security, consider setting a different, random
5572 password for each device. Typically, you do this as a separate
5573 step after you deploy the image onto the device.
5574
5575- Consider enabling a Mandatory Access Control (MAC) framework such as
5576 SMACK or SELinux and tuning it appropriately for your device's usage.
5577 You can find more information in the
5578 ```meta-selinux`` <http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/>`__
5579 layer.
5580
5581Tools for Hardening Your Image
5582------------------------------
5583
5584The Yocto Project provides tools for making your image more secure. You
5585can find these tools in the ``meta-security`` layer of the `Yocto
5586Project Source Repositories <&YOCTO_GIT_URL;>`__.
5587
5588Creating Your Own Distribution
5589==============================
5590
5591When you build an image using the Yocto Project and do not alter any
5592distribution `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__, you are
5593creating a Poky distribution. If you wish to gain more control over
5594package alternative selections, compile-time options, and other
5595low-level configurations, you can create your own distribution.
5596
5597To create your own distribution, the basic steps consist of creating
5598your own distribution layer, creating your own distribution
5599configuration file, and then adding any needed code and Metadata to the
5600layer. The following steps provide some more detail:
5601
5602- *Create a layer for your new distro:* Create your distribution layer
5603 so that you can keep your Metadata and code for the distribution
5604 separate. It is strongly recommended that you create and use your own
5605 layer for configuration and code. Using your own layer as compared to
5606 just placing configurations in a ``local.conf`` configuration file
5607 makes it easier to reproduce the same build configuration when using
5608 multiple build machines. See the "`Creating a General Layer Using the
5609 ``bitbake-layers``
5610 Script <#creating-a-general-layer-using-the-bitbake-layers-script>`__"
5611 section for information on how to quickly set up a layer.
5612
5613- *Create the distribution configuration file:* The distribution
5614 configuration file needs to be created in the ``conf/distro``
5615 directory of your layer. You need to name it using your distribution
5616 name (e.g. ``mydistro.conf``).
5617
5618 .. note::
5619
5620 The
5621 DISTRO
5622 variable in your
5623 local.conf
5624 file determines the name of your distribution.
5625
5626 You can split out parts of your configuration file into include files
5627 and then "require" them from within your distribution configuration
5628 file. Be sure to place the include files in the
5629 ``conf/distro/include`` directory of your layer. A common example
5630 usage of include files would be to separate out the selection of
5631 desired version and revisions for individual recipes.
5632
5633 Your configuration file needs to set the following required
5634 variables: ```DISTRO_NAME`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME>`__
5635 ```DISTRO_VERSION`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION>`__
5636 These following variables are optional and you typically set them
5637 from the distribution configuration file:
5638 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
5639 ```DISTRO_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS>`__
5640 ```DISTRO_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS>`__
5641 ```TCLIBC`` <&YOCTO_DOCS_REF_URL;#var-TCLIBC>`__
5642
5643 .. tip::
5644
5645 If you want to base your distribution configuration file on the
5646 very basic configuration from OE-Core, you can use
5647 conf/distro/defaultsetup.conf
5648 as a reference and just include variables that differ as compared
5649 to
5650 defaultsetup.conf
5651 . Alternatively, you can create a distribution configuration file
5652 from scratch using the
5653 defaultsetup.conf
5654 file or configuration files from other distributions such as Poky
5655 or Angstrom as references.
5656
5657- *Provide miscellaneous variables:* Be sure to define any other
5658 variables for which you want to create a default or enforce as part
5659 of the distribution configuration. You can include nearly any
5660 variable from the ``local.conf`` file. The variables you use are not
5661 limited to the list in the previous bulleted item.
5662
5663- *Point to Your distribution configuration file:* In your
5664 ``local.conf`` file in the `Build
5665 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, set your
5666 ```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__ variable to point to
5667 your distribution's configuration file. For example, if your
5668 distribution's configuration file is named ``mydistro.conf``, then
5669 you point to it as follows: DISTRO = "mydistro"
5670
5671- *Add more to the layer if necessary:* Use your layer to hold other
5672 information needed for the distribution:
5673
5674 - Add recipes for installing distro-specific configuration files
5675 that are not already installed by another recipe. If you have
5676 distro-specific configuration files that are included by an
5677 existing recipe, you should add an append file (``.bbappend``) for
5678 those. For general information and recommendations on how to add
5679 recipes to your layer, see the "`Creating Your Own
5680 Layer <#creating-your-own-layer>`__" and "`Following Best
5681 Practices When Creating
5682 Layers <#best-practices-to-follow-when-creating-layers>`__"
5683 sections.
5684
5685 - Add any image recipes that are specific to your distribution.
5686
5687 - Add a ``psplash`` append file for a branded splash screen. For
5688 information on append files, see the "`Using .bbappend Files in
5689 Your Layer <#using-bbappend-files>`__" section.
5690
5691 - Add any other append files to make custom changes that are
5692 specific to individual recipes.
5693
5694Creating a Custom Template Configuration Directory
5695==================================================
5696
5697If you are producing your own customized version of the build system for
5698use by other users, you might want to customize the message shown by the
5699setup script or you might want to change the template configuration
5700files (i.e. ``local.conf`` and ``bblayers.conf``) that are created in a
5701new build directory.
5702
5703The OpenEmbedded build system uses the environment variable
5704``TEMPLATECONF`` to locate the directory from which it gathers
5705configuration information that ultimately ends up in the `Build
5706Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ ``conf`` directory.
5707By default, ``TEMPLATECONF`` is set as follows in the ``poky``
5708repository: TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} This is the
5709directory used by the build system to find templates from which to build
5710some key configuration files. If you look at this directory, you will
5711see the ``bblayers.conf.sample``, ``local.conf.sample``, and
5712``conf-notes.txt`` files. The build system uses these files to form the
5713respective ``bblayers.conf`` file, ``local.conf`` file, and display the
5714list of BitBake targets when running the setup script.
5715
5716To override these default configuration files with configurations you
5717want used within every new Build Directory, simply set the
5718``TEMPLATECONF`` variable to your directory. The ``TEMPLATECONF``
5719variable is set in the ``.templateconf`` file, which is in the top-level
5720`Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ folder
5721(e.g. ``poky``). Edit the ``.templateconf`` so that it can locate your
5722directory.
5723
5724Best practices dictate that you should keep your template configuration
5725directory in your custom distribution layer. For example, suppose you
5726have a layer named ``meta-mylayer`` located in your home directory and
5727you want your template configuration directory named ``myconf``.
5728Changing the ``.templateconf`` as follows causes the OpenEmbedded build
5729system to look in your directory and base its configuration files on the
5730``*.sample`` configuration files it finds. The final configuration files
5731(i.e. ``local.conf`` and ``bblayers.conf`` ultimately still end up in
5732your Build Directory, but they are based on your ``*.sample`` files.
5733TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf}
5734
5735Aside from the ``*.sample`` configuration files, the ``conf-notes.txt``
5736also resides in the default ``meta-poky/conf`` directory. The script
5737that sets up the build environment (i.e.
5738````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) uses this file to
5739display BitBake targets as part of the script output. Customizing this
5740``conf-notes.txt`` file is a good way to make sure your list of custom
5741targets appears as part of the script's output.
5742
5743Here is the default list of targets displayed as a result of running
5744either of the setup scripts: You can now run 'bitbake <target>' Common
5745targets are: core-image-minimal core-image-sato meta-toolchain
5746meta-ide-support
5747
5748Changing the listed common targets is as easy as editing your version of
5749``conf-notes.txt`` in your custom template configuration directory and
5750making sure you have ``TEMPLATECONF`` set to your directory.
5751
5752.. _dev-saving-memory-during-a-build:
5753
5754Conserving Disk Space During Builds
5755===================================
5756
5757To help conserve disk space during builds, you can add the following
5758statement to your project's ``local.conf`` configuration file found in
5759the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT
5760+= "rm_work" Adding this statement deletes the work directory used for
5761building a recipe once the recipe is built. For more information on
5762"rm_work", see the
5763```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ class in the
5764Yocto Project Reference Manual.
5765
5766Working with Packages
5767=====================
5768
5769This section describes a few tasks that involve packages:
5770
5771- `Excluding packages from an
5772 image <#excluding-packages-from-an-image>`__
5773
5774- `Incrementing a binary package
5775 version <#incrementing-a-binary-package-version>`__
5776
5777- `Handling optional module
5778 packaging <#handling-optional-module-packaging>`__
5779
5780- `Using runtime package
5781 management <#using-runtime-package-management>`__
5782
5783- `Generating and using signed
5784 packages <#generating-and-using-signed-packages>`__
5785
5786- `Setting up and running package test
5787 (ptest) <#testing-packages-with-ptest>`__
5788
5789- `Creating node package manager (NPM)
5790 packages <#creating-node-package-manager-npm-packages>`__
5791
5792- `Adding custom metadata to
5793 packages <#adding-custom-metadata-to-packages>`__
5794
5795Excluding Packages from an Image
5796--------------------------------
5797
5798You might find it necessary to prevent specific packages from being
5799installed into an image. If so, you can use several variables to direct
5800the build system to essentially ignore installing recommended packages
5801or to not install a package at all.
5802
5803The following list introduces variables you can use to prevent packages
5804from being installed into your image. Each of these variables only works
5805with IPK and RPM package types. Support for Debian packages does not
5806exist. Also, you can use these variables from your ``local.conf`` file
5807or attach them to a specific image recipe by using a recipe name
5808override. For more detail on the variables, see the descriptions in the
5809Yocto Project Reference Manual's glossary chapter.
5810
5811- ```BAD_RECOMMENDATIONS`` <&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS>`__:
5812 Use this variable to specify "recommended-only" packages that you do
5813 not want installed.
5814
5815- ```NO_RECOMMENDATIONS`` <&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS>`__:
5816 Use this variable to prevent all "recommended-only" packages from
5817 being installed.
5818
5819- ```PACKAGE_EXCLUDE`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE>`__:
5820 Use this variable to prevent specific packages from being installed
5821 regardless of whether they are "recommended-only" or not. You need to
5822 realize that the build process could fail with an error when you
5823 prevent the installation of a package whose presence is required by
5824 an installed package.
5825
5826.. _incrementing-a-binary-package-version:
5827
5828Incrementing a Package Version
5829------------------------------
5830
5831This section provides some background on how binary package versioning
5832is accomplished and presents some of the services, variables, and
5833terminology involved.
5834
5835In order to understand binary package versioning, you need to consider
5836the following:
5837
5838- Binary Package: The binary package that is eventually built and
5839 installed into an image.
5840
5841- Binary Package Version: The binary package version is composed of two
5842 components - a version and a revision.
5843
5844 .. note::
5845
5846 Technically, a third component, the "epoch" (i.e.
5847 PE
5848 ) is involved but this discussion for the most part ignores
5849 PE
5850 .
5851
5852 The version and revision are taken from the
5853 ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ and
5854 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ variables, respectively.
5855
5856- ``PV``: The recipe version. ``PV`` represents the version of the
5857 software being packaged. Do not confuse ``PV`` with the binary
5858 package version.
5859
5860- ``PR``: The recipe revision.
5861
5862- ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__: The OpenEmbedded
5863 build system uses this string to help define the value of ``PV`` when
5864 the source code revision needs to be included in it.
5865
5866- `PR Service <https://wiki.yoctoproject.org/wiki/PR_Service>`__: A
5867 network-based service that helps automate keeping package feeds
5868 compatible with existing package manager applications such as RPM,
5869 APT, and OPKG.
5870
5871Whenever the binary package content changes, the binary package version
5872must change. Changing the binary package version is accomplished by
5873changing or "bumping" the ``PR`` and/or ``PV`` values. Increasing these
5874values occurs one of two ways:
5875
5876- Automatically using a Package Revision Service (PR Service).
5877
5878- Manually incrementing the ``PR`` and/or ``PV`` variables.
5879
5880Given a primary challenge of any build system and its users is how to
5881maintain a package feed that is compatible with existing package manager
5882applications such as RPM, APT, and OPKG, using an automated system is
5883much preferred over a manual system. In either system, the main
5884requirement is that binary package version numbering increases in a
5885linear fashion and that a number of version components exist that
5886support that linear progression. For information on how to ensure
5887package revisioning remains linear, see the "`Automatically Incrementing
5888a Binary Package Revision
5889Number <#automatically-incrementing-a-binary-package-revision-number>`__"
5890section.
5891
5892The following three sections provide related information on the PR
5893Service, the manual method for "bumping" ``PR`` and/or ``PV``, and on
5894how to ensure binary package revisioning remains linear.
5895
5896Working With a PR Service
5897~~~~~~~~~~~~~~~~~~~~~~~~~
5898
5899As mentioned, attempting to maintain revision numbers in the
5900`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ is error prone, inaccurate,
5901and causes problems for people submitting recipes. Conversely, the PR
5902Service automatically generates increasing numbers, particularly the
5903revision field, which removes the human element.
5904
5905.. note::
5906
5907 For additional information on using a PR Service, you can see the
5908 PR Service
5909 wiki page.
5910
5911The Yocto Project uses variables in order of decreasing priority to
5912facilitate revision numbering (i.e.
5913```PE`` <&YOCTO_DOCS_REF_URL;#var-PE>`__,
5914```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__, and
5915```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ for epoch, version, and
5916revision, respectively). The values are highly dependent on the policies
5917and procedures of a given distribution and package feed.
5918
5919Because the OpenEmbedded build system uses
5920"`signatures <&YOCTO_DOCS_OM_URL;#overview-checksums>`__", which are
5921unique to a given build, the build system knows when to rebuild
5922packages. All the inputs into a given task are represented by a
5923signature, which can trigger a rebuild when different. Thus, the build
5924system itself does not rely on the ``PR``, ``PV``, and ``PE`` numbers to
5925trigger a rebuild. The signatures, however, can be used to generate
5926these values.
5927
5928The PR Service works with both ``OEBasic`` and ``OEBasicHash``
5929generators. The value of ``PR`` bumps when the checksum changes and the
5930different generator mechanisms change signatures under different
5931circumstances.
5932
5933As implemented, the build system includes values from the PR Service
5934into the ``PR`` field as an addition using the form "``.x``" so ``r0``
5935becomes ``r0.1``, ``r0.2`` and so forth. This scheme allows existing
5936``PR`` values to be used for whatever reasons, which include manual
5937``PR`` bumps, should it be necessary.
5938
5939By default, the PR Service is not enabled or running. Thus, the packages
5940generated are just "self consistent". The build system adds and removes
5941packages and there are no guarantees about upgrade paths but images will
5942be consistent and correct with the latest changes.
5943
5944The simplest form for a PR Service is for it to exist for a single host
5945development system that builds the package feed (building system). For
5946this scenario, you can enable a local PR Service by setting
5947```PRSERV_HOST`` <&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST>`__ in your
5948``local.conf`` file in the `Build
5949Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: PRSERV_HOST =
5950"localhost:0" Once the service is started, packages will automatically
5951get increasing ``PR`` values and BitBake takes care of starting and
5952stopping the server.
5953
5954If you have a more complex setup where multiple host development systems
5955work against a common, shared package feed, you have a single PR Service
5956running and it is connected to each building system. For this scenario,
5957you need to start the PR Service using the ``bitbake-prserv`` command:
5958bitbake-prserv --host ip --port port --start In addition to
5959hand-starting the service, you need to update the ``local.conf`` file of
5960each building system as described earlier so each system points to the
5961server and port.
5962
5963It is also recommended you use build history, which adds some sanity
5964checks to binary package versions, in conjunction with the server that
5965is running the PR Service. To enable build history, add the following to
5966each building system's ``local.conf`` file: # It is recommended to
5967activate "buildhistory" for testing the PR service INHERIT +=
5968"buildhistory" BUILDHISTORY_COMMIT = "1" For information on build
5969history, see the "`Maintaining Build Output
5970Quality <#maintaining-build-output-quality>`__" section.
5971
5972.. note::
5973
5974 The OpenEmbedded build system does not maintain ``PR`` information as
5975 part of the shared state (sstate) packages. If you maintain an sstate
5976 feed, its expected that either all your building systems that
5977 contribute to the sstate feed use a shared PR Service, or you do not
5978 run a PR Service on any of your building systems. Having some systems
5979 use a PR Service while others do not leads to obvious problems.
5980
5981 For more information on shared state, see the "`Shared State
5982 Cache <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__" section in the
5983 Yocto Project Overview and Concepts Manual.
5984
5985Manually Bumping PR
5986~~~~~~~~~~~~~~~~~~~
5987
5988The alternative to setting up a PR Service is to manually "bump" the
5989```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ variable.
5990
5991If a committed change results in changing the package output, then the
5992value of the PR variable needs to be increased (or "bumped") as part of
5993that commit. For new recipes you should add the ``PR`` variable and set
5994its initial value equal to "r0", which is the default. Even though the
5995default value is "r0", the practice of adding it to a new recipe makes
5996it harder to forget to bump the variable when you make changes to the
5997recipe in future.
5998
5999If you are sharing a common ``.inc`` file with multiple recipes, you can
6000also use the ``INC_PR`` variable to ensure that the recipes sharing the
6001``.inc`` file are rebuilt when the ``.inc`` file itself is changed. The
6002``.inc`` file must set ``INC_PR`` (initially to "r0"), and all recipes
6003referring to it should set ``PR`` to "${INC_PR}.0" initially,
6004incrementing the last number when the recipe is changed. If the ``.inc``
6005file is changed then its ``INC_PR`` should be incremented.
6006
6007When upgrading the version of a binary package, assuming the ``PV``
6008changes, the ``PR`` variable should be reset to "r0" (or "${INC_PR}.0"
6009if you are using ``INC_PR``).
6010
6011Usually, version increases occur only to binary packages. However, if
6012for some reason ``PV`` changes but does not increase, you can increase
6013the ``PE`` variable (Package Epoch). The ``PE`` variable defaults to
6014"0".
6015
6016Binary package version numbering strives to follow the `Debian Version
6017Field Policy
6018Guidelines <http://www.debian.org/doc/debian-policy/ch-controlfields.html>`__.
6019These guidelines define how versions are compared and what "increasing"
6020a version means.
6021
6022.. _automatically-incrementing-a-binary-package-revision-number:
6023
6024Automatically Incrementing a Package Version Number
6025~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6026
6027When fetching a repository, BitBake uses the
6028```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ variable to determine
6029the specific source code revision from which to build. You set the
6030``SRCREV`` variable to
6031```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__ to cause the
6032OpenEmbedded build system to automatically use the latest revision of
6033the software: SRCREV = "${AUTOREV}"
6034
6035Furthermore, you need to reference ``SRCPV`` in ``PV`` in order to
6036automatically update the version whenever the revision of the source
6037code changes. Here is an example: PV = "1.0+git${SRCPV}" The
6038OpenEmbedded build system substitutes ``SRCPV`` with the following:
6039AUTOINC+source_code_revision The build system replaces the ``AUTOINC``
6040with a number. The number used depends on the state of the PR Service:
6041
6042- If PR Service is enabled, the build system increments the number,
6043 which is similar to the behavior of
6044 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__. This behavior results in
6045 linearly increasing package versions, which is desirable. Here is an
6046 example: hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6047 hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk
6048
6049- If PR Service is not enabled, the build system replaces the
6050 ``AUTOINC`` placeholder with zero (i.e. "0"). This results in
6051 changing the package version since the source revision is included.
6052 However, package versions are not increased linearly. Here is an
6053 example: hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6054 hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk
6055
6056In summary, the OpenEmbedded build system does not track the history of
6057binary package versions for this purpose. ``AUTOINC``, in this case, is
6058comparable to ``PR``. If PR server is not enabled, ``AUTOINC`` in the
6059package version is simply replaced by "0". If PR server is enabled, the
6060build system keeps track of the package versions and bumps the number
6061when the package revision changes.
6062
6063Handling Optional Module Packaging
6064----------------------------------
6065
6066Many pieces of software split functionality into optional modules (or
6067plugins) and the plugins that are built might depend on configuration
6068options. To avoid having to duplicate the logic that determines what
6069modules are available in your recipe or to avoid having to package each
6070module by hand, the OpenEmbedded build system provides functionality to
6071handle module packaging dynamically.
6072
6073To handle optional module packaging, you need to do two things:
6074
6075- Ensure the module packaging is actually done.
6076
6077- Ensure that any dependencies on optional modules from other recipes
6078 are satisfied by your recipe.
6079
6080Making Sure the Packaging is Done
6081~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6082
6083To ensure the module packaging actually gets done, you use the
6084``do_split_packages`` function within the ``populate_packages`` Python
6085function in your recipe. The ``do_split_packages`` function searches for
6086a pattern of files or directories under a specified path and creates a
6087package for each one it finds by appending to the
6088```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ variable and
6089setting the appropriate values for ``FILES_packagename``,
6090``RDEPENDS_packagename``, ``DESCRIPTION_packagename``, and so forth.
6091Here is an example from the ``lighttpd`` recipe: python
6092populate_packages_prepend () { lighttpd_libdir = d.expand('${libdir}')
6093do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
6094'lighttpd-module-%s', 'Lighttpd module for %s', extra_depends='') } The
6095previous example specifies a number of things in the call to
6096``do_split_packages``.
6097
6098- A directory within the files installed by your recipe through
6099 ``do_install`` in which to search.
6100
6101- A regular expression used to match module files in that directory. In
6102 the example, note the parentheses () that mark the part of the
6103 expression from which the module name should be derived.
6104
6105- A pattern to use for the package names.
6106
6107- A description for each package.
6108
6109- An empty string for ``extra_depends``, which disables the default
6110 dependency on the main ``lighttpd`` package. Thus, if a file in
6111 ``${libdir}`` called ``mod_alias.so`` is found, a package called
6112 ``lighttpd-module-alias`` is created for it and the
6113 ```DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-DESCRIPTION>`__ is set to
6114 "Lighttpd module for alias".
6115
6116Often, packaging modules is as simple as the previous example. However,
6117more advanced options exist that you can use within
6118``do_split_packages`` to modify its behavior. And, if you need to, you
6119can add more logic by specifying a hook function that is called for each
6120package. It is also perfectly acceptable to call ``do_split_packages``
6121multiple times if you have more than one set of modules to package.
6122
6123For more examples that show how to use ``do_split_packages``, see the
6124``connman.inc`` file in the ``meta/recipes-connectivity/connman/``
6125directory of the ``poky`` `source
6126repository <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__. You can
6127also find examples in ``meta/classes/kernel.bbclass``.
6128
6129Following is a reference that shows ``do_split_packages`` mandatory and
6130optional arguments: Mandatory arguments root The path in which to search
6131file_regex Regular expression to match searched files. Use parentheses
6132() to mark the part of this expression that should be used to derive the
6133module name (to be substituted where %s is used in other function
6134arguments as noted below) output_pattern Pattern to use for the package
6135names. Must include %s. description Description to set for each package.
6136Must include %s. Optional arguments postinst Postinstall script to use
6137for all packages (as a string) recursive True to perform a recursive
6138search - default False hook A hook function to be called for every
6139match. The function will be called with the following arguments (in the
6140order listed): f Full path to the file/directory match pkg The package
6141name file_regex As above output_pattern As above modulename The module
6142name derived using file_regex extra_depends Extra runtime dependencies
6143(RDEPENDS) to be set for all packages. The default value of None causes
6144a dependency on the main package (${PN}) - if you do not want this, pass
6145empty string '' for this parameter. aux_files_pattern Extra item(s) to
6146be added to FILES for each package. Can be a single string item or a
6147list of strings for multiple items. Must include %s. postrm postrm
6148script to use for all packages (as a string) allow_dirs True to allow
6149directories to be matched - default False prepend If True, prepend
6150created packages to PACKAGES instead of the default False which appends
6151them match_path match file_regex on the whole relative path to the root
6152rather than just the file name aux_files_pattern_verbatim Extra item(s)
6153to be added to FILES for each package, using the actual derived module
6154name rather than converting it to something legal for a package name.
6155Can be a single string item or a list of strings for multiple items.
6156Must include %s. allow_links True to allow symlinks to be matched -
6157default False summary Summary to set for each package. Must include %s;
6158defaults to description if not set.
6159
6160Satisfying Dependencies
6161~~~~~~~~~~~~~~~~~~~~~~~
6162
6163The second part for handling optional module packaging is to ensure that
6164any dependencies on optional modules from other recipes are satisfied by
6165your recipe. You can be sure these dependencies are satisfied by using
6166the ```PACKAGES_DYNAMIC`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC>`__
6167variable. Here is an example that continues with the ``lighttpd`` recipe
6168shown earlier: PACKAGES_DYNAMIC = "lighttpd-module-.*" The name
6169specified in the regular expression can of course be anything. In this
6170example, it is ``lighttpd-module-`` and is specified as the prefix to
6171ensure that any ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ and
6172```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ on a package
6173name starting with the prefix are satisfied during build time. If you
6174are using ``do_split_packages`` as described in the previous section,
6175the value you put in ``PACKAGES_DYNAMIC`` should correspond to the name
6176pattern specified in the call to ``do_split_packages``.
6177
6178Using Runtime Package Management
6179--------------------------------
6180
6181During a build, BitBake always transforms a recipe into one or more
6182packages. For example, BitBake takes the ``bash`` recipe and produces a
6183number of packages (e.g. ``bash``, ``bash-bashbug``,
6184``bash-completion``, ``bash-completion-dbg``, ``bash-completion-dev``,
6185``bash-completion-extra``, ``bash-dbg``, and so forth). Not all
6186generated packages are included in an image.
6187
6188In several situations, you might need to update, add, remove, or query
6189the packages on a target device at runtime (i.e. without having to
6190generate a new image). Examples of such situations include:
6191
6192- You want to provide in-the-field updates to deployed devices (e.g.
6193 security updates).
6194
6195- You want to have a fast turn-around development cycle for one or more
6196 applications that run on your device.
6197
6198- You want to temporarily install the "debug" packages of various
6199 applications on your device so that debugging can be greatly improved
6200 by allowing access to symbols and source debugging.
6201
6202- You want to deploy a more minimal package selection of your device
6203 but allow in-the-field updates to add a larger selection for
6204 customization.
6205
6206In all these situations, you have something similar to a more
6207traditional Linux distribution in that in-field devices are able to
6208receive pre-compiled packages from a server for installation or update.
6209Being able to install these packages on a running, in-field device is
6210what is termed "runtime package management".
6211
6212In order to use runtime package management, you need a host or server
6213machine that serves up the pre-compiled packages plus the required
6214metadata. You also need package manipulation tools on the target. The
6215build machine is a likely candidate to act as the server. However, that
6216machine does not necessarily have to be the package server. The build
6217machine could push its artifacts to another machine that acts as the
6218server (e.g. Internet-facing). In fact, doing so is advantageous for a
6219production environment as getting the packages away from the development
6220system's build directory prevents accidental overwrites.
6221
6222A simple build that targets just one device produces more than one
6223package database. In other words, the packages produced by a build are
6224separated out into a couple of different package groupings based on
6225criteria such as the target's CPU architecture, the target board, or the
6226C library used on the target. For example, a build targeting the
6227``qemux86`` device produces the following three package databases:
6228``noarch``, ``i586``, and ``qemux86``. If you wanted your ``qemux86``
6229device to be aware of all the packages that were available to it, you
6230would need to point it to each of these databases individually. In a
6231similar way, a traditional Linux distribution usually is configured to
6232be aware of a number of software repositories from which it retrieves
6233packages.
6234
6235Using runtime package management is completely optional and not required
6236for a successful build or deployment in any way. But if you want to make
6237use of runtime package management, you need to do a couple things above
6238and beyond the basics. The remainder of this section describes what you
6239need to do.
6240
6241.. _runtime-package-management-build:
6242
6243Build Considerations
6244~~~~~~~~~~~~~~~~~~~~
6245
6246This section describes build considerations of which you need to be
6247aware in order to provide support for runtime package management.
6248
6249When BitBake generates packages, it needs to know what format or formats
6250to use. In your configuration, you use the
6251```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
6252variable to specify the format:
6253
62541. Open the ``local.conf`` file inside your `Build
6255 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (e.g.
6256 ``~/poky/build/conf/local.conf``).
6257
62582. Select the desired package format as follows: PACKAGE_CLASSES ?=
6259 “package_packageformat” where packageformat can be "ipk", "rpm",
6260 "deb", or "tar" which are the supported package formats.
6261
6262 .. note::
6263
6264 Because the Yocto Project supports four different package formats,
6265 you can set the variable with more than one argument. However, the
6266 OpenEmbedded build system only uses the first argument when
6267 creating an image or Software Development Kit (SDK).
6268
6269If you would like your image to start off with a basic package database
6270containing the packages in your current build as well as to have the
6271relevant tools available on the target for runtime package management,
6272you can include "package-management" in the
6273```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__
6274variable. Including "package-management" in this configuration variable
6275ensures that when the image is assembled for your target, the image
6276includes the currently-known package databases as well as the
6277target-specific tools required for runtime package management to be
6278performed on the target. However, this is not strictly necessary. You
6279could start your image off without any databases but only include the
6280required on-target package tool(s). As an example, you could include
6281"opkg" in your
6282```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__ variable
6283if you are using the IPK package format. You can then initialize your
6284target's package database(s) later once your image is up and running.
6285
6286Whenever you perform any sort of build step that can potentially
6287generate a package or modify existing package, it is always a good idea
6288to re-generate the package index after the build by using the following
6289command: $ bitbake package-index It might be tempting to build the
6290package and the package index at the same time with a command such as
6291the following: $ bitbake some-package package-index Do not do this as
6292BitBake does not schedule the package index for after the completion of
6293the package you are building. Consequently, you cannot be sure of the
6294package index including information for the package you just built.
6295Thus, be sure to run the package update step separately after building
6296any packages.
6297
6298You can use the
6299```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6300```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6301and
6302```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6303variables to pre-configure target images to use a package feed. If you
6304do not define these variables, then manual steps as described in the
6305subsequent sections are necessary to configure the target. You should
6306set these variables before building the image in order to produce a
6307correctly configured image.
6308
6309When your build is complete, your packages reside in the
6310``${TMPDIR}/deploy/packageformat`` directory. For example, if
6311``${``\ ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__\ ``}`` is
6312``tmp`` and your selected package type is RPM, then your RPM packages
6313are available in ``tmp/deploy/rpm``.
6314
6315.. _runtime-package-management-server:
6316
6317Host or Server Machine Setup
6318~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6319
6320Although other protocols are possible, a server using HTTP typically
6321serves packages. If you want to use HTTP, then set up and configure a
6322web server such as Apache 2, lighttpd, or SimpleHTTPServer on the
6323machine serving the packages.
6324
6325To keep things simple, this section describes how to set up a
6326SimpleHTTPServer web server to share package feeds from the developer's
6327machine. Although this server might not be the best for a production
6328environment, the setup is simple and straight forward. Should you want
6329to use a different server more suited for production (e.g. Apache 2,
6330Lighttpd, or Nginx), take the appropriate steps to do so.
6331
6332From within the build directory where you have built an image based on
6333your packaging choice (i.e. the
6334```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
6335setting), simply start the server. The following example assumes a build
6336directory of ``~/poky/build/tmp/deploy/rpm`` and a ``PACKAGE_CLASSES``
6337setting of "package_rpm": $ cd ~/poky/build/tmp/deploy/rpm $ python -m
6338SimpleHTTPServer
6339
6340.. _runtime-package-management-target:
6341
6342Target Setup
6343~~~~~~~~~~~~
6344
6345Setting up the target differs depending on the package management
6346system. This section provides information for RPM, IPK, and DEB.
6347
6348.. _runtime-package-management-target-rpm:
6349
6350Using RPM
6351^^^^^^^^^
6352
6353The `Dandified Packaging
6354Tool <https://en.wikipedia.org/wiki/DNF_(software)>`__ (DNF) performs
6355runtime package management of RPM packages. In order to use DNF for
6356runtime package management, you must perform an initial setup on the
6357target machine for cases where the ``PACKAGE_FEED_*`` variables were not
6358set as part of the image that is running on the target. This means if
6359you built your image and did not not use these variables as part of the
6360build and your image is now running on the target, you need to perform
6361the steps in this section if you want to use runtime package management.
6362
6363.. note::
6364
6365 For information on the
6366 PACKAGE_FEED_\*
6367 variables, see
6368 PACKAGE_FEED_ARCHS
6369 ,
6370 PACKAGE_FEED_BASE_PATHS
6371 , and
6372 PACKAGE_FEED_URIS
6373 in the Yocto Project Reference Manual variables glossary.
6374
6375On the target, you must inform DNF that package databases are available.
6376You do this by creating a file named
6377``/etc/yum.repos.d/oe-packages.repo`` and defining the ``oe-packages``.
6378
6379As an example, assume the target is able to use the following package
6380databases: ``all``, ``i586``, and ``qemux86`` from a server named
6381``my.server``. The specifics for setting up the web server are up to
6382you. The critical requirement is that the URIs in the target repository
6383configuration point to the correct remote location for the feeds.
6384
6385.. note::
6386
6387 For development purposes, you can point the web server to the build
6388 system's
6389 deploy
6390 directory. However, for production use, it is better to copy the
6391 package directories to a location outside of the build area and use
6392 that location. Doing so avoids situations where the build system
6393 overwrites or changes the
6394 deploy
6395 directory.
6396
6397When telling DNF where to look for the package databases, you must
6398declare individual locations per architecture or a single location used
6399for all architectures. You cannot do both:
6400
6401- *Create an Explicit List of Architectures:* Define individual base
6402 URLs to identify where each package database is located:
6403 [oe-packages] baseurl=http://my.server/rpm/i586
6404 http://my.server/rpm/qemux86 http://my.server/rpm/all This example
6405 informs DNF about individual package databases for all three
6406 architectures.
6407
6408- *Create a Single (Full) Package Index:* Define a single base URL that
6409 identifies where a full package database is located: [oe-packages]
6410 baseurl=http://my.server/rpm This example informs DNF about a single
6411 package database that contains all the package index information for
6412 all supported architectures.
6413
6414Once you have informed DNF where to find the package databases, you need
6415to fetch them: # dnf makecache DNF is now able to find, install, and
6416upgrade packages from the specified repository or repositories.
6417
6418.. note::
6419
6420 See the
6421 DNF documentation
6422 for additional information.
6423
6424.. _runtime-package-management-target-ipk:
6425
6426Using IPK
6427^^^^^^^^^
6428
6429The ``opkg`` application performs runtime package management of IPK
6430packages. You must perform an initial setup for ``opkg`` on the target
6431machine if the
6432```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6433```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6434and
6435```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6436variables have not been set or the target image was built before the
6437variables were set.
6438
6439The ``opkg`` application uses configuration files to find available
6440package databases. Thus, you need to create a configuration file inside
6441the ``/etc/opkg/`` direction, which informs ``opkg`` of any repository
6442you want to use.
6443
6444As an example, suppose you are serving packages from a ``ipk/``
6445directory containing the ``i586``, ``all``, and ``qemux86`` databases
6446through an HTTP server named ``my.server``. On the target, create a
6447configuration file (e.g. ``my_repo.conf``) inside the ``/etc/opkg/``
6448directory containing the following: src/gz all http://my.server/ipk/all
6449src/gz i586 http://my.server/ipk/i586 src/gz qemux86
6450http://my.server/ipk/qemux86 Next, instruct ``opkg`` to fetch the
6451repository information: # opkg update The ``opkg`` application is now
6452able to find, install, and upgrade packages from the specified
6453repository.
6454
6455.. _runtime-package-management-target-deb:
6456
6457Using DEB
6458^^^^^^^^^
6459
6460The ``apt`` application performs runtime package management of DEB
6461packages. This application uses a source list file to find available
6462package databases. You must perform an initial setup for ``apt`` on the
6463target machine if the
6464```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6465```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6466and
6467```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6468variables have not been set or the target image was built before the
6469variables were set.
6470
6471To inform ``apt`` of the repository you want to use, you might create a
6472list file (e.g. ``my_repo.list``) inside the
6473``/etc/apt/sources.list.d/`` directory. As an example, suppose you are
6474serving packages from a ``deb/`` directory containing the ``i586``,
6475``all``, and ``qemux86`` databases through an HTTP server named
6476``my.server``. The list file should contain: deb
6477http://my.server/deb/all ./ deb http://my.server/deb/i586 ./ deb
6478http://my.server/deb/qemux86 ./ Next, instruct the ``apt`` application
6479to fetch the repository information: # apt-get update After this step,
6480``apt`` is able to find, install, and upgrade packages from the
6481specified repository.
6482
6483Generating and Using Signed Packages
6484------------------------------------
6485
6486In order to add security to RPM packages used during a build, you can
6487take steps to securely sign them. Once a signature is verified, the
6488OpenEmbedded build system can use the package in the build. If security
6489fails for a signed package, the build system aborts the build.
6490
6491This section describes how to sign RPM packages during a build and how
6492to use signed package feeds (repositories) when doing a build.
6493
6494Signing RPM Packages
6495~~~~~~~~~~~~~~~~~~~~
6496
6497To enable signing RPM packages, you must set up the following
6498configurations in either your ``local.config`` or ``distro.config``
6499file: # Inherit sign_rpm.bbclass to enable signing functionality INHERIT
6500+= " sign_rpm" # Define the GPG key that will be used for signing.
6501RPM_GPG_NAME = "key_name" # Provide passphrase for the key
6502RPM_GPG_PASSPHRASE = "passphrase"
6503
6504.. note::
6505
6506 Be sure to supply appropriate values for both
6507 key_name
6508 and
6509 passphrase
6510
6511Aside from the ``RPM_GPG_NAME`` and ``RPM_GPG_PASSPHRASE`` variables in
6512the previous example, two optional variables related to signing exist:
6513
6514- *``GPG_BIN``:* Specifies a ``gpg`` binary/wrapper that is executed
6515 when the package is signed.
6516
6517- *``GPG_PATH``:* Specifies the ``gpg`` home directory used when the
6518 package is signed.
6519
6520Processing Package Feeds
6521~~~~~~~~~~~~~~~~~~~~~~~~
6522
6523In addition to being able to sign RPM packages, you can also enable
6524signed package feeds for IPK and RPM packages.
6525
6526The steps you need to take to enable signed package feed use are similar
6527to the steps used to sign RPM packages. You must define the following in
6528your ``local.config`` or ``distro.config`` file: INHERIT +=
6529"sign_package_feed" PACKAGE_FEED_GPG_NAME = "key_name"
6530PACKAGE_FEED_GPG_PASSPHRASE_FILE = "path_to_file_containing_passphrase"
6531For signed package feeds, the passphrase must exist in a separate file,
6532which is pointed to by the ``PACKAGE_FEED_GPG_PASSPHRASE_FILE``
6533variable. Regarding security, keeping a plain text passphrase out of the
6534configuration is more secure.
6535
6536Aside from the ``PACKAGE_FEED_GPG_NAME`` and
6537``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` variables, three optional variables
6538related to signed package feeds exist:
6539
6540- *``GPG_BIN``:* Specifies a ``gpg`` binary/wrapper that is executed
6541 when the package is signed.
6542
6543- *``GPG_PATH``:* Specifies the ``gpg`` home directory used when the
6544 package is signed.
6545
6546- *``PACKAGE_FEED_GPG_SIGNATURE_TYPE``:* Specifies the type of ``gpg``
6547 signature. This variable applies only to RPM and IPK package feeds.
6548 Allowable values for the ``PACKAGE_FEED_GPG_SIGNATURE_TYPE`` are
6549 "ASC", which is the default and specifies ascii armored, and "BIN",
6550 which specifies binary.
6551
6552Testing Packages With ptest
6553---------------------------
6554
6555A Package Test (ptest) runs tests against packages built by the
6556OpenEmbedded build system on the target machine. A ptest contains at
6557least two items: the actual test, and a shell script (``run-ptest``)
6558that starts the test. The shell script that starts the test must not
6559contain the actual test - the script only starts the test. On the other
6560hand, the test can be anything from a simple shell script that runs a
6561binary and checks the output to an elaborate system of test binaries and
6562data files.
6563
6564The test generates output in the format used by Automake: result:
6565testname where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
6566the testname can be any identifying string.
6567
6568For a list of Yocto Project recipes that are already enabled with ptest,
6569see the `Ptest <https://wiki.yoctoproject.org/wiki/Ptest>`__ wiki page.
6570
6571.. note::
6572
6573 A recipe is "ptest-enabled" if it inherits the
6574 ptest
6575 class.
6576
6577Adding ptest to Your Build
6578~~~~~~~~~~~~~~~~~~~~~~~~~~
6579
6580To add package testing to your build, add the
6581```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__ and
6582```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
6583variables to your ``local.conf`` file, which is found in the `Build
6584Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__:
6585DISTRO_FEATURES_append = " ptest" EXTRA_IMAGE_FEATURES += "ptest-pkgs"
6586Once your build is complete, the ptest files are installed into the
6587``/usr/lib/package/ptest`` directory within the image, where ``package``
6588is the name of the package.
6589
6590Running ptest
6591~~~~~~~~~~~~~
6592
6593The ``ptest-runner`` package installs a shell script that loops through
6594all installed ptest test suites and runs them in sequence. Consequently,
6595you might want to add this package to your image.
6596
6597Getting Your Package Ready
6598~~~~~~~~~~~~~~~~~~~~~~~~~~
6599
6600In order to enable a recipe to run installed ptests on target hardware,
6601you need to prepare the recipes that build the packages you want to
6602test. Here is what you have to do for each recipe:
6603
6604- *Be sure the recipe inherits
6605 the*\ ```ptest`` <&YOCTO_DOCS_REF_URL;#ref-classes-ptest>`__\ *class:*
6606 Include the following line in each recipe: inherit ptest
6607
6608- *Create ``run-ptest``:* This script starts your test. Locate the
6609 script where you will refer to it using
6610 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__. Here is an
6611 example that starts a test for ``dbus``: #!/bin/sh cd test make -k
6612 runtest-TESTS
6613
6614- *Ensure dependencies are met:* If the test adds build or runtime
6615 dependencies that normally do not exist for the package (such as
6616 requiring "make" to run the test suite), use the
6617 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ and
6618 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ variables in
6619 your recipe in order for the package to meet the dependencies. Here
6620 is an example where the package has a runtime dependency on "make":
6621 RDEPENDS_${PN}-ptest += "make"
6622
6623- *Add a function to build the test suite:* Not many packages support
6624 cross-compilation of their test suites. Consequently, you usually
6625 need to add a cross-compilation function to the package.
6626
6627 Many packages based on Automake compile and run the test suite by
6628 using a single command such as ``make check``. However, the host
6629 ``make check`` builds and runs on the same computer, while
6630 cross-compiling requires that the package is built on the host but
6631 executed for the target architecture (though often, as in the case
6632 for ptest, the execution occurs on the host). The built version of
6633 Automake that ships with the Yocto Project includes a patch that
6634 separates building and execution. Consequently, packages that use the
6635 unaltered, patched version of ``make check`` automatically
6636 cross-compiles.
6637
6638 Regardless, you still must add a ``do_compile_ptest`` function to
6639 build the test suite. Add a function similar to the following to your
6640 recipe: do_compile_ptest() { oe_runmake buildtest-TESTS }
6641
6642- *Ensure special configurations are set:* If the package requires
6643 special configurations prior to compiling the test code, you must
6644 insert a ``do_configure_ptest`` function into the recipe.
6645
6646- *Install the test suite:* The ``ptest`` class automatically copies
6647 the file ``run-ptest`` to the target and then runs make
6648 ``install-ptest`` to run the tests. If this is not enough, you need
6649 to create a ``do_install_ptest`` function and make sure it gets
6650 called after the "make install-ptest" completes.
6651
6652Creating Node Package Manager (NPM) Packages
6653--------------------------------------------
6654
6655`NPM <https://en.wikipedia.org/wiki/Npm_(software)>`__ is a package
6656manager for the JavaScript programming language. The Yocto Project
6657supports the NPM `fetcher <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__. You can
6658use this fetcher in combination with
6659```devtool`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__ to create
6660recipes that produce NPM packages.
6661
6662Two workflows exist that allow you to create NPM packages using
6663``devtool``: the NPM registry modules method and the NPM project code
6664method.
6665
6666.. note::
6667
6668 While it is possible to create NPM recipes manually, using
6669 devtool
6670 is far simpler.
6671
6672Additionally, some requirements and caveats exist.
6673
6674.. _npm-package-creation-requirements:
6675
6676Requirements and Caveats
6677~~~~~~~~~~~~~~~~~~~~~~~~
6678
6679You need to be aware of the following before using ``devtool`` to create
6680NPM packages:
6681
6682- Of the two methods that you can use ``devtool`` to create NPM
6683 packages, the registry approach is slightly simpler. However, you
6684 might consider the project approach because you do not have to
6685 publish your module in the NPM registry
6686 (```npm-registry`` <https://docs.npmjs.com/misc/registry>`__), which
6687 is NPM's public registry.
6688
6689- Be familiar with
6690 ```devtool`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__.
6691
6692- The NPM host tools need the native ``nodejs-npm`` package, which is
6693 part of the OpenEmbedded environment. You need to get the package by
6694 cloning the ` <https://github.com/openembedded/meta-openembedded>`__
6695 repository out of GitHub. Be sure to add the path to your local copy
6696 to your ``bblayers.conf`` file.
6697
6698- ``devtool`` cannot detect native libraries in module dependencies.
6699 Consequently, you must manually add packages to your recipe.
6700
6701- While deploying NPM packages, ``devtool`` cannot determine which
6702 dependent packages are missing on the target (e.g. the node runtime
6703 ``nodejs``). Consequently, you need to find out what files are
6704 missing and be sure they are on the target.
6705
6706- Although you might not need NPM to run your node package, it is
6707 useful to have NPM on your target. The NPM package name is
6708 ``nodejs-npm``.
6709
6710.. _npm-using-the-registry-modules-method:
6711
6712Using the Registry Modules Method
6713~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6714
6715This section presents an example that uses the ``cute-files`` module,
6716which is a file browser web application.
6717
6718.. note::
6719
6720 You must know the
6721 cute-files
6722 module version.
6723
6724The first thing you need to do is use ``devtool`` and the NPM fetcher to
6725create the recipe: $ devtool add
6726"npm://registry.npmjs.org;package=cute-files;version=1.0.2" The
6727``devtool add`` command runs ``recipetool create`` and uses the same
6728fetch URI to download each dependency and capture license details where
6729possible. The result is a generated recipe.
6730
6731The recipe file is fairly simple and contains every license that
6732``recipetool`` finds and includes the licenses in the recipe's
6733```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
6734variables. You need to examine the variables and look for those with
6735"unknown" in the ```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__
6736field. You need to track down the license information for "unknown"
6737modules and manually add the information to the recipe.
6738
6739``recipetool`` creates a "shrinkwrap" file for your recipe. Shrinkwrap
6740files capture the version of all dependent modules. Many packages do not
6741provide shrinkwrap files. ``recipetool`` create a shrinkwrap file as it
6742runs.
6743
6744.. note::
6745
6746 A package is created for each sub-module. This policy is the only
6747 practical way to have the licenses for all of the dependencies
6748 represented in the license manifest of the image.
6749
6750The ``devtool edit-recipe`` command lets you take a look at the recipe:
6751$ devtool edit-recipe cute-files SUMMARY = "Turn any folder on your
6752computer into a cute file browser, available on the local network."
6753LICENSE = "MIT & ISC & Unknown" LIC_FILES_CHKSUM =
6754"file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \\
6755file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502
6756\\
6757file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99
6758\\ ... SRC_URI = " \\
6759npm://registry.npmjs.org/;package=cute-files;version=${PV} \\
6760npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \\ " S = "${WORKDIR}/npm"
6761inherit npm LICENSE_${PN} = "MIT" LICENSE_${PN}-accepts = "MIT"
6762LICENSE_${PN}-array-flatten = "MIT" ... LICENSE_${PN}-vary = "MIT" Three
6763key points exist in the previous example:
6764
6765- ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ uses the NPM
6766 scheme so that the NPM fetcher is used.
6767
6768- ``recipetool`` collects all the license information. If a
6769 sub-module's license is unavailable, the sub-module's name appears in
6770 the comments.
6771
6772- The ``inherit npm`` statement causes the
6773 ```npm`` <&YOCTO_DOCS_REF_URL;#ref-classes-npm>`__ class to package
6774 up all the modules.
6775
6776You can run the following command to build the ``cute-files`` package: $
6777devtool build cute-files Remember that ``nodejs`` must be installed on
6778the target before your package.
6779
6780Assuming 192.168.7.2 for the target's IP address, use the following
6781command to deploy your package: $ devtool deploy-target -s cute-files
6782root@192.168.7.2 Once the package is installed on the target, you can
6783test the application:
6784
6785.. note::
6786
6787 Because of a know issue, you cannot simply run
6788 cute-files
6789 as you would if you had run
6790 npm install
6791 .
6792
6793$ cd /usr/lib/node_modules/cute-files $ node cute-files.js On a browser,
6794go to ``http://192.168.7.2:3000`` and you see the following:
6795
6796You can find the recipe in ``workspace/recipes/cute-files``. You can use
6797the recipe in any layer you choose.
6798
6799.. _npm-using-the-npm-projects-method:
6800
6801Using the NPM Projects Code Method
6802~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6803
6804Although it is useful to package modules already in the NPM registry,
6805adding ``node.js`` projects under development is a more common developer
6806use case.
6807
6808This section covers the NPM projects code method, which is very similar
6809to the "registry" approach described in the previous section. In the NPM
6810projects method, you provide ``devtool`` with an URL that points to the
6811source files.
6812
6813Replicating the same example, (i.e. ``cute-files``) use the following
6814command: $ devtool add https://github.com/martinaglv/cute-files.git The
6815recipe this command generates is very similar to the recipe created in
6816the previous section. However, the ``SRC_URI`` looks like the following:
6817SRC_URI = " \\ git://github.com/martinaglv/cute-files.git;protocol=https
6818\\ npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \\ " In this example,
6819the main module is taken from the Git repository and dependents are
6820taken from the NPM registry. Other than those differences, the recipe is
6821basically the same between the two methods. You can build and deploy the
6822package exactly as described in the previous section that uses the
6823registry modules method.
6824
6825Adding custom metadata to packages
6826----------------------------------
6827
6828The variable
6829```PACKAGE_ADD_METADATA`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ADD_METADATA>`__
6830can be used to add additional metadata to packages. This is reflected in
6831the package control/spec file. To take the ipk format for example, the
6832CONTROL file stored inside would contain the additional metadata as
6833additional lines.
6834
6835The variable can be used in multiple ways, including using suffixes to
6836set it for a specific package type and/or package. Note that the order
6837of precedence is the same as this list:
6838
6839- ``PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>``
6840
6841- ``PACKAGE_ADD_METADATA_<PKGTYPE>``
6842
6843- ``PACKAGE_ADD_METADATA_<PN>``
6844
6845- ``PACKAGE_ADD_METADATA``
6846
6847<PKGTYPE> is a parameter and expected to be a distinct name of specific
6848package type:
6849
6850- IPK for .ipk packages
6851
6852- DEB for .deb packages
6853
6854- RPM for .rpm packages
6855
6856<PN> is a parameter and expected to be a package name.
6857
6858The variable can contain multiple [one-line] metadata fields separated
6859by the literal sequence '\n'. The separator can be redefined using the
6860variable flag ``separator``.
6861
6862The following is an example that adds two custom fields for ipk
6863packages: PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup:
6864Applications/Spreadsheets"
6865
6866Efficiently Fetching Source Files During a Build
6867================================================
6868
6869The OpenEmbedded build system works with source files located through
6870the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. When
6871you build something using BitBake, a big part of the operation is
6872locating and downloading all the source tarballs. For images,
6873downloading all the source for various packages can take a significant
6874amount of time.
6875
6876This section shows you how you can use mirrors to speed up fetching
6877source files and how you can pre-fetch files all of which leads to more
6878efficient use of resources and time.
6879
6880Setting up Effective Mirrors
6881----------------------------
6882
6883A good deal that goes into a Yocto Project build is simply downloading
6884all of the source tarballs. Maybe you have been working with another
6885build system (OpenEmbedded or Angstrom) for which you have built up a
6886sizable directory of source tarballs. Or, perhaps someone else has such
6887a directory for which you have read access. If so, you can save time by
6888adding statements to your configuration file so that the build process
6889checks local directories first for existing tarballs before checking the
6890Internet.
6891
6892Here is an efficient way to set it up in your ``local.conf`` file:
6893SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" INHERIT +=
6894"own-mirrors" BB_GENERATE_MIRROR_TARBALLS = "1" # BB_NO_NETWORK = "1"
6895
6896In the previous example, the
6897```BB_GENERATE_MIRROR_TARBALLS`` <&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS>`__
6898variable causes the OpenEmbedded build system to generate tarballs of
6899the Git repositories and store them in the
6900```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ directory. Due to
6901performance reasons, generating and storing these tarballs is not the
6902build system's default behavior.
6903
6904You can also use the
6905```PREMIRRORS`` <&YOCTO_DOCS_REF_URL;#var-PREMIRRORS>`__ variable. For
6906an example, see the variable's glossary entry in the Yocto Project
6907Reference Manual.
6908
6909Getting Source Files and Suppressing the Build
6910----------------------------------------------
6911
6912Another technique you can use to ready yourself for a successive string
6913of build operations, is to pre-fetch all the source files without
6914actually starting a build. This technique lets you work through any
6915download issues and ultimately gathers all the source files into your
6916download directory
6917```build/downloads`` <&YOCTO_DOCS_REF_URL;#structure-build-downloads>`__,
6918which is located with ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__.
6919
6920Use the following BitBake command form to fetch all the necessary
6921sources without starting the build: $ bitbake target --runall=fetch This
6922variation of the BitBake command guarantees that you have all the
6923sources for that BitBake target should you disconnect from the Internet
6924and want to do the build later offline.
6925
6926Selecting an Initialization Manager
6927===================================
6928
6929By default, the Yocto Project uses SysVinit as the initialization
6930manager. However, support also exists for systemd, which is a full
6931replacement for init with parallel starting of services, reduced shell
6932overhead and other features that are used by many distributions.
6933
6934Within the system, SysVinit treats system components as services. These
6935services are maintained as shell scripts stored in the ``/etc/init.d/``
6936directory. Services organize into different run levels. This
6937organization is maintained by putting links to the services in the
6938``/etc/rcN.d/`` directories, where N/ is one of the following options:
6939"S", "0", "1", "2", "3", "4", "5", or "6".
6940
6941.. note::
6942
6943 Each runlevel has a dependency on the previous runlevel. This
6944 dependency allows the services to work properly.
6945
6946In comparison, systemd treats components as units. Using units is a
6947broader concept as compared to using a service. A unit includes several
6948different types of entities. Service is one of the types of entities.
6949The runlevel concept in SysVinit corresponds to the concept of a target
6950in systemd, where target is also a type of supported unit.
6951
6952In a SysVinit-based system, services load sequentially (i.e. one by one)
6953during and parallelization is not supported. With systemd, services
6954start in parallel. Needless to say, the method can have an impact on
6955system startup performance.
6956
6957If you want to use SysVinit, you do not have to do anything. But, if you
6958want to use systemd, you must take some steps as described in the
6959following sections.
6960
6961Using systemd Exclusively
6962-------------------------
6963
6964Set these variables in your distribution configuration file as follows:
6965DISTRO_FEATURES_append = " systemd" VIRTUAL-RUNTIME_init_manager =
6966"systemd" You can also prevent the SysVinit distribution feature from
6967being automatically enabled as follows:
6968DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" Doing so removes any
6969redundant SysVinit scripts.
6970
6971To remove initscripts from your image altogether, set this variable
6972also: VIRTUAL-RUNTIME_initscripts = ""
6973
6974For information on the backfill variable, see
6975```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__.
6976
6977Using systemd for the Main Image and Using SysVinit for the Rescue Image
6978------------------------------------------------------------------------
6979
6980Set these variables in your distribution configuration file as follows:
6981DISTRO_FEATURES_append = " systemd" VIRTUAL-RUNTIME_init_manager =
6982"systemd" Doing so causes your main image to use the
6983``packagegroup-core-boot.bb`` recipe and systemd. The rescue/minimal
6984image cannot use this package group. However, it can install SysVinit
6985and the appropriate packages will have support for both systemd and
6986SysVinit.
6987
6988.. _selecting-dev-manager:
6989
6990Selecting a Device Manager
6991==========================
6992
6993The Yocto Project provides multiple ways to manage the device manager
6994(``/dev``):
6995
6996- *Persistent and Pre-Populated\ ``/dev``:* For this case, the ``/dev``
6997 directory is persistent and the required device nodes are created
6998 during the build.
6999
7000- *Use ``devtmpfs`` with a Device Manager:* For this case, the ``/dev``
7001 directory is provided by the kernel as an in-memory file system and
7002 is automatically populated by the kernel at runtime. Additional
7003 configuration of device nodes is done in user space by a device
7004 manager like ``udev`` or ``busybox-mdev``.
7005
7006.. _static-dev-management:
7007
7008Using Persistent and Pre-Populated\ ``/dev``
7009--------------------------------------------
7010
7011To use the static method for device population, you need to set the
7012```USE_DEVFS`` <&YOCTO_DOCS_REF_URL;#var-USE_DEVFS>`__ variable to "0"
7013as follows: USE_DEVFS = "0"
7014
7015The content of the resulting ``/dev`` directory is defined in a Device
7016Table file. The
7017```IMAGE_DEVICE_TABLES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES>`__
7018variable defines the Device Table to use and should be set in the
7019machine or distro configuration file. Alternatively, you can set this
7020variable in your ``local.conf`` configuration file.
7021
7022If you do not define the ``IMAGE_DEVICE_TABLES`` variable, the default
7023``device_table-minimal.txt`` is used: IMAGE_DEVICE_TABLES =
7024"device_table-mymachine.txt"
7025
7026The population is handled by the ``makedevs`` utility during image
7027creation:
7028
7029.. _devtmpfs-dev-management:
7030
7031Using ``devtmpfs`` and a Device Manager
7032---------------------------------------
7033
7034To use the dynamic method for device population, you need to use (or be
7035sure to set) the ```USE_DEVFS`` <&YOCTO_DOCS_REF_URL;#var-USE_DEVFS>`__
7036variable to "1", which is the default: USE_DEVFS = "1" With this
7037setting, the resulting ``/dev`` directory is populated by the kernel
7038using ``devtmpfs``. Make sure the corresponding kernel configuration
7039variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux
7040kernel.
7041
7042All devices created by ``devtmpfs`` will be owned by ``root`` and have
7043permissions ``0600``.
7044
7045To have more control over the device nodes, you can use a device manager
7046like ``udev`` or ``busybox-mdev``. You choose the device manager by
7047defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or
7048distro configuration file. Alternatively, you can set this variable in
7049your ``local.conf`` configuration file: VIRTUAL-RUNTIME_dev_manager =
7050"udev" # Some alternative values # VIRTUAL-RUNTIME_dev_manager =
7051"busybox-mdev" # VIRTUAL-RUNTIME_dev_manager = "systemd"
7052
7053.. _platdev-appdev-srcrev:
7054
7055Using an External SCM
7056=====================
7057
7058If you're working on a recipe that pulls from an external Source Code
7059Manager (SCM), it is possible to have the OpenEmbedded build system
7060notice new recipe changes added to the SCM and then build the resulting
7061packages that depend on the new recipes by using the latest versions.
7062This only works for SCMs from which it is possible to get a sensible
7063revision number for changes. Currently, you can do this with Apache
7064Subversion (SVN), Git, and Bazaar (BZR) repositories.
7065
7066To enable this behavior, the ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ of
7067the recipe needs to reference
7068```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__. Here is an example: PV =
7069"1.2.3+git${SRCPV}" Then, you can add the following to your
7070``local.conf``: SRCREV_pn-PN = "${AUTOREV}"
7071```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ is the name of the recipe for
7072which you want to enable automatic source revision updating.
7073
7074If you do not want to update your local configuration file, you can add
7075the following directly to the recipe to finish enabling the feature:
7076SRCREV = "${AUTOREV}"
7077
7078The Yocto Project provides a distribution named ``poky-bleeding``, whose
7079configuration file contains the line: require
7080conf/distro/include/poky-floating-revisions.inc This line pulls in the
7081listed include file that contains numerous lines of exactly that form:
7082#SRCREV_pn-opkg-native ?= "${AUTOREV}" #SRCREV_pn-opkg-sdk ?=
7083"${AUTOREV}" #SRCREV_pn-opkg ?= "${AUTOREV}"
7084#SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" #SRCREV_pn-opkg-utils ?=
7085"${AUTOREV}" SRCREV_pn-gconf-dbus ?= "${AUTOREV}"
7086SRCREV_pn-matchbox-common ?= "${AUTOREV}" SRCREV_pn-matchbox-config-gtk
7087?= "${AUTOREV}" SRCREV_pn-matchbox-desktop ?= "${AUTOREV}"
7088SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" SRCREV_pn-matchbox-panel-2
7089?= "${AUTOREV}" SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}"
7090SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" SRCREV_pn-matchbox-wm ?=
7091"${AUTOREV}" SRCREV_pn-settings-daemon ?= "${AUTOREV}"
7092SRCREV_pn-screenshot ?= "${AUTOREV}" . . . These lines allow you to
7093experiment with building a distribution that tracks the latest
7094development source for numerous packages.
7095
7096.. note::
7097
7098 The
7099 poky-bleeding
7100 distribution is not tested on a regular basis. Keep this in mind if
7101 you use it.
7102
7103Creating a Read-Only Root Filesystem
7104====================================
7105
7106Suppose, for security reasons, you need to disable your target device's
7107root filesystem's write permissions (i.e. you need a read-only root
7108filesystem). Or, perhaps you are running the device's operating system
7109from a read-only storage device. For either case, you can customize your
7110image for that behavior.
7111
7112.. note::
7113
7114 Supporting a read-only root filesystem requires that the system and
7115 applications do not try to write to the root filesystem. You must
7116 configure all parts of the target system to write elsewhere, or to
7117 gracefully fail in the event of attempting to write to the root
7118 filesystem.
7119
7120Creating the Root Filesystem
7121----------------------------
7122
7123To create the read-only root filesystem, simply add the
7124"read-only-rootfs" feature to your image, normally in one of two ways.
7125The first way is to add the "read-only-rootfs" image feature in the
7126image's recipe file via the ``IMAGE_FEATURES`` variable: IMAGE_FEATURES
7127+= "read-only-rootfs" As an alternative, you can add the same feature
7128from within your build directory's ``local.conf`` file with the
7129associated ``EXTRA_IMAGE_FEATURES`` variable, as in:
7130EXTRA_IMAGE_FEATURES = "read-only-rootfs"
7131
7132For more information on how to use these variables, see the
7133"`Customizing Images Using Custom ``IMAGE_FEATURES`` and
7134``EXTRA_IMAGE_FEATURES`` <#usingpoky-extend-customimage-imagefeatures>`__"
7135section. For information on the variables, see
7136```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ and
7137```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__.
7138
7139Post-Installation Scripts
7140-------------------------
7141
7142It is very important that you make sure all post-Installation
7143(``pkg_postinst``) scripts for packages that are installed into the
7144image can be run at the time when the root filesystem is created during
7145the build on the host system. These scripts cannot attempt to run during
7146first-boot on the target device. With the "read-only-rootfs" feature
7147enabled, the build system checks during root filesystem creation to make
7148sure all post-installation scripts succeed. If any of these scripts
7149still need to be run after the root filesystem is created, the build
7150immediately fails. These build-time checks ensure that the build fails
7151rather than the target device fails later during its initial boot
7152operation.
7153
7154Most of the common post-installation scripts generated by the build
7155system for the out-of-the-box Yocto Project are engineered so that they
7156can run during root filesystem creation (e.g. post-installation scripts
7157for caching fonts). However, if you create and add custom scripts, you
7158need to be sure they can be run during this file system creation.
7159
7160Here are some common problems that prevent post-installation scripts
7161from running during root filesystem creation:
7162
7163- *Not using $D in front of absolute paths:* The build system defines
7164 ``$``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__ when the root
7165 filesystem is created. Furthermore, ``$D`` is blank when the script
7166 is run on the target device. This implies two purposes for ``$D``:
7167 ensuring paths are valid in both the host and target environments,
7168 and checking to determine which environment is being used as a method
7169 for taking appropriate actions.
7170
7171- *Attempting to run processes that are specific to or dependent on the
7172 target architecture:* You can work around these attempts by using
7173 native tools, which run on the host system, to accomplish the same
7174 tasks, or by alternatively running the processes under QEMU, which
7175 has the ``qemu_run_binary`` function. For more information, see the
7176 ```qemu`` <&YOCTO_DOCS_REF_URL;#ref-classes-qemu>`__ class.
7177
7178Areas With Write Access
7179-----------------------
7180
7181With the "read-only-rootfs" feature enabled, any attempt by the target
7182to write to the root filesystem at runtime fails. Consequently, you must
7183make sure that you configure processes and applications that attempt
7184these types of writes do so to directories with write access (e.g.
7185``/tmp`` or ``/var/run``).
7186
7187Maintaining Build Output Quality
7188================================
7189
7190Many factors can influence the quality of a build. For example, if you
7191upgrade a recipe to use a new version of an upstream software package or
7192you experiment with some new configuration options, subtle changes can
7193occur that you might not detect until later. Consider the case where
7194your recipe is using a newer version of an upstream package. In this
7195case, a new version of a piece of software might introduce an optional
7196dependency on another library, which is auto-detected. If that library
7197has already been built when the software is building, the software will
7198link to the built library and that library will be pulled into your
7199image along with the new software even if you did not want the library.
7200
7201The ```buildhistory`` <&YOCTO_DOCS_REF_URL;#ref-classes-buildhistory>`__
7202class exists to help you maintain the quality of your build output. You
7203can use the class to highlight unexpected and possibly unwanted changes
7204in the build output. When you enable build history, it records
7205information about the contents of each package and image and then
7206commits that information to a local Git repository where you can examine
7207the information.
7208
7209The remainder of this section describes the following:
7210
7211- How you can enable and disable build history
7212
7213- How to understand what the build history contains
7214
7215- How to limit the information used for build history
7216
7217- How to examine the build history from both a command-line and web
7218 interface
7219
7220Enabling and Disabling Build History
7221------------------------------------
7222
7223Build history is disabled by default. To enable it, add the following
7224``INHERIT`` statement and set the
7225```BUILDHISTORY_COMMIT`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT>`__
7226variable to "1" at the end of your ``conf/local.conf`` file found in the
7227`Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
7228"buildhistory" BUILDHISTORY_COMMIT = "1" Enabling build history as
7229previously described causes the OpenEmbedded build system to collect
7230build output information and commit it as a single commit to a local
7231`Git <&YOCTO_DOCS_OM_URL;#git>`__ repository.
7232
7233.. note::
7234
7235 Enabling build history increases your build times slightly,
7236 particularly for images, and increases the amount of disk space used
7237 during the build.
7238
7239You can disable build history by removing the previous statements from
7240your ``conf/local.conf`` file.
7241
7242Understanding What the Build History Contains
7243---------------------------------------------
7244
7245Build history information is kept in
7246``${``\ ```TOPDIR`` <&YOCTO_DOCS_REF_URL;#var-TOPDIR>`__\ ``}/buildhistory``
7247in the Build Directory as defined by the
7248```BUILDHISTORY_DIR`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_DIR>`__
7249variable. The following is an example abbreviated listing:
7250
7251At the top level, a ``metadata-revs`` file exists that lists the
7252revisions of the repositories for the enabled layers when the build was
7253produced. The rest of the data splits into separate ``packages``,
7254``images`` and ``sdk`` directories, the contents of which are described
7255as follows.
7256
7257Build History Package Information
7258~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7259
7260The history for each package contains a text file that has name-value
7261pairs with information about the package. For example,
7262``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
7263contains the following: PV = 1.22.1 PR = r32 RPROVIDES = RDEPENDS =
7264glibc (>= 2.20) update-alternatives-opkg RRECOMMENDS = busybox-syslog
7265busybox-udhcpc update-rc.d PKGSIZE = 540168 FILES = /usr/bin/\*
7266/usr/sbin/\* /usr/lib/busybox/\* /usr/lib/lib*.so.\* \\ /etc /com /var
7267/bin/\* /sbin/\* /lib/*.so.\* /lib/udev/rules.d \\ /usr/lib/udev/rules.d
7268/usr/share/busybox /usr/lib/busybox/\* \\ /usr/share/pixmaps
7269/usr/share/applications /usr/share/idl \\ /usr/share/omf
7270/usr/share/sounds /usr/lib/bonobo/servers FILELIST = /bin/busybox
7271/bin/busybox.nosuid /bin/busybox.suid /bin/sh \\
7272/etc/busybox.links.nosuid /etc/busybox.links.suid Most of these
7273name-value pairs correspond to variables used to produce the package.
7274The exceptions are ``FILELIST``, which is the actual list of files in
7275the package, and ``PKGSIZE``, which is the total size of files in the
7276package in bytes.
7277
7278A file also exists that corresponds to the recipe from which the package
7279came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``): PV
7280= 1.22.1 PR = r32 DEPENDS = initscripts kern-tools-native
7281update-rc.d-native \\ virtual/i586-poky-linux-compilerlibs
7282virtual/i586-poky-linux-gcc \\ virtual/libc virtual/update-alternatives
7283PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \\
7284busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \\
7285busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
7286
7287Finally, for those recipes fetched from a version control system (e.g.,
7288Git), a file exists that lists source revisions that are specified in
7289the recipe and lists the actual revisions used during the build. Listed
7290and actual revisions might differ when
7291```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ is set to
7292${```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__}. Here is an
7293example assuming
7294``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``):
7295# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
7296SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" #
7297SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" SRCREV_meta =
7298"a227f20eff056e511d504b2e490f3774ab260d6f" You can use the
7299``buildhistory-collect-srcrevs`` command with the ``-a`` option to
7300collect the stored ``SRCREV`` values from build history and report them
7301in a format suitable for use in global configuration (e.g.,
7302``local.conf`` or a distro include file) to override floating
7303``AUTOREV`` values to a fixed set of revisions. Here is some example
7304output from this command: $ buildhistory-collect-srcrevs -a #
7305i586-poky-linux SRCREV_pn-glibc =
7306"b8079dd0d360648e4e8de48656c5c38972621072" SRCREV_pn-glibc-initial =
7307"b8079dd0d360648e4e8de48656c5c38972621072" SRCREV_pn-opkg-utils =
7308"53274f087565fd45d8452c5367997ba6a682a37a" SRCREV_pn-kmod =
7309"fd56638aed3fe147015bfa10ed4a5f7491303cb4" # x86_64-linux
7310SRCREV_pn-gtk-doc-stub-native =
7311"1dea266593edb766d6d898c79451ef193eb17cfa" SRCREV_pn-dtc-native =
7312"65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" SRCREV_pn-update-rc.d-native
7313= "eca680ddf28d024954895f59a241a622dd575c11"
7314SRCREV_glibc_pn-cross-localedef-native =
7315"b8079dd0d360648e4e8de48656c5c38972621072"
7316SRCREV_localedef_pn-cross-localedef-native =
7317"c833367348d39dad7ba018990bfdaffaec8e9ed3" SRCREV_pn-prelink-native =
7318"faa069deec99bf61418d0bab831c83d7c1b797ca" SRCREV_pn-opkg-utils-native =
7319"53274f087565fd45d8452c5367997ba6a682a37a" SRCREV_pn-kern-tools-native =
7320"23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" SRCREV_pn-kmod-native =
7321"fd56638aed3fe147015bfa10ed4a5f7491303cb4" # qemux86-poky-linux
7322SRCREV_machine_pn-linux-yocto =
7323"38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" SRCREV_meta_pn-linux-yocto =
7324"a227f20eff056e511d504b2e490f3774ab260d6f" # all-poky-linux
7325SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
7326
7327.. note::
7328
7329 Here are some notes on using the
7330 buildhistory-collect-srcrevs
7331 command:
7332
7333 - By default, only values where the ``SRCREV`` was not hardcoded
7334 (usually when ``AUTOREV`` is used) are reported. Use the ``-a``
7335 option to see all ``SRCREV`` values.
7336
7337 - The output statements might not have any effect if overrides are
7338 applied elsewhere in the build system configuration. Use the
7339 ``-f`` option to add the ``forcevariable`` override to each output
7340 line if you need to work around this restriction.
7341
7342 - The script does apply special handling when building for multiple
7343 machines. However, the script does place a comment before each set
7344 of values that specifies which triplet to which they belong as
7345 previously shown (e.g., ``i586-poky-linux``).
7346
7347Build History Image Information
7348~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7349
7350The files produced for each image are as follows:
7351
7352- ``image-files:`` A directory containing selected files from the root
7353 filesystem. The files are defined by
7354 ```BUILDHISTORY_IMAGE_FILES`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_IMAGE_FILES>`__.
7355
7356- ``build-id.txt:`` Human-readable information about the build
7357 configuration and metadata source revisions. This file contains the
7358 full build header as printed by BitBake.
7359
7360- ``*.dot:`` Dependency graphs for the image that are compatible with
7361 ``graphviz``.
7362
7363- ``files-in-image.txt:`` A list of files in the image with
7364 permissions, owner, group, size, and symlink information.
7365
7366- ``image-info.txt:`` A text file containing name-value pairs with
7367 information about the image. See the following listing example for
7368 more information.
7369
7370- ``installed-package-names.txt:`` A list of installed packages by name
7371 only.
7372
7373- ``installed-package-sizes.txt:`` A list of installed packages ordered
7374 by size.
7375
7376- ``installed-packages.txt:`` A list of installed packages with full
7377 package filenames.
7378
7379.. note::
7380
7381 Installed package information is able to be gathered and produced
7382 even if package management is disabled for the final image.
7383
7384Here is an example of ``image-info.txt``: DISTRO = poky DISTRO_VERSION =
73851.7 USER_CLASSES = buildstats image-mklibs image-prelink IMAGE_CLASSES =
7386image_types IMAGE_FEATURES = debug-tweaks IMAGE_LINGUAS = IMAGE_INSTALL
7387= packagegroup-core-boot run-postinsts BAD_RECOMMENDATIONS =
7388NO_RECOMMENDATIONS = PACKAGE_EXCLUDE = ROOTFS_POSTPROCESS_COMMAND =
7389write_package_manifest; license_create_manifest; \\ write_image_manifest
7390; buildhistory_list_installed_image ; \\
7391buildhistory_get_image_installed ; ssh_allow_empty_password; \\
7392postinst_enable_logging; rootfs_update_timestamp ;
7393ssh_disable_dns_lookup ; IMAGE_POSTPROCESS_COMMAND =
7394buildhistory_get_imageinfo ; IMAGESIZE = 6900 Other than ``IMAGESIZE``,
7395which is the total size of the files in the image in Kbytes, the
7396name-value pairs are variables that may have influenced the content of
7397the image. This information is often useful when you are trying to
7398determine why a change in the package or file listings has occurred.
7399
7400Using Build History to Gather Image Information Only
7401~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7402
7403As you can see, build history produces image information, including
7404dependency graphs, so you can see why something was pulled into the
7405image. If you are just interested in this information and not interested
7406in collecting specific package or SDK information, you can enable
7407writing only image information without any history by adding the
7408following to your ``conf/local.conf`` file found in the `Build
7409Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
7410"buildhistory" BUILDHISTORY_COMMIT = "0" BUILDHISTORY_FEATURES = "image"
7411Here, you set the
7412```BUILDHISTORY_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_FEATURES>`__
7413variable to use the image feature only.
7414
7415Build History SDK Information
7416~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7417
7418Build history collects similar information on the contents of SDKs (e.g.
7419``bitbake -c populate_sdk imagename``) as compared to information it
7420collects for images. Furthermore, this information differs depending on
7421whether an extensible or standard SDK is being produced.
7422
7423The following list shows the files produced for SDKs:
7424
7425- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
7426 owner, group, size, and symlink information. This list includes both
7427 the host and target parts of the SDK.
7428
7429- ``sdk-info.txt:`` A text file containing name-value pairs with
7430 information about the SDK. See the following listing example for more
7431 information.
7432
7433- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
7434 with information about task group sizes (e.g. ``do_populate_sysroot``
7435 tasks have a total size). The ``sstate-task-sizes.txt`` file exists
7436 only when an extensible SDK is created.
7437
7438- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
7439 with information for the shared-state packages and sizes in the SDK.
7440 The ``sstate-package-sizes.txt`` file exists only when an extensible
7441 SDK is created.
7442
7443- ``sdk-files:`` A folder that contains copies of the files mentioned
7444 in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
7445 Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
7446 specific to the extensible SDK although you can set it differently if
7447 you would like to pull in specific files from the standard SDK.
7448
7449 The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
7450 ``conf/auto.conf``, ``conf/locked-sigs.inc``, and
7451 ``conf/devtool.conf``. Thus, for an extensible SDK, these files get
7452 copied into the ``sdk-files`` directory.
7453
7454- The following information appears under each of the ``host`` and
7455 ``target`` directories for the portions of the SDK that run on the
7456 host and on the target, respectively:
7457
7458 .. note::
7459
7460 The following files for the most part are empty when producing an
7461 extensible SDK because this type of SDK is not constructed from
7462 packages as is the standard SDK.
7463
7464 - ``depends.dot:`` Dependency graph for the SDK that is compatible
7465 with ``graphviz``.
7466
7467 - ``installed-package-names.txt:`` A list of installed packages by
7468 name only.
7469
7470 - ``installed-package-sizes.txt:`` A list of installed packages
7471 ordered by size.
7472
7473 - ``installed-packages.txt:`` A list of installed packages with full
7474 package filenames.
7475
7476Here is an example of ``sdk-info.txt``: DISTRO = poky DISTRO_VERSION =
74771.3+snapshot-20130327 SDK_NAME = poky-glibc-i686-arm SDK_VERSION =
74781.3+snapshot SDKMACHINE = SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
7479BAD_RECOMMENDATIONS = SDKSIZE = 352712 Other than ``SDKSIZE``, which is
7480the total size of the files in the SDK in Kbytes, the name-value pairs
7481are variables that might have influenced the content of the SDK. This
7482information is often useful when you are trying to determine why a
7483change in the package or file listings has occurred.
7484
7485Examining Build History Information
7486~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7487
7488You can examine build history output from the command line or from a web
7489interface.
7490
7491To see any changes that have occurred (assuming you have
7492```BUILDHISTORY_COMMIT`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT>`__\ `` = "1"``),
7493you can simply use any Git command that allows you to view the history
7494of a repository. Here is one method: $ git log -p You need to realize,
7495however, that this method does show changes that are not significant
7496(e.g. a package's size changing by a few bytes).
7497
7498A command-line tool called ``buildhistory-diff`` does exist, though,
7499that queries the Git repository and prints just the differences that
7500might be significant in human-readable form. Here is an example: $
7501~/poky/poky/scripts/buildhistory-diff . HEAD^ Changes to
7502images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
7503/etc/anotherpkg.conf was added /sbin/anotherpkg was added \*
7504(installed-package-names.txt): \* anotherpkg was added Changes to
7505images/qemux86_64/glibc/core-image-minimal
7506(installed-package-names.txt): anotherpkg was added
7507packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" \* PR
7508changed from "r0" to "r1" \* PV changed from "0.1.10" to "0.1.12"
7509packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to
7510144381 (+30%) \* PR changed from "r0" to "r1" \* PV changed from
7511"0.1.10" to "0.1.12"
7512
7513.. note::
7514
7515 The
7516 buildhistory-diff
7517 tool requires the
7518 GitPython
7519 package. Be sure to install it using Pip3 as follows:
7520 ::
7521
7522 $ pip3 install GitPython --user
7523
7524
7525 Alternatively, you can install
7526 python3-git
7527 using the appropriate distribution package manager (e.g.
7528 apt-get
7529 ,
7530 dnf
7531 , or
7532 zipper
7533 ).
7534
7535To see changes to the build history using a web interface, follow the
7536instruction in the ``README`` file here.
7537` <http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/>`__.
7538
7539Here is a sample screenshot of the interface:
7540
7541Performing Automated Runtime Testing
7542====================================
7543
7544The OpenEmbedded build system makes available a series of automated
7545tests for images to verify runtime functionality. You can run these
7546tests on either QEMU or actual target hardware. Tests are written in
7547Python making use of the ``unittest`` module, and the majority of them
7548run commands on the target system over SSH. This section describes how
7549you set up the environment to use these tests, run available tests, and
7550write and add your own tests.
7551
7552For information on the test and QA infrastructure available within the
7553Yocto Project, see the "`Testing and Quality
7554Assurance <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__"
7555section in the Yocto Project Reference Manual.
7556
7557Enabling Tests
7558--------------
7559
7560Depending on whether you are planning to run tests using QEMU or on the
7561hardware, you have to take different steps to enable the tests. See the
7562following subsections for information on how to enable both types of
7563tests.
7564
7565.. _qemu-image-enabling-tests:
7566
7567Enabling Runtime Tests on QEMU
7568~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7569
7570In order to run tests, you need to do the following:
7571
7572- *Set up to avoid interaction with ``sudo`` for networking:* To
7573 accomplish this, you must do one of the following:
7574
7575 - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all
7576 commands or just for ``runqemu-ifup``. You must provide the full
7577 path as that can change if you are using multiple clones of the
7578 source repository.
7579
7580 .. note::
7581
7582 On some distributions, you also need to comment out "Defaults
7583 requiretty" in
7584 /etc/sudoers
7585 .
7586
7587 - Manually configure a tap interface for your system.
7588
7589 - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which
7590 should generate a list of tap devices. This is the option
7591 typically chosen for Autobuilder-type environments.
7592
7593 .. note::
7594
7595 - Be sure to use an absolute path when calling this script
7596 with sudo.
7597
7598 - The package recipe ``qemu-helper-native`` is required to run
7599 this script. Build the package using the following command:
7600 $ bitbake qemu-helper-native
7601
7602- *Set the ``DISPLAY`` variable:* You need to set this variable so that
7603 you have an X server available (e.g. start ``vncserver`` for a
7604 headless machine).
7605
7606- *Be sure your host's firewall accepts incoming connections from
7607 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an
7608 HTTP server on a random high number port, which is used to serve
7609 files to the target. The DNF module serves
7610 ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands.
7611 That means your host's firewall must accept incoming connections from
7612 192.168.7.0/24, which is the default IP range used for tap devices by
7613 ``runqemu``.
7614
7615- *Be sure your host has the correct packages installed:* Depending
7616 your host's distribution, you need to have the following packages
7617 installed:
7618
7619 - Ubuntu and Debian: ``sysstat`` and ``iproute2``
7620
7621 - OpenSUSE: ``sysstat`` and ``iproute2``
7622
7623 - Fedora: ``sysstat`` and ``iproute``
7624
7625 - CentOS: ``sysstat`` and ``iproute``
7626
7627Once you start running the tests, the following happens:
7628
76291. A copy of the root filesystem is written to ``${WORKDIR}/testimage``.
7630
76312. The image is booted under QEMU using the standard ``runqemu`` script.
7632
76333. A default timeout of 500 seconds occurs to allow for the boot process
7634 to reach the login prompt. You can change the timeout period by
7635 setting
7636 ```TEST_QEMUBOOT_TIMEOUT`` <&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT>`__
7637 in the ``local.conf`` file.
7638
76394. Once the boot process is reached and the login prompt appears, the
7640 tests run. The full boot log is written to
7641 ``${WORKDIR}/testimage/qemu_boot_log``.
7642
76435. Each test module loads in the order found in ``TEST_SUITES``. You can
7644 find the full output of the commands run over SSH in
7645 ``${WORKDIR}/testimgage/ssh_target_log``.
7646
76476. If no failures occur, the task running the tests ends successfully.
7648 You can find the output from the ``unittest`` in the task log at
7649 ``${WORKDIR}/temp/log.do_testimage``.
7650
7651.. _hardware-image-enabling-tests:
7652
7653Enabling Runtime Tests on Hardware
7654~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7655
7656The OpenEmbedded build system can run tests on real hardware, and for
7657certain devices it can also deploy the image to be tested onto the
7658device beforehand.
7659
7660For automated deployment, a "master image" is installed onto the
7661hardware once as part of setup. Then, each time tests are to be run, the
7662following occurs:
7663
76641. The master image is booted into and used to write the image to be
7665 tested to a second partition.
7666
76672. The device is then rebooted using an external script that you need to
7668 provide.
7669
76703. The device boots into the image to be tested.
7671
7672When running tests (independent of whether the image has been deployed
7673automatically or not), the device is expected to be connected to a
7674network on a pre-determined IP address. You can either use static IP
7675addresses written into the image, or set the image to use DHCP and have
7676your DHCP server on the test network assign a known IP address based on
7677the MAC address of the device.
7678
7679In order to run tests on hardware, you need to set ``TEST_TARGET`` to an
7680appropriate value. For QEMU, you do not have to change anything, the
7681default value is "qemu". For running tests on hardware, the following
7682options exist:
7683
7684- *"simpleremote":* Choose "simpleremote" if you are going to run tests
7685 on a target system that is already running the image to be tested and
7686 is available on the network. You can use "simpleremote" in
7687 conjunction with either real hardware or an image running within a
7688 separately started QEMU or any other virtual machine manager.
7689
7690- *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is
7691 an EFI-based machine with ``systemd-boot`` as bootloader and
7692 ``core-image-testmaster`` (or something similar) is installed. Also,
7693 your hardware under test must be in a DHCP-enabled network that gives
7694 it the same IP address for each reboot.
7695
7696 If you choose "SystemdbootTarget", there are additional requirements
7697 and considerations. See the "`Selecting
7698 SystemdbootTarget <#selecting-systemdboottarget>`__" section, which
7699 follows, for more information.
7700
7701- *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying
7702 images and running tests on the BeagleBone "Black" or original
7703 "White" hardware. For information on how to use these tests, see the
7704 comments at the top of the BeagleBoneTarget
7705 ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file.
7706
7707- *"EdgeRouterTarget":* Choose "EdgeRouterTarget" is you are deploying
7708 images and running tests on the Ubiquiti Networks EdgeRouter Lite.
7709 For information on how to use these tests, see the comments at the
7710 top of the EdgeRouterTarget
7711 ``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file.
7712
7713- *"GrubTarget":* Choose the "supports deploying images and running
7714 tests on any generic PC that boots using GRUB. For information on how
7715 to use these tests, see the comments at the top of the GrubTarget
7716 ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file.
7717
7718- *"your-target":* Create your own custom target if you want to run
7719 tests when you are deploying images and running tests on a custom
7720 machine within your BSP layer. To do this, you need to add a Python
7721 unit that defines the target class under ``lib/oeqa/controllers/``
7722 within your layer. You must also provide an empty ``__init__.py``.
7723 For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``.
7724
7725Selecting SystemdbootTarget
7726~~~~~~~~~~~~~~~~~~~~~~~~~~~
7727
7728If you did not set ``TEST_TARGET`` to "SystemdbootTarget", then you do
7729not need any information in this section. You can skip down to the
7730"`Running Tests <#qemu-image-running-tests>`__" section.
7731
7732If you did set ``TEST_TARGET`` to "SystemdbootTarget", you also need to
7733perform a one-time setup of your master image by doing the following:
7734
77351. *Set ``EFI_PROVIDER``:* Be sure that ``EFI_PROVIDER`` is as follows:
7736 EFI_PROVIDER = "systemd-boot"
7737
77382. *Build the master image:* Build the ``core-image-testmaster`` image.
7739 The ``core-image-testmaster`` recipe is provided as an example for a
7740 "master" image and you can customize the image recipe as you would
7741 any other recipe.
7742
7743 Here are the image recipe requirements:
7744
7745 - Inherits ``core-image`` so that kernel modules are installed.
7746
7747 - Installs normal linux utilities not busybox ones (e.g. ``bash``,
7748 ``coreutils``, ``tar``, ``gzip``, and ``kmod``).
7749
7750 - Uses a custom Initial RAM Disk (initramfs) image with a custom
7751 installer. A normal image that you can install usually creates a
7752 single rootfs partition. This image uses another installer that
7753 creates a specific partition layout. Not all Board Support
7754 Packages (BSPs) can use an installer. For such cases, you need to
7755 manually create the following partition layout on the target:
7756
7757 - First partition mounted under ``/boot``, labeled "boot".
7758
7759 - The main rootfs partition where this image gets installed,
7760 which is mounted under ``/``.
7761
7762 - Another partition labeled "testrootfs" where test images get
7763 deployed.
7764
77653. *Install image:* Install the image that you just built on the target
7766 system.
7767
7768The final thing you need to do when setting ``TEST_TARGET`` to
7769"SystemdbootTarget" is to set up the test image:
7770
77711. *Set up your ``local.conf`` file:* Make sure you have the following
7772 statements in your ``local.conf`` file: IMAGE_FSTYPES += "tar.gz"
7773 INHERIT += "testimage" TEST_TARGET = "SystemdbootTarget"
7774 TEST_TARGET_IP = "192.168.2.3"
7775
77762. *Build your test image:* Use BitBake to build the image: $ bitbake
7777 core-image-sato
7778
7779Power Control
7780~~~~~~~~~~~~~
7781
7782For most hardware targets other than "simpleremote", you can control
7783power:
7784
7785- You can use ``TEST_POWERCONTROL_CMD`` together with
7786 ``TEST_POWERCONTROL_EXTRA_ARGS`` as a command that runs on the host
7787 and does power cycling. The test code passes one argument to that
7788 command: off, on or cycle (off then on). Here is an example that
7789 could appear in your ``local.conf`` file: TEST_POWERCONTROL_CMD =
7790 "powercontrol.exp test 10.11.12.1 nuc1" In this example, the expect
7791 script does the following: ssh test@10.11.12.1 "pyctl nuc1 arg" It
7792 then runs a Python script that controls power for a label called
7793 ``nuc1``.
7794
7795 .. note::
7796
7797 You need to customize
7798 TEST_POWERCONTROL_CMD
7799 and
7800 TEST_POWERCONTROL_EXTRA_ARGS
7801 for your own setup. The one requirement is that it accepts "on",
7802 "off", and "cycle" as the last argument.
7803
7804- When no command is defined, it connects to the device over SSH and
7805 uses the classic reboot command to reboot the device. Classic reboot
7806 is fine as long as the machine actually reboots (i.e. the SSH test
7807 has not failed). It is useful for scenarios where you have a simple
7808 setup, typically with a single board, and where some manual
7809 interaction is okay from time to time.
7810
7811If you have no hardware to automatically perform power control but still
7812wish to experiment with automated hardware testing, you can use the
7813dialog-power-control script that shows a dialog prompting you to perform
7814the required power action. This script requires either KDialog or Zenity
7815to be installed. To use this script, set the
7816```TEST_POWERCONTROL_CMD`` <&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD>`__
7817variable as follows: TEST_POWERCONTROL_CMD =
7818"${COREBASE}/scripts/contrib/dialog-power-control"
7819
7820Serial Console Connection
7821~~~~~~~~~~~~~~~~~~~~~~~~~
7822
7823For test target classes requiring a serial console to interact with the
7824bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget),
7825you need to specify a command to use to connect to the serial console of
7826the target machine by using the
7827```TEST_SERIALCONTROL_CMD`` <&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD>`__
7828variable and optionally the
7829```TEST_SERIALCONTROL_EXTRA_ARGS`` <&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS>`__
7830variable.
7831
7832These cases could be a serial terminal program if the machine is
7833connected to a local serial port, or a ``telnet`` or ``ssh`` command
7834connecting to a remote console server. Regardless of the case, the
7835command simply needs to connect to the serial console and forward that
7836connection to standard input and output as any normal terminal program
7837does. For example, to use the picocom terminal program on serial device
7838``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:
7839TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" For local
7840devices where the serial port device disappears when the device reboots,
7841an additional "serdevtry" wrapper script is provided. To use this
7842wrapper, simply prefix the terminal command with
7843``${COREBASE}/scripts/contrib/serdevtry``: TEST_SERIALCONTROL_CMD =
7844"${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0"
7845
7846.. _qemu-image-running-tests:
7847
7848Running Tests
7849-------------
7850
7851You can start the tests automatically or manually:
7852
7853- *Automatically running tests:* To run the tests automatically after
7854 the OpenEmbedded build system successfully creates an image, first
7855 set the
7856 ```TESTIMAGE_AUTO`` <&YOCTO_DOCS_REF_URL;#var-TESTIMAGE_AUTO>`__
7857 variable to "1" in your ``local.conf`` file in the `Build
7858 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: TESTIMAGE_AUTO =
7859 "1" Next, build your image. If the image successfully builds, the
7860 tests run: bitbake core-image-sato
7861
7862- *Manually running tests:* To manually run the tests, first globally
7863 inherit the
7864 ```testimage`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__ class
7865 by editing your ``local.conf`` file: INHERIT += "testimage" Next, use
7866 BitBake to run the tests: bitbake -c testimage image
7867
7868All test files reside in ``meta/lib/oeqa/runtime`` in the `Source
7869Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. A test name maps
7870directly to a Python module. Each test module may contain a number of
7871individual tests. Tests are usually grouped together by the area tested
7872(e.g tests for systemd reside in ``meta/lib/oeqa/runtime/systemd.py``).
7873
7874You can add tests to any layer provided you place them in the proper
7875area and you extend ```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__ in
7876the ``local.conf`` file as normal. Be sure that tests reside in
7877``layer/lib/oeqa/runtime``.
7878
7879.. note::
7880
7881 Be sure that module names do not collide with module names used in
7882 the default set of test modules in
7883 meta/lib/oeqa/runtime
7884 .
7885
7886You can change the set of tests run by appending or overriding
7887```TEST_SUITES`` <&YOCTO_DOCS_REF_URL;#var-TEST_SUITES>`__ variable in
7888``local.conf``. Each name in ``TEST_SUITES`` represents a required test
7889for the image. Test modules named within ``TEST_SUITES`` cannot be
7890skipped even if a test is not suitable for an image (e.g. running the
7891RPM tests on an image without ``rpm``). Appending "auto" to
7892``TEST_SUITES`` causes the build system to try to run all tests that are
7893suitable for the image (i.e. each test module may elect to skip itself).
7894
7895The order you list tests in ``TEST_SUITES`` is important and influences
7896test dependencies. Consequently, tests that depend on other tests should
7897be added after the test on which they depend. For example, since the
7898``ssh`` test depends on the ``ping`` test, "ssh" needs to come after
7899"ping" in the list. The test class provides no re-ordering or dependency
7900handling.
7901
7902.. note::
7903
7904 Each module can have multiple classes with multiple test methods.
7905 And, Python
7906 unittest
7907 rules apply.
7908
7909Here are some things to keep in mind when running tests:
7910
7911- The default tests for the image are defined as:
7912 DEFAULT_TEST_SUITES_pn-image = "ping ssh df connman syslog xorg scp
7913 vnc date rpm dnf dmesg"
7914
7915- Add your own test to the list of the by using the following:
7916 TEST_SUITES_append = " mytest"
7917
7918- Run a specific list of tests as follows: TEST_SUITES = "test1 test2
7919 test3" Remember, order is important. Be sure to place a test that is
7920 dependent on another test later in the order.
7921
7922Exporting Tests
7923---------------
7924
7925You can export tests so that they can run independently of the build
7926system. Exporting tests is required if you want to be able to hand the
7927test execution off to a scheduler. You can only export tests that are
7928defined in ```TEST_SUITES`` <&YOCTO_DOCS_REF_URL;#var-TEST_SUITES>`__.
7929
7930If your image is already built, make sure the following are set in your
7931``local.conf`` file: INHERIT +="testexport" TEST_TARGET_IP =
7932"IP-address-for-the-test-target" TEST_SERVER_IP =
7933"IP-address-for-the-test-server" You can then export the tests with the
7934following BitBake command form: $ bitbake image -c testexport Exporting
7935the tests places them in the `Build
7936Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in
7937``tmp/testexport/``\ image, which is controlled by the
7938``TEST_EXPORT_DIR`` variable.
7939
7940You can now run the tests outside of the build environment: $ cd
7941tmp/testexport/image $ ./runexported.py testdata.json
7942
7943Here is a complete example that shows IP addresses and uses the
7944``core-image-sato`` image: INHERIT +="testexport" TEST_TARGET_IP =
7945"192.168.7.2" TEST_SERVER_IP = "192.168.7.1" Use BitBake to export the
7946tests: $ bitbake core-image-sato -c testexport Run the tests outside of
7947the build environment using the following: $ cd
7948tmp/testexport/core-image-sato $ ./runexported.py testdata.json
7949
7950.. _qemu-image-writing-new-tests:
7951
7952Writing New Tests
7953-----------------
7954
7955As mentioned previously, all new test files need to be in the proper
7956place for the build system to find them. New tests for additional
7957functionality outside of the core should be added to the layer that adds
7958the functionality, in ``layer/lib/oeqa/runtime`` (as long as
7959```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__ is extended in the
7960layer's ``layer.conf`` file as normal). Just remember the following:
7961
7962- Filenames need to map directly to test (module) names.
7963
7964- Do not use module names that collide with existing core tests.
7965
7966- Minimally, an empty ``__init__.py`` file must exist in the runtime
7967 directory.
7968
7969To create a new test, start by copying an existing module (e.g.
7970``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use
7971code from ``meta/lib/oeqa/utils``, which are helper classes.
7972
7973.. note::
7974
7975 Structure shell commands such that you rely on them and they return a
7976 single code for success. Be aware that sometimes you will need to
7977 parse the output. See the
7978 df.py
7979 and
7980 date.py
7981 modules for examples.
7982
7983You will notice that all test classes inherit ``oeRuntimeTest``, which
7984is found in ``meta/lib/oetest.py``. This base class offers some helper
7985attributes, which are described in the following sections:
7986
7987.. _qemu-image-writing-tests-class-methods:
7988
7989Class Methods
7990~~~~~~~~~~~~~
7991
7992Class methods are as follows:
7993
7994- *``hasPackage(pkg)``:* Returns "True" if ``pkg`` is in the installed
7995 package list of the image, which is based on the manifest file that
7996 is generated during the ``do_rootfs`` task.
7997
7998- *``hasFeature(feature)``:* Returns "True" if the feature is in
7999 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ or
8000 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__.
8001
8002.. _qemu-image-writing-tests-class-attributes:
8003
8004Class Attributes
8005~~~~~~~~~~~~~~~~
8006
8007Class attributes are as follows:
8008
8009- *``pscmd``:* Equals "ps -ef" if ``procps`` is installed in the image.
8010 Otherwise, ``pscmd`` equals "ps" (busybox).
8011
8012- *``tc``:* The called test context, which gives access to the
8013 following attributes:
8014
8015 - *``d``:* The BitBake datastore, which allows you to use stuff such
8016 as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``.
8017
8018 - *``testslist`` and ``testsrequired``:* Used internally. The tests
8019 do not need these.
8020
8021 - *``filesdir``:* The absolute path to
8022 ``meta/lib/oeqa/runtime/files``, which contains helper files for
8023 tests meant for copying on the target such as small files written
8024 in C for compilation.
8025
8026 - *``target``:* The target controller object used to deploy and
8027 start an image on a particular target (e.g. Qemu, SimpleRemote,
8028 and SystemdbootTarget). Tests usually use the following:
8029
8030 - *``ip``:* The target's IP address.
8031
8032 - *``server_ip``:* The host's IP address, which is usually used
8033 by the DNF test suite.
8034
8035 - *``run(cmd, timeout=None)``:* The single, most used method.
8036 This command is a wrapper for: ``ssh root@host "cmd"``. The
8037 command returns a tuple: (status, output), which are what their
8038 names imply - the return code of "cmd" and whatever output it
8039 produces. The optional timeout argument represents the number
8040 of seconds the test should wait for "cmd" to return. If the
8041 argument is "None", the test uses the default instance's
8042 timeout period, which is 300 seconds. If the argument is "0",
8043 the test runs until the command returns.
8044
8045 - *``copy_to(localpath, remotepath)``:*
8046 ``scp localpath root@ip:remotepath``.
8047
8048 - *``copy_from(remotepath, localpath)``:*
8049 ``scp root@host:remotepath localpath``.
8050
8051.. _qemu-image-writing-tests-instance-attributes:
8052
8053Instance Attributes
8054~~~~~~~~~~~~~~~~~~~
8055
8056A single instance attribute exists, which is ``target``. The ``target``
8057instance attribute is identical to the class attribute of the same name,
8058which is described in the previous section. This attribute exists as
8059both an instance and class attribute so tests can use
8060``self.target.run(cmd)`` in instance methods instead of
8061``oeRuntimeTest.tc.target.run(cmd)``.
8062
8063Installing Packages in the DUT Without the Package Manager
8064----------------------------------------------------------
8065
8066When a test requires a package built by BitBake, it is possible to
8067install that package. Installing the package does not require a package
8068manager be installed in the device under test (DUT). It does, however,
8069require an SSH connection and the target must be using the
8070``sshcontrol`` class.
8071
8072.. note::
8073
8074 This method uses
8075 scp
8076 to copy files from the host to the target, which causes permissions
8077 and special attributes to be lost.
8078
8079A JSON file is used to define the packages needed by a test. This file
8080must be in the same path as the file used to define the tests.
8081Furthermore, the filename must map directly to the test module name with
8082a ``.json`` extension.
8083
8084The JSON file must include an object with the test name as keys of an
8085object or an array. This object (or array of objects) uses the following
8086data:
8087
8088- "pkg" - A mandatory string that is the name of the package to be
8089 installed.
8090
8091- "rm" - An optional boolean, which defaults to "false", that specifies
8092 to remove the package after the test.
8093
8094- "extract" - An optional boolean, which defaults to "false", that
8095 specifies if the package must be extracted from the package format.
8096 When set to "true", the package is not automatically installed into
8097 the DUT.
8098
8099Following is an example JSON file that handles test "foo" installing
8100package "bar" and test "foobar" installing packages "foo" and "bar".
8101Once the test is complete, the packages are removed from the DUT. {
8102"foo": { "pkg": "bar" }, "foobar": [ { "pkg": "foo", "rm": true }, {
8103"pkg": "bar", "rm": true } ] }
8104
8105.. _usingpoky-debugging-tools-and-techniques:
8106
8107Debugging Tools and Techniques
8108==============================
8109
8110The exact method for debugging build failures depends on the nature of
8111the problem and on the system's area from which the bug originates.
8112Standard debugging practices such as comparison against the last known
8113working version with examination of the changes and the re-application
8114of steps to identify the one causing the problem are valid for the Yocto
8115Project just as they are for any other system. Even though it is
8116impossible to detail every possible potential failure, this section
8117provides some general tips to aid in debugging given a variety of
8118situations.
8119
8120.. note::
8121
8122 A useful feature for debugging is the error reporting tool.
8123 Configuring the Yocto Project to use this tool causes the
8124 OpenEmbedded build system to produce error reporting commands as part
8125 of the console output. You can enter the commands after the build
8126 completes to log error information into a common database, that can
8127 help you figure out what might be going wrong. For information on how
8128 to enable and use this feature, see the "
8129 Using the Error Reporting Tool
8130 " section.
8131
8132The following list shows the debugging topics in the remainder of this
8133section:
8134
8135- "`Viewing Logs from Failed
8136 Tasks <#dev-debugging-viewing-logs-from-failed-tasks>`__" describes
8137 how to find and view logs from tasks that failed during the build
8138 process.
8139
8140- "`Viewing Variable
8141 Values <#dev-debugging-viewing-variable-values>`__" describes how to
8142 use the BitBake ``-e`` option to examine variable values after a
8143 recipe has been parsed.
8144
8145- "`Viewing Package Information with
8146 ``oe-pkgdata-util`` <#viewing-package-information-with-oe-pkgdata-util>`__"
8147 describes how to use the ``oe-pkgdata-util`` utility to query
8148 ```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__ and
8149 display package-related information for built packages.
8150
8151- "`Viewing Dependencies Between Recipes and
8152 Tasks <#dev-viewing-dependencies-between-recipes-and-tasks>`__"
8153 describes how to use the BitBake ``-g`` option to display recipe
8154 dependency information used during the build.
8155
8156- "`Viewing Task Variable
8157 Dependencies <#dev-viewing-task-variable-dependencies>`__" describes
8158 how to use the ``bitbake-dumpsig`` command in conjunction with key
8159 subdirectories in the `Build
8160 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ to determine
8161 variable dependencies.
8162
8163- "`Running Specific Tasks <#dev-debugging-taskrunning>`__" describes
8164 how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``)
8165 to run specific tasks in the build chain. It can be useful to run
8166 tasks "out-of-order" when trying isolate build issues.
8167
8168- "`General BitBake Problems <#dev-debugging-bitbake>`__" describes how
8169 to use BitBake's ``-D`` debug output option to reveal more about what
8170 BitBake is doing during the build.
8171
8172- "`Building with No Dependencies <#dev-debugging-buildfile>`__"
8173 describes how to use the BitBake ``-b`` option to build a recipe
8174 while ignoring dependencies.
8175
8176- "`Recipe Logging Mechanisms <#recipe-logging-mechanisms>`__"
8177 describes how to use the many recipe logging functions to produce
8178 debugging output and report errors and warnings.
8179
8180- "`Debugging Parallel Make Races <#debugging-parallel-make-races>`__"
8181 describes how to debug situations where the build consists of several
8182 parts that are run simultaneously and when the output or result of
8183 one part is not ready for use with a different part of the build that
8184 depends on that output.
8185
8186- "`Debugging With the GNU Project Debugger (GDB)
8187 Remotely <#platdev-gdb-remotedebug>`__" describes how to use GDB to
8188 allow you to examine running programs, which can help you fix
8189 problems.
8190
8191- "`Debugging with the GNU Project Debugger (GDB) on the
8192 Target <#debugging-with-the-gnu-project-debugger-gdb-on-the-target>`__"
8193 describes how to use GDB directly on target hardware for debugging.
8194
8195- "`Other Debugging Tips <#dev-other-debugging-others>`__" describes
8196 miscellaneous debugging tips that can be useful.
8197
8198.. _dev-debugging-viewing-logs-from-failed-tasks:
8199
8200Viewing Logs from Failed Tasks
8201------------------------------
8202
8203You can find the log for a task in the file
8204``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/temp/log.do_``\ taskname.
8205For example, the log for the
8206```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ task of the
8207QEMU minimal image for the x86 machine (``qemux86``) might be in
8208``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``.
8209To see the commands `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ ran
8210to generate a log, look at the corresponding ``run.do_``\ taskname file
8211in the same directory.
8212
8213``log.do_``\ taskname and ``run.do_``\ taskname are actually symbolic
8214links to ``log.do_``\ taskname\ ``.``\ pid and
8215``log.run_``\ taskname\ ``.``\ pid, where pid is the PID the task had
8216when it ran. The symlinks always point to the files corresponding to the
8217most recent run.
8218
8219.. _dev-debugging-viewing-variable-values:
8220
8221Viewing Variable Values
8222-----------------------
8223
8224Sometimes you need to know the value of a variable as a result of
8225BitBake's parsing step. This could be because some unexpected behavior
8226occurred in your project. Perhaps an attempt to `modify a
8227variable <&YOCTO_DOCS_BB_URL;#modifying-existing-variables>`__ did not
8228work out as expected.
8229
8230BitBake's ``-e`` option is used to display variable values after
8231parsing. The following command displays the variable values after the
8232configuration files (i.e. ``local.conf``, ``bblayers.conf``,
8233``bitbake.conf`` and so forth) have been parsed: $ bitbake -e The
8234following command displays variable values after a specific recipe has
8235been parsed. The variables include those from the configuration as well:
8236$ bitbake -e recipename
8237
8238.. note::
8239
8240 Each recipe has its own private set of variables (datastore).
8241 Internally, after parsing the configuration, a copy of the resulting
8242 datastore is made prior to parsing each recipe. This copying implies
8243 that variables set in one recipe will not be visible to other
8244 recipes.
8245
8246 Likewise, each task within a recipe gets a private datastore based on
8247 the recipe datastore, which means that variables set within one task
8248 will not be visible to other tasks.
8249
8250In the output of ``bitbake -e``, each variable is preceded by a
8251description of how the variable got its value, including temporary
8252values that were later overriden. This description also includes
8253variable flags (varflags) set on the variable. The output can be very
8254helpful during debugging.
8255
8256Variables that are exported to the environment are preceded by
8257``export`` in the output of ``bitbake -e``. See the following example:
8258export CC="i586-poky-linux-gcc -m32 -march=i586
8259--sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86"
8260
8261In addition to variable values, the output of the ``bitbake -e`` and
8262``bitbake -e`` recipe commands includes the following information:
8263
8264- The output starts with a tree listing all configuration files and
8265 classes included globally, recursively listing the files they include
8266 or inherit in turn. Much of the behavior of the OpenEmbedded build
8267 system (including the behavior of the `normal recipe build
8268 tasks <&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks>`__) is
8269 implemented in the
8270 ```base`` <&YOCTO_DOCS_REF_URL;#ref-classes-base>`__ class and the
8271 classes it inherits, rather than being built into BitBake itself.
8272
8273- After the variable values, all functions appear in the output. For
8274 shell functions, variables referenced within the function body are
8275 expanded. If a function has been modified using overrides or using
8276 override-style operators like ``_append`` and ``_prepend``, then the
8277 final assembled function body appears in the output.
8278
8279Viewing Package Information with ``oe-pkgdata-util``
8280----------------------------------------------------
8281
8282You can use the ``oe-pkgdata-util`` command-line utility to query
8283```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__ and display
8284various package-related information. When you use the utility, you must
8285use it to view information on packages that have already been built.
8286
8287Following are a few of the available ``oe-pkgdata-util`` subcommands.
8288
8289.. note::
8290
8291 You can use the standard \* and ? globbing wildcards as part of
8292 package names and paths.
8293
8294- ``oe-pkgdata-util list-pkgs [``\ pattern\ ``]``: Lists all packages
8295 that have been built, optionally limiting the match to packages that
8296 match pattern.
8297
8298- ``oe-pkgdata-util list-pkg-files ``\ package\ `` ...``: Lists the
8299 files and directories contained in the given packages.
8300
8301 .. note::
8302
8303 A different way to view the contents of a package is to look at
8304 the
8305 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/packages-split``
8306 directory of the recipe that generates the package. This directory
8307 is created by the
8308 ```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ task
8309 and has one subdirectory for each package the recipe generates,
8310 which contains the files stored in that package.
8311
8312 If you want to inspect the ``${WORKDIR}/packages-split``
8313 directory, make sure that
8314 ```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ is not
8315 enabled when you build the recipe.
8316
8317- ``oe-pkgdata-util find-path ``\ path\ `` ...``: Lists the names of
8318 the packages that contain the given paths. For example, the following
8319 tells us that ``/usr/share/man/man1/make.1`` is contained in the
8320 ``make-doc`` package: $ oe-pkgdata-util find-path
8321 /usr/share/man/man1/make.1 make-doc: /usr/share/man/man1/make.1
8322
8323- ``oe-pkgdata-util lookup-recipe ``\ package\ `` ...``: Lists the name
8324 of the recipes that produce the given packages.
8325
8326For more information on the ``oe-pkgdata-util`` command, use the help
8327facility: $ oe-pkgdata-util DASHDASHhelp $ oe-pkgdata-util subcommand
8328--help
8329
8330.. _dev-viewing-dependencies-between-recipes-and-tasks:
8331
8332Viewing Dependencies Between Recipes and Tasks
8333----------------------------------------------
8334
8335Sometimes it can be hard to see why BitBake wants to build other recipes
8336before the one you have specified. Dependency information can help you
8337understand why a recipe is built.
8338
8339To generate dependency information for a recipe, run the following
8340command: $ bitbake -g recipename This command writes the following files
8341in the current directory:
8342
8343- ``pn-buildlist``: A list of recipes/targets involved in building
8344 recipename. "Involved" here means that at least one task from the
8345 recipe needs to run when building recipename from scratch. Targets
8346 that are in
8347 ```ASSUME_PROVIDED`` <&YOCTO_DOCS_REF_URL;#var-ASSUME_PROVIDED>`__
8348 are not listed.
8349
8350- ``task-depends.dot``: A graph showing dependencies between tasks.
8351
8352The graphs are in
8353`DOT <https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29>`__
8354format and can be converted to images (e.g. using the ``dot`` tool from
8355`Graphviz <http://www.graphviz.org/>`__).
8356
8357.. note::
8358
8359 - DOT files use a plain text format. The graphs generated using the
8360 ``bitbake -g`` command are often so large as to be difficult to
8361 read without special pruning (e.g. with Bitbake's ``-I`` option)
8362 and processing. Despite the form and size of the graphs, the
8363 corresponding ``.dot`` files can still be possible to read and
8364 provide useful information.
8365
8366 As an example, the ``task-depends.dot`` file contains lines such
8367 as the following: "libxslt.do_configure" ->
8368 "libxml2.do_populate_sysroot" The above example line reveals that
8369 the
8370 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__
8371 task in ``libxslt`` depends on the
8372 ```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
8373 task in ``libxml2``, which is a normal
8374 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ dependency
8375 between the two recipes.
8376
8377 - For an example of how ``.dot`` files can be processed, see the
8378 ``scripts/contrib/graph-tool`` Python script, which finds and
8379 displays paths between graph nodes.
8380
8381You can use a different method to view dependency information by using
8382the following command: $ bitbake -g -u taskexp recipename This command
8383displays a GUI window from which you can view build-time and runtime
8384dependencies for the recipes involved in building recipename.
8385
8386.. _dev-viewing-task-variable-dependencies:
8387
8388Viewing Task Variable Dependencies
8389----------------------------------
8390
8391As mentioned in the "`Checksums
8392(Signatures) <&YOCTO_DOCS_BB_URL;#checksums>`__" section of the BitBake
8393User Manual, BitBake tries to automatically determine what variables a
8394task depends on so that it can rerun the task if any values of the
8395variables change. This determination is usually reliable. However, if
8396you do things like construct variable names at runtime, then you might
8397have to manually declare dependencies on those variables using
8398``vardeps`` as described in the "`Variable
8399Flags <&YOCTO_DOCS_BB_URL;#variable-flags>`__" section of the BitBake
8400User Manual.
8401
8402If you are unsure whether a variable dependency is being picked up
8403automatically for a given task, you can list the variable dependencies
8404BitBake has determined by doing the following:
8405
84061. Build the recipe containing the task: $ bitbake recipename
8407
84082. Inside the ```STAMPS_DIR`` <&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR>`__
8409 directory, find the signature data (``sigdata``) file that
8410 corresponds to the task. The ``sigdata`` files contain a pickled
8411 Python database of all the metadata that went into creating the input
8412 checksum for the task. As an example, for the
8413 ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ task of the
8414 ``db`` recipe, the ``sigdata`` file might be found in the following
8415 location:
8416 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
8417 For tasks that are accelerated through the shared state
8418 (`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__) cache, an
8419 additional ``siginfo`` file is written into
8420 ```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ along with
8421 the cached task output. The ``siginfo`` files contain exactly the
8422 same information as ``sigdata`` files.
8423
84243. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here
8425 is an example: $ bitbake-dumpsig
8426 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
8427 In the output of the above command, you will find a line like the
8428 following, which lists all the (inferred) variable dependencies for
8429 the task. This list also includes indirect dependencies from
8430 variables depending on other variables, recursively. Task
8431 dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]',
8432 'SRC_URI[sha256sum]', 'base_do_fetch']
8433
8434 .. note::
8435
8436 Functions (e.g.
8437 base_do_fetch
8438 ) also count as variable dependencies. These functions in turn
8439 depend on the variables they reference.
8440
8441 The output of ``bitbake-dumpsig`` also includes the value each
8442 variable had, a list of dependencies for each variable, and
8443 ```BB_HASHBASE_WHITELIST`` <&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST>`__
8444 information.
8445
8446There is also a ``bitbake-diffsigs`` command for comparing two
8447``siginfo`` or ``sigdata`` files. This command can be helpful when
8448trying to figure out what changed between two versions of a task. If you
8449call ``bitbake-diffsigs`` with just one file, the command behaves like
8450``bitbake-dumpsig``.
8451
8452You can also use BitBake to dump out the signature construction
8453information without executing tasks by using either of the following
8454BitBake command-line options: DASHDASHdump-signatures=SIGNATURE_HANDLER
8455-S SIGNATURE_HANDLER
8456
8457.. note::
8458
8459 Two common values for
8460 SIGNATURE_HANDLER
8461 are "none" and "printdiff", which dump only the signature or compare
8462 the dumped signature with the cached one, respectively.
8463
8464Using BitBake with either of these options causes BitBake to dump out
8465``sigdata`` files in the ``stamps`` directory for every task it would
8466have executed instead of building the specified target package.
8467
8468.. _dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task:
8469
8470Viewing Metadata Used to Create the Input Signature of a Shared State Task
8471--------------------------------------------------------------------------
8472
8473Seeing what metadata went into creating the input signature of a shared
8474state (sstate) task can be a useful debugging aid. This information is
8475available in signature information (``siginfo``) files in
8476```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__. For
8477information on how to view and interpret information in ``siginfo``
8478files, see the "`Viewing Task Variable
8479Dependencies <#dev-viewing-task-variable-dependencies>`__" section.
8480
8481For conceptual information on shared state, see the "`Shared
8482State <&YOCTO_DOCS_OM_URL;#shared-state>`__" section in the Yocto
8483Project Overview and Concepts Manual.
8484
8485.. _dev-invalidating-shared-state-to-force-a-task-to-run:
8486
8487Invalidating Shared State to Force a Task to Run
8488------------------------------------------------
8489
8490The OpenEmbedded build system uses
8491`checksums <&YOCTO_DOCS_OM_URL;#overview-checksums>`__ and `shared
8492state <&YOCTO_DOCS_OM_URL;#shared-state>`__ cache to avoid unnecessarily
8493rebuilding tasks. Collectively, this scheme is known as "shared state
8494code."
8495
8496As with all schemes, this one has some drawbacks. It is possible that
8497you could make implicit changes to your code that the checksum
8498calculations do not take into account. These implicit changes affect a
8499task's output but do not trigger the shared state code into rebuilding a
8500recipe. Consider an example during which a tool changes its output.
8501Assume that the output of ``rpmdeps`` changes. The result of the change
8502should be that all the ``package`` and ``package_write_rpm`` shared
8503state cache items become invalid. However, because the change to the
8504output is external to the code and therefore implicit, the associated
8505shared state cache items do not become invalidated. In this case, the
8506build process uses the cached items rather than running the task again.
8507Obviously, these types of implicit changes can cause problems.
8508
8509To avoid these problems during the build, you need to understand the
8510effects of any changes you make. Realize that changes you make directly
8511to a function are automatically factored into the checksum calculation.
8512Thus, these explicit changes invalidate the associated area of shared
8513state cache. However, you need to be aware of any implicit changes that
8514are not obvious changes to the code and could affect the output of a
8515given task.
8516
8517When you identify an implicit change, you can easily take steps to
8518invalidate the cache and force the tasks to run. The steps you can take
8519are as simple as changing a function's comments in the source code. For
8520example, to invalidate package shared state files, change the comment
8521statements of
8522```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ or the
8523comments of one of the functions it calls. Even though the change is
8524purely cosmetic, it causes the checksum to be recalculated and forces
8525the build system to run the task again.
8526
8527.. note::
8528
8529 For an example of a commit that makes a cosmetic change to invalidate
8530 shared state, see this
8531 commit
8532 .
8533
8534.. _dev-debugging-taskrunning:
8535
8536Running Specific Tasks
8537----------------------
8538
8539Any given recipe consists of a set of tasks. The standard BitBake
8540behavior in most cases is: ``do_fetch``, ``do_unpack``, ``do_patch``,
8541``do_configure``, ``do_compile``, ``do_install``, ``do_package``,
8542``do_package_write_*``, and ``do_build``. The default task is
8543``do_build`` and any tasks on which it depends build first. Some tasks,
8544such as ``do_devshell``, are not part of the default build chain. If you
8545wish to run a task that is not part of the default build chain, you can
8546use the ``-c`` option in BitBake. Here is an example: $ bitbake
8547matchbox-desktop -c devshell
8548
8549The ``-c`` option respects task dependencies, which means that all other
8550tasks (including tasks from other recipes) that the specified task
8551depends on will be run before the task. Even when you manually specify a
8552task to run with ``-c``, BitBake will only run the task if it considers
8553it "out of date". See the "`Stamp Files and the Rerunning of
8554Tasks <&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks>`__"
8555section in the Yocto Project Overview and Concepts Manual for how
8556BitBake determines whether a task is "out of date".
8557
8558If you want to force an up-to-date task to be rerun (e.g. because you
8559made manual modifications to the recipe's
8560```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__ that you want to try
8561out), then you can use the ``-f`` option.
8562
8563.. note::
8564
8565 The reason
8566 -f
8567 is never required when running the
8568 do_devshell
8569 task is because the
8570 [
8571 nostamp
8572 ]
8573 variable flag is already set for the task.
8574
8575The following example shows one way you can use the ``-f`` option: $
8576bitbake matchbox-desktop . . make some changes to the source code in the
8577work directory . . $ bitbake matchbox-desktop -c compile -f $ bitbake
8578matchbox-desktop
8579
8580This sequence first builds and then recompiles ``matchbox-desktop``. The
8581last command reruns all tasks (basically the packaging tasks) after the
8582compile. BitBake recognizes that the ``do_compile`` task was rerun and
8583therefore understands that the other tasks also need to be run again.
8584
8585Another, shorter way to rerun a task and all `normal recipe build
8586tasks <&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks>`__ that depend on
8587it is to use the ``-C`` option.
8588
8589.. note::
8590
8591 This option is upper-cased and is separate from the
8592 -c
8593 option, which is lower-cased.
8594
8595Using this option invalidates the given task and then runs the
8596```do_build`` <&YOCTO_DOCS_REF_URL;#ref-tasks-build>`__ task, which is
8597the default task if no task is given, and the tasks on which it depends.
8598You could replace the final two commands in the previous example with
8599the following single command: $ bitbake matchbox-desktop -C compile
8600Internally, the ``-f`` and ``-C`` options work by tainting (modifying)
8601the input checksum of the specified task. This tainting indirectly
8602causes the task and its dependent tasks to be rerun through the normal
8603task dependency mechanisms.
8604
8605.. note::
8606
8607 BitBake explicitly keeps track of which tasks have been tainted in
8608 this fashion, and will print warnings such as the following for
8609 builds involving such tasks:
8610 ::
8611
8612 WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run
8613
8614
8615 The purpose of the warning is to let you know that the work directory
8616 and build output might not be in the clean state they would be in for
8617 a "normal" build, depending on what actions you took. To get rid of
8618 such warnings, you can remove the work directory and rebuild the
8619 recipe, as follows:
8620 ::
8621
8622 $ bitbake matchbox-desktop -c clean
8623 $ bitbake matchbox-desktop
8624
8625
8626You can view a list of tasks in a given package by running the
8627``do_listtasks`` task as follows: $ bitbake matchbox-desktop -c
8628listtasks The results appear as output to the console and are also in
8629the file ``${WORKDIR}/temp/log.do_listtasks``.
8630
8631.. _dev-debugging-bitbake:
8632
8633General BitBake Problems
8634------------------------
8635
8636You can see debug output from BitBake by using the ``-D`` option. The
8637debug output gives more information about what BitBake is doing and the
8638reason behind it. Each ``-D`` option you use increases the logging
8639level. The most common usage is ``-DDD``.
8640
8641The output from ``bitbake -DDD -v`` targetname can reveal why BitBake
8642chose a certain version of a package or why BitBake picked a certain
8643provider. This command could also help you in a situation where you
8644think BitBake did something unexpected.
8645
8646.. _dev-debugging-buildfile:
8647
8648Building with No Dependencies
8649-----------------------------
8650
8651To build a specific recipe (``.bb`` file), you can use the following
8652command form: $ bitbake -b somepath/somerecipe.bb This command form does
8653not check for dependencies. Consequently, you should use it only when
8654you know existing dependencies have been met.
8655
8656.. note::
8657
8658 You can also specify fragments of the filename. In this case, BitBake
8659 checks for a unique match.
8660
8661Recipe Logging Mechanisms
8662-------------------------
8663
8664The Yocto Project provides several logging functions for producing
8665debugging output and reporting errors and warnings. For Python
8666functions, the following logging functions exist. All of these functions
8667log to ``${T}/log.do_``\ task, and can also log to standard output
8668(stdout) with the right settings:
8669
8670- ``bb.plain(``\ msg\ ``)``: Writes msg as is to the log while also
8671 logging to stdout.
8672
8673- ``bb.note(``\ msg\ ``)``: Writes "NOTE: msg" to the log. Also logs to
8674 stdout if BitBake is called with "-v".
8675
8676- ``bb.debug(``\ level\ ``, ``\ msg\ ``)``: Writes "DEBUG: msg" to the
8677 log. Also logs to stdout if the log level is greater than or equal to
8678 level. See the "`-D <&YOCTO_DOCS_BB_URL;#usage-and-syntax>`__" option
8679 in the BitBake User Manual for more information.
8680
8681- ``bb.warn(``\ msg\ ``)``: Writes "WARNING: msg" to the log while also
8682 logging to stdout.
8683
8684- ``bb.error(``\ msg\ ``)``: Writes "ERROR: msg" to the log while also
8685 logging to standard out (stdout).
8686
8687 .. note::
8688
8689 Calling this function does not cause the task to fail.
8690
8691- ``bb.fatal(``\ msg\ ``)``: This logging function is similar to
8692 ``bb.error(``\ msg\ ``)`` but also causes the calling task to fail.
8693
8694 .. note::
8695
8696 bb.fatal()
8697 raises an exception, which means you do not need to put a "return"
8698 statement after the function.
8699
8700The same logging functions are also available in shell functions, under
8701the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``,
8702and ``bbfatal``. The
8703```logging`` <&YOCTO_DOCS_REF_URL;#ref-classes-logging>`__ class
8704implements these functions. See that class in the ``meta/classes``
8705folder of the `Source
8706Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ for information.
8707
8708Logging With Python
8709~~~~~~~~~~~~~~~~~~~
8710
8711When creating recipes using Python and inserting code that handles build
8712logs, keep in mind the goal is to have informative logs while keeping
8713the console as "silent" as possible. Also, if you want status messages
8714in the log, use the "debug" loglevel.
8715
8716Following is an example written in Python. The code handles logging for
8717a function that determines the number of tasks needed to be run. See the
8718"```do_listtasks`` <&YOCTO_DOCS_REF_URL;#ref-tasks-listtasks>`__"
8719section for additional information: python do_listtasks() { bb.debug(2,
8720"Starting to figure out the task list") if noteworthy_condition:
8721bb.note("There are 47 tasks to run") bb.debug(2, "Got to point xyz") if
8722warning_trigger: bb.warn("Detected warning_trigger, this might be a
8723problem later.") if recoverable_error: bb.error("Hit recoverable_error,
8724you really need to fix this!") if fatal_error: bb.fatal("fatal_error
8725detected, unable to print the task list") bb.plain("The tasks present
8726are abc") bb.debug(2, "Finished figuring out the tasklist") }
8727
8728Logging With Bash
8729~~~~~~~~~~~~~~~~~
8730
8731When creating recipes using Bash and inserting code that handles build
8732logs, you have the same goals - informative with minimal console output.
8733The syntax you use for recipes written in Bash is similar to that of
8734recipes written in Python described in the previous section.
8735
8736Following is an example written in Bash. The code logs the progress of
8737the ``do_my_function`` function. do_my_function() { bbdebug 2 "Running
8738do_my_function" if [ exceptional_condition ]; then bbnote "Hit
8739exceptional_condition" fi bbdebug 2 "Got to point xyz" if [
8740warning_trigger ]; then bbwarn "Detected warning_trigger, this might
8741cause a problem later." fi if [ recoverable_error ]; then bberror "Hit
8742recoverable_error, correcting" fi if [ fatal_error ]; then bbfatal
8743"fatal_error detected" fi bbdebug 2 "Completed do_my_function" }
8744
8745Debugging Parallel Make Races
8746-----------------------------
8747
8748A parallel ``make`` race occurs when the build consists of several parts
8749that are run simultaneously and a situation occurs when the output or
8750result of one part is not ready for use with a different part of the
8751build that depends on that output. Parallel make races are annoying and
8752can sometimes be difficult to reproduce and fix. However, some simple
8753tips and tricks exist that can help you debug and fix them. This section
8754presents a real-world example of an error encountered on the Yocto
8755Project autobuilder and the process used to fix it.
8756
8757.. note::
8758
8759 If you cannot properly fix a
8760 make
8761 race condition, you can work around it by clearing either the
8762 PARALLEL_MAKE
8763 or
8764 PARALLEL_MAKEINST
8765 variables.
8766
8767The Failure
8768~~~~~~~~~~~
8769
8770For this example, assume that you are building an image that depends on
8771the "neard" package. And, during the build, BitBake runs into problems
8772and creates the following output.
8773
8774.. note::
8775
8776 This example log file has longer lines artificially broken to make
8777 the listing easier to read.
8778
8779If you examine the output or the log file, you see the failure during
8780``make``: \| DEBUG: SITE files ['endian-little', 'bit-32',
8781'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
8782\| DEBUG: Executing shell function do_compile \| NOTE: make -j 16 \|
8783make --no-print-directory all-am \| /bin/mkdir -p include/near \|
8784/bin/mkdir -p include/near \| /bin/mkdir -p include/near \| ln -s
8785/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87860.14-r0/neard-0.14/include/types.h include/near/types.h \| ln -s
8787/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87880.14-r0/neard-0.14/include/log.h include/near/log.h \| ln -s
8789/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87900.14-r0/neard-0.14/include/plugin.h include/near/plugin.h \| /bin/mkdir
8791-p include/near \| /bin/mkdir -p include/near \| /bin/mkdir -p
8792include/near \| ln -s
8793/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87940.14-r0/neard-0.14/include/tag.h include/near/tag.h \| /bin/mkdir -p
8795include/near \| ln -s
8796/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87970.14-r0/neard-0.14/include/adapter.h include/near/adapter.h \|
8798/bin/mkdir -p include/near \| ln -s
8799/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88000.14-r0/neard-0.14/include/ndef.h include/near/ndef.h \| ln -s
8801/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88020.14-r0/neard-0.14/include/tlv.h include/near/tlv.h \| /bin/mkdir -p
8803include/near \| /bin/mkdir -p include/near \| ln -s
8804/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88050.14-r0/neard-0.14/include/setting.h include/near/setting.h \|
8806/bin/mkdir -p include/near \| /bin/mkdir -p include/near \| /bin/mkdir
8807-p include/near \| ln -s
8808/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88090.14-r0/neard-0.14/include/device.h include/near/device.h \| ln -s
8810/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88110.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h \| ln -s
8812/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88130.14-r0/neard-0.14/include/snep.h include/near/snep.h \| ln -s
8814/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88150.14-r0/neard-0.14/include/version.h include/near/version.h \| ln -s
8816/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88170.14-r0/neard-0.14/include/dbus.h include/near/dbus.h \|
8818./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h
8819\| i586-poky-linux-gcc -m32 -march=i586
8820--sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/
8821build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src
8822-I./gdbus -I/home/pokybuild/
8823yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0
8824-I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/
8825lib/glib-2.0/include
8826-I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/
8827tmp/sysroots/qemux86/usr/include/dbus-1.0
8828-I/home/pokybuild/yocto-autobuilder/yocto-slave/
8829nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include
8830-I/home/pokybuild/yocto-autobuilder/
8831yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3
8832-DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\"
8833-DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types
8834-c -o tools/snep-send.o tools/snep-send.c \| In file included from
8835tools/snep-send.c:16:0: \| tools/../src/near.h:41:23: fatal error:
8836near/dbus.h: No such file or directory \| #include <near/dbus.h> \| ^ \|
8837compilation terminated. \| make[1]: \**\* [tools/snep-send.o] Error 1 \|
8838make[1]: \**\* Waiting for unfinished jobs.... \| make: \**\* [all]
8839Error 2 \| ERROR: oe_runmake failed
8840
8841Reproducing the Error
8842~~~~~~~~~~~~~~~~~~~~~
8843
8844Because race conditions are intermittent, they do not manifest
8845themselves every time you do the build. In fact, most times the build
8846will complete without problems even though the potential race condition
8847exists. Thus, once the error surfaces, you need a way to reproduce it.
8848
8849In this example, compiling the "neard" package is causing the problem.
8850So the first thing to do is build "neard" locally. Before you start the
8851build, set the
8852```PARALLEL_MAKE`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ variable
8853in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a
8854high value for ``PARALLEL_MAKE`` increases the chances of the race
8855condition showing up: $ bitbake neard
8856
8857Once the local build for "neard" completes, start a ``devshell`` build:
8858$ bitbake neard -c devshell For information on how to use a
8859``devshell``, see the "`Using a Development
8860Shell <#platdev-appdev-devshell>`__" section.
8861
8862In the ``devshell``, do the following: $ make clean $ make
8863tools/snep-send.o The ``devshell`` commands cause the failure to clearly
8864be visible. In this case, a missing dependency exists for the "neard"
8865Makefile target. Here is some abbreviated, sample output with the
8866missing dependency clearly visible at the end: i586-poky-linux-gcc -m32
8867-march=i586 --sysroot=/home/scott-lenovo/...... . . . tools/snep-send.c
8868In file included from tools/snep-send.c:16:0: tools/../src/near.h:41:23:
8869fatal error: near/dbus.h: No such file or directory #include
8870<near/dbus.h> ^ compilation terminated. make: \**\* [tools/snep-send.o]
8871Error 1 $
8872
8873Creating a Patch for the Fix
8874~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8875
8876Because there is a missing dependency for the Makefile target, you need
8877to patch the ``Makefile.am`` file, which is generated from
8878``Makefile.in``. You can use Quilt to create the patch: $ quilt new
8879parallelmake.patch Patch patches/parallelmake.patch is now on top $
8880quilt add Makefile.am File Makefile.am added to patch
8881patches/parallelmake.patch For more information on using Quilt, see the
8882"`Using Quilt in Your Workflow <#using-a-quilt-workflow>`__" section.
8883
8884At this point you need to make the edits to ``Makefile.am`` to add the
8885missing dependency. For our example, you have to add the following line
8886to the file: tools/snep-send.$(OBJEXT): include/near/dbus.h
8887
8888Once you have edited the file, use the ``refresh`` command to create the
8889patch: $ quilt refresh Refreshed patch patches/parallelmake.patch Once
8890the patch file exists, you need to add it back to the originating recipe
8891folder. Here is an example assuming a top-level `Source
8892Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ named ``poky``: $
8893cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard
8894The final thing you need to do to implement the fix in the build is to
8895update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the
8896```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement includes
8897the patch file. The recipe file is in the folder above the patch. Here
8898is what the edited ``SRC_URI`` statement would look like: SRC_URI =
8899"${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \\
8900file://neard.in \\ file://neard.service.in \\ file://parallelmake.patch
8901\\ "
8902
8903With the patch complete and moved to the correct folder and the
8904``SRC_URI`` statement updated, you can exit the ``devshell``: $ exit
8905
8906Testing the Build
8907~~~~~~~~~~~~~~~~~
8908
8909With everything in place, you can get back to trying the build again
8910locally: $ bitbake neard This build should succeed.
8911
8912Now you can open up a ``devshell`` again and repeat the clean and make
8913operations as follows: $ bitbake neard -c devshell $ make clean $ make
8914tools/snep-send.o The build should work without issue.
8915
8916As with all solved problems, if they originated upstream, you need to
8917submit the fix for the recipe in OE-Core and upstream so that the
8918problem is taken care of at its source. See the "`Submitting a Change to
8919the Yocto Project <#how-to-submit-a-change>`__" section for more
8920information.
8921
8922.. _platdev-gdb-remotedebug:
8923
8924Debugging With the GNU Project Debugger (GDB) Remotely
8925------------------------------------------------------
8926
8927GDB allows you to examine running programs, which in turn helps you to
8928understand and fix problems. It also allows you to perform post-mortem
8929style analysis of program crashes. GDB is available as a package within
8930the Yocto Project and is installed in SDK images by default. See the
8931"`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
8932Project Reference Manual for a description of these images. You can find
8933information on GDB at ` <http://sourceware.org/gdb/>`__.
8934
8935.. note::
8936
8937 For best results, install debug (
8938 -dbg
8939 ) packages for the applications you are going to debug. Doing so
8940 makes extra debug symbols available that give you more meaningful
8941 output.
8942
8943Sometimes, due to memory or disk space constraints, it is not possible
8944to use GDB directly on the remote target to debug applications. These
8945constraints arise because GDB needs to load the debugging information
8946and the binaries of the process being debugged. Additionally, GDB needs
8947to perform many computations to locate information such as function
8948names, variable names and values, stack traces and so forth - even
8949before starting the debugging process. These extra computations place
8950more load on the target system and can alter the characteristics of the
8951program being debugged.
8952
8953To help get past the previously mentioned constraints, you can use
8954gdbserver, which runs on the remote target and does not load any
8955debugging information from the debugged process. Instead, a GDB instance
8956processes the debugging information that is run on a remote computer -
8957the host GDB. The host GDB then sends control commands to gdbserver to
8958make it stop or start the debugged program, as well as read or write
8959memory regions of that debugged program. All the debugging information
8960loaded and processed as well as all the heavy debugging is done by the
8961host GDB. Offloading these processes gives the gdbserver running on the
8962target a chance to remain small and fast.
8963
8964Because the host GDB is responsible for loading the debugging
8965information and for doing the necessary processing to make actual
8966debugging happen, you have to make sure the host can access the
8967unstripped binaries complete with their debugging information and also
8968be sure the target is compiled with no optimizations. The host GDB must
8969also have local access to all the libraries used by the debugged
8970program. Because gdbserver does not need any local debugging
8971information, the binaries on the remote target can remain stripped.
8972However, the binaries must also be compiled without optimization so they
8973match the host's binaries.
8974
8975To remain consistent with GDB documentation and terminology, the binary
8976being debugged on the remote target machine is referred to as the
8977"inferior" binary. For documentation on GDB see the `GDB
8978site <http://sourceware.org/gdb/documentation/>`__.
8979
8980The following steps show you how to debug using the GNU project
8981debugger.
8982
89831. *Configure your build system to construct the companion debug
8984 filesystem:*
8985
8986 In your ``local.conf`` file, set the following: IMAGE_GEN_DEBUGFS =
8987 "1" IMAGE_FSTYPES_DEBUGFS = "tar.bz2" These options cause the
8988 OpenEmbedded build system to generate a special companion filesystem
8989 fragment, which contains the matching source and debug symbols to
8990 your deployable filesystem. The build system does this by looking at
8991 what is in the deployed filesystem, and pulling the corresponding
8992 ``-dbg`` packages.
8993
8994 The companion debug filesystem is not a complete filesystem, but only
8995 contains the debug fragments. This filesystem must be combined with
8996 the full filesystem for debugging. Subsequent steps in this procedure
8997 show how to combine the partial filesystem with the full filesystem.
8998
89992. *Configure the system to include gdbserver in the target filesystem:*
9000
9001 Make the following addition in either your ``local.conf`` file or in
9002 an image recipe: IMAGE_INSTALL_append = “ gdbserver" The change makes
9003 sure the ``gdbserver`` package is included.
9004
90053. *Build the environment:*
9006
9007 Use the following command to construct the image and the companion
9008 Debug Filesystem: $ bitbake image Build the cross GDB component and
9009 make it available for debugging. Build the SDK that matches the
9010 image. Building the SDK is best for a production build that can be
9011 used later for debugging, especially during long term maintenance: $
9012 bitbake -c populate_sdk image
9013
9014 Alternatively, you can build the minimal toolchain components that
9015 match the target. Doing so creates a smaller than typical SDK and
9016 only contains a minimal set of components with which to build simple
9017 test applications, as well as run the debugger: $ bitbake
9018 meta-toolchain
9019
9020 A final method is to build Gdb itself within the build system: $
9021 bitbake gdb-cross-architecture Doing so produces a temporary copy of
9022 ``cross-gdb`` you can use for debugging during development. While
9023 this is the quickest approach, the two previous methods in this step
9024 are better when considering long-term maintenance strategies.
9025
9026 .. note::
9027
9028 If you run
9029 bitbake gdb-cross
9030 , the OpenEmbedded build system suggests the actual image (e.g.
9031 gdb-cross-i586
9032 ). The suggestion is usually the actual name you want to use.
9033
90344. *Set up the* ``debugfs``
9035
9036 Run the following commands to set up the ``debugfs``: $ mkdir debugfs
9037 $ cd debugfs $ tar xvfj
9038 build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2 $ tar
9039 xvfj
9040 build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2
9041
90425. *Set up GDB*
9043
9044 Install the SDK (if you built one) and then source the correct
9045 environment file. Sourcing the environment file puts the SDK in your
9046 ``PATH`` environment variable.
9047
9048 If you are using the build system, Gdb is located in
9049 build-dir/tmp/sysroots/host/usr/bin/architecture/architecture-gdb
9050
90516. *Boot the target:*
9052
9053 For information on how to run QEMU, see the `QEMU
9054 Documentation <http://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__.
9055
9056 .. note::
9057
9058 Be sure to verify that your host can access the target via TCP.
9059
90607. *Debug a program:*
9061
9062 Debugging a program involves running gdbserver on the target and then
9063 running Gdb on the host. The example in this step debugs ``gzip``:
9064 root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help For
9065 additional gdbserver options, see the `GDB Server
9066 Documentation <https://www.gnu.org/software/gdb/documentation/>`__.
9067
9068 After running gdbserver on the target, you need to run Gdb on the
9069 host and configure it and connect to the target. Use these commands:
9070 $ cd directory-holding-the-debugfs-directory $ arch-gdb (gdb) set
9071 sysroot debugfs (gdb) set substitute-path /usr/src/debug
9072 debugfs/usr/src/debug (gdb) target remote IP-of-target:1234 At this
9073 point, everything should automatically load (i.e. matching binaries,
9074 symbols and headers).
9075
9076 .. note::
9077
9078 The Gdb
9079 set
9080 commands in the previous example can be placed into the users
9081 ~/.gdbinit
9082 file. Upon starting, Gdb automatically runs whatever commands are
9083 in that file.
9084
90858. *Deploying without a full image rebuild:*
9086
9087 In many cases, during development you want a quick method to deploy a
9088 new binary to the target and debug it, without waiting for a full
9089 image build.
9090
9091 One approach to solving this situation is to just build the component
9092 you want to debug. Once you have built the component, copy the
9093 executable directly to both the target and the host ``debugfs``.
9094
9095 If the binary is processed through the debug splitting in
9096 OpenEmbedded, you should also copy the debug items (i.e. ``.debug``
9097 contents and corresponding ``/usr/src/debug`` files) from the work
9098 directory. Here is an example: $ bitbake bash $ bitbake -c devshell
9099 bash $ cd .. $ scp packages-split/bash/bin/bash target:/bin/bash $ cp
9100 -a packages-split/bash-dbg/\* path/debugfs
9101
9102Debugging with the GNU Project Debugger (GDB) on the Target
9103-----------------------------------------------------------
9104
9105The previous section addressed using GDB remotely for debugging
9106purposes, which is the most usual case due to the inherent hardware
9107limitations on many embedded devices. However, debugging in the target
9108hardware itself is also possible with more powerful devices. This
9109section describes what you need to do in order to support using GDB to
9110debug on the target hardware.
9111
9112To support this kind of debugging, you need do the following:
9113
9114- Ensure that GDB is on the target. You can do this by adding "gdb" to
9115 ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__:
9116 IMAGE_INSTALL_append = " gdb" Alternatively, you can add
9117 "tools-debug" to
9118 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__:
9119 IMAGE_FEATURES_append = " tools-debug"
9120
9121- Ensure that debug symbols are present. You can make sure these
9122 symbols are present by installing ``-dbg``: IMAGE_INSTALL_append = "
9123 packagename-dbg" Alternatively, you can do the following to include
9124 all the debug symbols: IMAGE_FEATURES_append = " dbg-pkgs"
9125
9126.. note::
9127
9128 To improve the debug information accuracy, you can reduce the level
9129 of optimization used by the compiler. For example, when adding the
9130 following line to your
9131 local.conf
9132 file, you will reduce optimization from
9133 FULL_OPTIMIZATION
9134 of "-O2" to
9135 DEBUG_OPTIMIZATION
9136 of "-O -fno-omit-frame-pointer":
9137 ::
9138
9139 DEBUG_BUILD = "1"
9140
9141
9142 Consider that this will reduce the application's performance and is
9143 recommended only for debugging purposes.
9144
9145.. _dev-other-debugging-others:
9146
9147Other Debugging Tips
9148--------------------
9149
9150Here are some other tips that you might find useful:
9151
9152- When adding new packages, it is worth watching for undesirable items
9153 making their way into compiler command lines. For example, you do not
9154 want references to local system files like ``/usr/lib/`` or
9155 ``/usr/include/``.
9156
9157- If you want to remove the ``psplash`` boot splashscreen, add
9158 ``psplash=false`` to the kernel command line. Doing so prevents
9159 ``psplash`` from loading and thus allows you to see the console. It
9160 is also possible to switch out of the splashscreen by switching the
9161 virtual console (e.g. Fn+Left or Fn+Right on a Zaurus).
9162
9163- Removing ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ (usually
9164 ``tmp/``, within the `Build
9165 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__) can often fix
9166 temporary build issues. Removing ``TMPDIR`` is usually a relatively
9167 cheap operation, because task output will be cached in
9168 ```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ (usually
9169 ``sstate-cache/``, which is also in the Build Directory).
9170
9171 .. note::
9172
9173 Removing
9174 TMPDIR
9175 might be a workaround rather than a fix. Consequently, trying to
9176 determine the underlying cause of an issue before removing the
9177 directory is a good idea.
9178
9179- Understanding how a feature is used in practice within existing
9180 recipes can be very helpful. It is recommended that you configure
9181 some method that allows you to quickly search through files.
9182
9183 Using GNU Grep, you can use the following shell function to
9184 recursively search through common recipe-related files, skipping
9185 binary files, ``.git`` directories, and the Build Directory (assuming
9186 its name starts with "build"): g() { grep -Ir \\ --exclude-dir=.git
9187 \\ --exclude-dir='build*' \\ --include='*.bb*' \\ --include='*.inc*'
9188 \\ --include='*.conf*' \\ --include='*.py*' \\ "$@" } Following are
9189 some usage examples: $ g FOO # Search recursively for "FOO" $ g -i
9190 foo # Search recursively for "foo", ignoring case $ g -w FOO # Search
9191 recursively for "FOO" as a word, ignoring e.g. "FOOBAR" If figuring
9192 out how some feature works requires a lot of searching, it might
9193 indicate that the documentation should be extended or improved. In
9194 such cases, consider filing a documentation bug using the Yocto
9195 Project implementation of
9196 `Bugzilla <https://bugzilla.yoctoproject.org/>`__. For information on
9197 how to submit a bug against the Yocto Project, see the Yocto Project
9198 Bugzilla `wiki
9199 page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__
9200 and the "`Submitting a Defect Against the Yocto
9201 Project <#submitting-a-defect-against-the-yocto-project>`__" section.
9202
9203 .. note::
9204
9205 The manuals might not be the right place to document variables
9206 that are purely internal and have a limited scope (e.g. internal
9207 variables used to implement a single
9208 .bbclass
9209 file).
9210
9211Making Changes to the Yocto Project
9212===================================
9213
9214Because the Yocto Project is an open-source, community-based project,
9215you can effect changes to the project. This section presents procedures
9216that show you how to submit a defect against the project and how to
9217submit a change.
9218
9219Submitting a Defect Against the Yocto Project
9220---------------------------------------------
9221
9222Use the Yocto Project implementation of
9223`Bugzilla <http://www.bugzilla.org/about/>`__ to submit a defect (bug)
9224against the Yocto Project. For additional information on this
9225implementation of Bugzilla see the "`Yocto Project
9226Bugzilla <&YOCTO_DOCS_REF_URL;#resources-bugtracker>`__" section in the
9227Yocto Project Reference Manual. For more detail on any of the following
9228steps, see the Yocto Project `Bugzilla wiki
9229page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__.
9230
9231Use the following general steps to submit a bug"
9232
92331. Open the Yocto Project implementation of
9234 `Bugzilla <&YOCTO_BUGZILLA_URL;>`__.
9235
92362. Click "File a Bug" to enter a new bug.
9237
92383. Choose the appropriate "Classification", "Product", and "Component"
9239 for which the bug was found. Bugs for the Yocto Project fall into
9240 one of several classifications, which in turn break down into
9241 several products and components. For example, for a bug against the
9242 ``meta-intel`` layer, you would choose "Build System, Metadata &
9243 Runtime", "BSPs", and "bsps-meta-intel", respectively.
9244
92454. Choose the "Version" of the Yocto Project for which you found the
9246 bug (e.g. DISTRO).
9247
92485. Determine and select the "Severity" of the bug. The severity
9249 indicates how the bug impacted your work.
9250
92516. Choose the "Hardware" that the bug impacts.
9252
92537. Choose the "Architecture" that the bug impacts.
9254
92558. Choose a "Documentation change" item for the bug. Fixing a bug might
9256 or might not affect the Yocto Project documentation. If you are
9257 unsure of the impact to the documentation, select "Don't Know".
9258
92599. Provide a brief "Summary" of the bug. Try to limit your summary to
9260 just a line or two and be sure to capture the essence of the bug.
9261
926210. Provide a detailed "Description" of the bug. You should provide as
9263 much detail as you can about the context, behavior, output, and so
9264 forth that surrounds the bug. You can even attach supporting files
9265 for output from logs by using the "Add an attachment" button.
9266
926711. Click the "Submit Bug" button submit the bug. A new Bugzilla number
9268 is assigned to the bug and the defect is logged in the bug tracking
9269 system.
9270
9271Once you file a bug, the bug is processed by the Yocto Project Bug
9272Triage Team and further details concerning the bug are assigned (e.g.
9273priority and owner). You are the "Submitter" of the bug and any further
9274categorization, progress, or comments on the bug result in Bugzilla
9275sending you an automated email concerning the particular change or
9276progress to the bug.
9277
9278.. _how-to-submit-a-change:
9279
9280Submitting a Change to the Yocto Project
9281----------------------------------------
9282
9283Contributions to the Yocto Project and OpenEmbedded are very welcome.
9284Because the system is extremely configurable and flexible, we recognize
9285that developers will want to extend, configure or optimize it for their
9286specific uses.
9287
9288The Yocto Project uses a mailing list and a patch-based workflow that is
9289similar to the Linux kernel but contains important differences. In
9290general, a mailing list exists through which you can submit patches. You
9291should send patches to the appropriate mailing list so that they can be
9292reviewed and merged by the appropriate maintainer. The specific mailing
9293list you need to use depends on the location of the code you are
9294changing. Each component (e.g. layer) should have a ``README`` file that
9295indicates where to send the changes and which process to follow.
9296
9297You can send the patch to the mailing list using whichever approach you
9298feel comfortable with to generate the patch. Once sent, the patch is
9299usually reviewed by the community at large. If somebody has concerns
9300with the patch, they will usually voice their concern over the mailing
9301list. If a patch does not receive any negative reviews, the maintainer
9302of the affected layer typically takes the patch, tests it, and then
9303based on successful testing, merges the patch.
9304
9305The "poky" repository, which is the Yocto Project's reference build
9306environment, is a hybrid repository that contains several individual
9307pieces (e.g. BitBake, Metadata, documentation, and so forth) built using
9308the combo-layer tool. The upstream location used for submitting changes
9309varies by component:
9310
9311- *Core Metadata:* Send your patch to the
9312 `openembedded-core <http://lists.openembedded.org/mailman/listinfo/openembedded-core>`__
9313 mailing list. For example, a change to anything under the ``meta`` or
9314 ``scripts`` directories should be sent to this mailing list.
9315
9316- *BitBake:* For changes to BitBake (i.e. anything under the
9317 ``bitbake`` directory), send your patch to the
9318 `bitbake-devel <http://lists.openembedded.org/mailman/listinfo/bitbake-devel>`__
9319 mailing list.
9320
9321- *"meta-*" trees:* These trees contain Metadata. Use the
9322 `poky <https://lists.yoctoproject.org/listinfo/poky>`__ mailing list.
9323
9324For changes to other layers hosted in the Yocto Project source
9325repositories (i.e. ``yoctoproject.org``), tools, and the Yocto Project
9326documentation, use the `Yocto
9327Project <https://lists.yoctoproject.org/listinfo/yocto>`__ general
9328mailing list.
9329
9330.. note::
9331
9332 Sometimes a layer's documentation specifies to use a particular
9333 mailing list. If so, use that list.
9334
9335For additional recipes that do not fit into the core Metadata, you
9336should determine which layer the recipe should go into and submit the
9337change in the manner recommended by the documentation (e.g. the
9338``README`` file) supplied with the layer. If in doubt, please ask on the
9339Yocto general mailing list or on the openembedded-devel mailing list.
9340
9341You can also push a change upstream and request a maintainer to pull the
9342change into the component's upstream repository. You do this by pushing
9343to a contribution repository that is upstream. See the "`Git Workflows
9344and the Yocto
9345Project <&YOCTO_DOCS_OM_URL;#gs-git-workflows-and-the-yocto-project>`__"
9346section in the Yocto Project Overview and Concepts Manual for additional
9347concepts on working in the Yocto Project development environment.
9348
9349Two commonly used testing repositories exist for OpenEmbedded-Core:
9350
9351- *"ross/mut" branch:* The "mut" (master-under-test) tree exists in the
9352 ``poky-contrib`` repository in the `Yocto Project source
9353 repositories <&YOCTO_GIT_URL;>`__.
9354
9355- *"master-next" branch:* This branch is part of the main "poky"
9356 repository in the Yocto Project source repositories.
9357
9358Maintainers use these branches to test submissions prior to merging
9359patches. Thus, you can get an idea of the status of a patch based on
9360whether the patch has been merged into one of these branches.
9361
9362.. note::
9363
9364 This system is imperfect and changes can sometimes get lost in the
9365 flow. Asking about the status of a patch or change is reasonable if
9366 the change has been idle for a while with no feedback. The Yocto
9367 Project does have plans to use
9368 Patchwork
9369 to track the status of patches and also to automatically preview
9370 patches.
9371
9372The following sections provide procedures for submitting a change.
9373
9374.. _pushing-a-change-upstream:
9375
9376Using Scripts to Push a Change Upstream and Request a Pull
9377~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9378
9379Follow this procedure to push a change to an upstream "contrib" Git
9380repository:
9381
9382.. note::
9383
9384 You can find general Git information on how to push a change upstream
9385 in the
9386 Git Community Book
9387 .
9388
93891. *Make Your Changes Locally:* Make your changes in your local Git
9390 repository. You should make small, controlled, isolated changes.
9391 Keeping changes small and isolated aids review, makes
9392 merging/rebasing easier and keeps the change history clean should
9393 anyone need to refer to it in future.
9394
93952. *Stage Your Changes:* Stage your changes by using the ``git add``
9396 command on each file you changed.
9397
93983. *Commit Your Changes:* Commit the change by using the ``git commit``
9399 command. Make sure your commit information follows standards by
9400 following these accepted conventions:
9401
9402 - Be sure to include a "Signed-off-by:" line in the same style as
9403 required by the Linux kernel. Adding this line signifies that you,
9404 the submitter, have agreed to the Developer's Certificate of
9405 Origin 1.1 as follows: Developer's Certificate of Origin 1.1 By
9406 making a contribution to this project, I certify that: (a) The
9407 contribution was created in whole or in part by me and I have the
9408 right to submit it under the open source license indicated in the
9409 file; or (b) The contribution is based upon previous work that, to
9410 the best of my knowledge, is covered under an appropriate open
9411 source license and I have the right under that license to submit
9412 that work with modifications, whether created in whole or in part
9413 by me, under the same open source license (unless I am permitted
9414 to submit under a different license), as indicated in the file; or
9415 (c) The contribution was provided directly to me by some other
9416 person who certified (a), (b) or (c) and I have not modified it.
9417 (d) I understand and agree that this project and the contribution
9418 are public and that a record of the contribution (including all
9419 personal information I submit with it, including my sign-off) is
9420 maintained indefinitely and may be redistributed consistent with
9421 this project or the open source license(s) involved.
9422
9423 - Provide a single-line summary of the change. and, if more
9424 explanation is needed, provide more detail in the body of the
9425 commit. This summary is typically viewable in the "shortlist" of
9426 changes. Thus, providing something short and descriptive that
9427 gives the reader a summary of the change is useful when viewing a
9428 list of many commits. You should prefix this short description
9429 with the recipe name (if changing a recipe), or else with the
9430 short form path to the file being changed.
9431
9432 - For the body of the commit message, provide detailed information
9433 that describes what you changed, why you made the change, and the
9434 approach you used. It might also be helpful if you mention how you
9435 tested the change. Provide as much detail as you can in the body
9436 of the commit message.
9437
9438 .. note::
9439
9440 You do not need to provide a more detailed explanation of a
9441 change if the change is minor to the point of the single line
9442 summary providing all the information.
9443
9444 - If the change addresses a specific bug or issue that is associated
9445 with a bug-tracking ID, include a reference to that ID in your
9446 detailed description. For example, the Yocto Project uses a
9447 specific convention for bug references - any commit that addresses
9448 a specific bug should use the following form for the detailed
9449 description. Be sure to use the actual bug-tracking ID from
9450 Bugzilla for bug-id: Fixes [YOCTO #bug-id] detailed description of
9451 change
9452
94534. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for
9454 permissions to push to an upstream contrib repository, push the
9455 change to that repository: $ git push upstream_remote_repo
9456 local_branch_name For example, suppose you have permissions to push
9457 into the upstream ``meta-intel-contrib`` repository and you are
9458 working in a local branch named your_name\ ``/README``. The following
9459 command pushes your local commits to the ``meta-intel-contrib``
9460 upstream repository and puts the commit in a branch named
9461 your_name\ ``/README``: $ git push meta-intel-contrib
9462 your_name/README
9463
94645. *Determine Who to Notify:* Determine the maintainer or the mailing
9465 list that you need to notify for the change.
9466
9467 Before submitting any change, you need to be sure who the maintainer
9468 is or what mailing list that you need to notify. Use either these
9469 methods to find out:
9470
9471 - *Maintenance File:* Examine the ``maintainers.inc`` file, which is
9472 located in the `Source
9473 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ at
9474 ``meta/conf/distro/include``, to see who is responsible for code.
9475
9476 - *Search by File:* Using `Git <&YOCTO_DOCS_OM_URL;#git>`__, you can
9477 enter the following command to bring up a short list of all
9478 commits against a specific file: git shortlog -- filename Just
9479 provide the name of the file for which you are interested. The
9480 information returned is not ordered by history but does include a
9481 list of everyone who has committed grouped by name. From the list,
9482 you can see who is responsible for the bulk of the changes against
9483 the file.
9484
9485 - *Examine the List of Mailing Lists:* For a list of the Yocto
9486 Project and related mailing lists, see the "`Mailing
9487 lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__" section in
9488 the Yocto Project Reference Manual.
9489
94906. *Make a Pull Request:* Notify the maintainer or the mailing list that
9491 you have pushed a change by making a pull request.
9492
9493 The Yocto Project provides two scripts that conveniently let you
9494 generate and send pull requests to the Yocto Project. These scripts
9495 are ``create-pull-request`` and ``send-pull-request``. You can find
9496 these scripts in the ``scripts`` directory within the `Source
9497 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
9498 ``~/poky/scripts``).
9499
9500 Using these scripts correctly formats the requests without
9501 introducing any whitespace or HTML formatting. The maintainer that
9502 receives your patches either directly or through the mailing list
9503 needs to be able to save and apply them directly from your emails.
9504 Using these scripts is the preferred method for sending patches.
9505
9506 First, create the pull request. For example, the following command
9507 runs the script, specifies the upstream repository in the contrib
9508 directory into which you pushed the change, and provides a subject
9509 line in the created patch files: $ ~/poky/scripts/create-pull-request
9510 -u meta-intel-contrib -s "Updated Manual Section Reference in README"
9511 Running this script forms ``*.patch`` files in a folder named
9512 ``pull-``\ PID in the current directory. One of the patch files is a
9513 cover letter.
9514
9515 Before running the ``send-pull-request`` script, you must edit the
9516 cover letter patch to insert information about your change. After
9517 editing the cover letter, send the pull request. For example, the
9518 following command runs the script and specifies the patch directory
9519 and email address. In this example, the email address is a mailing
9520 list: $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565
9521 -t meta-intel@yoctoproject.org You need to follow the prompts as the
9522 script is interactive.
9523
9524 .. note::
9525
9526 For help on using these scripts, simply provide the
9527 -h
9528 argument as follows:
9529 ::
9530
9531 $ poky/scripts/create-pull-request -h
9532 $ poky/scripts/send-pull-request -h
9533
9534
9535.. _submitting-a-patch:
9536
9537Using Email to Submit a Patch
9538~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9539
9540You can submit patches without using the ``create-pull-request`` and
9541``send-pull-request`` scripts described in the previous section.
9542However, keep in mind, the preferred method is to use the scripts.
9543
9544Depending on the components changed, you need to submit the email to a
9545specific mailing list. For some guidance on which mailing list to use,
9546see the `list <#figuring-out-the-mailing-list-to-use>`__ at the
9547beginning of this section. For a description of all the available
9548mailing lists, see the "`Mailing
9549Lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__" section in the
9550Yocto Project Reference Manual.
9551
9552Here is the general procedure on how to submit a patch through email
9553without using the scripts:
9554
95551. *Make Your Changes Locally:* Make your changes in your local Git
9556 repository. You should make small, controlled, isolated changes.
9557 Keeping changes small and isolated aids review, makes
9558 merging/rebasing easier and keeps the change history clean should
9559 anyone need to refer to it in future.
9560
95612. *Stage Your Changes:* Stage your changes by using the ``git add``
9562 command on each file you changed.
9563
95643. *Commit Your Changes:* Commit the change by using the
9565 ``git commit --signoff`` command. Using the ``--signoff`` option
9566 identifies you as the person making the change and also satisfies the
9567 Developer's Certificate of Origin (DCO) shown earlier.
9568
9569 When you form a commit, you must follow certain standards established
9570 by the Yocto Project development team. See `Step
9571 3 <#making-sure-you-have-correct-commit-information>`__ in the
9572 previous section for information on how to provide commit information
9573 that meets Yocto Project commit message standards.
9574
95754. *Format the Commit:* Format the commit into an email message. To
9576 format commits, use the ``git format-patch`` command. When you
9577 provide the command, you must include a revision list or a number of
9578 patches as part of the command. For example, either of these two
9579 commands takes your most recent single commit and formats it as an
9580 email message in the current directory: $ git format-patch -1 or $
9581 git format-patch HEAD~
9582
9583 After the command is run, the current directory contains a numbered
9584 ``.patch`` file for the commit.
9585
9586 If you provide several commits as part of the command, the
9587 ``git format-patch`` command produces a series of numbered files in
9588 the current directory – one for each commit. If you have more than
9589 one patch, you should also use the ``--cover`` option with the
9590 command, which generates a cover letter as the first "patch" in the
9591 series. You can then edit the cover letter to provide a description
9592 for the series of patches. For information on the
9593 ``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed
9594 using the ``man git-format-patch`` command.
9595
9596 .. note::
9597
9598 If you are or will be a frequent contributor to the Yocto Project
9599 or to OpenEmbedded, you might consider requesting a contrib area
9600 and the necessary associated rights.
9601
96025. *Import the Files Into Your Mail Client:* Import the files into your
9603 mail client by using the ``git send-email`` command.
9604
9605 .. note::
9606
9607 In order to use
9608 git send-email
9609 , you must have the proper Git packages installed on your host.
9610 For Ubuntu, Debian, and Fedora the package is
9611 git-email
9612 .
9613
9614 The ``git send-email`` command sends email by using a local or remote
9615 Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or
9616 through a direct ``smtp`` configuration in your Git ``~/.gitconfig``
9617 file. If you are submitting patches through email only, it is very
9618 important that you submit them without any whitespace or HTML
9619 formatting that either you or your mailer introduces. The maintainer
9620 that receives your patches needs to be able to save and apply them
9621 directly from your emails. A good way to verify that what you are
9622 sending will be applicable by the maintainer is to do a dry run and
9623 send them to yourself and then save and apply them as the maintainer
9624 would.
9625
9626 The ``git send-email`` command is the preferred method for sending
9627 your patches using email since there is no risk of compromising
9628 whitespace in the body of the message, which can occur when you use
9629 your own mail client. The command also has several options that let
9630 you specify recipients and perform further editing of the email
9631 message. For information on how to use the ``git send-email``
9632 command, see ``GIT-SEND-EMAIL(1)`` displayed using the
9633 ``man git-send-email`` command.
9634
9635Working With Licenses
9636=====================
9637
9638As mentioned in the "`Licensing <&YOCTO_DOCS_OM_URL;#licensing>`__"
9639section in the Yocto Project Overview and Concepts Manual, open source
9640projects are open to the public and they consequently have different
9641licensing structures in place. This section describes the mechanism by
9642which the `OpenEmbedded build
9643system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ tracks changes to
9644licensing text and covers how to maintain open source license compliance
9645during your project's lifecycle. The section also describes how to
9646enable commercially licensed recipes, which by default are disabled.
9647
9648.. _usingpoky-configuring-LIC_FILES_CHKSUM:
9649
9650Tracking License Changes
9651------------------------
9652
9653The license of an upstream project might change in the future. In order
9654to prevent these changes going unnoticed, the
9655```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
9656variable tracks changes to the license text. The checksums are validated
9657at the end of the configure step, and if the checksums do not match, the
9658build will fail.
9659
9660.. _usingpoky-specifying-LIC_FILES_CHKSUM:
9661
9662Specifying the ``LIC_FILES_CHKSUM`` Variable
9663~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9664
9665The ``LIC_FILES_CHKSUM`` variable contains checksums of the license text
9666in the source code for the recipe. Following is an example of how to
9667specify ``LIC_FILES_CHKSUM``: LIC_FILES_CHKSUM =
9668"file://COPYING;md5=xxxx \\
9669file://licfile1.txt;beginline=5;endline=29;md5=yyyy \\
9670file://licfile2.txt;endline=50;md5=zzzz \\ ..."
9671
9672.. note::
9673
9674 - When using "beginline" and "endline", realize that line numbering
9675 begins with one and not zero. Also, the included lines are
9676 inclusive (i.e. lines five through and including 29 in the
9677 previous example for ``licfile1.txt``).
9678
9679 - When a license check fails, the selected license text is included
9680 as part of the QA message. Using this output, you can determine
9681 the exact start and finish for the needed license text.
9682
9683The build system uses the ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__
9684variable as the default directory when searching files listed in
9685``LIC_FILES_CHKSUM``. The previous example employs the default
9686directory.
9687
9688Consider this next example: LIC_FILES_CHKSUM =
9689"file://src/ls.c;beginline=5;endline=16;\\
9690md5=bb14ed3c4cda583abc85401304b5cd4e" LIC_FILES_CHKSUM =
9691"file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
9692
9693The first line locates a file in ``${S}/src/ls.c`` and isolates lines
9694five through 16 as license text. The second line refers to a file in
9695```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__.
9696
9697Note that ``LIC_FILES_CHKSUM`` variable is mandatory for all recipes,
9698unless the ``LICENSE`` variable is set to "CLOSED".
9699
9700.. _usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax:
9701
9702Explanation of Syntax
9703~~~~~~~~~~~~~~~~~~~~~
9704
9705As mentioned in the previous section, the ``LIC_FILES_CHKSUM`` variable
9706lists all the important files that contain the license text for the
9707source code. It is possible to specify a checksum for an entire file, or
9708a specific section of a file (specified by beginning and ending line
9709numbers with the "beginline" and "endline" parameters, respectively).
9710The latter is useful for source files with a license notice header,
9711README documents, and so forth. If you do not use the "beginline"
9712parameter, then it is assumed that the text begins on the first line of
9713the file. Similarly, if you do not use the "endline" parameter, it is
9714assumed that the license text ends with the last line of the file.
9715
9716The "md5" parameter stores the md5 checksum of the license text. If the
9717license text changes in any way as compared to this parameter then a
9718mismatch occurs. This mismatch triggers a build failure and notifies the
9719developer. Notification allows the developer to review and address the
9720license text changes. Also note that if a mismatch occurs during the
9721build, the correct md5 checksum is placed in the build log and can be
9722easily copied to the recipe.
9723
9724There is no limit to how many files you can specify using the
9725``LIC_FILES_CHKSUM`` variable. Generally, however, every project
9726requires a few specifications for license tracking. Many projects have a
9727"COPYING" file that stores the license information for all the source
9728code files. This practice allows you to just track the "COPYING" file as
9729long as it is kept up to date.
9730
9731.. note::
9732
9733 - If you specify an empty or invalid "md5" parameter,
9734 `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ returns an md5
9735 mis-match error and displays the correct "md5" parameter value
9736 during the build. The correct parameter is also captured in the
9737 build log.
9738
9739 - If the whole file contains only license text, you do not need to
9740 use the "beginline" and "endline" parameters.
9741
9742Enabling Commercially Licensed Recipes
9743--------------------------------------
9744
9745By default, the OpenEmbedded build system disables components that have
9746commercial or other special licensing requirements. Such requirements
9747are defined on a recipe-by-recipe basis through the
9748```LICENSE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS>`__ variable
9749definition in the affected recipe. For instance, the
9750``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
9751contains the following statement: LICENSE_FLAGS = "commercial" Here is a
9752slightly more complicated example that contains both an explicit recipe
9753name and version (after variable expansion): LICENSE_FLAGS =
9754"license_${PN}_${PV}" In order for a component restricted by a
9755``LICENSE_FLAGS`` definition to be enabled and included in an image, it
9756needs to have a matching entry in the global
9757```LICENSE_FLAGS_WHITELIST`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST>`__
9758variable, which is a variable typically defined in your ``local.conf``
9759file. For example, to enable the
9760``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
9761could add either the string "commercial_gst-plugins-ugly" or the more
9762general string "commercial" to ``LICENSE_FLAGS_WHITELIST``. See the
9763"`License Flag Matching <#license-flag-matching>`__" section for a full
9764explanation of how ``LICENSE_FLAGS`` matching works. Here is the
9765example: LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
9766Likewise, to additionally enable the package built from the recipe
9767containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
9768the actual recipe name was ``emgd_1.10.bb``, the following string would
9769enable that package as well as the original ``gst-plugins-ugly``
9770package: LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly
9771license_emgd_1.10" As a convenience, you do not need to specify the
9772complete license string in the whitelist for every package. You can use
9773an abbreviated form, which consists of just the first portion or
9774portions of the license string before the initial underscore character
9775or characters. A partial string will match any license that contains the
9776given string as the first portion of its license. For example, the
9777following whitelist string will also match both of the packages
9778previously mentioned as well as any other packages that have licenses
9779starting with "commercial" or "license". LICENSE_FLAGS_WHITELIST =
9780"commercial license"
9781
9782License Flag Matching
9783~~~~~~~~~~~~~~~~~~~~~
9784
9785License flag matching allows you to control what recipes the
9786OpenEmbedded build system includes in the build. Fundamentally, the
9787build system attempts to match ``LICENSE_FLAGS`` strings found in
9788recipes against ``LICENSE_FLAGS_WHITELIST`` strings found in the
9789whitelist. A match causes the build system to include a recipe in the
9790build, while failure to find a match causes the build system to exclude
9791a recipe.
9792
9793In general, license flag matching is simple. However, understanding some
9794concepts will help you correctly and effectively use matching.
9795
9796Before a flag defined by a particular recipe is tested against the
9797contents of the whitelist, the expanded string ``_${PN}`` is appended to
9798the flag. This expansion makes each ``LICENSE_FLAGS`` value
9799recipe-specific. After expansion, the string is then matched against the
9800whitelist. Thus, specifying ``LICENSE_FLAGS = "commercial"`` in recipe
9801"foo", for example, results in the string ``"commercial_foo"``. And, to
9802create a match, that string must appear in the whitelist.
9803
9804Judicious use of the ``LICENSE_FLAGS`` strings and the contents of the
9805``LICENSE_FLAGS_WHITELIST`` variable allows you a lot of flexibility for
9806including or excluding recipes based on licensing. For example, you can
9807broaden the matching capabilities by using license flags string subsets
9808in the whitelist.
9809
9810.. note::
9811
9812 When using a string subset, be sure to use the part of the expanded
9813 string that precedes the appended underscore character (e.g.
9814 usethispart_1.3
9815 ,
9816 usethispart_1.4
9817 , and so forth).
9818
9819For example, simply specifying the string "commercial" in the whitelist
9820matches any expanded ``LICENSE_FLAGS`` definition that starts with the
9821string "commercial" such as "commercial_foo" and "commercial_bar", which
9822are the strings the build system automatically generates for
9823hypothetical recipes named "foo" and "bar" assuming those recipes simply
9824specify the following: LICENSE_FLAGS = "commercial" Thus, you can choose
9825to exhaustively enumerate each license flag in the whitelist and allow
9826only specific recipes into the image, or you can use a string subset
9827that causes a broader range of matches to allow a range of recipes into
9828the image.
9829
9830This scheme works even if the ``LICENSE_FLAGS`` string already has
9831``_${PN}`` appended. For example, the build system turns the license
9832flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
9833both the general "commercial" and the specific "commercial_1.2_foo"
9834strings found in the whitelist, as expected.
9835
9836Here are some other scenarios:
9837
9838- You can specify a versioned string in the recipe such as
9839 "commercial_foo_1.2" in a "foo" recipe. The build system expands this
9840 string to "commercial_foo_1.2_foo". Combine this license flag with a
9841 whitelist that has the string "commercial" and you match the flag
9842 along with any other flag that starts with the string "commercial".
9843
9844- Under the same circumstances, you can use "commercial_foo" in the
9845 whitelist and the build system not only matches "commercial_foo_1.2"
9846 but also matches any license flag with the string "commercial_foo",
9847 regardless of the version.
9848
9849- You can be very specific and use both the package and version parts
9850 in the whitelist (e.g. "commercial_foo_1.2") to specifically match a
9851 versioned recipe.
9852
9853Other Variables Related to Commercial Licenses
9854~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9855
9856Other helpful variables related to commercial license handling exist and
9857are defined in the
9858``poky/meta/conf/distro/include/default-distrovars.inc`` file:
9859COMMERCIAL_AUDIO_PLUGINS ?= "" COMMERCIAL_VIDEO_PLUGINS ?= "" If you
9860want to enable these components, you can do so by making sure you have
9861statements similar to the following in your ``local.conf`` configuration
9862file: COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \\
9863gst-plugins-ugly-mpegaudioparse" COMMERCIAL_VIDEO_PLUGINS =
9864"gst-plugins-ugly-mpeg2dec \\ gst-plugins-ugly-mpegstream
9865gst-plugins-bad-mpegvideoparse" LICENSE_FLAGS_WHITELIST =
9866"commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
9867Of course, you could also create a matching whitelist for those
9868components using the more general "commercial" in the whitelist, but
9869that would also enable all the other packages with ``LICENSE_FLAGS``
9870containing "commercial", which you may or may not want:
9871LICENSE_FLAGS_WHITELIST = "commercial"
9872
9873Specifying audio and video plugins as part of the
9874``COMMERCIAL_AUDIO_PLUGINS`` and ``COMMERCIAL_VIDEO_PLUGINS`` statements
9875(along with the enabling ``LICENSE_FLAGS_WHITELIST``) includes the
9876plugins or components into built images, thus adding support for media
9877formats or components.
9878
9879Maintaining Open Source License Compliance During Your Product's Lifecycle
9880--------------------------------------------------------------------------
9881
9882One of the concerns for a development organization using open source
9883software is how to maintain compliance with various open source
9884licensing during the lifecycle of the product. While this section does
9885not provide legal advice or comprehensively cover all scenarios, it does
9886present methods that you can use to assist you in meeting the compliance
9887requirements during a software release.
9888
9889With hundreds of different open source licenses that the Yocto Project
9890tracks, it is difficult to know the requirements of each and every
9891license. However, the requirements of the major FLOSS licenses can begin
9892to be covered by assuming that three main areas of concern exist:
9893
9894- Source code must be provided.
9895
9896- License text for the software must be provided.
9897
9898- Compilation scripts and modifications to the source code must be
9899 provided.
9900
9901There are other requirements beyond the scope of these three and the
9902methods described in this section (e.g. the mechanism through which
9903source code is distributed).
9904
9905As different organizations have different methods of complying with open
9906source licensing, this section is not meant to imply that there is only
9907one single way to meet your compliance obligations, but rather to
9908describe one method of achieving compliance. The remainder of this
9909section describes methods supported to meet the previously mentioned
9910three requirements. Once you take steps to meet these requirements, and
9911prior to releasing images, sources, and the build system, you should
9912audit all artifacts to ensure completeness.
9913
9914.. note::
9915
9916 The Yocto Project generates a license manifest during image creation
9917 that is located in
9918 ${DEPLOY_DIR}/licenses/
9919 image_name-datestamp
9920 to assist with any audits.
9921
9922Providing the Source Code
9923~~~~~~~~~~~~~~~~~~~~~~~~~
9924
9925Compliance activities should begin before you generate the final image.
9926The first thing you should look at is the requirement that tops the list
9927for most compliance groups - providing the source. The Yocto Project has
9928a few ways of meeting this requirement.
9929
9930One of the easiest ways to meet this requirement is to provide the
9931entire ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ used by the
9932build. This method, however, has a few issues. The most obvious is the
9933size of the directory since it includes all sources used in the build
9934and not just the source used in the released image. It will include
9935toolchain source, and other artifacts, which you would not generally
9936release. However, the more serious issue for most companies is
9937accidental release of proprietary software. The Yocto Project provides
9938an ```archiver`` <&YOCTO_DOCS_REF_URL;#ref-classes-archiver>`__ class to
9939help avoid some of these concerns.
9940
9941Before you employ ``DL_DIR`` or the ``archiver`` class, you need to
9942decide how you choose to provide source. The source ``archiver`` class
9943can generate tarballs and SRPMs and can create them with various levels
9944of compliance in mind.
9945
9946One way of doing this (but certainly not the only way) is to release
9947just the source as a tarball. You can do this by adding the following to
9948the ``local.conf`` file found in the `Build
9949Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
9950"archiver" ARCHIVER_MODE[src] = "original" During the creation of your
9951image, the source from all recipes that deploy packages to the image is
9952placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
9953```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ for each recipe.
9954Releasing the entire directory enables you to comply with requirements
9955concerning providing the unmodified source. It is important to note that
9956the size of the directory can get large.
9957
9958A way to help mitigate the size issue is to only release tarballs for
9959licenses that require the release of source. Let us assume you are only
9960concerned with GPL code as identified by running the following script: #
9961Script to archive a subset of packages matching specific license(s) #
9962Source and license files are copied into sub folders of package folder #
9963Must be run from build folder #!/bin/bash
9964src_release_dir="source-release" mkdir -p $src_release_dir for a in
9965tmp/deploy/sources/*; do for d in $a/*; do # Get package name from path
9966p=`basename $d\` p=${p%-*} p=${p%-*} # Only archive GPL packages (update
9967\*GPL\* regex for your license check) numfiles=`ls
9968tmp/deploy/licenses/$p/*GPL\* 2> /dev/null \| wc -l\` if [ $numfiles -gt
99691 ]; then echo Archiving $p mkdir -p $src_release_dir/$p/source cp $d/\*
9970$src_release_dir/$p/source 2> /dev/null mkdir -p
9971$src_release_dir/$p/license cp tmp/deploy/licenses/$p/\*
9972$src_release_dir/$p/license 2> /dev/null fi done done At this point, you
9973could create a tarball from the ``gpl_source_release`` directory and
9974provide that to the end user. This method would be a step toward
9975achieving compliance with section 3a of GPLv2 and with section 6 of
9976GPLv3.
9977
9978Providing License Text
9979~~~~~~~~~~~~~~~~~~~~~~
9980
9981One requirement that is often overlooked is inclusion of license text.
9982This requirement also needs to be dealt with prior to generating the
9983final image. Some licenses require the license text to accompany the
9984binary. You can achieve this by adding the following to your
9985``local.conf`` file: COPY_LIC_MANIFEST = "1" COPY_LIC_DIRS = "1"
9986LICENSE_CREATE_PACKAGE = "1" Adding these statements to the
9987configuration file ensures that the licenses collected during package
9988generation are included on your image.
9989
9990.. note::
9991
9992 Setting all three variables to "1" results in the image having two
9993 copies of the same license file. One copy resides in
9994 ``/usr/share/common-licenses`` and the other resides in
9995 ``/usr/share/license``.
9996
9997 The reason for this behavior is because
9998 ```COPY_LIC_DIRS`` <&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS>`__ and
9999 ```COPY_LIC_MANIFEST`` <&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST>`__
10000 add a copy of the license when the image is built but do not offer a
10001 path for adding licenses for newly installed packages to an image.
10002 ```LICENSE_CREATE_PACKAGE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE>`__
10003 adds a separate package and an upgrade path for adding licenses to an
10004 image.
10005
10006As the source ``archiver`` class has already archived the original
10007unmodified source that contains the license files, you would have
10008already met the requirements for inclusion of the license information
10009with source as defined by the GPL and other open source licenses.
10010
10011Providing Compilation Scripts and Source Code Modifications
10012~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10013
10014At this point, we have addressed all we need to prior to generating the
10015image. The next two requirements are addressed during the final
10016packaging of the release.
10017
10018By releasing the version of the OpenEmbedded build system and the layers
10019used during the build, you will be providing both compilation scripts
10020and the source code modifications in one step.
10021
10022If the deployment team has a `BSP
10023layer <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__ and a distro layer, and those
10024those layers are used to patch, compile, package, or modify (in any way)
10025any open source software included in your released images, you might be
10026required to release those layers under section 3 of GPLv2 or section 1
10027of GPLv3. One way of doing that is with a clean checkout of the version
10028of the Yocto Project and layers used during your build. Here is an
10029example: # We built using the DISTRO_NAME_NO_CAP branch of the poky repo
10030$ git clone -b DISTRO_NAME_NO_CAP git://git.yoctoproject.org/poky $ cd
10031poky # We built using the release_branch for our layers $ git clone -b
10032release_branch git://git.mycompany.com/meta-my-bsp-layer $ git clone -b
10033release_branch git://git.mycompany.com/meta-my-software-layer # clean up
10034the .git repos $ find . -name ".git" -type d -exec rm -rf {} \\; One
10035thing a development organization might want to consider for end-user
10036convenience is to modify ``meta-poky/conf/bblayers.conf.sample`` to
10037ensure that when the end user utilizes the released build system to
10038build an image, the development organization's layers are included in
10039the ``bblayers.conf`` file automatically: # POKY_BBLAYERS_CONF_VERSION
10040is increased each time build/conf/bblayers.conf # changes incompatibly
10041POKY_BBLAYERS_CONF_VERSION = "2" BBPATH = "${TOPDIR}" BBFILES ?= ""
10042BBLAYERS ?= " \\ ##OEROOT##/meta \\ ##OEROOT##/meta-poky \\
10043##OEROOT##/meta-yocto-bsp \\ ##OEROOT##/meta-mylayer \\ " Creating and
10044providing an archive of the `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__
10045layers (recipes, configuration files, and so forth) enables you to meet
10046your requirements to include the scripts to control compilation as well
10047as any modifications to the original source.
10048
10049Copying Licenses that Do Not Exist
10050----------------------------------
10051
10052Some packages, such as the linux-firmware package, have many licenses
10053that are not in any way common. You can avoid adding a lot of these
10054types of common license files, which are only applicable to a specific
10055package, by using the
10056```NO_GENERIC_LICENSE`` <&YOCTO_DOCS_REF_URL;#var-NO_GENERIC_LICENSE>`__
10057variable. Using this variable also avoids QA errors when you use a
10058non-common, non-CLOSED license in a recipe.
10059
10060The following is an example that uses the ``LICENSE.Abilis.txt`` file as
10061the license from the fetched source: NO_GENERIC_LICENSE[Firmware-Abilis]
10062= "LICENSE.Abilis.txt"
10063
10064Using the Error Reporting Tool
10065==============================
10066
10067The error reporting tool allows you to submit errors encountered during
10068builds to a central database. Outside of the build environment, you can
10069use a web interface to browse errors, view statistics, and query for
10070errors. The tool works using a client-server system where the client
10071portion is integrated with the installed Yocto Project `Source
10072Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
10073The server receives the information collected and saves it in a
10074database.
10075
10076A live instance of the error reporting server exists at
10077` <http://errors.yoctoproject.org>`__. This server exists so that when
10078you want to get help with build failures, you can submit all of the
10079information on the failure easily and then point to the URL in your bug
10080report or send an email to the mailing list.
10081
10082.. note::
10083
10084 If you send error reports to this server, the reports become publicly
10085 visible.
10086
10087Enabling and Using the Tool
10088---------------------------
10089
10090By default, the error reporting tool is disabled. You can enable it by
10091inheriting the
10092```report-error`` <&YOCTO_DOCS_REF_URL;#ref-classes-report-error>`__
10093class by adding the following statement to the end of your
10094``local.conf`` file in your `Build
10095Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. INHERIT +=
10096"report-error"
10097
10098By default, the error reporting feature stores information in
10099``${``\ ```LOG_DIR`` <&YOCTO_DOCS_REF_URL;#var-LOG_DIR>`__\ ``}/error-report``.
10100However, you can specify a directory to use by adding the following to
10101your ``local.conf`` file: ERR_REPORT_DIR = "path" Enabling error
10102reporting causes the build process to collect the errors and store them
10103in a file as previously described. When the build system encounters an
10104error, it includes a command as part of the console output. You can run
10105the command to send the error file to the server. For example, the
10106following command sends the errors to an upstream server: $
10107send-error-report
10108/home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
10109In the previous example, the errors are sent to a public database
10110available at ` <http://errors.yoctoproject.org>`__, which is used by the
10111entire community. If you specify a particular server, you can send the
10112errors to a different database. Use the following command for more
10113information on available options: $ send-error-report --help
10114
10115When sending the error file, you are prompted to review the data being
10116sent as well as to provide a name and optional email address. Once you
10117satisfy these prompts, the command returns a link from the server that
10118corresponds to your entry in the database. For example, here is a
10119typical link: http://errors.yoctoproject.org/Errors/Details/9522/
10120Following the link takes you to a web interface where you can browse,
10121query the errors, and view statistics.
10122
10123Disabling the Tool
10124------------------
10125
10126To disable the error reporting feature, simply remove or comment out the
10127following statement from the end of your ``local.conf`` file in your
10128`Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. INHERIT +=
10129"report-error"
10130
10131Setting Up Your Own Error Reporting Server
10132------------------------------------------
10133
10134If you want to set up your own error reporting server, you can obtain
10135the code from the Git repository at
10136` <http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/>`__.
10137Instructions on how to set it up are in the README document.
10138
10139.. _dev-using-wayland-and-weston:
10140
10141Using Wayland and Weston
10142========================
10143
10144`Wayland <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)>`__
10145is a computer display server protocol that provides a method for
10146compositing window managers to communicate directly with applications
10147and video hardware and expects them to communicate with input hardware
10148using other libraries. Using Wayland with supporting targets can result
10149in better control over graphics frame rendering than an application
10150might otherwise achieve.
10151
10152The Yocto Project provides the Wayland protocol libraries and the
10153reference
10154`Weston <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston>`__
10155compositor as part of its release. You can find the integrated packages
10156in the ``meta`` layer of the `Source
10157Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Specifically, you
10158can find the recipes that build both Wayland and Weston at
10159``meta/recipes-graphics/wayland``.
10160
10161You can build both the Wayland and Weston packages for use only with
10162targets that accept the `Mesa 3D and Direct Rendering
10163Infrastructure <https://en.wikipedia.org/wiki/Mesa_(computer_graphics)>`__,
10164which is also known as Mesa DRI. This implies that you cannot build and
10165use the packages if your target uses, for example, the Intel Embedded
10166Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI.
10167
10168.. note::
10169
10170 Due to lack of EGL support, Weston 1.0.3 will not run directly on the
10171 emulated QEMU hardware. However, this version of Weston will run
10172 under X emulation without issues.
10173
10174This section describes what you need to do to implement Wayland and use
10175the Weston compositor when building an image for a supporting target.
10176
10177Enabling Wayland in an Image
10178----------------------------
10179
10180To enable Wayland, you need to enable it to be built and enable it to be
10181included (installed) in the image.
10182
10183.. _enable-building:
10184
10185Building
10186~~~~~~~~
10187
10188To cause Mesa to build the ``wayland-egl`` platform and Weston to build
10189Wayland with Kernel Mode Setting
10190(`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__)
10191support, include the "wayland" flag in the
10192```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
10193statement in your ``local.conf`` file: DISTRO_FEATURES_append = "
10194wayland"
10195
10196.. note::
10197
10198 If X11 has been enabled elsewhere, Weston will build Wayland with X11
10199 support
10200
10201.. _enable-installation-in-an-image:
10202
10203Installing
10204~~~~~~~~~~
10205
10206To install the Wayland feature into an image, you must include the
10207following
10208```CORE_IMAGE_EXTRA_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL>`__
10209statement in your ``local.conf`` file: CORE_IMAGE_EXTRA_INSTALL +=
10210"wayland weston"
10211
10212Running Weston
10213--------------
10214
10215To run Weston inside X11, enabling it as described earlier and building
10216a Sato image is sufficient. If you are running your image under Sato, a
10217Weston Launcher appears in the "Utility" category.
10218
10219Alternatively, you can run Weston through the command-line interpretor
10220(CLI), which is better suited for development work. To run Weston under
10221the CLI, you need to do the following after your image is built:
10222
102231. Run these commands to export ``XDG_RUNTIME_DIR``: mkdir -p
10224 /tmp/$USER-weston chmod 0700 /tmp/$USER-weston export
10225 XDG_RUNTIME_DIR=/tmp/$USER-weston
10226
102272. Launch Weston in the shell: weston