summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual')
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml16081
-rw-r--r--documentation/dev-manual/dev-manual-customization.xsl29
-rw-r--r--documentation/dev-manual/dev-manual-intro.xml104
-rw-r--r--documentation/dev-manual/dev-manual-qemu.xml691
-rw-r--r--documentation/dev-manual/dev-manual-start.xml1288
-rwxr-xr-xdocumentation/dev-manual/dev-manual.xml195
-rw-r--r--documentation/dev-manual/dev-style.css991
7 files changed, 0 insertions, 19379 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
deleted file mode 100644
index 247f6abfd4..0000000000
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ /dev/null
@@ -1,16081 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='extendpoky'>
7
8<title>Common Tasks</title>
9 <para>
10 This chapter describes fundamental procedures such as creating layers,
11 adding new software packages, extending or customizing images,
12 porting work to new hardware (adding a new machine), and so forth.
13 You will find that the procedures documented here occur often in the
14 development cycle using the Yocto Project.
15 </para>
16
17 <section id="understanding-and-creating-layers">
18 <title>Understanding and Creating Layers</title>
19
20 <para>
21 The OpenEmbedded build system supports organizing
22 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink> into
23 multiple layers.
24 Layers allow you to isolate different types of customizations from
25 each other.
26 For introductory information on the Yocto Project Layer Model,
27 see the
28 "<ulink url='&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model'>The Yocto Project Layer Model</ulink>"
29 section in the Yocto Project Overview and Concepts Manual.
30 </para>
31
32 <section id='creating-your-own-layer'>
33 <title>Creating Your Own Layer</title>
34
35 <para>
36 It is very easy to create your own layers to use with the
37 OpenEmbedded build system.
38 The Yocto Project ships with tools that speed up creating
39 layers.
40 This section describes the steps you perform by hand to create
41 layers so that you can better understand them.
42 For information about the layer-creation tools, see the
43 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'>Creating a New BSP Layer Using the <filename>bitbake-layers</filename> Script</ulink>"
44 section in the Yocto Project Board Support Package (BSP)
45 Developer's Guide and the
46 "<link linkend='creating-a-general-layer-using-the-bitbake-layers-script'>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</link>"
47 section further down in this manual.
48 </para>
49
50 <para>
51 Follow these general steps to create your layer without using
52 tools:
53 <orderedlist>
54 <listitem><para>
55 <emphasis>Check Existing Layers:</emphasis>
56 Before creating a new layer, you should be sure someone
57 has not already created a layer containing the Metadata
58 you need.
59 You can see the
60 <ulink url='http://layers.openembedded.org/layerindex/layers/'>OpenEmbedded Metadata Index</ulink>
61 for a list of layers from the OpenEmbedded community
62 that can be used in the Yocto Project.
63 You could find a layer that is identical or close to
64 what you need.
65 </para></listitem>
66 <listitem><para>
67 <emphasis>Create a Directory:</emphasis>
68 Create the directory for your layer.
69 When you create the layer, be sure to create the
70 directory in an area not associated with the
71 Yocto Project
72 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
73 (e.g. the cloned <filename>poky</filename> repository).
74 </para>
75
76 <para>While not strictly required, prepend the name of
77 the directory with the string "meta-".
78 For example:
79 <literallayout class='monospaced'>
80 meta-mylayer
81 meta-GUI_xyz
82 meta-mymachine
83 </literallayout>
84 With rare exceptions, a layer's name follows this
85 form:
86 <literallayout class='monospaced'>
87 meta-<replaceable>root_name</replaceable>
88 </literallayout>
89 Following this layer naming convention can
90 save you trouble later when tools, components, or
91 variables "assume" your layer name begins with "meta-".
92 A notable example is in configuration files as
93 shown in the following step where layer names without
94 the "meta-" string are appended
95 to several variables used in the configuration.
96 </para></listitem>
97 <listitem><para id='dev-layer-config-file-description'>
98 <emphasis>Create a Layer Configuration File:</emphasis>
99 Inside your new layer folder, you need to create a
100 <filename>conf/layer.conf</filename> file.
101 It is easiest to take an existing layer configuration
102 file and copy that to your layer's
103 <filename>conf</filename> directory and then modify the
104 file as needed.</para>
105
106 <para>The
107 <filename>meta-yocto-bsp/conf/layer.conf</filename> file
108 in the Yocto Project
109 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf'>Source Repositories</ulink>
110 demonstrates the required syntax.
111 For your layer, you need to replace "yoctobsp" with
112 a unique identifier for your layer (e.g. "machinexyz"
113 for a layer named "meta-machinexyz"):
114 <literallayout class='monospaced'>
115 # We have a conf and classes directory, add to BBPATH
116 BBPATH .= ":${LAYERDIR}"
117
118 # We have recipes-* directories, add to BBFILES
119 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
120 ${LAYERDIR}/recipes-*/*/*.bbappend"
121
122 BBFILE_COLLECTIONS += "yoctobsp"
123 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
124 BBFILE_PRIORITY_yoctobsp = "5"
125 LAYERVERSION_yoctobsp = "4"
126 LAYERSERIES_COMPAT_yoctobsp = "&DISTRO_NAME_NO_CAP;"
127 </literallayout>
128 Following is an explanation of the layer configuration
129 file:
130 <itemizedlist>
131 <listitem><para>
132 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>:
133 Adds the layer's root directory to BitBake's
134 search path.
135 Through the use of the
136 <filename>BBPATH</filename> variable, BitBake
137 locates class files
138 (<filename>.bbclass</filename>),
139 configuration files, and files that are
140 included with <filename>include</filename> and
141 <filename>require</filename> statements.
142 For these cases, BitBake uses the first file
143 that matches the name found in
144 <filename>BBPATH</filename>.
145 This is similar to the way the
146 <filename>PATH</filename> variable is used for
147 binaries.
148 It is recommended, therefore, that you use
149 unique class and configuration filenames in
150 your custom layer.
151 </para></listitem>
152 <listitem><para>
153 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink>:
154 Defines the location for all recipes in the
155 layer.
156 </para></listitem>
157 <listitem><para>
158 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'><filename>BBFILE_COLLECTIONS</filename></ulink>:
159 Establishes the current layer through a
160 unique identifier that is used throughout the
161 OpenEmbedded build system to refer to the layer.
162 In this example, the identifier "yoctobsp" is
163 the representation for the container layer
164 named "meta-yocto-bsp".
165 </para></listitem>
166 <listitem><para>
167 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'><filename>BBFILE_PATTERN</filename></ulink>:
168 Expands immediately during parsing to
169 provide the directory of the layer.
170 </para></listitem>
171 <listitem><para>
172 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>:
173 Establishes a priority to use for
174 recipes in the layer when the OpenEmbedded build
175 finds recipes of the same name in different
176 layers.
177 </para></listitem>
178 <listitem><para>
179 <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'><filename>LAYERVERSION</filename></ulink>:
180 Establishes a version number for the layer.
181 You can use this version number to specify this
182 exact version of the layer as a dependency when
183 using the
184 <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></ulink>
185 variable.
186 </para></listitem>
187 <listitem><para>
188 <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></ulink>:
189 Lists all layers on which this layer depends (if any).
190 </para></listitem>
191 <listitem><para>
192 <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERSERIES_COMPAT'><filename>LAYERSERIES_COMPAT</filename></ulink>:
193 Lists the
194 <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Yocto Project</ulink>
195 releases for which the current version is
196 compatible.
197 This variable is a good way to indicate if
198 your particular layer is current.
199 </para></listitem>
200 </itemizedlist>
201 </para></listitem>
202 <listitem><para>
203 <emphasis>Add Content:</emphasis>
204 Depending on the type of layer, add the content.
205 If the layer adds support for a machine, add the machine
206 configuration in a <filename>conf/machine/</filename>
207 file within the layer.
208 If the layer adds distro policy, add the distro
209 configuration in a <filename>conf/distro/</filename>
210 file within the layer.
211 If the layer introduces new recipes, put the recipes
212 you need in <filename>recipes-*</filename>
213 subdirectories within the layer.
214 <note>
215 For an explanation of layer hierarchy that
216 is compliant with the Yocto Project, see
217 the
218 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout'>Example Filesystem Layout</ulink>"
219 section in the Yocto Project Board
220 Support Package (BSP) Developer's Guide.
221 </note>
222 </para></listitem>
223 <listitem><para>
224 <emphasis>Optionally Test for Compatibility:</emphasis>
225 If you want permission to use the Yocto Project
226 Compatibility logo with your layer or application that
227 uses your layer, perform the steps to apply for
228 compatibility.
229 See the
230 "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>"
231 section for more information.
232 </para></listitem>
233 </orderedlist>
234 </para>
235 </section>
236
237 <section id='best-practices-to-follow-when-creating-layers'>
238 <title>Following Best Practices When Creating Layers</title>
239
240 <para>
241 To create layers that are easier to maintain and that will
242 not impact builds for other machines, you should consider the
243 information in the following list:
244 <itemizedlist>
245 <listitem><para>
246 <emphasis>Avoid "Overlaying" Entire Recipes from Other Layers in Your Configuration:</emphasis>
247 In other words, do not copy an entire recipe into your
248 layer and then modify it.
249 Rather, use an append file
250 (<filename>.bbappend</filename>) to override only those
251 parts of the original recipe you need to modify.
252 </para></listitem>
253 <listitem><para>
254 <emphasis>Avoid Duplicating Include Files:</emphasis>
255 Use append files (<filename>.bbappend</filename>)
256 for each recipe that uses an include file.
257 Or, if you are introducing a new recipe that requires
258 the included file, use the path relative to the
259 original layer directory to refer to the file.
260 For example, use
261 <filename>require recipes-core/</filename><replaceable>package</replaceable><filename>/</filename><replaceable>file</replaceable><filename>.inc</filename>
262 instead of
263 <filename>require </filename><replaceable>file</replaceable><filename>.inc</filename>.
264 If you're finding you have to overlay the include file,
265 it could indicate a deficiency in the include file in
266 the layer to which it originally belongs.
267 If this is the case, you should try to address that
268 deficiency instead of overlaying the include file.
269 For example, you could address this by getting the
270 maintainer of the include file to add a variable or
271 variables to make it easy to override the parts needing
272 to be overridden.
273 </para></listitem>
274 <listitem><para>
275 <emphasis>Structure Your Layers:</emphasis>
276 Proper use of overrides within append files and
277 placement of machine-specific files within your layer
278 can ensure that a build is not using the wrong Metadata
279 and negatively impacting a build for a different
280 machine.
281 Following are some examples:
282 <itemizedlist>
283 <listitem><para>
284 <emphasis>Modify Variables to Support a
285 Different Machine:</emphasis>
286 Suppose you have a layer named
287 <filename>meta-one</filename> that adds support
288 for building machine "one".
289 To do so, you use an append file named
290 <filename>base-files.bbappend</filename> and
291 create a dependency on "foo" by altering the
292 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
293 variable:
294 <literallayout class='monospaced'>
295 DEPENDS = "foo"
296 </literallayout>
297 The dependency is created during any build that
298 includes the layer
299 <filename>meta-one</filename>.
300 However, you might not want this dependency
301 for all machines.
302 For example, suppose you are building for
303 machine "two" but your
304 <filename>bblayers.conf</filename> file has the
305 <filename>meta-one</filename> layer included.
306 During the build, the
307 <filename>base-files</filename> for machine
308 "two" will also have the dependency on
309 <filename>foo</filename>.</para>
310 <para>To make sure your changes apply only when
311 building machine "one", use a machine override
312 with the <filename>DEPENDS</filename> statement:
313 <literallayout class='monospaced'>
314 DEPENDS_one = "foo"
315 </literallayout>
316 You should follow the same strategy when using
317 <filename>_append</filename> and
318 <filename>_prepend</filename> operations:
319 <literallayout class='monospaced'>
320 DEPENDS_append_one = " foo"
321 DEPENDS_prepend_one = "foo "
322 </literallayout>
323 As an actual example, here's a snippet from the
324 generic kernel include file
325 <filename>linux-yocto.inc</filename>,
326 wherein the kernel compile and link options are
327 adjusted in the case of a subset of the supported
328 architectures:
329 <literallayout class='monospaced'>
330 DEPENDS_append_aarch64 = " libgcc"
331 KERNEL_CC_append_aarch64 = " ${TOOLCHAIN_OPTIONS}"
332 KERNEL_LD_append_aarch64 = " ${TOOLCHAIN_OPTIONS}"
333
334 DEPENDS_append_nios2 = " libgcc"
335 KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
336 KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
337
338 DEPENDS_append_arc = " libgcc"
339 KERNEL_CC_append_arc = " ${TOOLCHAIN_OPTIONS}"
340 KERNEL_LD_append_arc = " ${TOOLCHAIN_OPTIONS}"
341
342 KERNEL_FEATURES_append_qemuall=" features/debug/printk.scc"
343 </literallayout>
344 <note>
345 Avoiding "+=" and "=+" and using
346 machine-specific
347 <filename>_append</filename>
348 and <filename>_prepend</filename> operations
349 is recommended as well.
350 </note>
351 </para></listitem>
352 <listitem><para>
353 <emphasis>Place Machine-Specific Files in
354 Machine-Specific Locations:</emphasis>
355 When you have a base recipe, such as
356 <filename>base-files.bb</filename>, that
357 contains a
358 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
359 statement to a file, you can use an append file
360 to cause the build to use your own version of
361 the file.
362 For example, an append file in your layer at
363 <filename>meta-one/recipes-core/base-files/base-files.bbappend</filename>
364 could extend
365 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
366 using
367 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
368 as follows:
369 <literallayout class='monospaced'>
370 FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"
371 </literallayout>
372 The build for machine "one" will pick up your
373 machine-specific file as long as you have the
374 file in
375 <filename>meta-one/recipes-core/base-files/base-files/</filename>.
376 However, if you are building for a different
377 machine and the
378 <filename>bblayers.conf</filename> file includes
379 the <filename>meta-one</filename> layer and
380 the location of your machine-specific file is
381 the first location where that file is found
382 according to <filename>FILESPATH</filename>,
383 builds for all machines will also use that
384 machine-specific file.</para>
385 <para>You can make sure that a machine-specific
386 file is used for a particular machine by putting
387 the file in a subdirectory specific to the
388 machine.
389 For example, rather than placing the file in
390 <filename>meta-one/recipes-core/base-files/base-files/</filename>
391 as shown above, put it in
392 <filename>meta-one/recipes-core/base-files/base-files/one/</filename>.
393 Not only does this make sure the file is used
394 only when building for machine "one", but the
395 build process locates the file more quickly.</para>
396 <para>In summary, you need to place all files
397 referenced from <filename>SRC_URI</filename>
398 in a machine-specific subdirectory within the
399 layer in order to restrict those files to
400 machine-specific builds.
401 </para></listitem>
402 </itemizedlist>
403 </para></listitem>
404 <listitem><para>
405 <emphasis>Perform Steps to Apply for Yocto Project Compatibility:</emphasis>
406 If you want permission to use the
407 Yocto Project Compatibility logo with your layer
408 or application that uses your layer, perform the
409 steps to apply for compatibility.
410 See the
411 "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>"
412 section for more information.
413 </para></listitem>
414 <listitem><para>
415 <emphasis>Follow the Layer Naming Convention:</emphasis>
416 Store custom layers in a Git repository that use the
417 <filename>meta-<replaceable>layer_name</replaceable></filename>
418 format.
419 </para></listitem>
420 <listitem><para>
421 <emphasis>Group Your Layers Locally:</emphasis>
422 Clone your repository alongside other cloned
423 <filename>meta</filename> directories from the
424 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
425 </para></listitem>
426 </itemizedlist>
427 </para>
428 </section>
429
430 <section id='making-sure-your-layer-is-compatible-with-yocto-project'>
431 <title>Making Sure Your Layer is Compatible With Yocto Project</title>
432
433 <para>
434 When you create a layer used with the Yocto Project, it is
435 advantageous to make sure that the layer interacts well with
436 existing Yocto Project layers (i.e. the layer is compatible
437 with the Yocto Project).
438 Ensuring compatibility makes the layer easy to be consumed
439 by others in the Yocto Project community and could allow you
440 permission to use the Yocto Project Compatible Logo.
441 <note>
442 Only Yocto Project member organizations are permitted to
443 use the Yocto Project Compatible Logo.
444 The logo is not available for general use.
445 For information on how to become a Yocto Project member
446 organization, see the
447 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>.
448 </note>
449 </para>
450
451 <para>
452 The Yocto Project Compatibility Program consists of a layer
453 application process that requests permission to use the Yocto
454 Project Compatibility Logo for your layer and application.
455 The process consists of two parts:
456 <orderedlist>
457 <listitem><para>
458 Successfully passing a script
459 (<filename>yocto-check-layer</filename>) that
460 when run against your layer, tests it against
461 constraints based on experiences of how layers have
462 worked in the real world and where pitfalls have been
463 found.
464 Getting a "PASS" result from the script is required for
465 successful compatibility registration.
466 </para></listitem>
467 <listitem><para>
468 Completion of an application acceptance form, which
469 you can find at
470 <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>.
471 </para></listitem>
472 </orderedlist>
473 </para>
474
475 <para>
476 To be granted permission to use the logo, you need to satisfy
477 the following:
478 <itemizedlist>
479 <listitem><para>
480 Be able to check the box indicating that you
481 got a "PASS" when running the script against your
482 layer.
483 </para></listitem>
484 <listitem><para>
485 Answer "Yes" to the questions on the form or have an
486 acceptable explanation for any questions answered "No".
487 </para></listitem>
488 <listitem><para>
489 Be a Yocto Project Member Organization.
490 </para></listitem>
491 </itemizedlist>
492 </para>
493
494 <para>
495 The remainder of this section presents information on the
496 registration form and on the
497 <filename>yocto-check-layer</filename> script.
498 </para>
499
500 <section id='yocto-project-compatible-program-application'>
501 <title>Yocto Project Compatible Program Application</title>
502
503 <para>
504 Use the form to apply for your layer's approval.
505 Upon successful application, you can use the Yocto
506 Project Compatibility Logo with your layer and the
507 application that uses your layer.
508 </para>
509
510 <para>
511 To access the form, use this link:
512 <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>.
513 Follow the instructions on the form to complete your
514 application.
515 </para>
516
517 <para>
518 The application consists of the following sections:
519 <itemizedlist>
520 <listitem><para>
521 <emphasis>Contact Information:</emphasis>
522 Provide your contact information as the fields
523 require.
524 Along with your information, provide the
525 released versions of the Yocto Project for which
526 your layer is compatible.
527 </para></listitem>
528 <listitem><para>
529 <emphasis>Acceptance Criteria:</emphasis>
530 Provide "Yes" or "No" answers for each of the
531 items in the checklist.
532 Space exists at the bottom of the form for any
533 explanations for items for which you answered "No".
534 </para></listitem>
535 <listitem><para>
536 <emphasis>Recommendations:</emphasis>
537 Provide answers for the questions regarding Linux
538 kernel use and build success.
539 </para></listitem>
540 </itemizedlist>
541 </para>
542 </section>
543
544 <section id='yocto-check-layer-script'>
545 <title><filename>yocto-check-layer</filename> Script</title>
546
547 <para>
548 The <filename>yocto-check-layer</filename> script
549 provides you a way to assess how compatible your layer is
550 with the Yocto Project.
551 You should run this script prior to using the form to
552 apply for compatibility as described in the previous
553 section.
554 You need to achieve a "PASS" result in order to have
555 your application form successfully processed.
556 </para>
557
558 <para>
559 The script divides tests into three areas: COMMON, BSP,
560 and DISTRO.
561 For example, given a distribution layer (DISTRO), the
562 layer must pass both the COMMON and DISTRO related tests.
563 Furthermore, if your layer is a BSP layer, the layer must
564 pass the COMMON and BSP set of tests.
565 </para>
566
567 <para>
568 To execute the script, enter the following commands from
569 your build directory:
570 <literallayout class='monospaced'>
571 $ source oe-init-build-env
572 $ yocto-check-layer <replaceable>your_layer_directory</replaceable>
573 </literallayout>
574 Be sure to provide the actual directory for your layer
575 as part of the command.
576 </para>
577
578 <para>
579 Entering the command causes the script to determine the
580 type of layer and then to execute a set of specific
581 tests against the layer.
582 The following list overviews the test:
583 <itemizedlist>
584 <listitem><para>
585 <filename>common.test_readme</filename>:
586 Tests if a <filename>README</filename> file
587 exists in the layer and the file is not empty.
588 </para></listitem>
589 <listitem><para>
590 <filename>common.test_parse</filename>:
591 Tests to make sure that BitBake can parse the
592 files without error (i.e.
593 <filename>bitbake -p</filename>).
594 </para></listitem>
595 <listitem><para>
596 <filename>common.test_show_environment</filename>:
597 Tests that the global or per-recipe environment
598 is in order without errors (i.e.
599 <filename>bitbake -e</filename>).
600 </para></listitem>
601 <listitem><para>
602 <filename>common.test_world</filename>:
603 Verifies that <filename>bitbake world</filename> works.
604 </para></listitem>
605 <listitem><para>
606 <filename>common.test_signatures</filename>:
607 Tests to be sure that BSP and DISTRO layers do not
608 come with recipes that change signatures.
609 </para></listitem>
610 <listitem><para>
611 <filename>common.test_layerseries_compat</filename>:
612 Verifies layer compatibility is set properly.
613 </para></listitem>
614 <listitem><para>
615 <filename>bsp.test_bsp_defines_machines</filename>:
616 Tests if a BSP layer has machine configurations.
617 </para></listitem>
618 <listitem><para>
619 <filename>bsp.test_bsp_no_set_machine</filename>:
620 Tests to ensure a BSP layer does not set the
621 machine when the layer is added.
622 </para></listitem>
623 <listitem><para>
624 <filename>bsp.test_machine_world</filename>:
625 Verifies that <filename>bitbake world</filename>
626 works regardless of which machine is selected.
627 </para></listitem>
628 <listitem><para>
629 <filename>bsp.test_machine_signatures</filename>:
630 Verifies that building for a particular machine
631 affects only the signature of tasks specific to that
632 machine.
633 </para></listitem>
634 <listitem><para>
635 <filename>distro.test_distro_defines_distros</filename>:
636 Tests if a DISTRO layer has distro configurations.
637 </para></listitem>
638 <listitem><para>
639 <filename>distro.test_distro_no_set_distros</filename>:
640 Tests to ensure a DISTRO layer does not set the
641 distribution when the layer is added.
642 </para></listitem>
643 </itemizedlist>
644 </para>
645 </section>
646 </section>
647
648 <section id='enabling-your-layer'>
649 <title>Enabling Your Layer</title>
650
651 <para>
652 Before the OpenEmbedded build system can use your new layer,
653 you need to enable it.
654 To enable your layer, simply add your layer's path to the
655 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename>
656 variable in your <filename>conf/bblayers.conf</filename> file,
657 which is found in the
658 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
659 The following example shows how to enable a layer named
660 <filename>meta-mylayer</filename>:
661 <literallayout class='monospaced'>
662 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
663 # changes incompatibly
664 POKY_BBLAYERS_CONF_VERSION = "2"
665
666 BBPATH = "${TOPDIR}"
667 BBFILES ?= ""
668
669 BBLAYERS ?= " \
670 /home/<replaceable>user</replaceable>/poky/meta \
671 /home/<replaceable>user</replaceable>/poky/meta-poky \
672 /home/<replaceable>user</replaceable>/poky/meta-yocto-bsp \
673 /home/<replaceable>user</replaceable>/poky/meta-mylayer \
674 "
675 </literallayout>
676 </para>
677
678 <para>
679 BitBake parses each <filename>conf/layer.conf</filename> file
680 from the top down as specified in the
681 <filename>BBLAYERS</filename> variable
682 within the <filename>conf/bblayers.conf</filename> file.
683 During the processing of each
684 <filename>conf/layer.conf</filename> file, BitBake adds the
685 recipes, classes and configurations contained within the
686 particular layer to the source directory.
687 </para>
688 </section>
689
690 <section id='using-bbappend-files'>
691 <title>Using .bbappend Files in Your Layer</title>
692
693 <para>
694 A recipe that appends Metadata to another recipe is called a
695 BitBake append file.
696 A BitBake append file uses the <filename>.bbappend</filename>
697 file type suffix, while the corresponding recipe to which
698 Metadata is being appended uses the <filename>.bb</filename>
699 file type suffix.
700 </para>
701
702 <para>
703 You can use a <filename>.bbappend</filename> file in your
704 layer to make additions or changes to the content of another
705 layer's recipe without having to copy the other layer's
706 recipe into your layer.
707 Your <filename>.bbappend</filename> file resides in your layer,
708 while the main <filename>.bb</filename> recipe file to
709 which you are appending Metadata resides in a different layer.
710 </para>
711
712 <para>
713 Being able to append information to an existing recipe not only
714 avoids duplication, but also automatically applies recipe
715 changes from a different layer into your layer.
716 If you were copying recipes, you would have to manually merge
717 changes as they occur.
718 </para>
719
720 <para>
721 When you create an append file, you must use the same root
722 name as the corresponding recipe file.
723 For example, the append file
724 <filename>someapp_&DISTRO;.bbappend</filename> must apply to
725 <filename>someapp_&DISTRO;.bb</filename>.
726 This means the original recipe and append file names are
727 version number-specific.
728 If the corresponding recipe is renamed to update to a newer
729 version, you must also rename and possibly update
730 the corresponding <filename>.bbappend</filename> as well.
731 During the build process, BitBake displays an error on starting
732 if it detects a <filename>.bbappend</filename> file that does
733 not have a corresponding recipe with a matching name.
734 See the
735 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink>
736 variable for information on how to handle this error.
737 </para>
738
739 <para>
740 As an example, consider the main formfactor recipe and a
741 corresponding formfactor append file both from the
742 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
743 Here is the main formfactor recipe, which is named
744 <filename>formfactor_0.0.bb</filename> and located in the
745 "meta" layer at
746 <filename>meta/recipes-bsp/formfactor</filename>:
747 <literallayout class='monospaced'>
748 SUMMARY = "Device formfactor information"
749 SECTION = "base"
750 LICENSE = "MIT"
751 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
752 PR = "r45"
753
754 SRC_URI = "file://config file://machconfig"
755 S = "${WORKDIR}"
756
757 PACKAGE_ARCH = "${MACHINE_ARCH}"
758 INHIBIT_DEFAULT_DEPS = "1"
759
760 do_install() {
761 # Install file only if it has contents
762 install -d ${D}${sysconfdir}/formfactor/
763 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
764 if [ -s "${S}/machconfig" ]; then
765 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
766 fi
767 } </literallayout>
768 In the main recipe, note the
769 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
770 variable, which tells the OpenEmbedded build system where to
771 find files during the build.
772 </para>
773
774 <para>
775 Following is the append file, which is named
776 <filename>formfactor_0.0.bbappend</filename> and is from the
777 Raspberry Pi BSP Layer named
778 <filename>meta-raspberrypi</filename>.
779 The file is in the layer at
780 <filename>recipes-bsp/formfactor</filename>:
781 <literallayout class='monospaced'>
782 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
783 </literallayout>
784 </para>
785
786 <para>
787 By default, the build system uses the
788 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
789 variable to locate files.
790 This append file extends the locations by setting the
791 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
792 variable.
793 Setting this variable in the <filename>.bbappend</filename>
794 file is the most reliable and recommended method for adding
795 directories to the search path used by the build system
796 to find files.
797 </para>
798
799 <para>
800 The statement in this example extends the directories to
801 include
802 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>,
803 which resolves to a directory named
804 <filename>formfactor</filename> in the same directory
805 in which the append file resides (i.e.
806 <filename>meta-raspberrypi/recipes-bsp/formfactor</filename>.
807 This implies that you must have the supporting directory
808 structure set up that will contain any files or patches you
809 will be including from the layer.
810 </para>
811
812 <para>
813 Using the immediate expansion assignment operator
814 <filename>:=</filename> is important because of the reference
815 to <filename>THISDIR</filename>.
816 The trailing colon character is important as it ensures that
817 items in the list remain colon-separated.
818 <note>
819 <para>
820 BitBake automatically defines the
821 <filename>THISDIR</filename> variable.
822 You should never set this variable yourself.
823 Using "_prepend" as part of the
824 <filename>FILESEXTRAPATHS</filename> ensures your path
825 will be searched prior to other paths in the final
826 list.
827 </para>
828
829 <para>
830 Also, not all append files add extra files.
831 Many append files simply exist to add build options
832 (e.g. <filename>systemd</filename>).
833 For these cases, your append file would not even
834 use the <filename>FILESEXTRAPATHS</filename> statement.
835 </para>
836 </note>
837 </para>
838 </section>
839
840 <section id='prioritizing-your-layer'>
841 <title>Prioritizing Your Layer</title>
842
843 <para>
844 Each layer is assigned a priority value.
845 Priority values control which layer takes precedence if there
846 are recipe files with the same name in multiple layers.
847 For these cases, the recipe file from the layer with a higher
848 priority number takes precedence.
849 Priority values also affect the order in which multiple
850 <filename>.bbappend</filename> files for the same recipe are
851 applied.
852 You can either specify the priority manually, or allow the
853 build system to calculate it based on the layer's dependencies.
854 </para>
855
856 <para>
857 To specify the layer's priority manually, use the
858 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>
859 variable and append the layer's root name:
860 <literallayout class='monospaced'>
861 BBFILE_PRIORITY_mylayer = "1"
862 </literallayout>
863 </para>
864
865 <note>
866 <para>It is possible for a recipe with a lower version number
867 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
868 in a layer that has a higher priority to take precedence.</para>
869 <para>Also, the layer priority does not currently affect the
870 precedence order of <filename>.conf</filename>
871 or <filename>.bbclass</filename> files.
872 Future versions of BitBake might address this.</para>
873 </note>
874 </section>
875
876 <section id='managing-layers'>
877 <title>Managing Layers</title>
878
879 <para>
880 You can use the BitBake layer management tool
881 <filename>bitbake-layers</filename> to provide a view
882 into the structure of recipes across a multi-layer project.
883 Being able to generate output that reports on configured layers
884 with their paths and priorities and on
885 <filename>.bbappend</filename> files and their applicable
886 recipes can help to reveal potential problems.
887 </para>
888
889 <para>
890 For help on the BitBake layer management tool, use the
891 following command:
892 <literallayout class='monospaced'>
893 $ bitbake-layers --help
894 NOTE: Starting bitbake server...
895 usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] &lt;subcommand&gt; ...
896
897 BitBake layers utility
898
899 optional arguments:
900 -d, --debug Enable debug output
901 -q, --quiet Print only errors
902 -F, --force Force add without recipe parse verification
903 --color COLOR Colorize output (where COLOR is auto, always, never)
904 -h, --help show this help message and exit
905
906 subcommands:
907 &lt;subcommand&gt;
908 show-layers show current configured layers.
909 show-overlayed list overlayed recipes (where the same recipe exists
910 in another layer)
911 show-recipes list available recipes, showing the layer they are
912 provided by
913 show-appends list bbappend files and recipe files they apply to
914 show-cross-depends Show dependencies between recipes that cross layer
915 boundaries.
916 add-layer Add one or more layers to bblayers.conf.
917 remove-layer Remove one or more layers from bblayers.conf.
918 flatten flatten layer configuration into a separate output
919 directory.
920 layerindex-fetch Fetches a layer from a layer index along with its
921 dependent layers, and adds them to conf/bblayers.conf.
922 layerindex-show-depends
923 Find layer dependencies from layer index.
924 create-layer Create a basic layer
925
926 Use bitbake-layers &lt;subcommand&gt; --help to get help on a specific command
927 </literallayout>
928 </para>
929
930 <para>
931 The following list describes the available commands:
932 <itemizedlist>
933 <listitem><para>
934 <emphasis><filename>help:</filename></emphasis>
935 Displays general help or help on a specified command.
936 </para></listitem>
937 <listitem><para>
938 <emphasis><filename>show-layers:</filename></emphasis>
939 Shows the current configured layers.
940 </para></listitem>
941 <listitem><para>
942 <emphasis><filename>show-overlayed:</filename></emphasis>
943 Lists overlayed recipes.
944 A recipe is overlayed when a recipe with the same name
945 exists in another layer that has a higher layer
946 priority.
947 </para></listitem>
948 <listitem><para>
949 <emphasis><filename>show-recipes:</filename></emphasis>
950 Lists available recipes and the layers that provide them.
951 </para></listitem>
952 <listitem><para>
953 <emphasis><filename>show-appends:</filename></emphasis>
954 Lists <filename>.bbappend</filename> files and the
955 recipe files to which they apply.
956 </para></listitem>
957 <listitem><para>
958 <emphasis><filename>show-cross-depends:</filename></emphasis>
959 Lists dependency relationships between recipes that
960 cross layer boundaries.
961 </para></listitem>
962 <listitem><para>
963 <emphasis><filename>add-layer:</filename></emphasis>
964 Adds a layer to <filename>bblayers.conf</filename>.
965 </para></listitem>
966 <listitem><para>
967 <emphasis><filename>remove-layer:</filename></emphasis>
968 Removes a layer from <filename>bblayers.conf</filename>
969 </para></listitem>
970 <listitem><para>
971 <emphasis><filename>flatten:</filename></emphasis>
972 Flattens the layer configuration into a separate output
973 directory.
974 Flattening your layer configuration builds a "flattened"
975 directory that contains the contents of all layers,
976 with any overlayed recipes removed and any
977 <filename>.bbappend</filename> files appended to the
978 corresponding recipes.
979 You might have to perform some manual cleanup of the
980 flattened layer as follows:
981 <itemizedlist>
982 <listitem><para>
983 Non-recipe files (such as patches)
984 are overwritten.
985 The flatten command shows a warning for these
986 files.
987 </para></listitem>
988 <listitem><para>
989 Anything beyond the normal layer
990 setup has been added to the
991 <filename>layer.conf</filename> file.
992 Only the lowest priority layer's
993 <filename>layer.conf</filename> is used.
994 </para></listitem>
995 <listitem><para>
996 Overridden and appended items from
997 <filename>.bbappend</filename> files need to be
998 cleaned up.
999 The contents of each
1000 <filename>.bbappend</filename> end up in the
1001 flattened recipe.
1002 However, if there are appended or changed
1003 variable values, you need to tidy these up
1004 yourself.
1005 Consider the following example.
1006 Here, the <filename>bitbake-layers</filename>
1007 command adds the line
1008 <filename>#### bbappended ...</filename> so that
1009 you know where the following lines originate:
1010 <literallayout class='monospaced'>
1011 ...
1012 DESCRIPTION = "A useful utility"
1013 ...
1014 EXTRA_OECONF = "--enable-something"
1015 ...
1016
1017 #### bbappended from meta-anotherlayer ####
1018
1019 DESCRIPTION = "Customized utility"
1020 EXTRA_OECONF += "--enable-somethingelse"
1021 </literallayout>
1022 Ideally, you would tidy up these utilities as
1023 follows:
1024 <literallayout class='monospaced'>
1025 ...
1026 DESCRIPTION = "Customized utility"
1027 ...
1028 EXTRA_OECONF = "--enable-something --enable-somethingelse"
1029 ...
1030 </literallayout>
1031 </para></listitem>
1032 </itemizedlist>
1033 </para></listitem>
1034 <listitem><para>
1035 <emphasis><filename>layerindex-fetch</filename>:</emphasis>
1036 Fetches a layer from a layer index, along with its
1037 dependent layers, and adds the layers to the
1038 <filename>conf/bblayers.conf</filename> file.
1039 </para></listitem>
1040 <listitem><para>
1041 <emphasis><filename>layerindex-show-depends</filename>:</emphasis>
1042 Finds layer dependencies from the layer index.
1043 </para></listitem>
1044 <listitem><para>
1045 <emphasis><filename>create-layer</filename>:</emphasis>
1046 Creates a basic layer.
1047 </para></listitem>
1048 </itemizedlist>
1049 </para>
1050 </section>
1051
1052 <section id='creating-a-general-layer-using-the-bitbake-layers-script'>
1053 <title>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</title>
1054
1055 <para>
1056 The <filename>bitbake-layers</filename> script with the
1057 <filename>create-layer</filename> subcommand simplifies
1058 creating a new general layer.
1059 <note><title>Notes</title>
1060 <itemizedlist>
1061 <listitem><para>
1062 For information on BSP layers, see the
1063 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
1064 section in the Yocto Project Board Specific (BSP)
1065 Developer's Guide.
1066 </para></listitem>
1067 <listitem><para>
1068 In order to use a layer with the OpenEmbedded
1069 build system, you need to add the layer to your
1070 <filename>bblayers.conf</filename> configuration
1071 file.
1072 See the
1073 "<link linkend='adding-a-layer-using-the-bitbake-layers-script'>Adding a Layer Using the <filename>bitbake-layers</filename> Script</link>"
1074 section for more information.
1075 </para></listitem>
1076 </itemizedlist>
1077 </note>
1078 The default mode of the script's operation with this
1079 subcommand is to create a layer with the following:
1080 <itemizedlist>
1081 <listitem><para>A layer priority of 6.
1082 </para></listitem>
1083 <listitem><para>A <filename>conf</filename>
1084 subdirectory that contains a
1085 <filename>layer.conf</filename> file.
1086 </para></listitem>
1087 <listitem><para>
1088 A <filename>recipes-example</filename> subdirectory
1089 that contains a further subdirectory named
1090 <filename>example</filename>, which contains
1091 an <filename>example.bb</filename> recipe file.
1092 </para></listitem>
1093 <listitem><para>A <filename >COPYING.MIT</filename>,
1094 which is the license statement for the layer.
1095 The script assumes you want to use the MIT license,
1096 which is typical for most layers, for the contents of
1097 the layer itself.
1098 </para></listitem>
1099 <listitem><para>
1100 A <filename>README</filename> file, which is a file
1101 describing the contents of your new layer.
1102 </para></listitem>
1103 </itemizedlist>
1104 </para>
1105
1106 <para>
1107 In its simplest form, you can use the following command form
1108 to create a layer.
1109 The command creates a layer whose name corresponds to
1110 <replaceable>your_layer_name</replaceable> in the current
1111 directory:
1112 <literallayout class='monospaced'>
1113 $ bitbake-layers create-layer <replaceable>your_layer_name</replaceable>
1114 </literallayout>
1115 As an example, the following command creates a layer named
1116 <filename>meta-scottrif</filename> in your home directory:
1117 <literallayout class='monospaced'>
1118 $ cd /usr/home
1119 $ bitbake-layers create-layer meta-scottrif
1120 NOTE: Starting bitbake server...
1121 Add your new layer with 'bitbake-layers add-layer meta-scottrif'
1122 </literallayout>
1123 </para>
1124
1125 <para>
1126 If you want to set the priority of the layer to other than the
1127 default value of "6", you can either use the
1128 <filename>&dash;&dash;priority</filename> option or you can
1129 edit the
1130 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>
1131 value in the <filename>conf/layer.conf</filename> after the
1132 script creates it.
1133 Furthermore, if you want to give the example recipe file
1134 some name other than the default, you can
1135 use the
1136 <filename>&dash;&dash;example-recipe-name</filename> option.
1137 </para>
1138
1139 <para>
1140 The easiest way to see how the
1141 <filename>bitbake-layers create-layer</filename> command
1142 works is to experiment with the script.
1143 You can also read the usage information by entering the
1144 following:
1145 <literallayout class='monospaced'>
1146 $ bitbake-layers create-layer --help
1147 NOTE: Starting bitbake server...
1148 usage: bitbake-layers create-layer [-h] [--priority PRIORITY]
1149 [--example-recipe-name EXAMPLERECIPE]
1150 layerdir
1151
1152 Create a basic layer
1153
1154 positional arguments:
1155 layerdir Layer directory to create
1156
1157 optional arguments:
1158 -h, --help show this help message and exit
1159 --priority PRIORITY, -p PRIORITY
1160 Layer directory to create
1161 --example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE
1162 Filename of the example recipe
1163 </literallayout>
1164 </para>
1165 </section>
1166
1167 <section id='adding-a-layer-using-the-bitbake-layers-script'>
1168 <title>Adding a Layer Using the <filename>bitbake-layers</filename> Script</title>
1169
1170 <para>
1171 Once you create your general layer, you must add it to your
1172 <filename>bblayers.conf</filename> file.
1173 Adding the layer to this configuration file makes the
1174 OpenEmbedded build system aware of your layer so that it can
1175 search it for metadata.
1176 </para>
1177
1178 <para>
1179 Add your layer by using the
1180 <filename>bitbake-layers add-layer</filename> command:
1181 <literallayout class='monospaced'>
1182 $ bitbake-layers add-layer <replaceable>your_layer_name</replaceable>
1183 </literallayout>
1184 Here is an example that adds a layer named
1185 <filename>meta-scottrif</filename> to the configuration file.
1186 Following the command that adds the layer is another
1187 <filename>bitbake-layers</filename> command that shows the
1188 layers that are in your <filename>bblayers.conf</filename>
1189 file:
1190 <literallayout class='monospaced'>
1191 $ bitbake-layers add-layer meta-scottrif
1192 NOTE: Starting bitbake server...
1193 Parsing recipes: 100% |##########################################################| Time: 0:00:49
1194 Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors.
1195 $ bitbake-layers show-layers
1196 NOTE: Starting bitbake server...
1197 layer path priority
1198 ==========================================================================
1199 meta /home/scottrif/poky/meta 5
1200 meta-poky /home/scottrif/poky/meta-poky 5
1201 meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5
1202 workspace /home/scottrif/poky/build/workspace 99
1203 meta-scottrif /home/scottrif/poky/build/meta-scottrif 6
1204 </literallayout>
1205 Adding the layer to this file enables the build system to
1206 locate the layer during the build.
1207 <note>
1208 During a build, the OpenEmbedded build system looks in
1209 the layers from the top of the list down to the bottom
1210 in that order.
1211 </note>
1212 </para>
1213 </section>
1214 </section>
1215
1216 <section id='usingpoky-extend-customimage'>
1217 <title>Customizing Images</title>
1218
1219 <para>
1220 You can customize images to satisfy particular requirements.
1221 This section describes several methods and provides guidelines for each.
1222 </para>
1223
1224 <section id='usingpoky-extend-customimage-localconf'>
1225 <title>Customizing Images Using <filename>local.conf</filename></title>
1226
1227 <para>
1228 Probably the easiest way to customize an image is to add a
1229 package by way of the <filename>local.conf</filename>
1230 configuration file.
1231 Because it is limited to local use, this method generally only
1232 allows you to add packages and is not as flexible as creating
1233 your own customized image.
1234 When you add packages using local variables this way, you need
1235 to realize that these variable changes are in effect for every
1236 build and consequently affect all images, which might not
1237 be what you require.
1238 </para>
1239
1240 <para>
1241 To add a package to your image using the local configuration
1242 file, use the
1243 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1244 variable with the <filename>_append</filename> operator:
1245 <literallayout class='monospaced'>
1246 IMAGE_INSTALL_append = " strace"
1247 </literallayout>
1248 Use of the syntax is important - specifically, the space between
1249 the quote and the package name, which is
1250 <filename>strace</filename> in this example.
1251 This space is required since the <filename>_append</filename>
1252 operator does not add the space.
1253 </para>
1254
1255 <para>
1256 Furthermore, you must use <filename>_append</filename> instead
1257 of the <filename>+=</filename> operator if you want to avoid
1258 ordering issues.
1259 The reason for this is because doing so unconditionally appends
1260 to the variable and avoids ordering problems due to the
1261 variable being set in image recipes and
1262 <filename>.bbclass</filename> files with operators like
1263 <filename>?=</filename>.
1264 Using <filename>_append</filename> ensures the operation takes
1265 affect.
1266 </para>
1267
1268 <para>
1269 As shown in its simplest use,
1270 <filename>IMAGE_INSTALL_append</filename> affects all images.
1271 It is possible to extend the syntax so that the variable
1272 applies to a specific image only.
1273 Here is an example:
1274 <literallayout class='monospaced'>
1275 IMAGE_INSTALL_append_pn-core-image-minimal = " strace"
1276 </literallayout>
1277 This example adds <filename>strace</filename> to the
1278 <filename>core-image-minimal</filename> image only.
1279 </para>
1280
1281 <para>
1282 You can add packages using a similar approach through the
1283 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename>
1284 variable.
1285 If you use this variable, only
1286 <filename>core-image-*</filename> images are affected.
1287 </para>
1288 </section>
1289
1290 <section id='usingpoky-extend-customimage-imagefeatures'>
1291 <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and
1292 <filename>EXTRA_IMAGE_FEATURES</filename></title>
1293
1294 <para>
1295 Another method for customizing your image is to enable or
1296 disable high-level image features by using the
1297 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
1298 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
1299 variables.
1300 Although the functions for both variables are nearly equivalent,
1301 best practices dictate using <filename>IMAGE_FEATURES</filename>
1302 from within a recipe and using
1303 <filename>EXTRA_IMAGE_FEATURES</filename> from within
1304 your <filename>local.conf</filename> file, which is found in the
1305 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
1306 </para>
1307
1308 <para>
1309 To understand how these features work, the best reference is
1310 <filename>meta/classes/core-image.bbclass</filename>.
1311 This class lists out the available
1312 <filename>IMAGE_FEATURES</filename> of which most map to
1313 package groups while some, such as
1314 <filename>debug-tweaks</filename> and
1315 <filename>read-only-rootfs</filename>, resolve as general
1316 configuration settings.
1317 </para>
1318
1319 <para>
1320 In summary, the file looks at the contents of the
1321 <filename>IMAGE_FEATURES</filename> variable and then maps
1322 or configures the feature accordingly.
1323 Based on this information, the build system automatically
1324 adds the appropriate packages or configurations to the
1325 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
1326 variable.
1327 Effectively, you are enabling extra features by extending the
1328 class or creating a custom class for use with specialized image
1329 <filename>.bb</filename> files.
1330 </para>
1331
1332 <para>
1333 Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable
1334 from within your local configuration file.
1335 Using a separate area from which to enable features with
1336 this variable helps you avoid overwriting the features in the
1337 image recipe that are enabled with
1338 <filename>IMAGE_FEATURES</filename>.
1339 The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added
1340 to <filename>IMAGE_FEATURES</filename> within
1341 <filename>meta/conf/bitbake.conf</filename>.
1342 </para>
1343
1344 <para>
1345 To illustrate how you can use these variables to modify your
1346 image, consider an example that selects the SSH server.
1347 The Yocto Project ships with two SSH servers you can use
1348 with your images: Dropbear and OpenSSH.
1349 Dropbear is a minimal SSH server appropriate for
1350 resource-constrained environments, while OpenSSH is a
1351 well-known standard SSH server implementation.
1352 By default, the <filename>core-image-sato</filename> image
1353 is configured to use Dropbear.
1354 The <filename>core-image-full-cmdline</filename> and
1355 <filename>core-image-lsb</filename> images both
1356 include OpenSSH.
1357 The <filename>core-image-minimal</filename> image does not
1358 contain an SSH server.
1359 </para>
1360
1361 <para>
1362 You can customize your image and change these defaults.
1363 Edit the <filename>IMAGE_FEATURES</filename> variable
1364 in your recipe or use the
1365 <filename>EXTRA_IMAGE_FEATURES</filename> in your
1366 <filename>local.conf</filename> file so that it configures the
1367 image you are working with to include
1368 <filename>ssh-server-dropbear</filename> or
1369 <filename>ssh-server-openssh</filename>.
1370 </para>
1371
1372 <note>
1373 See the
1374 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
1375 section in the Yocto Project Reference Manual for a complete
1376 list of image features that ship with the Yocto Project.
1377 </note>
1378 </section>
1379
1380 <section id='usingpoky-extend-customimage-custombb'>
1381 <title>Customizing Images Using Custom .bb Files</title>
1382
1383 <para>
1384 You can also customize an image by creating a custom recipe
1385 that defines additional software as part of the image.
1386 The following example shows the form for the two lines you need:
1387 <literallayout class='monospaced'>
1388 IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
1389
1390 inherit core-image
1391 </literallayout>
1392 </para>
1393
1394 <para>
1395 Defining the software using a custom recipe gives you total
1396 control over the contents of the image.
1397 It is important to use the correct names of packages in the
1398 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1399 variable.
1400 You must use the OpenEmbedded notation and not the Debian notation for the names
1401 (e.g. <filename>glibc-dev</filename> instead of <filename>libc6-dev</filename>).
1402 </para>
1403
1404 <para>
1405 The other method for creating a custom image is to base it on an existing image.
1406 For example, if you want to create an image based on <filename>core-image-sato</filename>
1407 but add the additional package <filename>strace</filename> to the image,
1408 copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a
1409 new <filename>.bb</filename> and add the following line to the end of the copy:
1410 <literallayout class='monospaced'>
1411 IMAGE_INSTALL += "strace"
1412 </literallayout>
1413 </para>
1414 </section>
1415
1416 <section id='usingpoky-extend-customimage-customtasks'>
1417 <title>Customizing Images Using Custom Package Groups</title>
1418
1419 <para>
1420 For complex custom images, the best approach for customizing
1421 an image is to create a custom package group recipe that is
1422 used to build the image or images.
1423 A good example of a package group recipe is
1424 <filename>meta/recipes-core/packagegroups/packagegroup-base.bb</filename>.
1425 </para>
1426
1427 <para>
1428 If you examine that recipe, you see that the
1429 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename>
1430 variable lists the package group packages to produce.
1431 The <filename>inherit packagegroup</filename> statement
1432 sets appropriate default values and automatically adds
1433 <filename>-dev</filename>, <filename>-dbg</filename>, and
1434 <filename>-ptest</filename> complementary packages for each
1435 package specified in the <filename>PACKAGES</filename>
1436 statement.
1437 <note>
1438 The <filename>inherit packagegroup</filename> line should be
1439 located near the top of the recipe, certainly before
1440 the <filename>PACKAGES</filename> statement.
1441 </note>
1442 </para>
1443
1444 <para>
1445 For each package you specify in <filename>PACKAGES</filename>,
1446 you can use
1447 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename>
1448 and
1449 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename>
1450 entries to provide a list of packages the parent task package
1451 should contain.
1452 You can see examples of these further down in the
1453 <filename>packagegroup-base.bb</filename> recipe.
1454 </para>
1455
1456 <para>
1457 Here is a short, fabricated example showing the same basic
1458 pieces for a hypothetical packagegroup defined in
1459 <filename>packagegroup-custom.bb</filename>, where the
1460 variable <filename>PN</filename> is the standard way to
1461 abbreviate the reference to the full packagegroup name
1462 <filename>packagegroup-custom</filename>:
1463 <literallayout class='monospaced'>
1464 DESCRIPTION = "My Custom Package Groups"
1465
1466 inherit packagegroup
1467
1468 PACKAGES = "\
1469 ${PN}-apps \
1470 ${PN}-tools \
1471 "
1472
1473 RDEPENDS_${PN}-apps = "\
1474 dropbear \
1475 portmap \
1476 psplash"
1477
1478 RDEPENDS_${PN}-tools = "\
1479 oprofile \
1480 oprofileui-server \
1481 lttng-tools"
1482
1483 RRECOMMENDS_${PN}-tools = "\
1484 kernel-module-oprofile"
1485 </literallayout>
1486 </para>
1487
1488 <para>
1489 In the previous example, two package group packages are created with their dependencies and their
1490 recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and
1491 <filename>packagegroup-custom-tools</filename>.
1492 To build an image using these package group packages, you need to add
1493 <filename>packagegroup-custom-apps</filename> and/or
1494 <filename>packagegroup-custom-tools</filename> to
1495 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>.
1496 For other forms of image dependencies see the other areas of this section.
1497 </para>
1498 </section>
1499
1500 <section id='usingpoky-extend-customimage-image-name'>
1501 <title>Customizing an Image Hostname</title>
1502
1503 <para>
1504 By default, the configured hostname (i.e.
1505 <filename>/etc/hostname</filename>) in an image is the
1506 same as the machine name.
1507 For example, if
1508 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
1509 equals "qemux86", the configured hostname written to
1510 <filename>/etc/hostname</filename> is "qemux86".
1511 </para>
1512
1513 <para>
1514 You can customize this name by altering the value of the
1515 "hostname" variable in the
1516 <filename>base-files</filename> recipe using either
1517 an append file or a configuration file.
1518 Use the following in an append file:
1519 <literallayout class='monospaced'>
1520 hostname="myhostname"
1521 </literallayout>
1522 Use the following in a configuration file:
1523 <literallayout class='monospaced'>
1524 hostname_pn-base-files = "myhostname"
1525 </literallayout>
1526 </para>
1527
1528 <para>
1529 Changing the default value of the variable "hostname" can be
1530 useful in certain situations.
1531 For example, suppose you need to do extensive testing on an
1532 image and you would like to easily identify the image
1533 under test from existing images with typical default
1534 hostnames.
1535 In this situation, you could change the default hostname to
1536 "testme", which results in all the images using the name
1537 "testme".
1538 Once testing is complete and you do not need to rebuild the
1539 image for test any longer, you can easily reset the default
1540 hostname.
1541 </para>
1542
1543 <para>
1544 Another point of interest is that if you unset the variable,
1545 the image will have no default hostname in the filesystem.
1546 Here is an example that unsets the variable in a
1547 configuration file:
1548 <literallayout class='monospaced'>
1549 hostname_pn-base-files = ""
1550 </literallayout>
1551 Having no default hostname in the filesystem is suitable for
1552 environments that use dynamic hostnames such as virtual
1553 machines.
1554 </para>
1555 </section>
1556 </section>
1557
1558 <section id='new-recipe-writing-a-new-recipe'>
1559 <title>Writing a New Recipe</title>
1560
1561 <para>
1562 Recipes (<filename>.bb</filename> files) are fundamental components
1563 in the Yocto Project environment.
1564 Each software component built by the OpenEmbedded build system
1565 requires a recipe to define the component.
1566 This section describes how to create, write, and test a new
1567 recipe.
1568 <note>
1569 For information on variables that are useful for recipes and
1570 for information about recipe naming issues, see the
1571 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>"
1572 section of the Yocto Project Reference Manual.
1573 </note>
1574 </para>
1575
1576 <section id='new-recipe-overview'>
1577 <title>Overview</title>
1578
1579 <para>
1580 The following figure shows the basic process for creating a
1581 new recipe.
1582 The remainder of the section provides details for the steps.
1583 <imagedata fileref="figures/recipe-workflow.png" width="6in" depth="7in" align="center" scalefit="1" />
1584 </para>
1585 </section>
1586
1587 <section id='new-recipe-locate-or-automatically-create-a-base-recipe'>
1588 <title>Locate or Automatically Create a Base Recipe</title>
1589
1590 <para>
1591 You can always write a recipe from scratch.
1592 However, three choices exist that can help you quickly get a
1593 start on a new recipe:
1594 <itemizedlist>
1595 <listitem><para>
1596 <emphasis><filename>devtool add</filename>:</emphasis>
1597 A command that assists in creating a recipe and
1598 an environment conducive to development.
1599 </para></listitem>
1600 <listitem><para>
1601 <emphasis><filename>recipetool create</filename>:</emphasis>
1602 A command provided by the Yocto Project that automates
1603 creation of a base recipe based on the source
1604 files.
1605 </para></listitem>
1606 <listitem><para>
1607 <emphasis>Existing Recipes:</emphasis>
1608 Location and modification of an existing recipe that is
1609 similar in function to the recipe you need.
1610 </para></listitem>
1611 </itemizedlist>
1612 <note>
1613 For information on recipe syntax, see the
1614 "<link linkend='recipe-syntax'>Recipe Syntax</link>"
1615 section.
1616 </note>
1617 </para>
1618
1619 <section id='new-recipe-creating-the-base-recipe-using-devtool'>
1620 <title>Creating the Base Recipe Using <filename>devtool add</filename></title>
1621
1622 <para>
1623 The <filename>devtool add</filename> command uses the same
1624 logic for auto-creating the recipe as
1625 <filename>recipetool create</filename>, which is listed
1626 below.
1627 Additionally, however, <filename>devtool add</filename>
1628 sets up an environment that makes it easy for you to
1629 patch the source and to make changes to the recipe as
1630 is often necessary when adding a recipe to build a new
1631 piece of software to be included in a build.
1632 </para>
1633
1634 <para>
1635 You can find a complete description of the
1636 <filename>devtool add</filename> command in the
1637 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-a-closer-look-at-devtool-add'>A Closer Look at <filename>devtool</filename> add</ulink>"
1638 section in the Yocto Project Application Development
1639 and the Extensible Software Development Kit (eSDK) manual.
1640 </para>
1641 </section>
1642
1643 <section id='new-recipe-creating-the-base-recipe-using-recipetool'>
1644 <title>Creating the Base Recipe Using <filename>recipetool create</filename></title>
1645
1646 <para>
1647 <filename>recipetool create</filename> automates creation
1648 of a base recipe given a set of source code files.
1649 As long as you can extract or point to the source files,
1650 the tool will construct a recipe and automatically
1651 configure all pre-build information into the recipe.
1652 For example, suppose you have an application that builds
1653 using Autotools.
1654 Creating the base recipe using
1655 <filename>recipetool</filename> results in a recipe
1656 that has the pre-build dependencies, license requirements,
1657 and checksums configured.
1658 </para>
1659
1660 <para>
1661 To run the tool, you just need to be in your
1662 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
1663 and have sourced the build environment setup script
1664 (i.e.
1665 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink>).
1666 To get help on the tool, use the following command:
1667 <literallayout class='monospaced'>
1668 $ recipetool -h
1669 NOTE: Starting bitbake server...
1670 usage: recipetool [-d] [-q] [--color COLOR] [-h] &lt;subcommand&gt; ...
1671
1672 OpenEmbedded recipe tool
1673
1674 options:
1675 -d, --debug Enable debug output
1676 -q, --quiet Print only errors
1677 --color COLOR Colorize output (where COLOR is auto, always, never)
1678 -h, --help show this help message and exit
1679
1680 subcommands:
1681 create Create a new recipe
1682 newappend Create a bbappend for the specified target in the specified
1683 layer
1684 setvar Set a variable within a recipe
1685 appendfile Create/update a bbappend to replace a target file
1686 appendsrcfiles Create/update a bbappend to add or replace source files
1687 appendsrcfile Create/update a bbappend to add or replace a source file
1688 Use recipetool &lt;subcommand&gt; --help to get help on a specific command
1689 </literallayout>
1690 </para>
1691
1692 <para>
1693 Running
1694 <filename>recipetool create -o</filename>&nbsp;<replaceable>OUTFILE</replaceable>
1695 creates the base recipe and locates it properly in the
1696 layer that contains your source files.
1697 Following are some syntax examples:
1698 </para>
1699
1700 <para>
1701 Use this syntax to generate a recipe based on
1702 <replaceable>source</replaceable>.
1703 Once generated, the recipe resides in the existing source
1704 code layer:
1705 <literallayout class='monospaced'>
1706 recipetool create -o <replaceable>OUTFILE</replaceable>&nbsp;<replaceable>source</replaceable>
1707 </literallayout>
1708 Use this syntax to generate a recipe using code that you
1709 extract from <replaceable>source</replaceable>.
1710 The extracted code is placed in its own layer defined
1711 by <replaceable>EXTERNALSRC</replaceable>.
1712 <literallayout class='monospaced'>
1713 recipetool create -o <replaceable>OUTFILE</replaceable> -x <replaceable>EXTERNALSRC</replaceable> <replaceable>source</replaceable>
1714 </literallayout>
1715 Use this syntax to generate a recipe based on
1716 <replaceable>source</replaceable>.
1717 The options direct <filename>recipetool</filename> to
1718 generate debugging information.
1719 Once generated, the recipe resides in the existing source
1720 code layer:
1721 <literallayout class='monospaced'>
1722 recipetool create -d -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable>
1723 </literallayout>
1724 </para>
1725 </section>
1726
1727 <section id='new-recipe-locating-and-using-a-similar-recipe'>
1728 <title>Locating and Using a Similar Recipe</title>
1729
1730 <para>
1731 Before writing a recipe from scratch, it is often useful to
1732 discover whether someone else has already written one that
1733 meets (or comes close to meeting) your needs.
1734 The Yocto Project and OpenEmbedded communities maintain many
1735 recipes that might be candidates for what you are doing.
1736 You can find a good central index of these recipes in the
1737 <ulink url='http://layers.openembedded.org'>OpenEmbedded Layer Index</ulink>.
1738 </para>
1739
1740 <para>
1741 Working from an existing recipe or a skeleton recipe is the
1742 best way to get started.
1743 Here are some points on both methods:
1744 <itemizedlist>
1745 <listitem><para><emphasis>Locate and modify a recipe that
1746 is close to what you want to do:</emphasis>
1747 This method works when you are familiar with the
1748 current recipe space.
1749 The method does not work so well for those new to
1750 the Yocto Project or writing recipes.</para>
1751 <para>Some risks associated with this method are
1752 using a recipe that has areas totally unrelated to
1753 what you are trying to accomplish with your recipe,
1754 not recognizing areas of the recipe that you might
1755 have to add from scratch, and so forth.
1756 All these risks stem from unfamiliarity with the
1757 existing recipe space.</para></listitem>
1758 <listitem><para><emphasis>Use and modify the following
1759 skeleton recipe:</emphasis>
1760 If for some reason you do not want to use
1761 <filename>recipetool</filename> and you cannot
1762 find an existing recipe that is close to meeting
1763 your needs, you can use the following structure to
1764 provide the fundamental areas of a new recipe.
1765 <literallayout class='monospaced'>
1766 DESCRIPTION = ""
1767 HOMEPAGE = ""
1768 LICENSE = ""
1769 SECTION = ""
1770 DEPENDS = ""
1771 LIC_FILES_CHKSUM = ""
1772
1773 SRC_URI = ""
1774 </literallayout>
1775 </para></listitem>
1776 </itemizedlist>
1777 </para>
1778 </section>
1779 </section>
1780
1781 <section id='new-recipe-storing-and-naming-the-recipe'>
1782 <title>Storing and Naming the Recipe</title>
1783
1784 <para>
1785 Once you have your base recipe, you should put it in your
1786 own layer and name it appropriately.
1787 Locating it correctly ensures that the OpenEmbedded build
1788 system can find it when you use BitBake to process the
1789 recipe.
1790 </para>
1791
1792 <itemizedlist>
1793 <listitem><para><emphasis>Storing Your Recipe:</emphasis>
1794 The OpenEmbedded build system locates your recipe
1795 through the layer's <filename>conf/layer.conf</filename>
1796 file and the
1797 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink>
1798 variable.
1799 This variable sets up a path from which the build system can
1800 locate recipes.
1801 Here is the typical use:
1802 <literallayout class='monospaced'>
1803 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
1804 ${LAYERDIR}/recipes-*/*/*.bbappend"
1805 </literallayout>
1806 Consequently, you need to be sure you locate your new recipe
1807 inside your layer such that it can be found.</para>
1808 <para>You can find more information on how layers are
1809 structured in the
1810 "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>"
1811 section.</para></listitem>
1812 <listitem><para><emphasis>Naming Your Recipe:</emphasis>
1813 When you name your recipe, you need to follow this naming
1814 convention:
1815 <literallayout class='monospaced'>
1816 <replaceable>basename</replaceable>_<replaceable>version</replaceable>.bb
1817 </literallayout>
1818 Use lower-cased characters and do not include the reserved
1819 suffixes <filename>-native</filename>,
1820 <filename>-cross</filename>, <filename>-initial</filename>,
1821 or <filename>-dev</filename> casually (i.e. do not use them
1822 as part of your recipe name unless the string applies).
1823 Here are some examples:
1824 <literallayout class='monospaced'>
1825 cups_1.7.0.bb
1826 gawk_4.0.2.bb
1827 irssi_0.8.16-rc1.bb
1828 </literallayout></para></listitem>
1829 </itemizedlist>
1830 </section>
1831
1832 <section id='new-recipe-running-a-build-on-the-recipe'>
1833 <title>Running a Build on the Recipe</title>
1834
1835 <para>
1836 Creating a new recipe is usually an iterative process that
1837 requires using BitBake to process the recipe multiple times in
1838 order to progressively discover and add information to the
1839 recipe file.
1840 </para>
1841
1842 <para>
1843 Assuming you have sourced the build environment setup script (i.e.
1844 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>)
1845 and you are in the
1846 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>,
1847 use BitBake to process your recipe.
1848 All you need to provide is the
1849 <filename><replaceable>basename</replaceable></filename> of the recipe as described
1850 in the previous section:
1851 <literallayout class='monospaced'>
1852 $ bitbake <replaceable>basename</replaceable>
1853 </literallayout>
1854
1855 </para>
1856
1857 <para>
1858 During the build, the OpenEmbedded build system creates a
1859 temporary work directory for each recipe
1860 (<filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>)
1861 where it keeps extracted source files, log files, intermediate
1862 compilation and packaging files, and so forth.
1863 </para>
1864
1865 <para>
1866 The path to the per-recipe temporary work directory depends
1867 on the context in which it is being built.
1868 The quickest way to find this path is to have BitBake return it
1869 by running the following:
1870 <literallayout class='monospaced'>
1871 $ bitbake -e <replaceable>basename</replaceable> | grep ^WORKDIR=
1872 </literallayout>
1873 As an example, assume a Source Directory top-level folder named
1874 <filename>poky</filename>, a default Build Directory at
1875 <filename>poky/build</filename>, and a
1876 <filename>qemux86-poky-linux</filename> machine target system.
1877 Furthermore, suppose your recipe is named
1878 <filename>foo_1.3.0.bb</filename>.
1879 In this case, the work directory the build system uses to
1880 build the package would be as follows:
1881 <literallayout class='monospaced'>
1882 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
1883 </literallayout>
1884 Inside this directory you can find sub-directories such as
1885 <filename>image</filename>, <filename>packages-split</filename>,
1886 and <filename>temp</filename>.
1887 After the build, you can examine these to determine how well
1888 the build went.
1889 <note>
1890 You can find log files for each task in the recipe's
1891 <filename>temp</filename> directory (e.g.
1892 <filename>poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp</filename>).
1893 Log files are named <filename>log.<replaceable>taskname</replaceable></filename>
1894 (e.g. <filename>log.do_configure</filename>,
1895 <filename>log.do_fetch</filename>, and
1896 <filename>log.do_compile</filename>).
1897 </note>
1898 </para>
1899
1900 <para>
1901 You can find more information about the build process in
1902 "<ulink url='&YOCTO_DOCS_OM_URL;#overview-development-environment'>The Yocto Project Development Environment</ulink>"
1903 chapter of the Yocto Project Overview and Concepts Manual.
1904 </para>
1905 </section>
1906
1907 <section id='new-recipe-fetching-code'>
1908 <title>Fetching Code</title>
1909
1910 <para>
1911 The first thing your recipe must do is specify how to fetch
1912 the source files.
1913 Fetching is controlled mainly through the
1914 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
1915 variable.
1916 Your recipe must have a <filename>SRC_URI</filename> variable
1917 that points to where the source is located.
1918 For a graphical representation of source locations, see the
1919 "<ulink url='&YOCTO_DOCS_OM_URL;#sources-dev-environment'>Sources</ulink>"
1920 section in the Yocto Project Overview and Concepts Manual.
1921 </para>
1922
1923 <para>
1924 The
1925 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>
1926 task uses the prefix of each entry in the
1927 <filename>SRC_URI</filename> variable value to determine which
1928 <ulink url='&YOCTO_DOCS_BB_URL;#bb-fetchers'>fetcher</ulink>
1929 to use to get your source files.
1930 It is the <filename>SRC_URI</filename> variable that triggers
1931 the fetcher.
1932 The
1933 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
1934 task uses the variable after source is fetched to apply
1935 patches.
1936 The OpenEmbedded build system uses
1937 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES'><filename>FILESOVERRIDES</filename></ulink>
1938 for scanning directory locations for local files in
1939 <filename>SRC_URI</filename>.
1940 </para>
1941
1942 <para>
1943 The <filename>SRC_URI</filename> variable in your recipe must
1944 define each unique location for your source files.
1945 It is good practice to not hard-code version numbers in a URL used
1946 in <filename>SRC_URI</filename>.
1947 Rather than hard-code these values, use
1948 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>,
1949 which causes the fetch process to use the version specified in
1950 the recipe filename.
1951 Specifying the version in this manner means that upgrading the
1952 recipe to a future version is as simple as renaming the recipe
1953 to match the new version.
1954 </para>
1955
1956 <para>
1957 Here is a simple example from the
1958 <filename>meta/recipes-devtools/strace/strace_5.5.bb</filename>
1959 recipe where the source comes from a single tarball.
1960 Notice the use of the
1961 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
1962 variable:
1963 <literallayout class='monospaced'>
1964 SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \
1965 </literallayout>
1966 </para>
1967
1968 <para>
1969 Files mentioned in <filename>SRC_URI</filename> whose names end
1970 in a typical archive extension (e.g. <filename>.tar</filename>,
1971 <filename>.tar.gz</filename>, <filename>.tar.bz2</filename>,
1972 <filename>.zip</filename>, and so forth), are automatically
1973 extracted during the
1974 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
1975 task.
1976 For another example that specifies these types of files, see
1977 the
1978 "<link linkend='new-recipe-autotooled-package'>Autotooled Package</link>"
1979 section.
1980 </para>
1981
1982 <para>
1983 Another way of specifying source is from an SCM.
1984 For Git repositories, you must specify
1985 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
1986 and you should specify
1987 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
1988 to include the revision with
1989 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>.
1990 Here is an example from the recipe
1991 <filename>meta/recipes-kernel/blktrace/blktrace_git.bb</filename>:
1992 <literallayout class='monospaced'>
1993 SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385"
1994
1995 PR = "r6"
1996 PV = "1.0.5+git${SRCPV}"
1997
1998 SRC_URI = "git://git.kernel.dk/blktrace.git \
1999 file://ldflags.patch"
2000 </literallayout>
2001 </para>
2002
2003 <para>
2004 If your <filename>SRC_URI</filename> statement includes
2005 URLs pointing to individual files fetched from a remote server
2006 other than a version control system, BitBake attempts to
2007 verify the files against checksums defined in your recipe to
2008 ensure they have not been tampered with or otherwise modified
2009 since the recipe was written.
2010 Two checksums are used:
2011 <filename>SRC_URI[md5sum]</filename> and
2012 <filename>SRC_URI[sha256sum]</filename>.
2013 </para>
2014
2015 <para>
2016 If your <filename>SRC_URI</filename> variable points to
2017 more than a single URL (excluding SCM URLs), you need to
2018 provide the <filename>md5</filename> and
2019 <filename>sha256</filename> checksums for each URL.
2020 For these cases, you provide a name for each URL as part of
2021 the <filename>SRC_URI</filename> and then reference that name
2022 in the subsequent checksum statements.
2023 Here is an example combining lines from the files
2024 <filename>git.inc</filename> and
2025 <filename>git_2.24.1.bb</filename>:
2026 <literallayout class='monospaced'>
2027 SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \
2028 ${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages"
2029
2030 SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b"
2031 SRC_URI[tarball.sha256sum] = "ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02"
2032 SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d"
2033 SRC_URI[manpages.sha256sum] = "9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230"
2034 </literallayout>
2035 </para>
2036
2037 <para>
2038 Proper values for <filename>md5</filename> and
2039 <filename>sha256</filename> checksums might be available
2040 with other signatures on the download page for the upstream
2041 source (e.g. <filename>md5</filename>,
2042 <filename>sha1</filename>, <filename>sha256</filename>,
2043 <filename>GPG</filename>, and so forth).
2044 Because the OpenEmbedded build system only deals with
2045 <filename>sha256sum</filename> and <filename>md5sum</filename>,
2046 you should verify all the signatures you find by hand.
2047 </para>
2048
2049 <para>
2050 If no <filename>SRC_URI</filename> checksums are specified
2051 when you attempt to build the recipe, or you provide an
2052 incorrect checksum, the build will produce an error for each
2053 missing or incorrect checksum.
2054 As part of the error message, the build system provides
2055 the checksum string corresponding to the fetched file.
2056 Once you have the correct checksums, you can copy and paste
2057 them into your recipe and then run the build again to continue.
2058 <note>
2059 As mentioned, if the upstream source provides signatures
2060 for verifying the downloaded source code, you should
2061 verify those manually before setting the checksum values
2062 in the recipe and continuing with the build.
2063 </note>
2064 </para>
2065
2066 <para>
2067 This final example is a bit more complicated and is from the
2068 <filename>meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb</filename>
2069 recipe.
2070 The example's <filename>SRC_URI</filename> statement identifies
2071 multiple files as the source files for the recipe: a tarball, a
2072 patch file, a desktop file, and an icon.
2073 <literallayout class='monospaced'>
2074 SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \
2075 file://xwc.patch \
2076 file://rxvt.desktop \
2077 file://rxvt.png"
2078 </literallayout>
2079 </para>
2080
2081 <para>
2082 When you specify local files using the
2083 <filename>file://</filename> URI protocol, the build system
2084 fetches files from the local machine.
2085 The path is relative to the
2086 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
2087 variable and searches specific directories in a certain order:
2088 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink><filename>}</filename>,
2089 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}</filename>,
2090 and <filename>files</filename>.
2091 The directories are assumed to be subdirectories of the
2092 directory in which the recipe or append file resides.
2093 For another example that specifies these types of files, see the
2094 "<link linkend='new-recipe-single-c-file-package-hello-world'>Single .c File Package (Hello World!)</link>"
2095 section.
2096 </para>
2097
2098 <para>
2099 The previous example also specifies a patch file.
2100 Patch files are files whose names usually end in
2101 <filename>.patch</filename> or <filename>.diff</filename> but
2102 can end with compressed suffixes such as
2103 <filename>diff.gz</filename> and
2104 <filename>patch.bz2</filename>, for example.
2105 The build system automatically applies patches as described
2106 in the
2107 "<link linkend='new-recipe-patching-code'>Patching Code</link>" section.
2108 </para>
2109 </section>
2110
2111 <section id='new-recipe-unpacking-code'>
2112 <title>Unpacking Code</title>
2113
2114 <para>
2115 During the build, the
2116 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
2117 task unpacks the source with
2118 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>
2119 pointing to where it is unpacked.
2120 </para>
2121
2122 <para>
2123 If you are fetching your source files from an upstream source
2124 archived tarball and the tarball's internal structure matches
2125 the common convention of a top-level subdirectory named
2126 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}-${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>,
2127 then you do not need to set <filename>S</filename>.
2128 However, if <filename>SRC_URI</filename> specifies to fetch
2129 source from an archive that does not use this convention,
2130 or from an SCM like Git or Subversion, your recipe needs to
2131 define <filename>S</filename>.
2132 </para>
2133
2134 <para>
2135 If processing your recipe using BitBake successfully unpacks
2136 the source files, you need to be sure that the directory
2137 pointed to by <filename>${S}</filename> matches the structure
2138 of the source.
2139 </para>
2140 </section>
2141
2142 <section id='new-recipe-patching-code'>
2143 <title>Patching Code</title>
2144
2145 <para>
2146 Sometimes it is necessary to patch code after it has been
2147 fetched.
2148 Any files mentioned in <filename>SRC_URI</filename> whose
2149 names end in <filename>.patch</filename> or
2150 <filename>.diff</filename> or compressed versions of these
2151 suffixes (e.g. <filename>diff.gz</filename> are treated as
2152 patches.
2153 The
2154 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
2155 task automatically applies these patches.
2156 </para>
2157
2158 <para>
2159 The build system should be able to apply patches with the "-p1"
2160 option (i.e. one directory level in the path will be stripped
2161 off).
2162 If your patch needs to have more directory levels stripped off,
2163 specify the number of levels using the "striplevel" option in
2164 the <filename>SRC_URI</filename> entry for the patch.
2165 Alternatively, if your patch needs to be applied in a specific
2166 subdirectory that is not specified in the patch file, use the
2167 "patchdir" option in the entry.
2168 </para>
2169
2170 <para>
2171 As with all local files referenced in
2172 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
2173 using <filename>file://</filename>, you should place
2174 patch files in a directory next to the recipe either
2175 named the same as the base name of the recipe
2176 (<ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink>
2177 and
2178 <ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink>)
2179 or "files".
2180 </para>
2181 </section>
2182
2183 <section id='new-recipe-licensing'>
2184 <title>Licensing</title>
2185
2186 <para>
2187 Your recipe needs to have both the
2188 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
2189 and
2190 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
2191 variables:
2192 <itemizedlist>
2193 <listitem><para><emphasis><filename>LICENSE</filename>:</emphasis>
2194 This variable specifies the license for the software.
2195 If you do not know the license under which the software
2196 you are building is distributed, you should go to the
2197 source code and look for that information.
2198 Typical files containing this information include
2199 <filename>COPYING</filename>,
2200 <filename>LICENSE</filename>, and
2201 <filename>README</filename> files.
2202 You could also find the information near the top of
2203 a source file.
2204 For example, given a piece of software licensed under
2205 the GNU General Public License version 2, you would
2206 set <filename>LICENSE</filename> as follows:
2207 <literallayout class='monospaced'>
2208 LICENSE = "GPLv2"
2209 </literallayout></para>
2210 <para>The licenses you specify within
2211 <filename>LICENSE</filename> can have any name as long
2212 as you do not use spaces, since spaces are used as
2213 separators between license names.
2214 For standard licenses, use the names of the files in
2215 <filename>meta/files/common-licenses/</filename>
2216 or the <filename>SPDXLICENSEMAP</filename> flag names
2217 defined in <filename>meta/conf/licenses.conf</filename>.
2218 </para></listitem>
2219 <listitem><para><emphasis><filename>LIC_FILES_CHKSUM</filename>:</emphasis>
2220 The OpenEmbedded build system uses this variable to
2221 make sure the license text has not changed.
2222 If it has, the build produces an error and it affords
2223 you the chance to figure it out and correct the problem.
2224 </para>
2225 <para>You need to specify all applicable licensing
2226 files for the software.
2227 At the end of the configuration step, the build process
2228 will compare the checksums of the files to be sure
2229 the text has not changed.
2230 Any differences result in an error with the message
2231 containing the current checksum.
2232 For more explanation and examples of how to set the
2233 <filename>LIC_FILES_CHKSUM</filename> variable, see the
2234 "<link link='usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</link>"
2235 section.</para>
2236
2237 <para>To determine the correct checksum string, you
2238 can list the appropriate files in the
2239 <filename>LIC_FILES_CHKSUM</filename> variable with
2240 incorrect md5 strings, attempt to build the software,
2241 and then note the resulting error messages that will
2242 report the correct md5 strings.
2243 See the
2244 "<link linkend='new-recipe-fetching-code'>Fetching Code</link>"
2245 section for additional information.
2246 </para>
2247
2248 <para>
2249 Here is an example that assumes the software has a
2250 <filename>COPYING</filename> file:
2251 <literallayout class='monospaced'>
2252 LIC_FILES_CHKSUM = "file://COPYING;md5=xxx"
2253 </literallayout>
2254 When you try to build the software, the build system
2255 will produce an error and give you the correct string
2256 that you can substitute into the recipe file for a
2257 subsequent build.
2258 </para></listitem>
2259 </itemizedlist>
2260 </para>
2261
2262<!--
2263
2264 <para>
2265 For trying this out I created a new recipe named
2266 <filename>htop_1.0.2.bb</filename> and put it in
2267 <filename>poky/meta/recipes-extended/htop</filename>.
2268 There are two license type statements in my very simple
2269 recipe:
2270 <literallayout class='monospaced'>
2271 LICENSE = ""
2272
2273 LIC_FILES_CHKSUM = ""
2274
2275 SRC_URI[md5sum] = ""
2276 SRC_URI[sha256sum] = ""
2277 </literallayout>
2278 Evidently, you need to run a <filename>bitbake -c cleanall htop</filename>.
2279 Next, you delete or comment out the two <filename>SRC_URI</filename>
2280 lines at the end and then attempt to build the software with
2281 <filename>bitbake htop</filename>.
2282 Doing so causes BitBake to report some errors and and give
2283 you the actual strings you need for the last two
2284 <filename>SRC_URI</filename> lines.
2285 Prior to this, you have to dig around in the home page of the
2286 source for <filename>htop</filename> and determine that the
2287 software is released under GPLv2.
2288 You can provide that in the <filename>LICENSE</filename>
2289 statement.
2290 Now you edit your recipe to have those two strings for
2291 the <filename>SRC_URI</filename> statements:
2292 <literallayout class='monospaced'>
2293 LICENSE = "GPLv2"
2294
2295 LIC_FILES_CHKSUM = ""
2296
2297 SRC_URI = "${SOURCEFORGE_MIRROR}/htop/htop-${PV}.tar.gz"
2298 SRC_URI[md5sum] = "0d01cca8df3349c74569cefebbd9919e"
2299 SRC_URI[sha256sum] = "ee60657b044ece0df096c053060df7abf3cce3a568ab34d260049e6a37ccd8a1"
2300 </literallayout>
2301 At this point, you can build the software again using the
2302 <filename>bitbake htop</filename> command.
2303 There is just a set of errors now associated with the
2304 empty <filename>LIC_FILES_CHKSUM</filename> variable now.
2305 </para>
2306-->
2307
2308 </section>
2309
2310 <section id='new-dependencies'>
2311 <title>Dependencies</title>
2312
2313 <para>
2314 Most software packages have a short list of other packages
2315 that they require, which are called dependencies.
2316 These dependencies fall into two main categories: build-time
2317 dependencies, which are required when the software is built;
2318 and runtime dependencies, which are required to be installed
2319 on the target in order for the software to run.
2320 </para>
2321
2322 <para>
2323 Within a recipe, you specify build-time dependencies using the
2324 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
2325 variable.
2326 Although nuances exist, items specified in
2327 <filename>DEPENDS</filename> should be names of other recipes.
2328 It is important that you specify all build-time dependencies
2329 explicitly.
2330 If you do not, due to the parallel nature of BitBake's
2331 execution, you can end up with a race condition where the
2332 dependency is present for one task of a recipe (e.g.
2333 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>)
2334 and then gone when the next task runs (e.g.
2335 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>).
2336 </para>
2337
2338 <para>
2339 Another consideration is that configure scripts might
2340 automatically check for optional dependencies and enable
2341 corresponding functionality if those dependencies are found.
2342 This behavior means that to ensure deterministic results and
2343 thus avoid more race conditions, you need to either explicitly
2344 specify these dependencies as well, or tell the configure
2345 script explicitly to disable the functionality.
2346 If you wish to make a recipe that is more generally useful
2347 (e.g. publish the recipe in a layer for others to use),
2348 instead of hard-disabling the functionality, you can use the
2349 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG'><filename>PACKAGECONFIG</filename></ulink>
2350 variable to allow functionality and the corresponding
2351 dependencies to be enabled and disabled easily by other
2352 users of the recipe.
2353 </para>
2354
2355 <para>
2356 Similar to build-time dependencies, you specify runtime
2357 dependencies through a variable -
2358 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
2359 which is package-specific.
2360 All variables that are package-specific need to have the name
2361 of the package added to the end as an override.
2362 Since the main package for a recipe has the same name as the
2363 recipe, and the recipe's name can be found through the
2364 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
2365 variable, then you specify the dependencies for the main
2366 package by setting <filename>RDEPENDS_${PN}</filename>.
2367 If the package were named <filename>${PN}-tools</filename>,
2368 then you would set <filename>RDEPENDS_${PN}-tools</filename>,
2369 and so forth.
2370 </para>
2371
2372 <para>
2373 Some runtime dependencies will be set automatically at
2374 packaging time.
2375 These dependencies include any shared library dependencies
2376 (i.e. if a package "example" contains "libexample" and
2377 another package "mypackage" contains a binary that links to
2378 "libexample" then the OpenEmbedded build system will
2379 automatically add a runtime dependency to "mypackage" on
2380 "example").
2381 See the
2382 "<ulink url='&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>"
2383 section in the Yocto Project Overview and Concepts Manual for
2384 further details.
2385 </para>
2386 </section>
2387
2388 <section id='new-recipe-configuring-the-recipe'>
2389 <title>Configuring the Recipe</title>
2390
2391 <para>
2392 Most software provides some means of setting build-time
2393 configuration options before compilation.
2394 Typically, setting these options is accomplished by running a
2395 configure script with options, or by modifying a build
2396 configuration file.
2397 <note>
2398 As of Yocto Project Release 1.7, some of the core recipes
2399 that package binary configuration scripts now disable the
2400 scripts due to the scripts previously requiring error-prone
2401 path substitution.
2402 The OpenEmbedded build system uses
2403 <filename>pkg-config</filename> now, which is much more
2404 robust.
2405 You can find a list of the <filename>*-config</filename>
2406 scripts that are disabled list in the
2407 "<ulink url='&YOCTO_DOCS_REF_URL;#migration-1.7-binary-configuration-scripts-disabled'>Binary Configuration Scripts Disabled</ulink>"
2408 section in the Yocto Project Reference Manual.
2409 </note>
2410 </para>
2411
2412 <para>
2413 A major part of build-time configuration is about checking for
2414 build-time dependencies and possibly enabling optional
2415 functionality as a result.
2416 You need to specify any build-time dependencies for the
2417 software you are building in your recipe's
2418 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
2419 value, in terms of other recipes that satisfy those
2420 dependencies.
2421 You can often find build-time or runtime
2422 dependencies described in the software's documentation.
2423 </para>
2424
2425 <para>
2426 The following list provides configuration items of note based
2427 on how your software is built:
2428 <itemizedlist>
2429 <listitem><para><emphasis>Autotools:</emphasis>
2430 If your source files have a
2431 <filename>configure.ac</filename> file, then your
2432 software is built using Autotools.
2433 If this is the case, you just need to worry about
2434 modifying the configuration.</para>
2435
2436 <para>When using Autotools, your recipe needs to inherit
2437 the
2438 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
2439 class and your recipe does not have to contain a
2440 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2441 task.
2442 However, you might still want to make some adjustments.
2443 For example, you can set
2444 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink>
2445 or
2446 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink>
2447 to pass any needed configure options that are specific
2448 to the recipe.
2449 </para></listitem>
2450 <listitem><para><emphasis>CMake:</emphasis>
2451 If your source files have a
2452 <filename>CMakeLists.txt</filename> file, then your
2453 software is built using CMake.
2454 If this is the case, you just need to worry about
2455 modifying the configuration.</para>
2456
2457 <para>When you use CMake, your recipe needs to inherit
2458 the
2459 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink>
2460 class and your recipe does not have to contain a
2461 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2462 task.
2463 You can make some adjustments by setting
2464 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE'><filename>EXTRA_OECMAKE</filename></ulink>
2465 to pass any needed configure options that are specific
2466 to the recipe.
2467 <note>
2468 If you need to install one or more custom CMake
2469 toolchain files that are supplied by the
2470 application you are building, install the files to
2471 <filename>${D}${datadir}/cmake/</filename> Modules
2472 during
2473 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>.
2474 </note>
2475 </para></listitem>
2476 <listitem><para><emphasis>Other:</emphasis>
2477 If your source files do not have a
2478 <filename>configure.ac</filename> or
2479 <filename>CMakeLists.txt</filename> file, then your
2480 software is built using some method other than Autotools
2481 or CMake.
2482 If this is the case, you normally need to provide a
2483 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2484 task in your recipe
2485 unless, of course, there is nothing to configure.
2486 </para>
2487 <para>Even if your software is not being built by
2488 Autotools or CMake, you still might not need to deal
2489 with any configuration issues.
2490 You need to determine if configuration is even a required step.
2491 You might need to modify a Makefile or some configuration file
2492 used for the build to specify necessary build options.
2493 Or, perhaps you might need to run a provided, custom
2494 configure script with the appropriate options.</para>
2495 <para>For the case involving a custom configure
2496 script, you would run
2497 <filename>./configure --help</filename> and look for
2498 the options you need to set.</para></listitem>
2499 </itemizedlist>
2500 </para>
2501
2502 <para>
2503 Once configuration succeeds, it is always good practice to
2504 look at the <filename>log.do_configure</filename> file to
2505 ensure that the appropriate options have been enabled and no
2506 additional build-time dependencies need to be added to
2507 <filename>DEPENDS</filename>.
2508 For example, if the configure script reports that it found
2509 something not mentioned in <filename>DEPENDS</filename>, or
2510 that it did not find something that it needed for some
2511 desired optional functionality, then you would need to add
2512 those to <filename>DEPENDS</filename>.
2513 Looking at the log might also reveal items being checked for,
2514 enabled, or both that you do not want, or items not being found
2515 that are in <filename>DEPENDS</filename>, in which case
2516 you would need to look at passing extra options to the
2517 configure script as needed.
2518 For reference information on configure options specific to the
2519 software you are building, you can consult the output of the
2520 <filename>./configure --help</filename> command within
2521 <filename>${S}</filename> or consult the software's upstream
2522 documentation.
2523 </para>
2524 </section>
2525
2526 <section id='new-recipe-using-headers-to-interface-with-devices'>
2527 <title>Using Headers to Interface with Devices</title>
2528
2529 <para>
2530 If your recipe builds an application that needs to
2531 communicate with some device or needs an API into a custom
2532 kernel, you will need to provide appropriate header files.
2533 Under no circumstances should you ever modify the existing
2534 <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename>
2535 file.
2536 These headers are used to build <filename>libc</filename> and
2537 must not be compromised with custom or machine-specific
2538 header information.
2539 If you customize <filename>libc</filename> through modified
2540 headers all other applications that use
2541 <filename>libc</filename> thus become affected.
2542 <note><title>Warning</title>
2543 Never copy and customize the <filename>libc</filename>
2544 header file (i.e.
2545 <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename>).
2546 </note>
2547 The correct way to interface to a device or custom kernel is
2548 to use a separate package that provides the additional headers
2549 for the driver or other unique interfaces.
2550 When doing so, your application also becomes responsible for
2551 creating a dependency on that specific provider.
2552 </para>
2553
2554 <para>
2555 Consider the following:
2556 <itemizedlist>
2557 <listitem><para>
2558 Never modify
2559 <filename>linux-libc-headers.inc</filename>.
2560 Consider that file to be part of the
2561 <filename>libc</filename> system, and not something
2562 you use to access the kernel directly.
2563 You should access <filename>libc</filename> through
2564 specific <filename>libc</filename> calls.
2565 </para></listitem>
2566 <listitem><para>
2567 Applications that must talk directly to devices
2568 should either provide necessary headers themselves,
2569 or establish a dependency on a special headers package
2570 that is specific to that driver.
2571 </para></listitem>
2572 </itemizedlist>
2573 </para>
2574
2575 <para>
2576 For example, suppose you want to modify an existing header
2577 that adds I/O control or network support.
2578 If the modifications are used by a small number programs,
2579 providing a unique version of a header is easy and has little
2580 impact.
2581 When doing so, bear in mind the guidelines in the previous
2582 list.
2583 <note>
2584 If for some reason your changes need to modify the behavior
2585 of the <filename>libc</filename>, and subsequently all
2586 other applications on the system, use a
2587 <filename>.bbappend</filename> to modify the
2588 <filename>linux-kernel-headers.inc</filename> file.
2589 However, take care to not make the changes
2590 machine specific.
2591 </note>
2592 </para>
2593
2594 <para>
2595 Consider a case where your kernel is older and you need
2596 an older <filename>libc</filename> ABI.
2597 The headers installed by your recipe should still be a
2598 standard mainline kernel, not your own custom one.
2599 </para>
2600
2601 <para>
2602 When you use custom kernel headers you need to get them from
2603 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_KERNEL_DIR'><filename>STAGING_KERNEL_DIR</filename></ulink>,
2604 which is the directory with kernel headers that are
2605 required to build out-of-tree modules.
2606 Your recipe will also need the following:
2607 <literallayout class='monospaced'>
2608 do_configure[depends] += "virtual/kernel:do_shared_workdir"
2609 </literallayout>
2610 </para>
2611 </section>
2612
2613 <section id='new-recipe-compilation'>
2614 <title>Compilation</title>
2615
2616 <para>
2617 During a build, the <filename>do_compile</filename> task
2618 happens after source is fetched, unpacked, and configured.
2619 If the recipe passes through <filename>do_compile</filename>
2620 successfully, nothing needs to be done.
2621 </para>
2622
2623 <para>
2624 However, if the compile step fails, you need to diagnose the
2625 failure.
2626 Here are some common issues that cause failures.
2627 <note>
2628 For cases where improper paths are detected for
2629 configuration files or for when libraries/headers cannot
2630 be found, be sure you are using the more robust
2631 <filename>pkg-config</filename>.
2632 See the note in section
2633 "<link linkend='new-recipe-configuring-the-recipe'>Configuring the Recipe</link>"
2634 for additional information.
2635 </note>
2636 <itemizedlist>
2637 <listitem><para><emphasis>Parallel build failures:</emphasis>
2638 These failures manifest themselves as intermittent
2639 errors, or errors reporting that a file or directory
2640 that should be created by some other part of the build
2641 process could not be found.
2642 This type of failure can occur even if, upon inspection,
2643 the file or directory does exist after the build has
2644 failed, because that part of the build process happened
2645 in the wrong order.</para>
2646 <para>To fix the problem, you need to either satisfy
2647 the missing dependency in the Makefile or whatever
2648 script produced the Makefile, or (as a workaround)
2649 set
2650 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
2651 to an empty string:
2652 <literallayout class='monospaced'>
2653 PARALLEL_MAKE = ""
2654 </literallayout></para>
2655 <para>
2656 For information on parallel Makefile issues, see the
2657 "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>"
2658 section.
2659 </para></listitem>
2660 <listitem><para><emphasis>Improper host path usage:</emphasis>
2661 This failure applies to recipes building for the target
2662 or <filename>nativesdk</filename> only.
2663 The failure occurs when the compilation process uses
2664 improper headers, libraries, or other files from the
2665 host system when cross-compiling for the target.
2666 </para>
2667 <para>To fix the problem, examine the
2668 <filename>log.do_compile</filename> file to identify
2669 the host paths being used (e.g.
2670 <filename>/usr/include</filename>,
2671 <filename>/usr/lib</filename>, and so forth) and then
2672 either add configure options, apply a patch, or do both.
2673 </para></listitem>
2674 <listitem><para><emphasis>Failure to find required
2675 libraries/headers:</emphasis>
2676 If a build-time dependency is missing because it has
2677 not been declared in
2678 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
2679 or because the dependency exists but the path used by
2680 the build process to find the file is incorrect and the
2681 configure step did not detect it, the compilation
2682 process could fail.
2683 For either of these failures, the compilation process
2684 notes that files could not be found.
2685 In these cases, you need to go back and add additional
2686 options to the configure script as well as possibly
2687 add additional build-time dependencies to
2688 <filename>DEPENDS</filename>.</para>
2689 <para>Occasionally, it is necessary to apply a patch
2690 to the source to ensure the correct paths are used.
2691 If you need to specify paths to find files staged
2692 into the sysroot from other recipes, use the variables
2693 that the OpenEmbedded build system provides
2694 (e.g.
2695 <filename>STAGING_BINDIR</filename>,
2696 <filename>STAGING_INCDIR</filename>,
2697 <filename>STAGING_DATADIR</filename>, and so forth).
2698<!--
2699 (e.g.
2700 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_BINDIR'><filename>STAGING_BINDIR</filename></ulink>,
2701 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_INCDIR'><filename>STAGING_INCDIR</filename></ulink>,
2702 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DATADIR'><filename>STAGING_DATADIR</filename></ulink>,
2703 and so forth).
2704-->
2705 </para></listitem>
2706 </itemizedlist>
2707 </para>
2708 </section>
2709
2710 <section id='new-recipe-installing'>
2711 <title>Installing</title>
2712
2713 <para>
2714 During <filename>do_install</filename>, the task copies the
2715 built files along with their hierarchy to locations that
2716 would mirror their locations on the target device.
2717 The installation process copies files from the
2718 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>,
2719 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink><filename>}</filename>,
2720 and
2721 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>
2722 directories to the
2723 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
2724 directory to create the structure as it should appear on the
2725 target system.
2726 </para>
2727
2728 <para>
2729 How your software is built affects what you must do to be
2730 sure your software is installed correctly.
2731 The following list describes what you must do for installation
2732 depending on the type of build system used by the software
2733 being built:
2734 <itemizedlist>
2735 <listitem><para><emphasis>Autotools and CMake:</emphasis>
2736 If the software your recipe is building uses Autotools
2737 or CMake, the OpenEmbedded build
2738 system understands how to install the software.
2739 Consequently, you do not have to have a
2740 <filename>do_install</filename> task as part of your
2741 recipe.
2742 You just need to make sure the install portion of the
2743 build completes with no issues.
2744 However, if you wish to install additional files not
2745 already being installed by
2746 <filename>make install</filename>, you should do this
2747 using a <filename>do_install_append</filename> function
2748 using the install command as described in
2749 the "Manual" bulleted item later in this list.
2750 </para></listitem>
2751 <listitem><para><emphasis>Other (using
2752 <filename>make install</filename>):</emphasis>
2753 You need to define a
2754 <filename>do_install</filename> function in your
2755 recipe.
2756 The function should call
2757 <filename>oe_runmake install</filename> and will likely
2758 need to pass in the destination directory as well.
2759 How you pass that path is dependent on how the
2760 <filename>Makefile</filename> being run is written
2761 (e.g. <filename>DESTDIR=${D}</filename>,
2762 <filename>PREFIX=${D}</filename>,
2763 <filename>INSTALLROOT=${D}</filename>, and so forth).
2764 </para>
2765 <para>For an example recipe using
2766 <filename>make install</filename>, see the
2767 "<link linkend='new-recipe-makefile-based-package'>Makefile-Based Package</link>"
2768 section.</para></listitem>
2769 <listitem><para><emphasis>Manual:</emphasis>
2770 You need to define a
2771 <filename>do_install</filename> function in your
2772 recipe.
2773 The function must first use
2774 <filename>install -d</filename> to create the
2775 directories under
2776 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>.
2777 Once the directories exist, your function can use
2778 <filename>install</filename> to manually install the
2779 built software into the directories.</para>
2780 <para>You can find more information on
2781 <filename>install</filename> at
2782 <ulink url='http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html'></ulink>.
2783 </para></listitem>
2784 </itemizedlist>
2785 </para>
2786
2787 <para>
2788 For the scenarios that do not use Autotools or
2789 CMake, you need to track the installation
2790 and diagnose and fix any issues until everything installs
2791 correctly.
2792 You need to look in the default location of
2793 <filename>${D}</filename>, which is
2794 <filename>${WORKDIR}/image</filename>, to be sure your
2795 files have been installed correctly.
2796 </para>
2797
2798 <note><title>Notes</title>
2799 <itemizedlist>
2800 <listitem><para>
2801 During the installation process, you might need to
2802 modify some of the installed files to suit the target
2803 layout.
2804 For example, you might need to replace hard-coded paths
2805 in an initscript with values of variables provided by
2806 the build system, such as replacing
2807 <filename>/usr/bin/</filename> with
2808 <filename>${bindir}</filename>.
2809 If you do perform such modifications during
2810 <filename>do_install</filename>, be sure to modify the
2811 destination file after copying rather than before
2812 copying.
2813 Modifying after copying ensures that the build system
2814 can re-execute <filename>do_install</filename> if
2815 needed.
2816 </para></listitem>
2817 <listitem><para>
2818 <filename>oe_runmake install</filename>, which can be
2819 run directly or can be run indirectly by the
2820 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
2821 and
2822 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink>
2823 classes, runs <filename>make install</filename> in
2824 parallel.
2825 Sometimes, a Makefile can have missing dependencies
2826 between targets that can result in race conditions.
2827 If you experience intermittent failures during
2828 <filename>do_install</filename>, you might be able to
2829 work around them by disabling parallel Makefile
2830 installs by adding the following to the recipe:
2831 <literallayout class='monospaced'>
2832 PARALLEL_MAKEINST = ""
2833 </literallayout>
2834 See
2835 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink>
2836 for additional information.
2837 </para></listitem>
2838 <listitem><para>
2839 If you need to install one or more custom CMake
2840 toolchain files that are supplied by the
2841 application you are building, install the files to
2842 <filename>${D}${datadir}/cmake/</filename> Modules
2843 during
2844 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>.
2845 </para></listitem>
2846 </itemizedlist>
2847 </note>
2848 </section>
2849
2850 <section id='new-recipe-enabling-system-services'>
2851 <title>Enabling System Services</title>
2852
2853 <para>
2854 If you want to install a service, which is a process that
2855 usually starts on boot and runs in the background, then
2856 you must include some additional definitions in your recipe.
2857 </para>
2858
2859 <para>
2860 If you are adding services and the service initialization
2861 script or the service file itself is not installed, you must
2862 provide for that installation in your recipe using a
2863 <filename>do_install_append</filename> function.
2864 If your recipe already has a <filename>do_install</filename>
2865 function, update the function near its end rather than
2866 adding an additional <filename>do_install_append</filename>
2867 function.
2868 </para>
2869
2870 <para>
2871 When you create the installation for your services, you need
2872 to accomplish what is normally done by
2873 <filename>make install</filename>.
2874 In other words, make sure your installation arranges the output
2875 similar to how it is arranged on the target system.
2876 </para>
2877
2878 <para>
2879 The OpenEmbedded build system provides support for starting
2880 services two different ways:
2881 <itemizedlist>
2882 <listitem><para><emphasis>SysVinit:</emphasis>
2883 SysVinit is a system and service manager that
2884 manages the init system used to control the very basic
2885 functions of your system.
2886 The init program is the first program
2887 started by the Linux kernel when the system boots.
2888 Init then controls the startup, running and shutdown
2889 of all other programs.</para>
2890 <para>To enable a service using SysVinit, your recipe
2891 needs to inherit the
2892 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d'><filename>update-rc.d</filename></ulink>
2893 class.
2894 The class helps facilitate safely installing the
2895 package on the target.</para>
2896 <para>You will need to set the
2897 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES'><filename>INITSCRIPT_PACKAGES</filename></ulink>,
2898 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME'><filename>INITSCRIPT_NAME</filename></ulink>,
2899 and
2900 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS'><filename>INITSCRIPT_PARAMS</filename></ulink>
2901 variables within your recipe.</para></listitem>
2902 <listitem><para><emphasis>systemd:</emphasis>
2903 System Management Daemon (systemd) was designed to
2904 replace SysVinit and to provide
2905 enhanced management of services.
2906 For more information on systemd, see the systemd
2907 homepage at
2908 <ulink url='http://freedesktop.org/wiki/Software/systemd/'></ulink>.
2909 </para>
2910 <para>To enable a service using systemd, your recipe
2911 needs to inherit the
2912 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-systemd'><filename>systemd</filename></ulink>
2913 class.
2914 See the <filename>systemd.bbclass</filename> file
2915 located in your
2916 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
2917 section for more information.
2918 </para></listitem>
2919 </itemizedlist>
2920 </para>
2921 </section>
2922
2923 <section id='new-recipe-packaging'>
2924 <title>Packaging</title>
2925
2926 <para>
2927 Successful packaging is a combination of automated processes
2928 performed by the OpenEmbedded build system and some
2929 specific steps you need to take.
2930 The following list describes the process:
2931 <itemizedlist>
2932 <listitem><para><emphasis>Splitting Files</emphasis>:
2933 The <filename>do_package</filename> task splits the
2934 files produced by the recipe into logical components.
2935 Even software that produces a single binary might
2936 still have debug symbols, documentation, and other
2937 logical components that should be split out.
2938 The <filename>do_package</filename> task ensures
2939 that files are split up and packaged correctly.
2940 </para></listitem>
2941 <listitem><para><emphasis>Running QA Checks</emphasis>:
2942 The
2943 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink>
2944 class adds a step to
2945 the package generation process so that output quality
2946 assurance checks are generated by the OpenEmbedded
2947 build system.
2948 This step performs a range of checks to be sure the
2949 build's output is free of common problems that show
2950 up during runtime.
2951 For information on these checks, see the
2952 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink>
2953 class and the
2954 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-qa-checks'>QA Error and Warning Messages</ulink>"
2955 chapter in the Yocto Project Reference Manual.
2956 </para></listitem>
2957 <listitem><para><emphasis>Hand-Checking Your Packages</emphasis>:
2958 After you build your software, you need to be sure
2959 your packages are correct.
2960 Examine the
2961 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename>
2962 directory and make sure files are where you expect
2963 them to be.
2964 If you discover problems, you can set
2965 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>,
2966 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>,
2967 <filename>do_install(_append)</filename>, and so forth as
2968 needed.
2969 </para></listitem>
2970 <listitem><para><emphasis>Splitting an Application into Multiple Packages</emphasis>:
2971 If you need to split an application into several
2972 packages, see the
2973 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>"
2974 section for an example.
2975 </para></listitem>
2976 <listitem><para><emphasis>Installing a Post-Installation Script</emphasis>:
2977 For an example showing how to install a
2978 post-installation script, see the
2979 "<link linkend='new-recipe-post-installation-scripts'>Post-Installation Scripts</link>"
2980 section.
2981 </para></listitem>
2982 <listitem><para><emphasis>Marking Package Architecture</emphasis>:
2983 Depending on what your recipe is building and how it
2984 is configured, it might be important to mark the
2985 packages produced as being specific to a particular
2986 machine, or to mark them as not being specific to
2987 a particular machine or architecture at all.</para>
2988 <para>By default, packages apply to any machine with the
2989 same architecture as the target machine.
2990 When a recipe produces packages that are
2991 machine-specific (e.g. the
2992 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
2993 value is passed into the configure script or a patch
2994 is applied only for a particular machine), you should
2995 mark them as such by adding the following to the
2996 recipe:
2997 <literallayout class='monospaced'>
2998 PACKAGE_ARCH = "${MACHINE_ARCH}"
2999 </literallayout></para>
3000 <para>On the other hand, if the recipe produces packages
3001 that do not contain anything specific to the target
3002 machine or architecture at all (e.g. recipes
3003 that simply package script files or configuration
3004 files), you should use the
3005 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
3006 class to do this for you by adding this to your
3007 recipe:
3008 <literallayout class='monospaced'>
3009 inherit allarch
3010 </literallayout>
3011 Ensuring that the package architecture is correct is
3012 not critical while you are doing the first few builds
3013 of your recipe.
3014 However, it is important in order
3015 to ensure that your recipe rebuilds (or does not
3016 rebuild) appropriately in response to changes in
3017 configuration, and to ensure that you get the
3018 appropriate packages installed on the target machine,
3019 particularly if you run separate builds for more
3020 than one target machine.
3021 </para></listitem>
3022 </itemizedlist>
3023 </para>
3024 </section>
3025
3026 <section id='new-sharing-files-between-recipes'>
3027 <title>Sharing Files Between Recipes</title>
3028
3029 <para>
3030 Recipes often need to use files provided by other recipes on
3031 the build host.
3032 For example, an application linking to a common library needs
3033 access to the library itself and its associated headers.
3034 The way this access is accomplished is by populating a sysroot
3035 with files.
3036 Each recipe has two sysroots in its work directory, one for
3037 target files
3038 (<filename>recipe-sysroot</filename>) and one for files that
3039 are native to the build host
3040 (<filename>recipe-sysroot-native</filename>).
3041 <note>
3042 You could find the term "staging" used within the Yocto
3043 project regarding files populating sysroots (e.g. the
3044 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR'><filename>STAGING_DIR</filename></ulink>
3045 variable).
3046 </note>
3047 </para>
3048
3049 <para>
3050 Recipes should never populate the sysroot directly (i.e. write
3051 files into sysroot).
3052 Instead, files should be installed into standard locations
3053 during the
3054 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3055 task within the
3056 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
3057 directory.
3058 The reason for this limitation is that almost all files that
3059 populate the sysroot are cataloged in manifests in order to
3060 ensure the files can be removed later when a recipe is either
3061 modified or removed.
3062 Thus, the sysroot is able to remain free from stale files.
3063 </para>
3064
3065 <para>
3066 A subset of the files installed by the
3067 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3068 task are used by the
3069 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3070 task as defined by the the
3071 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink>
3072 variable to automatically populate the sysroot.
3073 It is possible to modify the list of directories that populate
3074 the sysroot.
3075 The following example shows how you could add the
3076 <filename>/opt</filename> directory to the list of
3077 directories within a recipe:
3078 <literallayout class='monospaced'>
3079 SYSROOT_DIRS += "/opt"
3080 </literallayout>
3081 </para>
3082
3083 <para>
3084 For a more complete description of the
3085 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3086 task and its associated functions, see the
3087 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-staging'><filename>staging</filename></ulink>
3088 class.
3089 </para>
3090 </section>
3091
3092 <section id='metadata-virtual-providers'>
3093 <title>Using Virtual Providers</title>
3094
3095 <para>
3096 Prior to a build, if you know that several different recipes
3097 provide the same functionality, you can use a virtual provider
3098 (i.e. <filename>virtual/*</filename>) as a placeholder for the
3099 actual provider.
3100 The actual provider is determined at build-time.
3101 </para>
3102
3103 <para>
3104 A common scenario where a virtual provider is used would be
3105 for the kernel recipe.
3106 Suppose you have three kernel recipes whose
3107 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>
3108 values map to <filename>kernel-big</filename>,
3109 <filename>kernel-mid</filename>, and
3110 <filename>kernel-small</filename>.
3111 Furthermore, each of these recipes in some way uses a
3112 <ulink url='&YOCTO_DOCS_REF_URL;#var-PROVIDES'><filename>PROVIDES</filename></ulink>
3113 statement that essentially identifies itself as being able
3114 to provide <filename>virtual/kernel</filename>.
3115 Here is one way through the
3116 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-kernel'><filename>kernel</filename></ulink>
3117 class:
3118 <literallayout class='monospaced'>
3119 PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }"
3120 </literallayout>
3121 Any recipe that inherits the <filename>kernel</filename> class
3122 is going to utilize a <filename>PROVIDES</filename> statement
3123 that identifies that recipe as being able to provide the
3124 <filename>virtual/kernel</filename> item.
3125 </para>
3126
3127 <para>
3128 Now comes the time to actually build an image and you need a
3129 kernel recipe, but which one?
3130 You can configure your build to call out the kernel recipe
3131 you want by using the
3132 <ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'><filename>PREFERRED_PROVIDER</filename></ulink>
3133 variable.
3134 As an example, consider the
3135 <ulink url='https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc'><filename>x86-base.inc</filename></ulink>
3136 include file, which is a machine
3137 (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>)
3138 configuration file.
3139 This include file is the reason all x86-based machines use the
3140 <filename>linux-yocto</filename> kernel.
3141 Here are the relevant lines from the include file:
3142 <literallayout class='monospaced'>
3143 PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto"
3144 PREFERRED_VERSION_linux-yocto ??= "4.15%"
3145 </literallayout>
3146 </para>
3147
3148 <para>
3149 When you use a virtual provider, you do not have to
3150 "hard code" a recipe name as a build dependency.
3151 You can use the
3152 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
3153 variable to state the build is dependent on
3154 <filename>virtual/kernel</filename> for example:
3155 <literallayout class='monospaced'>
3156 DEPENDS = "virtual/kernel"
3157 </literallayout>
3158 During the build, the OpenEmbedded build system picks
3159 the correct recipe needed for the
3160 <filename>virtual/kernel</filename> dependency based on the
3161 <filename>PREFERRED_PROVIDER</filename> variable.
3162 If you want to use the small kernel mentioned at the beginning
3163 of this section, configure your build as follows:
3164 <literallayout class='monospaced'>
3165 PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small"
3166 </literallayout>
3167 <note>
3168 Any recipe that
3169 <ulink url='&YOCTO_DOCS_REF_URL;#var-PROVIDES'><filename>PROVIDES</filename></ulink>
3170 a <filename>virtual/*</filename> item that is ultimately
3171 not selected through
3172 <filename>PREFERRED_PROVIDER</filename> does not get built.
3173 Preventing these recipes from building is usually the
3174 desired behavior since this mechanism's purpose is to
3175 select between mutually exclusive alternative providers.
3176 </note>
3177 </para>
3178
3179 <para>
3180 The following lists specific examples of virtual providers:
3181 <itemizedlist>
3182 <listitem><para>
3183 <filename>virtual/kernel</filename>:
3184 Provides the name of the kernel recipe to use when
3185 building a kernel image.
3186 </para></listitem>
3187 <listitem><para>
3188 <filename>virtual/bootloader</filename>:
3189 Provides the name of the bootloader to use when
3190 building an image.
3191 </para></listitem>
3192 <listitem><para>
3193 <filename>virtual/libgbm</filename>:
3194 Provides <filename>gbm.pc</filename>.
3195 </para></listitem>
3196 <listitem><para>
3197 <filename>virtual/egl</filename>:
3198 Provides <filename>egl.pc</filename> and possibly
3199 <filename>wayland-egl.pc</filename>.
3200 </para></listitem>
3201 <listitem><para>
3202 <filename>virtual/libgl</filename>:
3203 Provides <filename>gl.pc</filename> (i.e. libGL).
3204 </para></listitem>
3205 <listitem><para>
3206 <filename>virtual/libgles1</filename>:
3207 Provides <filename>glesv1_cm.pc</filename>
3208 (i.e. libGLESv1_CM).
3209 </para></listitem>
3210 <listitem><para>
3211 <filename>virtual/libgles2</filename>:
3212 Provides <filename>glesv2.pc</filename>
3213 (i.e. libGLESv2).
3214 </para></listitem>
3215 </itemizedlist>
3216 </para>
3217 </section>
3218
3219 <section id='properly-versioning-pre-release-recipes'>
3220 <title>Properly Versioning Pre-Release Recipes</title>
3221
3222 <para>
3223 Sometimes the name of a recipe can lead to versioning
3224 problems when the recipe is upgraded to a final release.
3225 For example, consider the
3226 <filename>irssi_0.8.16-rc1.bb</filename> recipe file in
3227 the list of example recipes in the
3228 "<link linkend='new-recipe-storing-and-naming-the-recipe'>Storing and Naming the Recipe</link>"
3229 section.
3230 This recipe is at a release candidate stage (i.e.
3231 "rc1").
3232 When the recipe is released, the recipe filename becomes
3233 <filename>irssi_0.8.16.bb</filename>.
3234 The version change from <filename>0.8.16-rc1</filename>
3235 to <filename>0.8.16</filename> is seen as a decrease by the
3236 build system and package managers, so the resulting packages
3237 will not correctly trigger an upgrade.
3238 </para>
3239
3240 <para>
3241 In order to ensure the versions compare properly, the
3242 recommended convention is to set
3243 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
3244 within the recipe to
3245 "<replaceable>previous_version</replaceable>+<replaceable>current_version</replaceable>".
3246 You can use an additional variable so that you can use the
3247 current version elsewhere.
3248 Here is an example:
3249 <literallayout class='monospaced'>
3250 REALPV = "0.8.16-rc1"
3251 PV = "0.8.15+${REALPV}"
3252 </literallayout>
3253 </para>
3254 </section>
3255
3256 <section id='new-recipe-post-installation-scripts'>
3257 <title>Post-Installation Scripts</title>
3258
3259 <para>
3260 Post-installation scripts run immediately after installing
3261 a package on the target or during image creation when a
3262 package is included in an image.
3263 To add a post-installation script to a package, add a
3264 <filename>pkg_postinst_</filename><replaceable>PACKAGENAME</replaceable><filename>()</filename> function to
3265 the recipe file (<filename>.bb</filename>) and replace
3266 <replaceable>PACKAGENAME</replaceable> with the name of the package
3267 you want to attach to the <filename>postinst</filename>
3268 script.
3269 To apply the post-installation script to the main package
3270 for the recipe, which is usually what is required, specify
3271 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
3272 in place of <replaceable>PACKAGENAME</replaceable>.
3273 </para>
3274
3275 <para>
3276 A post-installation function has the following structure:
3277 <literallayout class='monospaced'>
3278 pkg_postinst_<replaceable>PACKAGENAME</replaceable>() {
3279 # Commands to carry out
3280 }
3281 </literallayout>
3282 </para>
3283
3284 <para>
3285 The script defined in the post-installation function is
3286 called when the root filesystem is created.
3287 If the script succeeds, the package is marked as installed.
3288 <note>
3289 Any RPM post-installation script that runs on the target
3290 should return a 0 exit code.
3291 RPM does not allow non-zero exit codes for these scripts,
3292 and the RPM package manager will cause the package to fail
3293 installation on the target.
3294 </note>
3295 </para>
3296
3297 <para>
3298 Sometimes it is necessary for the execution of a
3299 post-installation script to be delayed until the first boot.
3300 For example, the script might need to be executed on the
3301 device itself.
3302 To delay script execution until boot time, you must explicitly
3303 mark post installs to defer to the target.
3304 You can use <filename>pkg_postinst_ontarget()</filename> or
3305 call
3306 <filename>postinst_intercept delay_to_first_boot</filename>
3307 from <filename>pkg_postinst()</filename>.
3308 Any failure of a <filename>pkg_postinst()</filename> script
3309 (including exit 1) triggers an error during the
3310 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink>
3311 task.
3312 </para>
3313
3314 <para>
3315 If you have recipes that use
3316 <filename>pkg_postinst</filename> function
3317 and they require the use of non-standard native
3318 tools that have dependencies during rootfs construction, you
3319 need to use the
3320 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_WRITE_DEPS'><filename>PACKAGE_WRITE_DEPS</filename></ulink>
3321 variable in your recipe to list these tools.
3322 If you do not use this variable, the tools might be missing and
3323 execution of the post-installation script is deferred until
3324 first boot.
3325 Deferring the script to first boot is undesirable and for
3326 read-only rootfs impossible.
3327 </para>
3328
3329 <note>
3330 Equivalent support for pre-install, pre-uninstall, and
3331 post-uninstall scripts exist by way of
3332 <filename>pkg_preinst</filename>,
3333 <filename>pkg_prerm</filename>, and
3334 <filename>pkg_postrm</filename>, respectively.
3335 These scrips work in exactly the same way as does
3336 <filename>pkg_postinst</filename> with the exception
3337 that they run at different times.
3338 Also, because of when they run, they are not applicable to
3339 being run at image creation time like
3340 <filename>pkg_postinst</filename>.
3341 </note>
3342 </section>
3343
3344 <section id='new-recipe-testing'>
3345 <title>Testing</title>
3346
3347 <para>
3348 The final step for completing your recipe is to be sure that
3349 the software you built runs correctly.
3350 To accomplish runtime testing, add the build's output
3351 packages to your image and test them on the target.
3352 </para>
3353
3354 <para>
3355 For information on how to customize your image by adding
3356 specific packages, see the
3357 "<link linkend='usingpoky-extend-customimage'>Customizing Images</link>"
3358 section.
3359 </para>
3360 </section>
3361
3362 <section id='new-recipe-testing-examples'>
3363 <title>Examples</title>
3364
3365 <para>
3366 To help summarize how to write a recipe, this section provides
3367 some examples given various scenarios:
3368 <itemizedlist>
3369 <listitem><para>Recipes that use local files</para></listitem>
3370 <listitem><para>Using an Autotooled package</para></listitem>
3371 <listitem><para>Using a Makefile-based package</para></listitem>
3372 <listitem><para>Splitting an application into multiple packages</para></listitem>
3373 <listitem><para>Adding binaries to an image</para></listitem>
3374 </itemizedlist>
3375 </para>
3376
3377 <section id='new-recipe-single-c-file-package-hello-world'>
3378 <title>Single .c File Package (Hello World!)</title>
3379
3380 <para>
3381 Building an application from a single file that is stored
3382 locally (e.g. under <filename>files</filename>) requires
3383 a recipe that has the file listed in the
3384 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>
3385 variable.
3386 Additionally, you need to manually write the
3387 <filename>do_compile</filename> and
3388 <filename>do_install</filename> tasks.
3389 The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
3390 variable defines the directory containing the source code,
3391 which is set to
3392 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
3393 in this case - the directory BitBake uses for the build.
3394 <literallayout class='monospaced'>
3395 SUMMARY = "Simple helloworld application"
3396 SECTION = "examples"
3397 LICENSE = "MIT"
3398 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
3399
3400 SRC_URI = "file://helloworld.c"
3401
3402 S = "${WORKDIR}"
3403
3404 do_compile() {
3405 ${CC} helloworld.c -o helloworld
3406 }
3407
3408 do_install() {
3409 install -d ${D}${bindir}
3410 install -m 0755 helloworld ${D}${bindir}
3411 }
3412 </literallayout>
3413 </para>
3414
3415 <para>
3416 By default, the <filename>helloworld</filename>,
3417 <filename>helloworld-dbg</filename>, and
3418 <filename>helloworld-dev</filename> packages are built.
3419 For information on how to customize the packaging process,
3420 see the
3421 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>"
3422 section.
3423 </para>
3424 </section>
3425
3426 <section id='new-recipe-autotooled-package'>
3427 <title>Autotooled Package</title>
3428 <para>
3429 Applications that use Autotools such as <filename>autoconf</filename> and
3430 <filename>automake</filename> require a recipe that has a source archive listed in
3431 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and
3432 also inherit the
3433 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
3434 class, which contains the definitions of all the steps
3435 needed to build an Autotool-based application.
3436 The result of the build is automatically packaged.
3437 And, if the application uses NLS for localization, packages with local information are
3438 generated (one package per language).
3439 Following is one example: (<filename>hello_2.3.bb</filename>)
3440 <literallayout class='monospaced'>
3441 SUMMARY = "GNU Helloworld application"
3442 SECTION = "examples"
3443 LICENSE = "GPLv2+"
3444 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
3445
3446 SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
3447
3448 inherit autotools gettext
3449 </literallayout>
3450 </para>
3451
3452 <para>
3453 The variable
3454 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename>
3455 is used to track source license changes as described in the
3456 "<link linkend='usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</link>"
3457 section in the Yocto Project Overview and Concepts Manual.
3458 You can quickly create Autotool-based recipes in a manner
3459 similar to the previous example.
3460 </para>
3461 </section>
3462
3463 <section id='new-recipe-makefile-based-package'>
3464 <title>Makefile-Based Package</title>
3465
3466 <para>
3467 Applications that use GNU <filename>make</filename> also require a recipe that has
3468 the source archive listed in
3469 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
3470 You do not need to add a <filename>do_compile</filename> step since by default BitBake
3471 starts the <filename>make</filename> command to compile the application.
3472 If you need additional <filename>make</filename> options, you should store them in the
3473 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'><filename>EXTRA_OEMAKE</filename></ulink>
3474 or
3475 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink>
3476 variables.
3477 BitBake passes these options into the GNU <filename>make</filename> invocation.
3478 Note that a <filename>do_install</filename> task is still required.
3479 Otherwise, BitBake runs an empty <filename>do_install</filename> task by default.
3480 </para>
3481
3482 <para>
3483 Some applications might require extra parameters to be passed to the compiler.
3484 For example, the application might need an additional header path.
3485 You can accomplish this by adding to the
3486 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable.
3487 The following example shows this:
3488 <literallayout class='monospaced'>
3489 CFLAGS_prepend = "-I ${S}/include "
3490 </literallayout>
3491 </para>
3492
3493 <para>
3494 In the following example, <filename>mtd-utils</filename> is a makefile-based package:
3495 <literallayout class='monospaced'>
3496 SUMMARY = "Tools for managing memory technology devices"
3497 SECTION = "base"
3498 DEPENDS = "zlib lzo e2fsprogs util-linux"
3499 HOMEPAGE = "http://www.linux-mtd.infradead.org/"
3500 LICENSE = "GPLv2+"
3501 LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
3502 file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
3503
3504 # Use the latest version at 26 Oct, 2013
3505 SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b"
3506 SRC_URI = "git://git.infradead.org/mtd-utils.git \
3507 file://add-exclusion-to-mkfs-jffs2-git-2.patch \
3508 "
3509
3510 PV = "1.5.1+git${SRCPV}"
3511
3512 S = "${WORKDIR}/git"
3513
3514 EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'"
3515
3516 do_install () {
3517 oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir}
3518 }
3519
3520 PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc"
3521
3522 FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool"
3523 FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*"
3524 FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image"
3525
3526 PARALLEL_MAKE = ""
3527
3528 BBCLASSEXTEND = "native"
3529 </literallayout>
3530 </para>
3531 </section>
3532
3533 <section id='splitting-an-application-into-multiple-packages'>
3534 <title>Splitting an Application into Multiple Packages</title>
3535
3536 <para>
3537 You can use the variables
3538 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and
3539 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename>
3540 to split an application into multiple packages.
3541 </para>
3542
3543 <para>
3544 Following is an example that uses the <filename>libxpm</filename> recipe.
3545 By default, this recipe generates a single package that contains the library along
3546 with a few binaries.
3547 You can modify the recipe to split the binaries into separate packages:
3548 <literallayout class='monospaced'>
3549 require xorg-lib-common.inc
3550
3551 SUMMARY = "Xpm: X Pixmap extension library"
3552 LICENSE = "BSD"
3553 LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7"
3554 DEPENDS += "libxext libsm libxt"
3555 PE = "1"
3556
3557 XORG_PN = "libXpm"
3558
3559 PACKAGES =+ "sxpm cxpm"
3560 FILES_cxpm = "${bindir}/cxpm"
3561 FILES_sxpm = "${bindir}/sxpm"
3562 </literallayout>
3563 </para>
3564
3565 <para>
3566 In the previous example, we want to ship the <filename>sxpm</filename>
3567 and <filename>cxpm</filename> binaries in separate packages.
3568 Since <filename>bindir</filename> would be packaged into the main
3569 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
3570 package by default, we prepend the <filename>PACKAGES</filename>
3571 variable so additional package names are added to the start of list.
3572 This results in the extra <filename>FILES_*</filename>
3573 variables then containing information that define which files and
3574 directories go into which packages.
3575 Files included by earlier packages are skipped by latter packages.
3576 Thus, the main <filename>PN</filename> package
3577 does not include the above listed files.
3578 </para>
3579 </section>
3580
3581 <section id='packaging-externally-produced-binaries'>
3582 <title>Packaging Externally Produced Binaries</title>
3583
3584 <para>
3585 Sometimes, you need to add pre-compiled binaries to an
3586 image.
3587 For example, suppose that binaries for proprietary code
3588 exist, which are created by a particular division of a
3589 company.
3590 Your part of the company needs to use those binaries as
3591 part of an image that you are building using the
3592 OpenEmbedded build system.
3593 Since you only have the binaries and not the source code,
3594 you cannot use a typical recipe that expects to fetch the
3595 source specified in
3596 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
3597 and then compile it.
3598 </para>
3599
3600 <para>
3601 One method is to package the binaries and then install them
3602 as part of the image.
3603 Generally, it is not a good idea to package binaries
3604 since, among other things, it can hinder the ability to
3605 reproduce builds and could lead to compatibility problems
3606 with ABI in the future.
3607 However, sometimes you have no choice.
3608 </para>
3609
3610 <para>
3611 The easiest solution is to create a recipe that uses
3612 the
3613 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-bin-package'><filename>bin_package</filename></ulink>
3614 class and to be sure that you are using default locations
3615 for build artifacts.
3616 In most cases, the <filename>bin_package</filename> class
3617 handles "skipping" the configure and compile steps as well
3618 as sets things up to grab packages from the appropriate
3619 area.
3620 In particular, this class sets <filename>noexec</filename>
3621 on both the
3622 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
3623 and
3624 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
3625 tasks, sets
3626 <filename>FILES_${PN}</filename> to "/" so that it picks
3627 up all files, and sets up a
3628 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3629 task, which effectively copies all files from
3630 <filename>${S}</filename> to <filename>${D}</filename>.
3631 The <filename>bin_package</filename> class works well when
3632 the files extracted into <filename>${S}</filename> are
3633 already laid out in the way they should be laid out
3634 on the target.
3635 For more information on these variables, see the
3636 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>,
3637 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>,
3638 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>,
3639 and
3640 <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
3641 variables in the Yocto Project Reference Manual's variable
3642 glossary.
3643 <note><title>Notes</title>
3644 <itemizedlist>
3645 <listitem><para>
3646 Using
3647 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
3648 is a good idea even for components distributed
3649 in binary form, and is often necessary for
3650 shared libraries.
3651 For a shared library, listing the library
3652 dependencies in
3653 <filename>DEPENDS</filename> makes sure that
3654 the libraries are available in the staging
3655 sysroot when other recipes link against the
3656 library, which might be necessary for
3657 successful linking.
3658 </para></listitem>
3659 <listitem><para>
3660 Using <filename>DEPENDS</filename> also
3661 allows runtime dependencies between packages
3662 to be added automatically.
3663 See the
3664 "<ulink url='&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>"
3665 section in the Yocto Project Overview and
3666 Concepts Manual for more information.
3667 </para></listitem>
3668 </itemizedlist>
3669 </note>
3670 </para>
3671
3672 <para>
3673 If you cannot use the <filename>bin_package</filename>
3674 class, you need to be sure you are doing the following:
3675 <itemizedlist>
3676 <listitem><para>
3677 Create a recipe where the
3678 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
3679 and
3680 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
3681 tasks do nothing:
3682 It is usually sufficient to just not define these
3683 tasks in the recipe, because the default
3684 implementations do nothing unless a Makefile is
3685 found in
3686 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>.
3687 </para>
3688
3689 <para>If
3690 <filename>${S}</filename> might contain a Makefile,
3691 or if you inherit some class that replaces
3692 <filename>do_configure</filename> and
3693 <filename>do_compile</filename> with custom
3694 versions, then you can use the
3695 <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>noexec</filename></ulink><filename>]</filename>
3696 flag to turn the tasks into no-ops, as follows:
3697 <literallayout class='monospaced'>
3698 do_configure[noexec] = "1"
3699 do_compile[noexec] = "1"
3700 </literallayout>
3701 Unlike
3702 <ulink url='&YOCTO_DOCS_BB_URL;#deleting-a-task'><filename>deleting the tasks</filename></ulink>,
3703 using the flag preserves the dependency chain from
3704 the
3705 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>, <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>,
3706 and
3707 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
3708 tasks to the
3709 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3710 task.
3711 </para></listitem>
3712 <listitem><para>Make sure your
3713 <filename>do_install</filename> task installs the
3714 binaries appropriately.
3715 </para></listitem>
3716 <listitem><para>Ensure that you set up
3717 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
3718 (usually
3719 <filename>FILES_${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>)
3720 to point to the files you have installed, which of
3721 course depends on where you have installed them
3722 and whether those files are in different locations
3723 than the defaults.
3724 </para></listitem>
3725 </itemizedlist>
3726 </para>
3727 </section>
3728 </section>
3729
3730 <section id="following-recipe-style-guidelines">
3731 <title>Following Recipe Style Guidelines</title>
3732
3733 <para>
3734 When writing recipes, it is good to conform to existing
3735 style guidelines.
3736 The
3737 <ulink url='http://www.openembedded.org/wiki/Styleguide'>OpenEmbedded Styleguide</ulink>
3738 wiki page provides rough guidelines for preferred recipe style.
3739 </para>
3740
3741 <para>
3742 It is common for existing recipes to deviate a bit from this
3743 style.
3744 However, aiming for at least a consistent style is a good idea.
3745 Some practices, such as omitting spaces around
3746 <filename>=</filename> operators in assignments or ordering
3747 recipe components in an erratic way, are widely seen as poor
3748 style.
3749 </para>
3750 </section>
3751
3752 <section id='recipe-syntax'>
3753 <title>Recipe Syntax</title>
3754
3755 <para>
3756 Understanding recipe file syntax is important for writing
3757 recipes.
3758 The following list overviews the basic items that make up a
3759 BitBake recipe file.
3760 For more complete BitBake syntax descriptions, see the
3761 "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>"
3762 chapter of the BitBake User Manual.
3763 <itemizedlist>
3764 <listitem><para>
3765 <emphasis>Variable Assignments and Manipulations:</emphasis>
3766 Variable assignments allow a value to be assigned to a
3767 variable.
3768 The assignment can be static text or might include
3769 the contents of other variables.
3770 In addition to the assignment, appending and prepending
3771 operations are also supported.</para>
3772
3773 <para>The following example shows some of the ways
3774 you can use variables in recipes:
3775 <literallayout class='monospaced'>
3776 S = "${WORKDIR}/postfix-${PV}"
3777 CFLAGS += "-DNO_ASM"
3778 SRC_URI_append = " file://fixup.patch"
3779 </literallayout>
3780 </para></listitem>
3781 <listitem><para>
3782 <emphasis>Functions:</emphasis>
3783 Functions provide a series of actions to be performed.
3784 You usually use functions to override the default
3785 implementation of a task function or to complement
3786 a default function (i.e. append or prepend to an
3787 existing function).
3788 Standard functions use <filename>sh</filename> shell
3789 syntax, although access to OpenEmbedded variables and
3790 internal methods are also available.</para>
3791
3792 <para>The following is an example function from the
3793 <filename>sed</filename> recipe:
3794 <literallayout class='monospaced'>
3795 do_install () {
3796 autotools_do_install
3797 install -d ${D}${base_bindir}
3798 mv ${D}${bindir}/sed ${D}${base_bindir}/sed
3799 rmdir ${D}${bindir}/
3800 }
3801 </literallayout>
3802 It is also possible to implement new functions that
3803 are called between existing tasks as long as the
3804 new functions are not replacing or complementing the
3805 default functions.
3806 You can implement functions in Python
3807 instead of shell.
3808 Both of these options are not seen in the majority of
3809 recipes.
3810 </para></listitem>
3811 <listitem><para><emphasis>Keywords:</emphasis>
3812 BitBake recipes use only a few keywords.
3813 You use keywords to include common
3814 functions (<filename>inherit</filename>), load parts
3815 of a recipe from other files
3816 (<filename>include</filename> and
3817 <filename>require</filename>) and export variables
3818 to the environment (<filename>export</filename>).
3819 </para>
3820
3821 <para>The following example shows the use of some of
3822 these keywords:
3823 <literallayout class='monospaced'>
3824 export POSTCONF = "${STAGING_BINDIR}/postconf"
3825 inherit autoconf
3826 require otherfile.inc
3827 </literallayout>
3828 </para></listitem>
3829 <listitem><para>
3830 <emphasis>Comments (#):</emphasis>
3831 Any lines that begin with the hash character
3832 (<filename>#</filename>) are treated as comment lines
3833 and are ignored:
3834 <literallayout class='monospaced'>
3835 # This is a comment
3836 </literallayout>
3837 </para></listitem>
3838 </itemizedlist>
3839 </para>
3840
3841 <para>
3842 This next list summarizes the most important and most commonly
3843 used parts of the recipe syntax.
3844 For more information on these parts of the syntax, you can
3845 reference the
3846 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>
3847 chapter in the BitBake User Manual.
3848 <itemizedlist>
3849 <listitem><para>
3850 <emphasis>Line Continuation (\):</emphasis>
3851 Use the backward slash (<filename>\</filename>)
3852 character to split a statement over multiple lines.
3853 Place the slash character at the end of the line that
3854 is to be continued on the next line:
3855 <literallayout class='monospaced'>
3856 VAR = "A really long \
3857 line"
3858 </literallayout>
3859 <note>
3860 You cannot have any characters including spaces
3861 or tabs after the slash character.
3862 </note>
3863 </para></listitem>
3864 <listitem><para>
3865 <emphasis>Using Variables (${<replaceable>VARNAME</replaceable>}):</emphasis>
3866 Use the <filename>${<replaceable>VARNAME</replaceable>}</filename>
3867 syntax to access the contents of a variable:
3868 <literallayout class='monospaced'>
3869 SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
3870 </literallayout>
3871 <note>
3872 It is important to understand that the value of a
3873 variable expressed in this form does not get
3874 substituted automatically.
3875 The expansion of these expressions happens
3876 on-demand later (e.g. usually when a function that
3877 makes reference to the variable executes).
3878 This behavior ensures that the values are most
3879 appropriate for the context in which they are
3880 finally used.
3881 On the rare occasion that you do need the variable
3882 expression to be expanded immediately, you can use
3883 the <filename>:=</filename> operator instead of
3884 <filename>=</filename> when you make the
3885 assignment, but this is not generally needed.
3886 </note>
3887 </para></listitem>
3888 <listitem><para>
3889 <emphasis>Quote All Assignments ("<replaceable>value</replaceable>"):</emphasis>
3890 Use double quotes around values in all variable
3891 assignments (e.g.
3892 <filename>"<replaceable>value</replaceable>"</filename>).
3893 Following is an example:
3894 <literallayout class='monospaced'>
3895 VAR1 = "${OTHERVAR}"
3896 VAR2 = "The version is ${PV}"
3897 </literallayout>
3898 </para></listitem>
3899 <listitem><para>
3900 <emphasis>Conditional Assignment (?=):</emphasis>
3901 Conditional assignment is used to assign a
3902 value to a variable, but only when the variable is
3903 currently unset.
3904 Use the question mark followed by the equal sign
3905 (<filename>?=</filename>) to make a "soft" assignment
3906 used for conditional assignment.
3907 Typically, "soft" assignments are used in the
3908 <filename>local.conf</filename> file for variables
3909 that are allowed to come through from the external
3910 environment.
3911 </para>
3912
3913 <para>Here is an example where
3914 <filename>VAR1</filename> is set to "New value" if
3915 it is currently empty.
3916 However, if <filename>VAR1</filename> has already been
3917 set, it remains unchanged:
3918 <literallayout class='monospaced'>
3919 VAR1 ?= "New value"
3920 </literallayout>
3921 In this next example, <filename>VAR1</filename>
3922 is left with the value "Original value":
3923 <literallayout class='monospaced'>
3924 VAR1 = "Original value"
3925 VAR1 ?= "New value"
3926 </literallayout>
3927 </para></listitem>
3928 <listitem><para>
3929 <emphasis>Appending (+=):</emphasis>
3930 Use the plus character followed by the equals sign
3931 (<filename>+=</filename>) to append values to existing
3932 variables.
3933 <note>
3934 This operator adds a space between the existing
3935 content of the variable and the new content.
3936 </note></para>
3937
3938 <para>Here is an example:
3939 <literallayout class='monospaced'>
3940 SRC_URI += "file://fix-makefile.patch"
3941 </literallayout>
3942 </para></listitem>
3943 <listitem><para>
3944 <emphasis>Prepending (=+):</emphasis>
3945 Use the equals sign followed by the plus character
3946 (<filename>=+</filename>) to prepend values to existing
3947 variables.
3948 <note>
3949 This operator adds a space between the new content
3950 and the existing content of the variable.
3951 </note></para>
3952
3953 <para>Here is an example:
3954 <literallayout class='monospaced'>
3955 VAR =+ "Starts"
3956 </literallayout>
3957 </para></listitem>
3958 <listitem><para>
3959 <emphasis>Appending (_append):</emphasis>
3960 Use the <filename>_append</filename> operator to
3961 append values to existing variables.
3962 This operator does not add any additional space.
3963 Also, the operator is applied after all the
3964 <filename>+=</filename>, and
3965 <filename>=+</filename> operators have been applied and
3966 after all <filename>=</filename> assignments have
3967 occurred.
3968 </para>
3969
3970 <para>The following example shows the space being
3971 explicitly added to the start to ensure the appended
3972 value is not merged with the existing value:
3973 <literallayout class='monospaced'>
3974 SRC_URI_append = " file://fix-makefile.patch"
3975 </literallayout>
3976 You can also use the <filename>_append</filename>
3977 operator with overrides, which results in the actions
3978 only being performed for the specified target or
3979 machine:
3980 <literallayout class='monospaced'>
3981 SRC_URI_append_sh4 = " file://fix-makefile.patch"
3982 </literallayout>
3983 </para></listitem>
3984 <listitem><para>
3985 <emphasis>Prepending (_prepend):</emphasis>
3986 Use the <filename>_prepend</filename> operator to
3987 prepend values to existing variables.
3988 This operator does not add any additional space.
3989 Also, the operator is applied after all the
3990 <filename>+=</filename>, and
3991 <filename>=+</filename> operators have been applied and
3992 after all <filename>=</filename> assignments have
3993 occurred.
3994 </para>
3995
3996 <para>The following example shows the space being
3997 explicitly added to the end to ensure the prepended
3998 value is not merged with the existing value:
3999 <literallayout class='monospaced'>
4000 CFLAGS_prepend = "-I${S}/myincludes "
4001 </literallayout>
4002 You can also use the <filename>_prepend</filename>
4003 operator with overrides, which results in the actions
4004 only being performed for the specified target or
4005 machine:
4006 <literallayout class='monospaced'>
4007 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
4008 </literallayout>
4009 </para></listitem>
4010 <listitem><para>
4011 <emphasis>Overrides:</emphasis>
4012 You can use overrides to set a value conditionally,
4013 typically based on how the recipe is being built.
4014 For example, to set the
4015 <ulink url='&YOCTO_DOCS_REF_URL;#var-KBRANCH'><filename>KBRANCH</filename></ulink>
4016 variable's value to "standard/base" for any target
4017 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>,
4018 except for qemuarm where it should be set to
4019 "standard/arm-versatile-926ejs", you would do the
4020 following:
4021 <literallayout class='monospaced'>
4022 KBRANCH = "standard/base"
4023 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
4024 </literallayout>
4025 Overrides are also used to separate alternate values
4026 of a variable in other situations.
4027 For example, when setting variables such as
4028 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
4029 and
4030 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
4031 that are specific to individual packages produced by
4032 a recipe, you should always use an override that
4033 specifies the name of the package.
4034 </para></listitem>
4035 <listitem><para>
4036 <emphasis>Indentation:</emphasis>
4037 Use spaces for indentation rather than than tabs.
4038 For shell functions, both currently work.
4039 However, it is a policy decision of the Yocto Project
4040 to use tabs in shell functions.
4041 Realize that some layers have a policy to use spaces
4042 for all indentation.
4043 </para></listitem>
4044 <listitem><para>
4045 <emphasis>Using Python for Complex Operations:</emphasis>
4046 For more advanced processing, it is possible to use
4047 Python code during variable assignments (e.g.
4048 search and replacement on a variable).</para>
4049
4050 <para>You indicate Python code using the
4051 <filename>${@<replaceable>python_code</replaceable>}</filename>
4052 syntax for the variable assignment:
4053 <literallayout class='monospaced'>
4054 SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz
4055 </literallayout>
4056 </para></listitem>
4057 <listitem><para>
4058 <emphasis>Shell Function Syntax:</emphasis>
4059 Write shell functions as if you were writing a shell
4060 script when you describe a list of actions to take.
4061 You should ensure that your script works with a generic
4062 <filename>sh</filename> and that it does not require
4063 any <filename>bash</filename> or other shell-specific
4064 functionality.
4065 The same considerations apply to various system
4066 utilities (e.g. <filename>sed</filename>,
4067 <filename>grep</filename>, <filename>awk</filename>,
4068 and so forth) that you might wish to use.
4069 If in doubt, you should check with multiple
4070 implementations - including those from BusyBox.
4071 </para></listitem>
4072 </itemizedlist>
4073 </para>
4074 </section>
4075 </section>
4076
4077 <section id="platdev-newmachine">
4078 <title>Adding a New Machine</title>
4079
4080 <para>
4081 Adding a new machine to the Yocto Project is a straightforward
4082 process.
4083 This section describes how to add machines that are similar
4084 to those that the Yocto Project already supports.
4085 <note>
4086 Although well within the capabilities of the Yocto Project,
4087 adding a totally new architecture might require
4088 changes to <filename>gcc/glibc</filename> and to the site
4089 information, which is beyond the scope of this manual.
4090 </note>
4091 </para>
4092
4093 <para>
4094 For a complete example that shows how to add a new machine,
4095 see the
4096 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'>Creating a New BSP Layer Using the <filename>bitbake-layers</filename> Script</ulink>"
4097 section in the Yocto Project Board Support Package (BSP)
4098 Developer's Guide.
4099 </para>
4100
4101 <section id="platdev-newmachine-conffile">
4102 <title>Adding the Machine Configuration File</title>
4103
4104 <para>
4105 To add a new machine, you need to add a new machine
4106 configuration file to the layer's
4107 <filename>conf/machine</filename> directory.
4108 This configuration file provides details about the device
4109 you are adding.
4110 </para>
4111
4112 <para>
4113 The OpenEmbedded build system uses the root name of the
4114 machine configuration file to reference the new machine.
4115 For example, given a machine configuration file named
4116 <filename>crownbay.conf</filename>, the build system
4117 recognizes the machine as "crownbay".
4118 </para>
4119
4120 <para>
4121 The most important variables you must set in your machine
4122 configuration file or include from a lower-level configuration
4123 file are as follows:
4124 <itemizedlist>
4125 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename>
4126 (e.g. "arm")</para></listitem>
4127 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename>
4128 </para></listitem>
4129 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename>
4130 (e.g. "apm screen wifi")</para></listitem>
4131 </itemizedlist>
4132 </para>
4133
4134 <para>
4135 You might also need these variables:
4136 <itemizedlist>
4137 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename>
4138 (e.g. "115200;ttyS0 115200;ttyS1")</para></listitem>
4139 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename>
4140 (e.g. "zImage")</para></listitem>
4141 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename>
4142 (e.g. "tar.gz jffs2")</para></listitem>
4143 </itemizedlist>
4144 </para>
4145
4146 <para>
4147 You can find full details on these variables in the reference
4148 section.
4149 You can leverage existing machine <filename>.conf</filename>
4150 files from <filename>meta-yocto-bsp/conf/machine/</filename>.
4151 </para>
4152 </section>
4153
4154 <section id="platdev-newmachine-kernel">
4155 <title>Adding a Kernel for the Machine</title>
4156
4157 <para>
4158 The OpenEmbedded build system needs to be able to build a kernel
4159 for the machine.
4160 You need to either create a new kernel recipe for this machine,
4161 or extend an existing kernel recipe.
4162 You can find several kernel recipe examples in the
4163 Source Directory at
4164 <filename>meta/recipes-kernel/linux</filename>
4165 that you can use as references.
4166 </para>
4167
4168 <para>
4169 If you are creating a new kernel recipe, normal recipe-writing
4170 rules apply for setting up a
4171 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
4172 Thus, you need to specify any necessary patches and set
4173 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
4174 to point at the source code.
4175 You need to create a <filename>do_configure</filename> task that
4176 configures the unpacked kernel with a
4177 <filename>defconfig</filename> file.
4178 You can do this by using a <filename>make defconfig</filename>
4179 command or, more commonly, by copying in a suitable
4180 <filename>defconfig</filename> file and then running
4181 <filename>make oldconfig</filename>.
4182 By making use of <filename>inherit kernel</filename> and
4183 potentially some of the <filename>linux-*.inc</filename> files,
4184 most other functionality is centralized and the defaults of the
4185 class normally work well.
4186 </para>
4187
4188 <para>
4189 If you are extending an existing kernel recipe, it is usually
4190 a matter of adding a suitable <filename>defconfig</filename>
4191 file.
4192 The file needs to be added into a location similar to
4193 <filename>defconfig</filename> files used for other machines
4194 in a given kernel recipe.
4195 A possible way to do this is by listing the file in the
4196 <filename>SRC_URI</filename> and adding the machine to the
4197 expression in
4198 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>:
4199 <literallayout class='monospaced'>
4200 COMPATIBLE_MACHINE = '(qemux86|qemumips)'
4201 </literallayout>
4202 For more information on <filename>defconfig</filename> files,
4203 see the
4204 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
4205 section in the Yocto Project Linux Kernel Development Manual.
4206 </para>
4207 </section>
4208
4209 <section id="platdev-newmachine-formfactor">
4210 <title>Adding a Formfactor Configuration File</title>
4211
4212 <para>
4213 A formfactor configuration file provides information about the
4214 target hardware for which the image is being built and information that
4215 the build system cannot obtain from other sources such as the kernel.
4216 Some examples of information contained in a formfactor configuration file include
4217 framebuffer orientation, whether or not the system has a keyboard,
4218 the positioning of the keyboard in relation to the screen, and
4219 the screen resolution.
4220 </para>
4221
4222 <para>
4223 The build system uses reasonable defaults in most cases.
4224 However, if customization is
4225 necessary, you need to create a <filename>machconfig</filename> file
4226 in the <filename>meta/recipes-bsp/formfactor/files</filename>
4227 directory.
4228 This directory contains directories for specific machines such as
4229 <filename>qemuarm</filename> and <filename>qemux86</filename>.
4230 For information about the settings available and the defaults, see the
4231 <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the
4232 same area.
4233 </para>
4234
4235 <para>
4236 Following is an example for "qemuarm" machine:
4237 <literallayout class='monospaced'>
4238 HAVE_TOUCHSCREEN=1
4239 HAVE_KEYBOARD=1
4240
4241 DISPLAY_CAN_ROTATE=0
4242 DISPLAY_ORIENTATION=0
4243 #DISPLAY_WIDTH_PIXELS=640
4244 #DISPLAY_HEIGHT_PIXELS=480
4245 #DISPLAY_BPP=16
4246 DISPLAY_DPI=150
4247 DISPLAY_SUBPIXEL_ORDER=vrgb
4248 </literallayout>
4249 </para>
4250 </section>
4251 </section>
4252
4253 <section id='gs-upgrading-recipes'>
4254 <title>Upgrading Recipes</title>
4255
4256 <para>
4257 Over time, upstream developers publish new versions for software
4258 built by layer recipes.
4259 It is recommended to keep recipes up-to-date with upstream
4260 version releases.
4261 </para>
4262
4263 <para>
4264 While several methods exist that allow you upgrade a recipe,
4265 you might consider checking on the upgrade status of a recipe
4266 first.
4267 You can do so using the
4268 <filename>devtool check-upgrade-status</filename> command.
4269 See the
4270 "<ulink url='&YOCTO_DOCS_REF_URL;#devtool-checking-on-the-upgrade-status-of-a-recipe'>Checking on the Upgrade Status of a Recipe</ulink>"
4271 section in the Yocto Project Reference Manual for more information.
4272 </para>
4273
4274 <para>
4275 The remainder of this section describes three ways you can
4276 upgrade a recipe.
4277 You can use the Automated Upgrade Helper (AUH) to set up
4278 automatic version upgrades.
4279 Alternatively, you can use <filename>devtool upgrade</filename>
4280 to set up semi-automatic version upgrades.
4281 Finally, you can manually upgrade a recipe by editing the
4282 recipe itself.
4283 </para>
4284
4285 <section id='gs-using-the-auto-upgrade-helper'>
4286 <title>Using the Auto Upgrade Helper (AUH)</title>
4287
4288 <para>
4289 The AUH utility works in conjunction with the
4290 OpenEmbedded build system in order to automatically generate
4291 upgrades for recipes based on new versions being
4292 published upstream.
4293 Use AUH when you want to create a service that performs the
4294 upgrades automatically and optionally sends you an email with
4295 the results.
4296 </para>
4297
4298 <para>
4299 AUH allows you to update several recipes with a single use.
4300 You can also optionally perform build and integration tests
4301 using images with the results saved to your hard drive and
4302 emails of results optionally sent to recipe maintainers.
4303 Finally, AUH creates Git commits with appropriate commit
4304 messages in the layer's tree for the changes made to recipes.
4305 <note>
4306 Conditions do exist when you should not use AUH to upgrade
4307 recipes and you should instead use either
4308 <filename>devtool upgrade</filename> or upgrade your
4309 recipes manually:
4310 <itemizedlist>
4311 <listitem><para>
4312 When AUH cannot complete the upgrade sequence.
4313 This situation usually results because custom
4314 patches carried by the recipe cannot be
4315 automatically rebased to the new version.
4316 In this case, <filename>devtool upgrade</filename>
4317 allows you to manually resolve conflicts.
4318 </para></listitem>
4319 <listitem><para>
4320 When for any reason you want fuller control over
4321 the upgrade process.
4322 For example, when you want special arrangements
4323 for testing.
4324 </para></listitem>
4325 </itemizedlist>
4326 </note>
4327 </para>
4328
4329 <para>
4330 The following steps describe how to set up the AUH utility:
4331 <orderedlist>
4332 <listitem><para>
4333 <emphasis>Be Sure the Development Host is Set Up:</emphasis>
4334 You need to be sure that your development host is
4335 set up to use the Yocto Project.
4336 For information on how to set up your host, see the
4337 "<link linkend='dev-preparing-the-build-host'>Preparing the Build Host</link>"
4338 section.
4339 </para></listitem>
4340 <listitem><para>
4341 <emphasis>Make Sure Git is Configured:</emphasis>
4342 The AUH utility requires Git to be configured because
4343 AUH uses Git to save upgrades.
4344 Thus, you must have Git user and email configured.
4345 The following command shows your configurations:
4346 <literallayout class='monospaced'>
4347 $ git config --list
4348 </literallayout>
4349 If you do not have the user and email configured, you
4350 can use the following commands to do so:
4351 <literallayout class='monospaced'>
4352 $ git config --global user.name <replaceable>some_name</replaceable>
4353 $ git config --global user.email <replaceable>username</replaceable>@<replaceable>domain</replaceable>.com
4354 </literallayout>
4355 </para></listitem>
4356 <listitem><para>
4357 <emphasis>Clone the AUH Repository:</emphasis>
4358 To use AUH, you must clone the repository onto your
4359 development host.
4360 The following command uses Git to create a local
4361 copy of the repository on your system:
4362 <literallayout class='monospaced'>
4363 $ git clone git://git.yoctoproject.org/auto-upgrade-helper
4364 Cloning into 'auto-upgrade-helper'...
4365 remote: Counting objects: 768, done.
4366 remote: Compressing objects: 100% (300/300), done.
4367 remote: Total 768 (delta 499), reused 703 (delta 434)
4368 Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
4369 Resolving deltas: 100% (499/499), done.
4370 Checking connectivity... done.
4371 </literallayout>
4372 AUH is not part of the
4373 <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core (OE-Core)</ulink>
4374 or
4375 <ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink>
4376 repositories.
4377 </para></listitem>
4378 <listitem><para>
4379 <emphasis>Create a Dedicated Build Directory:</emphasis>
4380 Run the
4381 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink>
4382 script to create a fresh build directory that you
4383 use exclusively for running the AUH utility:
4384 <literallayout class='monospaced'>
4385 $ cd ~/poky
4386 $ source oe-init-build-env <replaceable>your_AUH_build_directory</replaceable>
4387 </literallayout>
4388 Re-using an existing build directory and its
4389 configurations is not recommended as existing settings
4390 could cause AUH to fail or behave undesirably.
4391 </para></listitem>
4392 <listitem><para>
4393 <emphasis>Make Configurations in Your Local Configuration File:</emphasis>
4394 Several settings need to exist in the
4395 <filename>local.conf</filename> file in the build
4396 directory you just created for AUH.
4397 Make these following configurations:
4398 <itemizedlist>
4399 <listitem><para>
4400 If you want to enable
4401 <ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality'>Build History</ulink>,
4402 which is optional, you need the following
4403 lines in the
4404 <filename>conf/local.conf</filename> file:
4405 <literallayout class='monospaced'>
4406 INHERIT =+ "buildhistory"
4407 BUILDHISTORY_COMMIT = "1"
4408 </literallayout>
4409 With this configuration and a successful
4410 upgrade, a build history "diff" file appears in
4411 the
4412 <filename>upgrade-helper/work/recipe/buildhistory-diff.txt</filename>
4413 file found in your build directory.
4414 </para></listitem>
4415 <listitem><para>
4416 If you want to enable testing through the
4417 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink>
4418 class, which is optional, you need to have the
4419 following set in your
4420 <filename>conf/local.conf</filename> file:
4421 <literallayout class='monospaced'>
4422 INHERIT += "testimage"
4423 </literallayout>
4424 <note>
4425 If your distro does not enable by default
4426 ptest, which Poky does, you need the
4427 following in your
4428 <filename>local.conf</filename> file:
4429 <literallayout class='monospaced'>
4430 DISTRO_FEATURES_append = " ptest"
4431 </literallayout>
4432 </note>
4433 </para></listitem>
4434 </itemizedlist>
4435 </para></listitem>
4436 <listitem><para>
4437 <emphasis>Optionally Start a vncserver:</emphasis>
4438 If you are running in a server without an X11 session,
4439 you need to start a vncserver:
4440 <literallayout class='monospaced'>
4441 $ vncserver :1
4442 $ export DISPLAY=:1
4443 </literallayout>
4444 </para></listitem>
4445 <listitem><para>
4446 <emphasis>Create and Edit an AUH Configuration File:</emphasis>
4447 You need to have the
4448 <filename>upgrade-helper/upgrade-helper.conf</filename>
4449 configuration file in your build directory.
4450 You can find a sample configuration file in the
4451 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/'>AUH source repository</ulink>.
4452 </para>
4453
4454 <para>Read through the sample file and make
4455 configurations as needed.
4456 For example, if you enabled build history in your
4457 <filename>local.conf</filename> as described earlier,
4458 you must enable it in
4459 <filename>upgrade-helper.conf</filename>.</para>
4460
4461 <para>Also, if you are using the default
4462 <filename>maintainers.inc</filename> file supplied
4463 with Poky and located in
4464 <filename>meta-yocto</filename> and you do not set a
4465 "maintainers_whitelist" or "global_maintainer_override"
4466 in the <filename>upgrade-helper.conf</filename>
4467 configuration, and you specify "-e all" on the
4468 AUH command-line, the utility automatically sends out
4469 emails to all the default maintainers.
4470 Please avoid this.
4471 </para></listitem>
4472 </orderedlist>
4473 </para>
4474
4475 <para>
4476 This next set of examples describes how to use the AUH:
4477 <itemizedlist>
4478 <listitem><para>
4479 <emphasis>Upgrading a Specific Recipe:</emphasis>
4480 To upgrade a specific recipe, use the following
4481 form:
4482 <literallayout class='monospaced'>
4483 $ upgrade-helper.py <replaceable>recipe_name</replaceable>
4484 </literallayout>
4485 For example, this command upgrades the
4486 <filename>xmodmap</filename> recipe:
4487 <literallayout class='monospaced'>
4488 $ upgrade-helper.py xmodmap
4489 </literallayout>
4490 </para></listitem>
4491 <listitem><para>
4492 <emphasis>Upgrading a Specific Recipe to a Particular Version:</emphasis>
4493 To upgrade a specific recipe to a particular version,
4494 use the following form:
4495 <literallayout class='monospaced'>
4496 $ upgrade-helper.py <replaceable>recipe_name</replaceable> -t <replaceable>version</replaceable>
4497 </literallayout>
4498 For example, this command upgrades the
4499 <filename>xmodmap</filename> recipe to version
4500 1.2.3:
4501 <literallayout class='monospaced'>
4502 $ upgrade-helper.py xmodmap -t 1.2.3
4503 </literallayout>
4504 </para></listitem>
4505 <listitem><para>
4506 <emphasis>Upgrading all Recipes to the Latest Versions and Suppressing Email Notifications:</emphasis>
4507 To upgrade all recipes to their most recent versions
4508 and suppress the email notifications, use the following
4509 command:
4510 <literallayout class='monospaced'>
4511 $ upgrade-helper.py all
4512 </literallayout>
4513 </para></listitem>
4514 <listitem><para>
4515 <emphasis>Upgrading all Recipes to the Latest Versions and Send Email Notifications:</emphasis>
4516 To upgrade all recipes to their most recent versions
4517 and send email messages to maintainers for each
4518 attempted recipe as well as a status email, use the
4519 following command:
4520 <literallayout class='monospaced'>
4521 $ upgrade-helper.py -e all
4522 </literallayout>
4523 </para></listitem>
4524 </itemizedlist>
4525 </para>
4526
4527 <para>
4528 Once you have run the AUH utility, you can find the results
4529 in the AUH build directory:
4530 <literallayout class='monospaced'>
4531 ${BUILDDIR}/upgrade-helper/<replaceable>timestamp</replaceable>
4532 </literallayout>
4533 The AUH utility also creates recipe update commits from
4534 successful upgrade attempts in the layer tree.
4535 </para>
4536
4537 <para>
4538 You can easily set up to run the AUH utility on a regular
4539 basis by using a cron job.
4540 See the
4541 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh'><filename>weeklyjob.sh</filename></ulink>
4542 file distributed with the utility for an example.
4543 </para>
4544 </section>
4545
4546 <section id='gs-using-devtool-upgrade'>
4547 <title>Using <filename>devtool upgrade</filename></title>
4548
4549 <para>
4550 As mentioned earlier, an alternative method for upgrading
4551 recipes to newer versions is to use
4552 <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool upgrade</filename></ulink>.
4553 You can read about <filename>devtool upgrade</filename> in
4554 general in the
4555 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software'>Use <filename>devtool upgrade</filename> to Create a Version of the Recipe that Supports a Newer Version of the Software</ulink>"
4556 section in the Yocto Project Application Development and the
4557 Extensible Software Development Kit (eSDK) Manual.
4558 </para>
4559
4560 <para>
4561 To see all the command-line options available with
4562 <filename>devtool upgrade</filename>, use the following help
4563 command:
4564 <literallayout class='monospaced'>
4565 $ devtool upgrade -h
4566 </literallayout>
4567 </para>
4568
4569 <para>
4570 If you want to find out what version a recipe is currently at
4571 upstream without any attempt to upgrade your local version of
4572 the recipe, you can use the following command:
4573 <literallayout class='monospaced'>
4574 $ devtool latest-version <replaceable>recipe_name</replaceable>
4575 </literallayout>
4576 </para>
4577
4578 <para>
4579 As mentioned in the previous section describing AUH,
4580 <filename>devtool upgrade</filename> works in a
4581 less-automated manner than AUH.
4582 Specifically, <filename>devtool upgrade</filename> only
4583 works on a single recipe that you name on the command line,
4584 cannot perform build and integration testing using images,
4585 and does not automatically generate commits for changes in
4586 the source tree.
4587 Despite all these "limitations",
4588 <filename>devtool upgrade</filename> updates the recipe file
4589 to the new upstream version and attempts to rebase custom
4590 patches contained by the recipe as needed.
4591 <note>
4592 AUH uses much of <filename>devtool upgrade</filename>
4593 behind the scenes making AUH somewhat of a "wrapper"
4594 application for <filename>devtool upgrade</filename>.
4595 </note>
4596 </para>
4597
4598 <para>
4599 A typical scenario involves having used Git to clone an
4600 upstream repository that you use during build operations.
4601 Because you are (or have) built the recipe in the past, the
4602 layer is likely added to your configuration already.
4603 If for some reason, the layer is not added, you could add
4604 it easily using the
4605 <ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'><filename>bitbake-layers</filename></ulink>
4606 script.
4607 For example, suppose you use the <filename>nano.bb</filename>
4608 recipe from the <filename>meta-oe</filename> layer in the
4609 <filename>meta-openembedded</filename> repository.
4610 For this example, assume that the layer has been cloned into
4611 following area:
4612 <literallayout class='monospaced'>
4613 /home/scottrif/meta-openembedded
4614 </literallayout>
4615 The following command from your
4616 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
4617 adds the layer to your build configuration (i.e.
4618 <filename>${BUILDDIR}/conf/bblayers.conf</filename>):
4619 <literallayout class='monospaced'>
4620 $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
4621 NOTE: Starting bitbake server...
4622 Parsing recipes: 100% |##########################################| Time: 0:00:55
4623 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
4624 Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
4625 Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
4626 Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
4627 Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
4628 </literallayout>
4629 For this example, assume that the <filename>nano.bb</filename>
4630 recipe that is upstream has a 2.9.3 version number.
4631 However, the version in the local repository is 2.7.4.
4632 The following command from your build directory automatically
4633 upgrades the recipe for you:
4634 <note>
4635 Using the <filename>-V</filename> option is not necessary.
4636 Omitting the version number causes
4637 <filename>devtool upgrade</filename> to upgrade the recipe
4638 to the most recent version.
4639 </note>
4640 <literallayout class='monospaced'>
4641 $ devtool upgrade nano -V 2.9.3
4642 NOTE: Starting bitbake server...
4643 NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
4644 Parsing recipes: 100% |##########################################| Time: 0:00:46
4645 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
4646 NOTE: Extracting current version source...
4647 NOTE: Resolving any missing task queue dependencies
4648 .
4649 .
4650 .
4651 NOTE: Executing SetScene Tasks
4652 NOTE: Executing RunQueue Tasks
4653 NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
4654 Adding changed files: 100% |#####################################| Time: 0:00:00
4655 NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
4656 NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
4657 </literallayout>
4658 Continuing with this example, you can use
4659 <filename>devtool build</filename> to build the newly upgraded
4660 recipe:
4661 <literallayout class='monospaced'>
4662 $ devtool build nano
4663 NOTE: Starting bitbake server...
4664 Loading cache: 100% |################################################################################################| Time: 0:00:01
4665 Loaded 2040 entries from dependency cache.
4666 Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
4667 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
4668 NOTE: Resolving any missing task queue dependencies
4669 .
4670 .
4671 .
4672 NOTE: Executing SetScene Tasks
4673 NOTE: Executing RunQueue Tasks
4674 NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
4675 NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
4676 </literallayout>
4677 Within the <filename>devtool upgrade</filename> workflow,
4678 opportunity exists to deploy and test your rebuilt software.
4679 For this example, however, running
4680 <filename>devtool finish</filename> cleans up the workspace
4681 once the source in your workspace is clean.
4682 This usually means using Git to stage and submit commits
4683 for the changes generated by the upgrade process.
4684 </para>
4685
4686 <para>
4687 Once the tree is clean, you can clean things up in this
4688 example with the following command from the
4689 <filename>${BUILDDIR}/workspace/sources/nano</filename>
4690 directory:
4691 <literallayout class='monospaced'>
4692 $ devtool finish nano meta-oe
4693 NOTE: Starting bitbake server...
4694 Loading cache: 100% |################################################################################################| Time: 0:00:00
4695 Loaded 2040 entries from dependency cache.
4696 Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
4697 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
4698 NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
4699 NOTE: Updating recipe nano_2.9.3.bb
4700 NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
4701 NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
4702 NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
4703 </literallayout>
4704 Using the <filename>devtool finish</filename> command cleans
4705 up the workspace and creates a patch file based on your
4706 commits.
4707 The tool puts all patch files back into the source directory
4708 in a sub-directory named <filename>nano</filename> in this
4709 case.
4710 </para>
4711 </section>
4712
4713 <section id='dev-manually-upgrading-a-recipe'>
4714 <title>Manually Upgrading a Recipe</title>
4715
4716 <para>
4717 If for some reason you choose not to upgrade recipes using the
4718 <link linkend='gs-using-the-auto-upgrade-helper'>Auto Upgrade Helper (AUH)</link>
4719 or by using
4720 <link linkend='gs-using-devtool-upgrade'><filename>devtool upgrade</filename></link>,
4721 you can manually edit the recipe files to upgrade the versions.
4722 <note><title>Caution</title>
4723 Manually updating multiple recipes scales poorly and
4724 involves many steps.
4725 The recommendation to upgrade recipe versions is through
4726 AUH or <filename>devtool upgrade</filename>, both of which
4727 automate some steps and provide guidance for others needed
4728 for the manual process.
4729 </note>
4730 </para>
4731
4732 <para>
4733 To manually upgrade recipe versions, follow these general steps:
4734 <orderedlist>
4735 <listitem><para>
4736 <emphasis>Change the Version:</emphasis>
4737 Rename the recipe such that the version (i.e. the
4738 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
4739 part of the recipe name) changes appropriately.
4740 If the version is not part of the recipe name, change
4741 the value as it is set for <filename>PV</filename>
4742 within the recipe itself.
4743 </para></listitem>
4744 <listitem><para>
4745 <emphasis>Update <filename>SRCREV</filename> if Needed:</emphasis>
4746 If the source code your recipe builds is fetched from
4747 Git or some other version control system, update
4748 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
4749 to point to the commit hash that matches the new
4750 version.
4751 </para></listitem>
4752 <listitem><para>
4753 <emphasis>Build the Software:</emphasis>
4754 Try to build the recipe using BitBake.
4755 Typical build failures include the following:
4756 <itemizedlist>
4757 <listitem><para>
4758 License statements were updated for the new
4759 version.
4760 For this case, you need to review any changes
4761 to the license and update the values of
4762 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
4763 and
4764 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
4765 as needed.
4766 <note>
4767 License changes are often inconsequential.
4768 For example, the license text's copyright
4769 year might have changed.
4770 </note>
4771 </para></listitem>
4772 <listitem><para>
4773 Custom patches carried by the older version of
4774 the recipe might fail to apply to the new
4775 version.
4776 For these cases, you need to review the
4777 failures.
4778 Patches might not be necessary for the new
4779 version of the software if the upgraded version
4780 has fixed those issues.
4781 If a patch is necessary and failing, you need
4782 to rebase it into the new version.
4783 </para></listitem>
4784 </itemizedlist>
4785 </para></listitem>
4786 <listitem><para>
4787 <emphasis>Optionally Attempt to Build for Several Architectures:</emphasis>
4788 Once you successfully build the new software for a
4789 given architecture, you could test the build for
4790 other architectures by changing the
4791 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
4792 variable and rebuilding the software.
4793 This optional step is especially important if the
4794 recipe is to be released publicly.
4795 </para></listitem>
4796 <listitem><para>
4797 <emphasis>Check the Upstream Change Log or Release Notes:</emphasis>
4798 Checking both these reveals if new features exist that
4799 could break backwards-compatibility.
4800 If so, you need to take steps to mitigate or eliminate
4801 that situation.
4802 </para></listitem>
4803 <listitem><para>
4804 <emphasis>Optionally Create a Bootable Image and Test:</emphasis>
4805 If you want, you can test the new software by booting
4806 it onto actual hardware.
4807 </para></listitem>
4808 <listitem><para>
4809 <emphasis>Create a Commit with the Change in the Layer Repository:</emphasis>
4810 After all builds work and any testing is successful,
4811 you can create commits for any changes in the layer
4812 holding your upgraded recipe.
4813 </para></listitem>
4814 </orderedlist>
4815 </para>
4816 </section>
4817 </section>
4818
4819 <section id='finding-the-temporary-source-code'>
4820 <title>Finding Temporary Source Code</title>
4821
4822 <para>
4823 You might find it helpful during development to modify the
4824 temporary source code used by recipes to build packages.
4825 For example, suppose you are developing a patch and you need to
4826 experiment a bit to figure out your solution.
4827 After you have initially built the package, you can iteratively
4828 tweak the source code, which is located in the
4829 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>,
4830 and then you can force a re-compile and quickly test your altered
4831 code.
4832 Once you settle on a solution, you can then preserve your changes
4833 in the form of patches.
4834 </para>
4835
4836 <para>
4837 During a build, the unpacked temporary source code used by recipes
4838 to build packages is available in the Build Directory as
4839 defined by the
4840 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
4841 variable.
4842 Below is the default value for the <filename>S</filename> variable
4843 as defined in the
4844 <filename>meta/conf/bitbake.conf</filename> configuration file
4845 in the
4846 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>:
4847 <literallayout class='monospaced'>
4848 S = "${WORKDIR}/${BP}"
4849 </literallayout>
4850 You should be aware that many recipes override the
4851 <filename>S</filename> variable.
4852 For example, recipes that fetch their source from Git usually set
4853 <filename>S</filename> to <filename>${WORKDIR}/git</filename>.
4854 <note>
4855 The
4856 <ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink>
4857 represents the base recipe name, which consists of the name
4858 and version:
4859 <literallayout class='monospaced'>
4860 BP = "${BPN}-${PV}"
4861 </literallayout>
4862 </note>
4863 </para>
4864
4865 <para>
4866 The path to the work directory for the recipe
4867 (<ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>)
4868 is defined as follows:
4869 <literallayout class='monospaced'>
4870 ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}
4871 </literallayout>
4872 The actual directory depends on several things:
4873 <itemizedlist>
4874 <listitem><para>
4875 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>:
4876 The top-level build output directory.
4877 </para></listitem>
4878 <listitem><para>
4879 <ulink url='&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS'><filename>MULTIMACH_TARGET_SYS</filename></ulink>:
4880 The target system identifier.
4881 </para></listitem>
4882 <listitem><para>
4883 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>:
4884 The recipe name.
4885 </para></listitem>
4886 <listitem><para>
4887 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTENDPE'><filename>EXTENDPE</filename></ulink>:
4888 The epoch - (if
4889 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>
4890 is not specified, which is usually the case for most
4891 recipes, then <filename>EXTENDPE</filename> is blank).
4892 </para></listitem>
4893 <listitem><para>
4894 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>:
4895 The recipe version.
4896 </para></listitem>
4897 <listitem><para>
4898 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>:
4899 The recipe revision.
4900 </para></listitem>
4901 </itemizedlist>
4902 </para>
4903
4904 <para>
4905 As an example, assume a Source Directory top-level folder
4906 named <filename>poky</filename>, a default Build Directory at
4907 <filename>poky/build</filename>, and a
4908 <filename>qemux86-poky-linux</filename> machine target
4909 system.
4910 Furthermore, suppose your recipe is named
4911 <filename>foo_1.3.0.bb</filename>.
4912 In this case, the work directory the build system uses to
4913 build the package would be as follows:
4914 <literallayout class='monospaced'>
4915 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
4916 </literallayout>
4917 </para>
4918 </section>
4919
4920 <section id="using-a-quilt-workflow">
4921 <title>Using Quilt in Your Workflow</title>
4922
4923 <para>
4924 <ulink url='http://savannah.nongnu.org/projects/quilt'>Quilt</ulink>
4925 is a powerful tool that allows you to capture source code changes
4926 without having a clean source tree.
4927 This section outlines the typical workflow you can use to modify
4928 source code, test changes, and then preserve the changes in the
4929 form of a patch all using Quilt.
4930 <note><title>Tip</title>
4931 With regard to preserving changes to source files, if you
4932 clean a recipe or have <filename>rm_work</filename> enabled,
4933 the
4934 <ulink url='&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow'><filename>devtool</filename> workflow</ulink>
4935 as described in the Yocto Project Application Development
4936 and the Extensible Software Development Kit (eSDK) manual
4937 is a safer development flow than the flow that uses Quilt.
4938 </note>
4939 </para>
4940
4941 <para>
4942 Follow these general steps:
4943 <orderedlist>
4944 <listitem><para>
4945 <emphasis>Find the Source Code:</emphasis>
4946 Temporary source code used by the OpenEmbedded build system
4947 is kept in the
4948 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
4949 See the
4950 "<link linkend='finding-the-temporary-source-code'>Finding Temporary Source Code</link>"
4951 section to learn how to locate the directory that has the
4952 temporary source code for a particular package.
4953 </para></listitem>
4954 <listitem><para>
4955 <emphasis>Change Your Working Directory:</emphasis>
4956 You need to be in the directory that has the temporary
4957 source code.
4958 That directory is defined by the
4959 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
4960 variable.</para></listitem>
4961 <listitem><para>
4962 <emphasis>Create a New Patch:</emphasis>
4963 Before modifying source code, you need to create a new
4964 patch.
4965 To create a new patch file, use
4966 <filename>quilt new</filename> as below:
4967 <literallayout class='monospaced'>
4968 $ quilt new my_changes.patch
4969 </literallayout>
4970 </para></listitem>
4971 <listitem><para>
4972 <emphasis>Notify Quilt and Add Files:</emphasis>
4973 After creating the patch, you need to notify Quilt about
4974 the files you plan to edit.
4975 You notify Quilt by adding the files to the patch you
4976 just created:
4977 <literallayout class='monospaced'>
4978 $ quilt add file1.c file2.c file3.c
4979 </literallayout>
4980 </para></listitem>
4981 <listitem><para>
4982 <emphasis>Edit the Files:</emphasis>
4983 Make your changes in the source code to the files you added
4984 to the patch.
4985 </para></listitem>
4986 <listitem><para>
4987 <emphasis>Test Your Changes:</emphasis>
4988 Once you have modified the source code, the easiest way to
4989 test your changes is by calling the
4990 <filename>do_compile</filename> task as shown in the
4991 following example:
4992 <literallayout class='monospaced'>
4993 $ bitbake -c compile -f <replaceable>package</replaceable>
4994 </literallayout>
4995 The <filename>-f</filename> or <filename>--force</filename>
4996 option forces the specified task to execute.
4997 If you find problems with your code, you can just keep
4998 editing and re-testing iteratively until things work
4999 as expected.
5000 <note>
5001 All the modifications you make to the temporary
5002 source code disappear once you run the
5003 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-clean'><filename>do_clean</filename></ulink>
5004 or
5005 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleanall'><filename>do_cleanall</filename></ulink>
5006 tasks using BitBake (i.e.
5007 <filename>bitbake -c clean <replaceable>package</replaceable></filename>
5008 and
5009 <filename>bitbake -c cleanall <replaceable>package</replaceable></filename>).
5010 Modifications will also disappear if you use the
5011 <filename>rm_work</filename> feature as described
5012 in the
5013 "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-saving-memory-during-a-build'>Conserving Disk Space During Builds</ulink>"
5014 section.
5015 </note>
5016 </para></listitem>
5017 <listitem><para>
5018 <emphasis>Generate the Patch:</emphasis>
5019 Once your changes work as expected, you need to use Quilt
5020 to generate the final patch that contains all your
5021 modifications.
5022 <literallayout class='monospaced'>
5023 $ quilt refresh
5024 </literallayout>
5025 At this point, the <filename>my_changes.patch</filename>
5026 file has all your edits made to the
5027 <filename>file1.c</filename>, <filename>file2.c</filename>,
5028 and <filename>file3.c</filename> files.</para>
5029
5030 <para>You can find the resulting patch file in the
5031 <filename>patches/</filename> subdirectory of the source
5032 (<filename>S</filename>) directory.
5033 </para></listitem>
5034 <listitem><para>
5035 <emphasis>Copy the Patch File:</emphasis>
5036 For simplicity, copy the patch file into a directory
5037 named <filename>files</filename>, which you can create
5038 in the same directory that holds the recipe
5039 (<filename>.bb</filename>) file or the append
5040 (<filename>.bbappend</filename>) file.
5041 Placing the patch here guarantees that the OpenEmbedded
5042 build system will find the patch.
5043 Next, add the patch into the
5044 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>
5045 of the recipe.
5046 Here is an example:
5047 <literallayout class='monospaced'>
5048 SRC_URI += "file://my_changes.patch"
5049 </literallayout>
5050 </para></listitem>
5051 </orderedlist>
5052 </para>
5053 </section>
5054
5055 <section id="platdev-appdev-devshell">
5056 <title>Using a Development Shell</title>
5057
5058 <para>
5059 When debugging certain commands or even when just editing packages,
5060 <filename>devshell</filename> can be a useful tool.
5061 When you invoke <filename>devshell</filename>, all tasks up to and
5062 including
5063 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
5064 are run for the specified target.
5065 Then, a new terminal is opened and you are placed in
5066 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>,
5067 the source directory.
5068 In the new terminal, all the OpenEmbedded build-related environment variables are
5069 still defined so you can use commands such as <filename>configure</filename> and
5070 <filename>make</filename>.
5071 The commands execute just as if the OpenEmbedded build system were executing them.
5072 Consequently, working this way can be helpful when debugging a build or preparing
5073 software to be used with the OpenEmbedded build system.
5074 </para>
5075
5076 <para>
5077 Following is an example that uses <filename>devshell</filename> on a target named
5078 <filename>matchbox-desktop</filename>:
5079 <literallayout class='monospaced'>
5080 $ bitbake matchbox-desktop -c devshell
5081 </literallayout>
5082 </para>
5083
5084 <para>
5085 This command spawns a terminal with a shell prompt within the OpenEmbedded build environment.
5086 The <ulink url='&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL'><filename>OE_TERMINAL</filename></ulink>
5087 variable controls what type of shell is opened.
5088 </para>
5089
5090 <para>
5091 For spawned terminals, the following occurs:
5092 <itemizedlist>
5093 <listitem><para>The <filename>PATH</filename> variable includes the
5094 cross-toolchain.</para></listitem>
5095 <listitem><para>The <filename>pkgconfig</filename> variables find the correct
5096 <filename>.pc</filename> files.</para></listitem>
5097 <listitem><para>The <filename>configure</filename> command finds the
5098 Yocto Project site files as well as any other necessary files.</para></listitem>
5099 </itemizedlist>
5100 </para>
5101
5102 <para>
5103 Within this environment, you can run configure or compile
5104 commands as if they were being run by
5105 the OpenEmbedded build system itself.
5106 As noted earlier, the working directory also automatically changes to the
5107 Source Directory (<ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>).
5108 </para>
5109
5110 <para>
5111 To manually run a specific task using <filename>devshell</filename>,
5112 run the corresponding <filename>run.*</filename> script in
5113 the
5114 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/temp</filename>
5115 directory (e.g.,
5116 <filename>run.do_configure.</filename><replaceable>pid</replaceable>).
5117 If a task's script does not exist, which would be the case if the task was
5118 skipped by way of the sstate cache, you can create the task by first running
5119 it outside of the <filename>devshell</filename>:
5120 <literallayout class='monospaced'>
5121 $ bitbake -c <replaceable>task</replaceable>
5122 </literallayout>
5123 <note><title>Notes</title>
5124 <itemizedlist>
5125 <listitem><para>Execution of a task's <filename>run.*</filename>
5126 script and BitBake's execution of a task are identical.
5127 In other words, running the script re-runs the task
5128 just as it would be run using the
5129 <filename>bitbake -c</filename> command.
5130 </para></listitem>
5131 <listitem><para>Any <filename>run.*</filename> file that does not
5132 have a <filename>.pid</filename> extension is a
5133 symbolic link (symlink) to the most recent version of that
5134 file.
5135 </para></listitem>
5136 </itemizedlist>
5137 </note>
5138 </para>
5139
5140 <para>
5141 Remember, that the <filename>devshell</filename> is a mechanism that allows
5142 you to get into the BitBake task execution environment.
5143 And as such, all commands must be called just as BitBake would call them.
5144 That means you need to provide the appropriate options for
5145 cross-compilation and so forth as applicable.
5146 </para>
5147
5148 <para>
5149 When you are finished using <filename>devshell</filename>, exit the shell
5150 or close the terminal window.
5151 </para>
5152
5153 <note><title>Notes</title>
5154 <itemizedlist>
5155 <listitem><para>
5156 It is worth remembering that when using <filename>devshell</filename>
5157 you need to use the full compiler name such as <filename>arm-poky-linux-gnueabi-gcc</filename>
5158 instead of just using <filename>gcc</filename>.
5159 The same applies to other applications such as <filename>binutils</filename>,
5160 <filename>libtool</filename> and so forth.
5161 BitBake sets up environment variables such as <filename>CC</filename>
5162 to assist applications, such as <filename>make</filename> to find the correct tools.
5163 </para></listitem>
5164 <listitem><para>
5165 It is also worth noting that <filename>devshell</filename> still works over
5166 X11 forwarding and similar situations.
5167 </para></listitem>
5168 </itemizedlist>
5169 </note>
5170 </section>
5171
5172 <section id="platdev-appdev-devpyshell">
5173 <title>Using a Development Python Shell</title>
5174
5175 <para>
5176 Similar to working within a development shell as described in
5177 the previous section, you can also spawn and work within an
5178 interactive Python development shell.
5179 When debugging certain commands or even when just editing packages,
5180 <filename>devpyshell</filename> can be a useful tool.
5181 When you invoke <filename>devpyshell</filename>, all tasks up to and
5182 including
5183 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
5184 are run for the specified target.
5185 Then a new terminal is opened.
5186 Additionally, key Python objects and code are available in the same
5187 way they are to BitBake tasks, in particular, the data store 'd'.
5188 So, commands such as the following are useful when exploring the data
5189 store and running functions:
5190 <literallayout class='monospaced'>
5191 pydevshell> d.getVar("STAGING_DIR")
5192 '/media/build1/poky/build/tmp/sysroots'
5193 pydevshell> d.getVar("STAGING_DIR")
5194 '${TMPDIR}/sysroots'
5195 pydevshell> d.setVar("FOO", "bar")
5196 pydevshell> d.getVar("FOO")
5197 'bar'
5198 pydevshell> d.delVar("FOO")
5199 pydevshell> d.getVar("FOO")
5200 pydevshell> bb.build.exec_func("do_unpack", d)
5201 pydevshell>
5202 </literallayout>
5203 The commands execute just as if the OpenEmbedded build system were executing them.
5204 Consequently, working this way can be helpful when debugging a build or preparing
5205 software to be used with the OpenEmbedded build system.
5206 </para>
5207
5208 <para>
5209 Following is an example that uses <filename>devpyshell</filename> on a target named
5210 <filename>matchbox-desktop</filename>:
5211 <literallayout class='monospaced'>
5212 $ bitbake matchbox-desktop -c devpyshell
5213 </literallayout>
5214 </para>
5215
5216 <para>
5217 This command spawns a terminal and places you in an interactive
5218 Python interpreter within the OpenEmbedded build environment.
5219 The <ulink url='&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL'><filename>OE_TERMINAL</filename></ulink>
5220 variable controls what type of shell is opened.
5221 </para>
5222
5223 <para>
5224 When you are finished using <filename>devpyshell</filename>, you
5225 can exit the shell either by using Ctrl+d or closing the terminal
5226 window.
5227 </para>
5228 </section>
5229
5230 <section id='dev-building'>
5231 <title>Building</title>
5232
5233 <para>
5234 This section describes various build procedures.
5235 For example, the steps needed for a simple build, a target that
5236 uses multiple configurations, building an image for more than
5237 one machine, and so forth.
5238 </para>
5239
5240 <section id='dev-building-a-simple-image'>
5241 <title>Building a Simple Image</title>
5242
5243 <para>
5244 In the development environment, you need to build an image
5245 whenever you change hardware support, add or change system
5246 libraries, or add or change services that have dependencies.
5247 Several methods exist that allow you to build an image within
5248 the Yocto Project.
5249 This section presents the basic steps you need to build a
5250 simple image using BitBake from a build host running Linux.
5251 <note><title>Notes</title>
5252 <itemizedlist>
5253 <listitem><para>
5254 For information on how to build an image using
5255 <ulink url='&YOCTO_DOCS_REF_URL;#toaster-term'>Toaster</ulink>,
5256 see the
5257 <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>.
5258 </para></listitem>
5259 <listitem><para>
5260 For information on how to use
5261 <filename>devtool</filename> to build images, see
5262 the
5263 "<ulink url='&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow'>Using <filename>devtool</filename> in Your SDK Workflow</ulink>"
5264 section in the Yocto Project Application
5265 Development and the Extensible Software Development
5266 Kit (eSDK) manual.
5267 </para></listitem>
5268 <listitem><para>
5269 For a quick example on how to build an image using
5270 the OpenEmbedded build system, see the
5271 <ulink url='&YOCTO_DOCS_BRIEF_URL;'>Yocto Project Quick Build</ulink>
5272 document.
5273 </para></listitem>
5274 </itemizedlist>
5275 </note>
5276 </para>
5277
5278 <para>
5279 The build process creates an entire Linux distribution from
5280 source and places it in your
5281 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
5282 under <filename>tmp/deploy/images</filename>.
5283 For detailed information on the build process using BitBake,
5284 see the
5285 "<ulink url='&YOCTO_DOCS_OM_URL;#images-dev-environment'>Images</ulink>"
5286 section in the Yocto Project Overview and Concepts Manual.
5287 </para>
5288
5289 <para>
5290 The following figure and list overviews the build process:
5291 <imagedata fileref="figures/bitbake-build-flow.png" width="7in" depth="4in" align="center" scalefit="1" />
5292 <orderedlist>
5293 <listitem><para>
5294 <emphasis>Set up Your Host Development System to Support
5295 Development Using the Yocto Project</emphasis>:
5296 See the
5297 "<link linkend='dev-manual-start'>Setting Up to Use the Yocto Project</link>"
5298 section for options on how to get a build host ready to
5299 use the Yocto Project.
5300 </para></listitem>
5301 <listitem><para>
5302 <emphasis>Initialize the Build Environment:</emphasis>
5303 Initialize the build environment by sourcing the build
5304 environment script (i.e.
5305 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>):
5306 <literallayout class='monospaced'>
5307 $ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>]
5308 </literallayout></para>
5309
5310 <para>When you use the initialization script, the
5311 OpenEmbedded build system uses
5312 <filename>build</filename> as the default Build
5313 Directory in your current work directory.
5314 You can use a <replaceable>build_dir</replaceable>
5315 argument with the script to specify a different build
5316 directory.
5317 <note><title>Tip</title>
5318 A common practice is to use a different Build
5319 Directory for different targets.
5320 For example, <filename>~/build/x86</filename> for a
5321 <filename>qemux86</filename> target, and
5322 <filename>~/build/arm</filename> for a
5323 <filename>qemuarm</filename> target.
5324 </note>
5325 </para></listitem>
5326 <listitem><para>
5327 <emphasis>Make Sure Your <filename>local.conf</filename>
5328 File is Correct:</emphasis>
5329 Ensure the <filename>conf/local.conf</filename>
5330 configuration file, which is found in the Build
5331 Directory, is set up how you want it.
5332 This file defines many aspects of the build environment
5333 including the target machine architecture through the
5334 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'>MACHINE</ulink></filename> variable,
5335 the packaging format used during the build
5336 (<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>),
5337 and a centralized tarball download directory through the
5338 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> variable.
5339 </para></listitem>
5340 <listitem><para>
5341 <emphasis>Build the Image:</emphasis>
5342 Build the image using the <filename>bitbake</filename>
5343 command:
5344 <literallayout class='monospaced'>
5345 $ bitbake <replaceable>target</replaceable>
5346 </literallayout>
5347 <note>
5348 For information on BitBake, see the
5349 <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
5350 </note>
5351 The <replaceable>target</replaceable> is the name of the
5352 recipe you want to build.
5353 Common targets are the images in
5354 <filename>meta/recipes-core/images</filename>,
5355 <filename>meta/recipes-sato/images</filename>, and so
5356 forth all found in the
5357 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
5358 Or, the target can be the name of a recipe for a
5359 specific piece of software such as BusyBox.
5360 For more details about the images the OpenEmbedded build
5361 system supports, see the
5362 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
5363 chapter in the Yocto Project Reference Manual.</para>
5364
5365 <para>As an example, the following command builds the
5366 <filename>core-image-minimal</filename> image:
5367 <literallayout class='monospaced'>
5368 $ bitbake core-image-minimal
5369 </literallayout>
5370 Once an image has been built, it often needs to be
5371 installed.
5372 The images and kernels built by the OpenEmbedded
5373 build system are placed in the Build Directory in
5374 <filename class="directory">tmp/deploy/images</filename>.
5375 For information on how to run pre-built images such as
5376 <filename>qemux86</filename> and <filename>qemuarm</filename>,
5377 see the
5378 <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
5379 manual.
5380 For information about how to install these images,
5381 see the documentation for your particular board or
5382 machine.
5383 </para></listitem>
5384 </orderedlist>
5385 </para>
5386 </section>
5387
5388 <section id='dev-building-images-for-multiple-targets-using-multiple-configurations'>
5389 <title>Building Images for Multiple Targets Using Multiple Configurations</title>
5390
5391 <para>
5392 You can use a single <filename>bitbake</filename> command
5393 to build multiple images or packages for different targets
5394 where each image or package requires a different configuration
5395 (multiple configuration builds).
5396 The builds, in this scenario, are sometimes referred to as
5397 "multiconfigs", and this section uses that term throughout.
5398 </para>
5399
5400 <para>
5401 This section describes how to set up for multiple
5402 configuration builds and how to account for cross-build
5403 dependencies between the multiconfigs.
5404 </para>
5405
5406 <section id='dev-setting-up-and-running-a-multiple-configuration-build'>
5407 <title>Setting Up and Running a Multiple Configuration Build</title>
5408
5409 <para>
5410 To accomplish a multiple configuration build, you must
5411 define each target's configuration separately using
5412 a parallel configuration file in the
5413 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>,
5414 and you must follow a required file hierarchy.
5415 Additionally, you must enable the multiple configuration
5416 builds in your <filename>local.conf</filename> file.
5417 </para>
5418
5419 <para>
5420 Follow these steps to set up and execute multiple
5421 configuration builds:
5422 <itemizedlist>
5423 <listitem><para>
5424 <emphasis>Create Separate Configuration Files</emphasis>:
5425 You need to create a single configuration file for
5426 each build target (each multiconfig).
5427 Minimally, each configuration file must define the
5428 machine and the temporary directory BitBake uses
5429 for the build.
5430 Suggested practice dictates that you do not
5431 overlap the temporary directories
5432 used during the builds.
5433 However, it is possible that you can share the
5434 temporary directory
5435 (<ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>).
5436 For example, consider a scenario with two
5437 different multiconfigs for the same
5438 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>: "qemux86" built for
5439 two distributions such as "poky" and "poky-lsb".
5440 In this case, you might want to use the same
5441 <filename>TMPDIR</filename>.</para>
5442
5443 <para>Here is an example showing the minimal
5444 statements needed in a configuration file for
5445 a "qemux86" target whose temporary build directory
5446 is <filename>tmpmultix86</filename>:
5447 <literallayout class='monospaced'>
5448 MACHINE="qemux86"
5449 TMPDIR="${TOPDIR}/tmpmultix86"
5450 </literallayout></para>
5451
5452 <para>The location for these multiconfig
5453 configuration files is specific.
5454 They must reside in the current build directory in
5455 a sub-directory of <filename>conf</filename> named
5456 <filename>multiconfig</filename>.
5457 Following is an example that defines two
5458 configuration files for the "x86" and "arm"
5459 multiconfigs:
5460 <imagedata fileref="figures/multiconfig_files.png" align="center" width="4in" depth="3in" />
5461 </para>
5462
5463 <para>The reason for this required file hierarchy
5464 is because the <filename>BBPATH</filename> variable
5465 is not constructed until the layers are parsed.
5466 Consequently, using the configuration file as a
5467 pre-configuration file is not possible unless it is
5468 located in the current working directory.
5469 </para></listitem>
5470 <listitem><para>
5471 <emphasis>Add the BitBake Multi-configuration Variable to the Local Configuration File</emphasis>:
5472 Use the
5473 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBMULTICONFIG'><filename>BBMULTICONFIG</filename></ulink>
5474 variable in your
5475 <filename>conf/local.conf</filename> configuration
5476 file to specify each multiconfig.
5477 Continuing with the example from the previous
5478 figure, the <filename>BBMULTICONFIG</filename>
5479 variable needs to enable two multiconfigs: "x86"
5480 and "arm" by specifying each configuration file:
5481 <literallayout class='monospaced'>
5482 BBMULTICONFIG = "x86 arm"
5483 </literallayout>
5484 <note>
5485 A "default" configuration already exists by
5486 definition.
5487 This configuration is named: "" (i.e. empty
5488 string) and is defined by the variables coming
5489 from your <filename>local.conf</filename> file.
5490 Consequently, the previous example actually
5491 adds two additional configurations to your
5492 build: "arm" and "x86" along with "".
5493 </note>
5494 </para></listitem>
5495 <listitem><para>
5496 <emphasis>Launch BitBake</emphasis>:
5497 Use the following BitBake command form to launch the
5498 multiple configuration build:
5499 <literallayout class='monospaced'>
5500 $ bitbake [mc:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable> [[[mc:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable>] ... ]
5501 </literallayout>
5502 For the example in this section, the following
5503 command applies:
5504 <literallayout class='monospaced'>
5505 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
5506 </literallayout>
5507 The previous BitBake command builds a
5508 <filename>core-image-minimal</filename> image that
5509 is configured through the
5510 <filename>x86.conf</filename> configuration file,
5511 a <filename>core-image-sato</filename>
5512 image that is configured through the
5513 <filename>arm.conf</filename> configuration file
5514 and a <filename>core-image-base</filename> that is
5515 configured through your
5516 <filename>local.conf</filename> configuration file.
5517 </para></listitem>
5518 </itemizedlist>
5519 <note>
5520 Support for multiple configuration builds in the
5521 Yocto Project &DISTRO; (&DISTRO_NAME;) Release does
5522 not include Shared State (sstate) optimizations.
5523 Consequently, if a build uses the same object twice
5524 in, for example, two different
5525 <filename>TMPDIR</filename> directories, the build
5526 either loads from an existing sstate cache for that
5527 build at the start or builds the object fresh.
5528 </note>
5529 </para>
5530 </section>
5531
5532 <section id='dev-enabling-multiple-configuration-build-dependencies'>
5533 <title>Enabling Multiple Configuration Build Dependencies</title>
5534
5535 <para>
5536 Sometimes dependencies can exist between targets
5537 (multiconfigs) in a multiple configuration build.
5538 For example, suppose that in order to build a
5539 <filename>core-image-sato</filename> image for an "x86"
5540 multiconfig, the root filesystem of an "arm"
5541 multiconfig must exist.
5542 This dependency is essentially that the
5543 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image'><filename>do_image</filename></ulink>
5544 task in the <filename>core-image-sato</filename> recipe
5545 depends on the completion of the
5546 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink>
5547 task of the <filename>core-image-minimal</filename>
5548 recipe.
5549 </para>
5550
5551 <para>
5552 To enable dependencies in a multiple configuration
5553 build, you must declare the dependencies in the recipe
5554 using the following statement form:
5555 <literallayout class='monospaced'>
5556 <replaceable>task_or_package</replaceable>[mcdepends] = "mc:<replaceable>from_multiconfig</replaceable>:<replaceable>to_multiconfig</replaceable>:<replaceable>recipe_name</replaceable>:<replaceable>task_on_which_to_depend</replaceable>"
5557 </literallayout>
5558 To better show how to use this statement, consider the
5559 example scenario from the first paragraph of this section.
5560 The following statement needs to be added to the recipe
5561 that builds the <filename>core-image-sato</filename>
5562 image:
5563 <literallayout class='monospaced'>
5564 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
5565 </literallayout>
5566 In this example, the
5567 <replaceable>from_multiconfig</replaceable> is "x86".
5568 The <replaceable>to_multiconfig</replaceable> is "arm".
5569 The task on which the <filename>do_image</filename> task
5570 in the recipe depends is the <filename>do_rootfs</filename>
5571 task from the <filename>core-image-minimal</filename>
5572 recipe associated with the "arm" multiconfig.
5573 </para>
5574
5575 <para>
5576 Once you set up this dependency, you can build the
5577 "x86" multiconfig using a BitBake command as follows:
5578 <literallayout class='monospaced'>
5579 $ bitbake mc:x86:core-image-sato
5580 </literallayout>
5581 This command executes all the tasks needed to create
5582 the <filename>core-image-sato</filename> image for the
5583 "x86" multiconfig.
5584 Because of the dependency, BitBake also executes through
5585 the <filename>do_rootfs</filename> task for the "arm"
5586 multiconfig build.
5587 </para>
5588
5589 <para>
5590 Having a recipe depend on the root filesystem of another
5591 build might not seem that useful.
5592 Consider this change to the statement in the
5593 <filename>core-image-sato</filename> recipe:
5594 <literallayout class='monospaced'>
5595 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
5596 </literallayout>
5597 In this case, BitBake must create the
5598 <filename>core-image-minimal</filename> image for the
5599 "arm" build since the "x86" build depends on it.
5600 </para>
5601
5602 <para>
5603 Because "x86" and "arm" are enabled for multiple
5604 configuration builds and have separate configuration
5605 files, BitBake places the artifacts for each build in the
5606 respective temporary build directories (i.e.
5607 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>).
5608 </para>
5609 </section>
5610 </section>
5611
5612 <section id='building-an-initramfs-image'>
5613 <title>Building an Initial RAM Filesystem (initramfs) Image</title>
5614
5615 <para>
5616 An initial RAM filesystem (initramfs) image provides a temporary
5617 root filesystem used for early system initialization (e.g.
5618 loading of modules needed to locate and mount the "real" root
5619 filesystem).
5620 <note>
5621 The initramfs image is the successor of initial RAM disk
5622 (initrd).
5623 It is a "copy in and out" (cpio) archive of the initial
5624 filesystem that gets loaded into memory during the Linux
5625 startup process.
5626 Because Linux uses the contents of the archive during
5627 initialization, the initramfs image needs to contain all of the
5628 device drivers and tools needed to mount the final root
5629 filesystem.
5630 </note>
5631 </para>
5632
5633 <para>
5634 Follow these steps to create an initramfs image:
5635 <orderedlist>
5636 <listitem><para>
5637 <emphasis>Create the initramfs Image Recipe:</emphasis>
5638 You can reference the
5639 <filename>core-image-minimal-initramfs.bb</filename>
5640 recipe found in the <filename>meta/recipes-core</filename>
5641 directory of the
5642 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
5643 as an example from which to work.
5644 </para></listitem>
5645 <listitem><para>
5646 <emphasis>Decide if You Need to Bundle the initramfs Image
5647 Into the Kernel Image:</emphasis>
5648 If you want the initramfs image that is built to be
5649 bundled in with the kernel image, set the
5650 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink>
5651 variable to "1" in your <filename>local.conf</filename>
5652 configuration file and set the
5653 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE'><filename>INITRAMFS_IMAGE</filename></ulink>
5654 variable in the recipe that builds the kernel image.
5655 <note><title>Tip</title>
5656 It is recommended that you do bundle the initramfs
5657 image with the kernel image to avoid circular
5658 dependencies between the kernel recipe and the
5659 initramfs recipe should the initramfs image
5660 include kernel modules.
5661 </note>
5662 Setting the <filename>INITRAMFS_IMAGE_BUNDLE</filename>
5663 flag causes the initramfs image to be unpacked
5664 into the <filename>${B}/usr/</filename> directory.
5665 The unpacked initramfs image is then passed to the kernel's
5666 <filename>Makefile</filename> using the
5667 <ulink url='&YOCTO_DOCS_REF_URL;#var-CONFIG_INITRAMFS_SOURCE'><filename>CONFIG_INITRAMFS_SOURCE</filename></ulink>
5668 variable, allowing the initramfs image to be built into
5669 the kernel normally.
5670 <note>
5671 If you choose to not bundle the initramfs image with
5672 the kernel image, you are essentially using an
5673 <ulink url='https://en.wikipedia.org/wiki/Initrd'>Initial RAM Disk (initrd)</ulink>.
5674 Creating an initrd is handled primarily through the
5675 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRD_IMAGE'><filename>INITRD_IMAGE</filename></ulink>,
5676 <filename>INITRD_LIVE</filename>, and
5677 <filename>INITRD_IMAGE_LIVE</filename> variables.
5678 For more information, see the
5679 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/image-live.bbclass'><filename>image-live.bbclass</filename></ulink>
5680 file.
5681 </note>
5682 </para></listitem>
5683 <listitem><para>
5684 <emphasis>Optionally Add Items to the initramfs Image
5685 Through the initramfs Image Recipe:</emphasis>
5686 If you add items to the initramfs image by way of its
5687 recipe, you should use
5688 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></ulink>
5689 rather than
5690 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>.
5691 <filename>PACKAGE_INSTALL</filename> gives more direct
5692 control of what is added to the image as compared to
5693 the defaults you might not necessarily want that are
5694 set by the
5695 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image'><filename>image</filename></ulink>
5696 or
5697 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-core-image'><filename>core-image</filename></ulink>
5698 classes.
5699 </para></listitem>
5700 <listitem><para>
5701 <emphasis>Build the Kernel Image and the initramfs
5702 Image:</emphasis>
5703 Build your kernel image using BitBake.
5704 Because the initramfs image recipe is a dependency of the
5705 kernel image, the initramfs image is built as well and
5706 bundled with the kernel image if you used the
5707 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink>
5708 variable described earlier.
5709 </para></listitem>
5710 </orderedlist>
5711 </para>
5712 </section>
5713
5714 <section id='building-a-tiny-system'>
5715 <title>Building a Tiny System</title>
5716
5717 <para>
5718 Very small distributions have some significant advantages such
5719 as requiring less on-die or in-package memory (cheaper), better
5720 performance through efficient cache usage, lower power requirements
5721 due to less memory, faster boot times, and reduced development
5722 overhead.
5723 Some real-world examples where a very small distribution gives
5724 you distinct advantages are digital cameras, medical devices,
5725 and small headless systems.
5726 </para>
5727
5728 <para>
5729 This section presents information that shows you how you can
5730 trim your distribution to even smaller sizes than the
5731 <filename>poky-tiny</filename> distribution, which is around
5732 5 Mbytes, that can be built out-of-the-box using the Yocto Project.
5733 </para>
5734
5735 <section id='tiny-system-overview'>
5736 <title>Overview</title>
5737
5738 <para>
5739 The following list presents the overall steps you need to
5740 consider and perform to create distributions with smaller
5741 root filesystems, achieve faster boot times, maintain your critical
5742 functionality, and avoid initial RAM disks:
5743 <itemizedlist>
5744 <listitem><para>
5745 <link linkend='goals-and-guiding-principles'>Determine your goals and guiding principles.</link>
5746 </para></listitem>
5747 <listitem><para>
5748 <link linkend='understand-what-gives-your-image-size'>Understand what contributes to your image size.</link>
5749 </para></listitem>
5750 <listitem><para>
5751 <link linkend='trim-the-root-filesystem'>Reduce the size of the root filesystem.</link>
5752 </para></listitem>
5753 <listitem><para>
5754 <link linkend='trim-the-kernel'>Reduce the size of the kernel.</link>
5755 </para></listitem>
5756 <listitem><para>
5757 <link linkend='remove-package-management-requirements'>Eliminate packaging requirements.</link>
5758 </para></listitem>
5759 <listitem><para>
5760 <link linkend='look-for-other-ways-to-minimize-size'>Look for other ways to minimize size.</link>
5761 </para></listitem>
5762 <listitem><para>
5763 <link linkend='iterate-on-the-process'>Iterate on the process.</link>
5764 </para></listitem>
5765 </itemizedlist>
5766 </para>
5767 </section>
5768
5769 <section id='goals-and-guiding-principles'>
5770 <title>Goals and Guiding Principles</title>
5771
5772 <para>
5773 Before you can reach your destination, you need to know
5774 where you are going.
5775 Here is an example list that you can use as a guide when
5776 creating very small distributions:
5777 <itemizedlist>
5778 <listitem><para>Determine how much space you need
5779 (e.g. a kernel that is 1 Mbyte or less and
5780 a root filesystem that is 3 Mbytes or less).
5781 </para></listitem>
5782 <listitem><para>Find the areas that are currently
5783 taking 90% of the space and concentrate on reducing
5784 those areas.
5785 </para></listitem>
5786 <listitem><para>Do not create any difficult "hacks"
5787 to achieve your goals.</para></listitem>
5788 <listitem><para>Leverage the device-specific
5789 options.</para></listitem>
5790 <listitem><para>Work in a separate layer so that you
5791 keep changes isolated.
5792 For information on how to create layers, see
5793 the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section.
5794 </para></listitem>
5795 </itemizedlist>
5796 </para>
5797 </section>
5798
5799 <section id='understand-what-gives-your-image-size'>
5800 <title>Understand What Contributes to Your Image Size</title>
5801
5802 <para>
5803 It is easiest to have something to start with when creating
5804 your own distribution.
5805 You can use the Yocto Project out-of-the-box to create the
5806 <filename>poky-tiny</filename> distribution.
5807 Ultimately, you will want to make changes in your own
5808 distribution that are likely modeled after
5809 <filename>poky-tiny</filename>.
5810 <note>
5811 To use <filename>poky-tiny</filename> in your build,
5812 set the
5813 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
5814 variable in your
5815 <filename>local.conf</filename> file to "poky-tiny"
5816 as described in the
5817 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
5818 section.
5819 </note>
5820 </para>
5821
5822 <para>
5823 Understanding some memory concepts will help you reduce the
5824 system size.
5825 Memory consists of static, dynamic, and temporary memory.
5826 Static memory is the TEXT (code), DATA (initialized data
5827 in the code), and BSS (uninitialized data) sections.
5828 Dynamic memory represents memory that is allocated at runtime:
5829 stacks, hash tables, and so forth.
5830 Temporary memory is recovered after the boot process.
5831 This memory consists of memory used for decompressing
5832 the kernel and for the <filename>__init__</filename>
5833 functions.
5834 </para>
5835
5836 <para>
5837 To help you see where you currently are with kernel and root
5838 filesystem sizes, you can use two tools found in the
5839 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> in
5840 the <filename>scripts/tiny/</filename> directory:
5841 <itemizedlist>
5842 <listitem><para><filename>ksize.py</filename>: Reports
5843 component sizes for the kernel build objects.
5844 </para></listitem>
5845 <listitem><para><filename>dirsize.py</filename>: Reports
5846 component sizes for the root filesystem.</para></listitem>
5847 </itemizedlist>
5848 This next tool and command help you organize configuration
5849 fragments and view file dependencies in a human-readable form:
5850 <itemizedlist>
5851 <listitem><para><filename>merge_config.sh</filename>:
5852 Helps you manage configuration files and fragments
5853 within the kernel.
5854 With this tool, you can merge individual configuration
5855 fragments together.
5856 The tool allows you to make overrides and warns you
5857 of any missing configuration options.
5858 The tool is ideal for allowing you to iterate on
5859 configurations, create minimal configurations, and
5860 create configuration files for different machines
5861 without having to duplicate your process.</para>
5862 <para>The <filename>merge_config.sh</filename> script is
5863 part of the Linux Yocto kernel Git repositories
5864 (i.e. <filename>linux-yocto-3.14</filename>,
5865 <filename>linux-yocto-3.10</filename>,
5866 <filename>linux-yocto-3.8</filename>, and so forth)
5867 in the
5868 <filename>scripts/kconfig</filename> directory.</para>
5869 <para>For more information on configuration fragments,
5870 see the
5871 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#creating-config-fragments'>Creating Configuration Fragments</ulink>"
5872 section in the Yocto Project Linux Kernel Development
5873 Manual.
5874 </para></listitem>
5875 <listitem><para><filename>bitbake -u taskexp -g <replaceable>bitbake_target</replaceable></filename>:
5876 Using the BitBake command with these options brings up
5877 a Dependency Explorer from which you can view file
5878 dependencies.
5879 Understanding these dependencies allows you to make
5880 informed decisions when cutting out various pieces of the
5881 kernel and root filesystem.</para></listitem>
5882 </itemizedlist>
5883 </para>
5884 </section>
5885
5886 <section id='trim-the-root-filesystem'>
5887 <title>Trim the Root Filesystem</title>
5888
5889 <para>
5890 The root filesystem is made up of packages for booting,
5891 libraries, and applications.
5892 To change things, you can configure how the packaging happens,
5893 which changes the way you build them.
5894 You can also modify the filesystem itself or select a different
5895 filesystem.
5896 </para>
5897
5898 <para>
5899 First, find out what is hogging your root filesystem by running the
5900 <filename>dirsize.py</filename> script from your root directory:
5901 <literallayout class='monospaced'>
5902 $ cd <replaceable>root-directory-of-image</replaceable>
5903 $ dirsize.py 100000 > dirsize-100k.log
5904 $ cat dirsize-100k.log
5905 </literallayout>
5906 You can apply a filter to the script to ignore files under
5907 a certain size.
5908 The previous example filters out any files below 100 Kbytes.
5909 The sizes reported by the tool are uncompressed, and thus
5910 will be smaller by a relatively constant factor in a
5911 compressed root filesystem.
5912 When you examine your log file, you can focus on areas of the
5913 root filesystem that take up large amounts of memory.
5914 </para>
5915
5916 <para>
5917 You need to be sure that what you eliminate does not cripple
5918 the functionality you need.
5919 One way to see how packages relate to each other is by using
5920 the Dependency Explorer UI with the BitBake command:
5921 <literallayout class='monospaced'>
5922 $ cd <replaceable>image-directory</replaceable>
5923 $ bitbake -u taskexp -g <replaceable>image</replaceable>
5924 </literallayout>
5925 Use the interface to select potential packages you wish to
5926 eliminate and see their dependency relationships.
5927 </para>
5928
5929 <para>
5930 When deciding how to reduce the size, get rid of packages that
5931 result in minimal impact on the feature set.
5932 For example, you might not need a VGA display.
5933 Or, you might be able to get by with <filename>devtmpfs</filename>
5934 and <filename>mdev</filename> instead of
5935 <filename>udev</filename>.
5936 </para>
5937
5938 <para>
5939 Use your <filename>local.conf</filename> file to make changes.
5940 For example, to eliminate <filename>udev</filename> and
5941 <filename>glib</filename>, set the following in the
5942 local configuration file:
5943 <literallayout class='monospaced'>
5944 VIRTUAL-RUNTIME_dev_manager = ""
5945 </literallayout>
5946 </para>
5947
5948 <para>
5949 Finally, you should consider exactly the type of root
5950 filesystem you need to meet your needs while also reducing
5951 its size.
5952 For example, consider <filename>cramfs</filename>,
5953 <filename>squashfs</filename>, <filename>ubifs</filename>,
5954 <filename>ext2</filename>, or an <filename>initramfs</filename>
5955 using <filename>initramfs</filename>.
5956 Be aware that <filename>ext3</filename> requires a 1 Mbyte
5957 journal.
5958 If you are okay with running read-only, you do not need this
5959 journal.
5960 </para>
5961
5962 <note>
5963 After each round of elimination, you need to rebuild your
5964 system and then use the tools to see the effects of your
5965 reductions.
5966 </note>
5967 </section>
5968
5969 <section id='trim-the-kernel'>
5970 <title>Trim the Kernel</title>
5971
5972 <para>
5973 The kernel is built by including policies for hardware-independent
5974 aspects.
5975 What subsystems do you enable?
5976 For what architecture are you building?
5977 Which drivers do you build by default?
5978 <note>You can modify the kernel source if you want to help
5979 with boot time.
5980 </note>
5981 </para>
5982
5983 <para>
5984 Run the <filename>ksize.py</filename> script from the top-level
5985 Linux build directory to get an idea of what is making up
5986 the kernel:
5987 <literallayout class='monospaced'>
5988 $ cd <replaceable>top-level-linux-build-directory</replaceable>
5989 $ ksize.py > ksize.log
5990 $ cat ksize.log
5991 </literallayout>
5992 When you examine the log, you will see how much space is
5993 taken up with the built-in <filename>.o</filename> files for
5994 drivers, networking, core kernel files, filesystem, sound,
5995 and so forth.
5996 The sizes reported by the tool are uncompressed, and thus
5997 will be smaller by a relatively constant factor in a compressed
5998 kernel image.
5999 Look to reduce the areas that are large and taking up around
6000 the "90% rule."
6001 </para>
6002
6003 <para>
6004 To examine, or drill down, into any particular area, use the
6005 <filename>-d</filename> option with the script:
6006 <literallayout class='monospaced'>
6007 $ ksize.py -d > ksize.log
6008 </literallayout>
6009 Using this option breaks out the individual file information
6010 for each area of the kernel (e.g. drivers, networking, and
6011 so forth).
6012 </para>
6013
6014 <para>
6015 Use your log file to see what you can eliminate from the kernel
6016 based on features you can let go.
6017 For example, if you are not going to need sound, you do not
6018 need any drivers that support sound.
6019 </para>
6020
6021 <para>
6022 After figuring out what to eliminate, you need to reconfigure
6023 the kernel to reflect those changes during the next build.
6024 You could run <filename>menuconfig</filename> and make all your
6025 changes at once.
6026 However, that makes it difficult to see the effects of your
6027 individual eliminations and also makes it difficult to replicate
6028 the changes for perhaps another target device.
6029 A better method is to start with no configurations using
6030 <filename>allnoconfig</filename>, create configuration
6031 fragments for individual changes, and then manage the
6032 fragments into a single configuration file using
6033 <filename>merge_config.sh</filename>.
6034 The tool makes it easy for you to iterate using the
6035 configuration change and build cycle.
6036 </para>
6037
6038 <para>
6039 Each time you make configuration changes, you need to rebuild
6040 the kernel and check to see what impact your changes had on
6041 the overall size.
6042 </para>
6043 </section>
6044
6045 <section id='remove-package-management-requirements'>
6046 <title>Remove Package Management Requirements</title>
6047
6048 <para>
6049 Packaging requirements add size to the image.
6050 One way to reduce the size of the image is to remove all the
6051 packaging requirements from the image.
6052 This reduction includes both removing the package manager
6053 and its unique dependencies as well as removing the package
6054 management data itself.
6055 </para>
6056
6057 <para>
6058 To eliminate all the packaging requirements for an image,
6059 be sure that "package-management" is not part of your
6060 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
6061 statement for the image.
6062 When you remove this feature, you are removing the package
6063 manager as well as its dependencies from the root filesystem.
6064 </para>
6065 </section>
6066
6067 <section id='look-for-other-ways-to-minimize-size'>
6068 <title>Look for Other Ways to Minimize Size</title>
6069
6070 <para>
6071 Depending on your particular circumstances, other areas that you
6072 can trim likely exist.
6073 The key to finding these areas is through tools and methods
6074 described here combined with experimentation and iteration.
6075 Here are a couple of areas to experiment with:
6076 <itemizedlist>
6077 <listitem><para><filename>glibc</filename>:
6078 In general, follow this process:
6079 <orderedlist>
6080 <listitem><para>Remove <filename>glibc</filename>
6081 features from
6082 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
6083 that you think you do not need.</para></listitem>
6084 <listitem><para>Build your distribution.
6085 </para></listitem>
6086 <listitem><para>If the build fails due to missing
6087 symbols in a package, determine if you can
6088 reconfigure the package to not need those
6089 features.
6090 For example, change the configuration to not
6091 support wide character support as is done for
6092 <filename>ncurses</filename>.
6093 Or, if support for those characters is needed,
6094 determine what <filename>glibc</filename>
6095 features provide the support and restore the
6096 configuration.
6097 </para></listitem>
6098 <listitem><para>Rebuild and repeat the process.
6099 </para></listitem>
6100 </orderedlist></para></listitem>
6101 <listitem><para><filename>busybox</filename>:
6102 For BusyBox, use a process similar as described for
6103 <filename>glibc</filename>.
6104 A difference is you will need to boot the resulting
6105 system to see if you are able to do everything you
6106 expect from the running system.
6107 You need to be sure to integrate configuration fragments
6108 into Busybox because BusyBox handles its own core
6109 features and then allows you to add configuration
6110 fragments on top.
6111 </para></listitem>
6112 </itemizedlist>
6113 </para>
6114 </section>
6115
6116 <section id='iterate-on-the-process'>
6117 <title>Iterate on the Process</title>
6118
6119 <para>
6120 If you have not reached your goals on system size, you need
6121 to iterate on the process.
6122 The process is the same.
6123 Use the tools and see just what is taking up 90% of the root
6124 filesystem and the kernel.
6125 Decide what you can eliminate without limiting your device
6126 beyond what you need.
6127 </para>
6128
6129 <para>
6130 Depending on your system, a good place to look might be
6131 Busybox, which provides a stripped down
6132 version of Unix tools in a single, executable file.
6133 You might be able to drop virtual terminal services or perhaps
6134 ipv6.
6135 </para>
6136 </section>
6137 </section>
6138
6139 <section id='building-images-for-more-than-one-machine'>
6140 <title>Building Images for More than One Machine</title>
6141
6142 <para>
6143 A common scenario developers face is creating images for several
6144 different machines that use the same software environment.
6145 In this situation, it is tempting to set the
6146 tunings and optimization flags for each build specifically for
6147 the targeted hardware (i.e. "maxing out" the tunings).
6148 Doing so can considerably add to build times and package feed
6149 maintenance collectively for the machines.
6150 For example, selecting tunes that are extremely specific to a
6151 CPU core used in a system might enable some micro optimizations
6152 in GCC for that particular system but would otherwise not gain
6153 you much of a performance difference across the other systems
6154 as compared to using a more general tuning across all the builds
6155 (e.g. setting
6156 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink>
6157 specifically for each machine's build).
6158 Rather than "max out" each build's tunings, you can take steps that
6159 cause the OpenEmbedded build system to reuse software across the
6160 various machines where it makes sense.
6161 </para>
6162
6163 <para>
6164 If build speed and package feed maintenance are considerations,
6165 you should consider the points in this section that can help you
6166 optimize your tunings to best consider build times and package
6167 feed maintenance.
6168 <itemizedlist>
6169 <listitem><para>
6170 <emphasis>Share the Build Directory:</emphasis>
6171 If at all possible, share the
6172 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
6173 across builds.
6174 The Yocto Project supports switching between different
6175 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
6176 values in the same <filename>TMPDIR</filename>.
6177 This practice is well supported and regularly used by
6178 developers when building for multiple machines.
6179 When you use the same <filename>TMPDIR</filename> for
6180 multiple machine builds, the OpenEmbedded build system can
6181 reuse the existing native and often cross-recipes for
6182 multiple machines.
6183 Thus, build time decreases.
6184 <note>
6185 If
6186 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
6187 settings change or fundamental configuration settings
6188 such as the filesystem layout, you need to work with
6189 a clean <filename>TMPDIR</filename>.
6190 Sharing <filename>TMPDIR</filename> under these
6191 circumstances might work but since it is not
6192 guaranteed, you should use a clean
6193 <filename>TMPDIR</filename>.
6194 </note>
6195 </para></listitem>
6196 <listitem><para>
6197 <emphasis>Enable the Appropriate Package Architecture:</emphasis>
6198 By default, the OpenEmbedded build system enables three
6199 levels of package architectures: "all", "tune" or "package",
6200 and "machine".
6201 Any given recipe usually selects one of these package
6202 architectures (types) for its output.
6203 Depending for what a given recipe creates packages, making
6204 sure you enable the appropriate package architecture can
6205 directly impact the build time.</para>
6206
6207 <para>A recipe that just generates scripts can enable
6208 "all" architecture because there are no binaries to build.
6209 To specifically enable "all" architecture, be sure your
6210 recipe inherits the
6211 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
6212 class.
6213 This class is useful for "all" architectures because it
6214 configures many variables so packages can be used across
6215 multiple architectures.</para>
6216
6217 <para>If your recipe needs to generate packages that are
6218 machine-specific or when one of the build or runtime
6219 dependencies is already machine-architecture dependent,
6220 which makes your recipe also machine-architecture dependent,
6221 make sure your recipe enables the "machine" package
6222 architecture through the
6223 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>
6224 variable:
6225 <literallayout class='monospaced'>
6226 PACKAGE_ARCH = "${MACHINE_ARCH}"
6227 </literallayout>
6228 When you do not specifically enable a package
6229 architecture through the
6230 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>,
6231 The OpenEmbedded build system defaults to the
6232 <ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH'><filename>TUNE_PKGARCH</filename></ulink>
6233 setting:
6234 <literallayout class='monospaced'>
6235 PACKAGE_ARCH = "${TUNE_PKGARCH}"
6236 </literallayout>
6237 </para></listitem>
6238 <listitem><para>
6239 <emphasis>Choose a Generic Tuning File if Possible:</emphasis>
6240 Some tunes are more generic and can run on multiple targets
6241 (e.g. an <filename>armv5</filename> set of packages could
6242 run on <filename>armv6</filename> and
6243 <filename>armv7</filename> processors in most cases).
6244 Similarly, <filename>i486</filename> binaries could work
6245 on <filename>i586</filename> and higher processors.
6246 You should realize, however, that advances on newer
6247 processor versions would not be used.</para>
6248
6249 <para>If you select the same tune for several different
6250 machines, the OpenEmbedded build system reuses software
6251 previously built, thus speeding up the overall build time.
6252 Realize that even though a new sysroot for each machine is
6253 generated, the software is not recompiled and only one
6254 package feed exists.
6255 </para></listitem>
6256 <listitem><para>
6257 <emphasis>Manage Granular Level Packaging:</emphasis>
6258 Sometimes cases exist where injecting another level of
6259 package architecture beyond the three higher levels noted
6260 earlier can be useful.
6261 For example, consider how NXP (formerly Freescale) allows
6262 for the easy reuse of binary packages in their layer
6263 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/'><filename>meta-freescale</filename></ulink>.
6264 In this example, the
6265 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass'><filename>fsl-dynamic-packagearch</filename></ulink>
6266 class shares GPU packages for i.MX53 boards because
6267 all boards share the AMD GPU.
6268 The i.MX6-based boards can do the same because all boards
6269 share the Vivante GPU.
6270 This class inspects the BitBake datastore to identify if
6271 the package provides or depends on one of the
6272 sub-architecture values.
6273 If so, the class sets the
6274 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>
6275 value based on the <filename>MACHINE_SUBARCH</filename>
6276 value.
6277 If the package does not provide or depend on one of the
6278 sub-architecture values but it matches a value in the
6279 machine-specific filter, it sets
6280 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>.
6281 This behavior reduces the number of packages built and
6282 saves build time by reusing binaries.
6283 </para></listitem>
6284 <listitem><para>
6285 <emphasis>Use Tools to Debug Issues:</emphasis>
6286 Sometimes you can run into situations where software is
6287 being rebuilt when you think it should not be.
6288 For example, the OpenEmbedded build system might not be
6289 using shared state between machines when you think it
6290 should be.
6291 These types of situations are usually due to references
6292 to machine-specific variables such as
6293 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>,
6294 <ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'><filename>SERIAL_CONSOLES</filename></ulink>,
6295 <ulink url='&YOCTO_DOCS_REF_URL;#var-XSERVER'><filename>XSERVER</filename></ulink>,
6296 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'><filename>MACHINE_FEATURES</filename></ulink>,
6297 and so forth in code that is supposed to only be
6298 tune-specific or when the recipe depends
6299 (<ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
6300 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
6301 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
6302 <ulink url='&YOCTO_DOCS_REF_URL;#var-RSUGGESTS'><filename>RSUGGESTS</filename></ulink>,
6303 and so forth) on some other recipe that already has
6304 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>
6305 defined as "${MACHINE_ARCH}".
6306 <note>
6307 Patches to fix any issues identified are most welcome
6308 as these issues occasionally do occur.
6309 </note></para>
6310
6311 <para>For such cases, you can use some tools to help you
6312 sort out the situation:
6313 <itemizedlist>
6314 <listitem><para>
6315 <emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis>
6316 You can find this tool in the
6317 <filename>scripts</filename> directory of the
6318 Source Repositories.
6319 See the comments in the script for information on
6320 how to use the tool.
6321 </para></listitem>
6322 <listitem><para>
6323 <emphasis>BitBake's "-S printdiff" Option:</emphasis>
6324 Using this option causes BitBake to try to
6325 establish the closest signature match it can
6326 (e.g. in the shared state cache) and then run
6327 <filename>bitbake-diffsigs</filename> over the
6328 matches to determine the stamps and delta where
6329 these two stamp trees diverge.
6330 </para></listitem>
6331 </itemizedlist>
6332 </para></listitem>
6333 </itemizedlist>
6334 </para>
6335 </section>
6336
6337 <section id="building-software-from-an-external-source">
6338 <title>Building Software from an External Source</title>
6339
6340 <para>
6341 By default, the OpenEmbedded build system uses the
6342 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
6343 when building source code.
6344 The build process involves fetching the source files, unpacking
6345 them, and then patching them if necessary before the build takes
6346 place.
6347 </para>
6348
6349 <para>
6350 Situations exist where you might want to build software from source
6351 files that are external to and thus outside of the
6352 OpenEmbedded build system.
6353 For example, suppose you have a project that includes a new BSP with
6354 a heavily customized kernel.
6355 And, you want to minimize exposing the build system to the
6356 development team so that they can focus on their project and
6357 maintain everyone's workflow as much as possible.
6358 In this case, you want a kernel source directory on the development
6359 machine where the development occurs.
6360 You want the recipe's
6361 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
6362 variable to point to the external directory and use it as is, not
6363 copy it.
6364 </para>
6365
6366 <para>
6367 To build from software that comes from an external source, all you
6368 need to do is inherit the
6369 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink>
6370 class and then set the
6371 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>
6372 variable to point to your external source code.
6373 Here are the statements to put in your
6374 <filename>local.conf</filename> file:
6375 <literallayout class='monospaced'>
6376 INHERIT += "externalsrc"
6377 EXTERNALSRC_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>"
6378 </literallayout>
6379 </para>
6380
6381 <para>
6382 This next example shows how to accomplish the same thing by setting
6383 <filename>EXTERNALSRC</filename> in the recipe itself or in the
6384 recipe's append file:
6385 <literallayout class='monospaced'>
6386 EXTERNALSRC = "<replaceable>path</replaceable>"
6387 EXTERNALSRC_BUILD = "<replaceable>path</replaceable>"
6388 </literallayout>
6389 <note>
6390 In order for these settings to take effect, you must globally
6391 or locally inherit the
6392 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink>
6393 class.
6394 </note>
6395 </para>
6396
6397 <para>
6398 By default, <filename>externalsrc.bbclass</filename> builds
6399 the source code in a directory separate from the external source
6400 directory as specified by
6401 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>.
6402 If you need to have the source built in the same directory in
6403 which it resides, or some other nominated directory, you can set
6404 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink>
6405 to point to that directory:
6406 <literallayout class='monospaced'>
6407 EXTERNALSRC_BUILD_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>"
6408 </literallayout>
6409 </para>
6410 </section>
6411
6412 <section id="replicating-a-build-offline">
6413 <title>Replicating a Build Offline</title>
6414
6415 <para>
6416 It can be useful to take a "snapshot" of upstream sources
6417 used in a build and then use that "snapshot" later to
6418 replicate the build offline.
6419 To do so, you need to first prepare and populate your downloads
6420 directory your "snapshot" of files.
6421 Once your downloads directory is ready, you can use it at
6422 any time and from any machine to replicate your build.
6423 </para>
6424
6425 <para>
6426 Follow these steps to populate your Downloads directory:
6427 <orderedlist>
6428 <listitem><para>
6429 <emphasis>Create a Clean Downloads Directory:</emphasis>
6430 Start with an empty downloads directory
6431 (<ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>).
6432 You start with an empty downloads directory by either
6433 removing the files in the existing directory or by
6434 setting
6435 <filename>DL_DIR</filename> to point to either an
6436 empty location or one that does not yet exist.
6437 </para></listitem>
6438 <listitem><para>
6439 <emphasis>Generate Tarballs of the Source Git Repositories:</emphasis>
6440 Edit your <filename>local.conf</filename> configuration
6441 file as follows:
6442 <literallayout class='monospaced'>
6443 DL_DIR = "/home/<replaceable>your-download-dir</replaceable>/"
6444 BB_GENERATE_MIRROR_TARBALLS = "1"
6445 </literallayout>
6446 During the fetch process in the next step, BitBake
6447 gathers the source files and creates tarballs in
6448 the directory pointed to by <filename>DL_DIR</filename>.
6449 See the
6450 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink>
6451 variable for more information.
6452 </para></listitem>
6453 <listitem><para>
6454 <emphasis>Populate Your Downloads Directory Without Building:</emphasis>
6455 Use BitBake to fetch your sources but inhibit the
6456 build:
6457 <literallayout class='monospaced'>
6458 $ bitbake <replaceable>target</replaceable> --runonly=fetch
6459 </literallayout>
6460 The downloads directory (i.e.
6461 <filename>${DL_DIR}</filename>) now has a "snapshot" of
6462 the source files in the form of tarballs, which can
6463 be used for the build.
6464 </para></listitem>
6465 <listitem><para>
6466 <emphasis>Optionally Remove Any Git or other SCM Subdirectories From the Downloads Directory:</emphasis>
6467 If you want, you can clean up your downloads directory
6468 by removing any Git or other Source Control Management
6469 (SCM) subdirectories such as
6470 <filename>${DL_DIR}/git2/*</filename>.
6471 The tarballs already contain these subdirectories.
6472 </para></listitem>
6473 </orderedlist>
6474 </para>
6475
6476 <para>
6477 Once your downloads directory has everything it needs regarding
6478 source files, you can create your "own-mirror" and build
6479 your target.
6480 Understand that you can use the files to build the target
6481 offline from any machine and at any time.
6482 </para>
6483
6484 <para>
6485 Follow these steps to build your target using the files in the
6486 downloads directory:
6487 <orderedlist>
6488 <listitem><para>
6489 <emphasis>Using Local Files Only:</emphasis>
6490 Inside your <filename>local.conf</filename> file, add
6491 the
6492 <ulink url='&YOCTO_DOCS_REF_URL;#var-SOURCE_MIRROR_URL'><filename>SOURCE_MIRROR_URL</filename></ulink>
6493 variable,
6494 inherit the <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-own-mirrors'><filename>own-mirrors</filename></ulink>
6495 class, and use the
6496 <ulink url='&YOCTO_DOCS_BB_URL;#var-bb-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></ulink>
6497 variable to your <filename>local.conf</filename>.
6498 <literallayout class='monospaced'>
6499 SOURCE_MIRROR_URL ?= "file:///home/<replaceable>your-download-dir</replaceable>/"
6500 INHERIT += "own-mirrors"
6501 BB_NO_NETWORK = "1"
6502 </literallayout>
6503 The <filename>SOURCE_MIRROR_URL</filename> and
6504 <filename>own-mirror</filename> class set up the system
6505 to use the downloads directory as your "own mirror".
6506 Using the <filename>BB_NO_NETWORK</filename>
6507 variable makes sure that BitBake's fetching process
6508 in step 3 stays local, which means files from
6509 your "own-mirror" are used.
6510 </para></listitem>
6511 <listitem><para>
6512 <emphasis>Start With a Clean Build:</emphasis>
6513 You can start with a clean build by removing the
6514 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink><filename>}</filename>
6515 directory or using a new
6516 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
6517 </para></listitem>
6518 <listitem><para>
6519 <emphasis>Build Your Target:</emphasis>
6520 Use BitBake to build your target:
6521 <literallayout class='monospaced'>
6522 $ bitbake <replaceable>target</replaceable>
6523 </literallayout>
6524 The build completes using the known local "snapshot" of
6525 source files from your mirror.
6526 The resulting tarballs for your "snapshot" of source
6527 files are in the downloads directory.
6528 <note>
6529 <para>The offline build does not work if recipes
6530 attempt to find the latest version of software
6531 by setting
6532 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
6533 to
6534 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink><filename>}</filename>:
6535 <literallayout class='monospaced'>
6536 SRCREV = "${AUTOREV}"
6537 </literallayout>
6538 When a recipe sets
6539 <filename>SRCREV</filename> to
6540 <filename>${AUTOREV}</filename>, the build system
6541 accesses the network in an attempt to determine the
6542 latest version of software from the SCM.
6543 Typically, recipes that use
6544 <filename>AUTOREV</filename> are custom or
6545 modified recipes.
6546 Recipes that reside in public repositories
6547 usually do not use <filename>AUTOREV</filename>.
6548 </para>
6549
6550 <para>If you do have recipes that use
6551 <filename>AUTOREV</filename>, you can take steps to
6552 still use the recipes in an offline build.
6553 Do the following:
6554 <orderedlist>
6555 <listitem><para>
6556 Use a configuration generated by
6557 enabling
6558 <link linkend='maintaining-build-output-quality'>build history</link>.
6559 </para></listitem>
6560 <listitem><para>
6561 Use the
6562 <filename>buildhistory-collect-srcrevs</filename>
6563 command to collect the stored
6564 <filename>SRCREV</filename> values from
6565 the build's history.
6566 For more information on collecting these
6567 values, see the
6568 "<link linkend='build-history-package-information'>Build History Package Information</link>"
6569 section.
6570 </para></listitem>
6571 <listitem><para>
6572 Once you have the correct source
6573 revisions, you can modify those recipes
6574 to to set <filename>SRCREV</filename>
6575 to specific versions of the software.
6576 </para></listitem>
6577 </orderedlist>
6578 </para>
6579 </note>
6580 </para></listitem>
6581 </orderedlist>
6582 </para>
6583 </section>
6584 </section>
6585
6586 <section id='speeding-up-a-build'>
6587 <title>Speeding Up a Build</title>
6588
6589 <para>
6590 Build time can be an issue.
6591 By default, the build system uses simple controls to try and maximize
6592 build efficiency.
6593 In general, the default settings for all the following variables
6594 result in the most efficient build times when dealing with single
6595 socket systems (i.e. a single CPU).
6596 If you have multiple CPUs, you might try increasing the default
6597 values to gain more speed.
6598 See the descriptions in the glossary for each variable for more
6599 information:
6600 <itemizedlist>
6601 <listitem><para>
6602 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</ulink>
6603 The maximum number of threads BitBake simultaneously executes.
6604 </para></listitem>
6605 <listitem><para>
6606 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink>
6607 The number of threads BitBake uses during parsing.
6608 </para></listitem>
6609 <listitem><para>
6610 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</ulink>
6611 Extra options passed to the <filename>make</filename> command
6612 during the
6613 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
6614 task in order to specify parallel compilation on the
6615 local build host.
6616 </para></listitem>
6617 <listitem><para>
6618 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</ulink>
6619 Extra options passed to the <filename>make</filename> command
6620 during the
6621 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
6622 task in order to specify parallel installation on the
6623 local build host.
6624 </para></listitem>
6625 </itemizedlist>
6626 As mentioned, these variables all scale to the number of processor
6627 cores available on the build system.
6628 For single socket systems, this auto-scaling ensures that the build
6629 system fundamentally takes advantage of potential parallel operations
6630 during the build based on the build machine's capabilities.
6631 </para>
6632
6633 <para>
6634 Following are additional factors that can affect build speed:
6635 <itemizedlist>
6636 <listitem><para>
6637 File system type:
6638 The file system type that the build is being performed on can
6639 also influence performance.
6640 Using <filename>ext4</filename> is recommended as compared
6641 to <filename>ext2</filename> and <filename>ext3</filename>
6642 due to <filename>ext4</filename> improved features
6643 such as extents.
6644 </para></listitem>
6645 <listitem><para>
6646 Disabling the updating of access time using
6647 <filename>noatime</filename>:
6648 The <filename>noatime</filename> mount option prevents the
6649 build system from updating file and directory access times.
6650 </para></listitem>
6651 <listitem><para>
6652 Setting a longer commit:
6653 Using the "commit=" mount option increases the interval
6654 in seconds between disk cache writes.
6655 Changing this interval from the five second default to
6656 something longer increases the risk of data loss but decreases
6657 the need to write to the disk, thus increasing the build
6658 performance.
6659 </para></listitem>
6660 <listitem><para>
6661 Choosing the packaging backend:
6662 Of the available packaging backends, IPK is the fastest.
6663 Additionally, selecting a singular packaging backend also
6664 helps.
6665 </para></listitem>
6666 <listitem><para>
6667 Using <filename>tmpfs</filename> for
6668 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
6669 as a temporary file system:
6670 While this can help speed up the build, the benefits are
6671 limited due to the compiler using
6672 <filename>-pipe</filename>.
6673 The build system goes to some lengths to avoid
6674 <filename>sync()</filename> calls into the
6675 file system on the principle that if there was a significant
6676 failure, the
6677 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
6678 contents could easily be rebuilt.
6679 </para></listitem>
6680 <listitem><para>
6681 Inheriting the
6682 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink>
6683 class:
6684 Inheriting this class has shown to speed up builds due to
6685 significantly lower amounts of data stored in the data
6686 cache as well as on disk.
6687 Inheriting this class also makes cleanup of
6688 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
6689 faster, at the expense of being easily able to dive into the
6690 source code.
6691 File system maintainers have recommended that the fastest way
6692 to clean up large numbers of files is to reformat partitions
6693 rather than delete files due to the linear nature of
6694 partitions.
6695 This, of course, assumes you structure the disk partitions and
6696 file systems in a way that this is practical.
6697 </para></listitem>
6698 </itemizedlist>
6699 Aside from the previous list, you should keep some trade offs in
6700 mind that can help you speed up the build:
6701 <itemizedlist>
6702 <listitem><para>
6703 Remove items from
6704 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
6705 that you might not need.
6706 </para></listitem>
6707 <listitem><para>
6708 Exclude debug symbols and other debug information:
6709 If you do not need these symbols and other debug information,
6710 disabling the <filename>*-dbg</filename> package generation
6711 can speed up the build.
6712 You can disable this generation by setting the
6713 <ulink url='&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></ulink>
6714 variable to "1".
6715 </para></listitem>
6716 <listitem><para>
6717 Disable static library generation for recipes derived from
6718 <filename>autoconf</filename> or <filename>libtool</filename>:
6719 Following is an example showing how to disable static
6720 libraries and still provide an override to handle exceptions:
6721 <literallayout class='monospaced'>
6722 STATICLIBCONF = "--disable-static"
6723 STATICLIBCONF_sqlite3-native = ""
6724 EXTRA_OECONF += "${STATICLIBCONF}"
6725 </literallayout>
6726 <note><title>Notes</title>
6727 <itemizedlist>
6728 <listitem><para>
6729 Some recipes need static libraries in order to work
6730 correctly (e.g. <filename>pseudo-native</filename>
6731 needs <filename>sqlite3-native</filename>).
6732 Overrides, as in the previous example, account for
6733 these kinds of exceptions.
6734 </para></listitem>
6735 <listitem><para>
6736 Some packages have packaging code that assumes the
6737 presence of the static libraries.
6738 If so, you might need to exclude them as well.
6739 </para></listitem>
6740 </itemizedlist>
6741 </note>
6742 </para></listitem>
6743 </itemizedlist>
6744 </para>
6745 </section>
6746
6747 <section id="platdev-working-with-libraries">
6748 <title>Working With Libraries</title>
6749
6750 <para>
6751 Libraries are an integral part of your system.
6752 This section describes some common practices you might find
6753 helpful when working with libraries to build your system:
6754 <itemizedlist>
6755 <listitem><para><link linkend='including-static-library-files'>How to include static library files</link>
6756 </para></listitem>
6757 <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link>
6758 </para></listitem>
6759 <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link>
6760 </para></listitem>
6761 </itemizedlist>
6762 </para>
6763
6764 <section id='including-static-library-files'>
6765 <title>Including Static Library Files</title>
6766
6767 <para>
6768 If you are building a library and the library offers static linking, you can control
6769 which static library files (<filename>*.a</filename> files) get included in the
6770 built library.
6771 </para>
6772
6773 <para>
6774 The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
6775 and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink>
6776 variables in the
6777 <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed
6778 by the <filename>do_install</filename> task are packaged.
6779 By default, the <filename>PACKAGES</filename> variable includes
6780 <filename>${PN}-staticdev</filename>, which represents all static library files.
6781 <note>
6782 Some previously released versions of the Yocto Project
6783 defined the static library files through
6784 <filename>${PN}-dev</filename>.
6785 </note>
6786 Following is part of the BitBake configuration file, where
6787 you can see how the static library files are defined:
6788 <literallayout class='monospaced'>
6789 PACKAGE_BEFORE_PN ?= ""
6790 PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
6791 PACKAGES_DYNAMIC = "^${PN}-locale-.*"
6792 FILES = ""
6793
6794 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
6795 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
6796 ${base_bindir}/* ${base_sbindir}/* \
6797 ${base_libdir}/*${SOLIBS} \
6798 ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \
6799 ${datadir}/${BPN} ${libdir}/${BPN}/* \
6800 ${datadir}/pixmaps ${datadir}/applications \
6801 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
6802 ${libdir}/bonobo/servers"
6803
6804 FILES_${PN}-bin = "${bindir}/* ${sbindir}/*"
6805
6806 FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
6807 ${datadir}/gnome/help"
6808 SECTION_${PN}-doc = "doc"
6809
6810 FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
6811 FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
6812 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
6813 ${datadir}/aclocal ${base_libdir}/*.o \
6814 ${libdir}/${BPN}/*.la ${base_libdir}/*.la"
6815 SECTION_${PN}-dev = "devel"
6816 ALLOW_EMPTY_${PN}-dev = "1"
6817 RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
6818
6819 FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
6820 SECTION_${PN}-staticdev = "devel"
6821 RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
6822 </literallayout>
6823 </para>
6824 </section>
6825
6826 <section id="combining-multiple-versions-library-files-into-one-image">
6827 <title>Combining Multiple Versions of Library Files into One Image</title>
6828
6829 <para>
6830 The build system offers the ability to build libraries with different
6831 target optimizations or architecture formats and combine these together
6832 into one system image.
6833 You can link different binaries in the image
6834 against the different libraries as needed for specific use cases.
6835 This feature is called "Multilib."
6836 </para>
6837
6838 <para>
6839 An example would be where you have most of a system compiled in 32-bit
6840 mode using 32-bit libraries, but you have something large, like a database
6841 engine, that needs to be a 64-bit application and uses 64-bit libraries.
6842 Multilib allows you to get the best of both 32-bit and 64-bit libraries.
6843 </para>
6844
6845 <para>
6846 While the Multilib feature is most commonly used for 32 and 64-bit differences,
6847 the approach the build system uses facilitates different target optimizations.
6848 You could compile some binaries to use one set of libraries and other binaries
6849 to use a different set of libraries.
6850 The libraries could differ in architecture, compiler options, or other
6851 optimizations.
6852 </para>
6853
6854 <para>
6855 Several examples exist in the
6856 <filename>meta-skeleton</filename> layer found in the
6857 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>:
6858 <itemizedlist>
6859 <listitem><para><filename>conf/multilib-example.conf</filename>
6860 configuration file</para></listitem>
6861 <listitem><para><filename>conf/multilib-example2.conf</filename>
6862 configuration file</para></listitem>
6863 <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename>
6864 recipe</para></listitem>
6865 </itemizedlist>
6866 </para>
6867
6868 <section id='preparing-to-use-multilib'>
6869 <title>Preparing to Use Multilib</title>
6870
6871 <para>
6872 User-specific requirements drive the Multilib feature.
6873 Consequently, there is no one "out-of-the-box" configuration that likely
6874 exists to meet your needs.
6875 </para>
6876
6877 <para>
6878 In order to enable Multilib, you first need to ensure your recipe is
6879 extended to support multiple libraries.
6880 Many standard recipes are already extended and support multiple libraries.
6881 You can check in the <filename>meta/conf/multilib.conf</filename>
6882 configuration file in the
6883 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> to see how this is
6884 done using the
6885 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink>
6886 variable.
6887 Eventually, all recipes will be covered and this list will
6888 not be needed.
6889 </para>
6890
6891 <para>
6892 For the most part, the Multilib class extension works automatically to
6893 extend the package name from <filename>${PN}</filename> to
6894 <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename>
6895 is the particular multilib (e.g. "lib32-" or "lib64-").
6896 Standard variables such as
6897 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
6898 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
6899 <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>,
6900 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
6901 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, and
6902 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink>
6903 are automatically extended by the system.
6904 If you are extending any manual code in the recipe, you can use the
6905 <filename>${MLPREFIX}</filename> variable to ensure those names are extended
6906 correctly.
6907 This automatic extension code resides in <filename>multilib.bbclass</filename>.
6908 </para>
6909 </section>
6910
6911 <section id='using-multilib'>
6912 <title>Using Multilib</title>
6913
6914 <para>
6915 After you have set up the recipes, you need to define the actual
6916 combination of multiple libraries you want to build.
6917 You accomplish this through your <filename>local.conf</filename>
6918 configuration file in the
6919 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
6920 An example configuration would be as follows:
6921 <literallayout class='monospaced'>
6922 MACHINE = "qemux86-64"
6923 require conf/multilib.conf
6924 MULTILIBS = "multilib:lib32"
6925 DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
6926 IMAGE_INSTALL_append = " lib32-glib-2.0"
6927 </literallayout>
6928 This example enables an
6929 additional library named <filename>lib32</filename> alongside the
6930 normal target packages.
6931 When combining these "lib32" alternatives, the example uses "x86" for tuning.
6932 For information on this particular tuning, see
6933 <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>.
6934 </para>
6935
6936 <para>
6937 The example then includes <filename>lib32-glib-2.0</filename>
6938 in all the images, which illustrates one method of including a
6939 multiple library dependency.
6940 You can use a normal image build to include this dependency,
6941 for example:
6942 <literallayout class='monospaced'>
6943 $ bitbake core-image-sato
6944 </literallayout>
6945 You can also build Multilib packages specifically with a command like this:
6946 <literallayout class='monospaced'>
6947 $ bitbake lib32-glib-2.0
6948 </literallayout>
6949 </para>
6950 </section>
6951
6952 <section id='additional-implementation-details'>
6953 <title>Additional Implementation Details</title>
6954
6955 <para>
6956 Generic implementation details as well as details that are
6957 specific to package management systems exist.
6958 Following are implementation details that exist regardless
6959 of the package management system:
6960 <itemizedlist>
6961 <listitem><para>The typical convention used for the
6962 class extension code as used by
6963 Multilib assumes that all package names specified
6964 in
6965 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
6966 that contain <filename>${PN}</filename> have
6967 <filename>${PN}</filename> at the start of the name.
6968 When that convention is not followed and
6969 <filename>${PN}</filename> appears at
6970 the middle or the end of a name, problems occur.
6971 </para></listitem>
6972 <listitem><para>The
6973 <ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR'><filename>TARGET_VENDOR</filename></ulink>
6974 value under Multilib will be extended to
6975 "-<replaceable>vendor</replaceable>ml<replaceable>multilib</replaceable>"
6976 (e.g. "-pokymllib32" for a "lib32" Multilib with
6977 Poky).
6978 The reason for this slightly unwieldy contraction
6979 is that any "-" characters in the vendor
6980 string presently break Autoconf's
6981 <filename>config.sub</filename>, and
6982 other separators are problematic for different
6983 reasons.
6984 </para></listitem>
6985 </itemizedlist>
6986 </para>
6987
6988 <para>
6989 For the RPM Package Management System, the following implementation details
6990 exist:
6991 <itemizedlist>
6992 <listitem><para>A unique architecture is defined for the Multilib packages,
6993 along with creating a unique deploy folder under
6994 <filename>tmp/deploy/rpm</filename> in the
6995 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
6996 For example, consider <filename>lib32</filename> in a
6997 <filename>qemux86-64</filename> image.
6998 The possible architectures in the system are "all", "qemux86_64",
6999 "lib32_qemux86_64", and "lib32_x86".</para></listitem>
7000 <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from
7001 <filename>${PN}</filename> during RPM packaging.
7002 The naming for a normal RPM package and a Multilib RPM package in a
7003 <filename>qemux86-64</filename> system resolves to something similar to
7004 <filename>bash-4.1-r2.x86_64.rpm</filename> and
7005 <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively.
7006 </para></listitem>
7007 <listitem><para>When installing a Multilib image, the RPM backend first
7008 installs the base image and then installs the Multilib libraries.
7009 </para></listitem>
7010 <listitem><para>The build system relies on RPM to resolve the identical files in the
7011 two (or more) Multilib packages.</para></listitem>
7012 </itemizedlist>
7013 </para>
7014
7015 <para>
7016 For the IPK Package Management System, the following implementation details exist:
7017 <itemizedlist>
7018 <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from
7019 <filename>${PN}</filename> during IPK packaging.
7020 The naming for a normal RPM package and a Multilib IPK package in a
7021 <filename>qemux86-64</filename> system resolves to something like
7022 <filename>bash_4.1-r2.x86_64.ipk</filename> and
7023 <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively.
7024 </para></listitem>
7025 <listitem><para>The IPK deploy folder is not modified with
7026 <filename>${MLPREFIX}</filename> because packages with and without
7027 the Multilib feature can exist in the same folder due to the
7028 <filename>${PN}</filename> differences.</para></listitem>
7029 <listitem><para>IPK defines a sanity check for Multilib installation
7030 using certain rules for file comparison, overridden, etc.
7031 </para></listitem>
7032 </itemizedlist>
7033 </para>
7034 </section>
7035 </section>
7036
7037 <section id='installing-multiple-versions-of-the-same-library'>
7038 <title>Installing Multiple Versions of the Same Library</title>
7039
7040 <para>
7041 Situations can exist where you need to install and use
7042 multiple versions of the same library on the same system
7043 at the same time.
7044 These situations almost always exist when a library API
7045 changes and you have multiple pieces of software that
7046 depend on the separate versions of the library.
7047 To accommodate these situations, you can install multiple
7048 versions of the same library in parallel on the same system.
7049 </para>
7050
7051 <para>
7052 The process is straightforward as long as the libraries use
7053 proper versioning.
7054 With properly versioned libraries, all you need to do to
7055 individually specify the libraries is create separate,
7056 appropriately named recipes where the
7057 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the
7058 name includes a portion that differentiates each library version
7059 (e.g.the major part of the version number).
7060 Thus, instead of having a single recipe that loads one version
7061 of a library (e.g. <filename>clutter</filename>), you provide
7062 multiple recipes that result in different versions
7063 of the libraries you want.
7064 As an example, the following two recipes would allow the
7065 two separate versions of the <filename>clutter</filename>
7066 library to co-exist on the same system:
7067 <literallayout class='monospaced'>
7068 clutter-1.6_1.6.20.bb
7069 clutter-1.8_1.8.4.bb
7070 </literallayout>
7071 Additionally, if you have other recipes that depend on a given
7072 library, you need to use the
7073 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
7074 variable to create the dependency.
7075 Continuing with the same example, if you want to have a recipe
7076 depend on the 1.8 version of the <filename>clutter</filename>
7077 library, use the following in your recipe:
7078 <literallayout class='monospaced'>
7079 DEPENDS = "clutter-1.8"
7080 </literallayout>
7081 </para>
7082 </section>
7083 </section>
7084
7085 <section id='using-x32-psabi'>
7086 <title>Using x32 psABI</title>
7087
7088 <para>
7089 x32 processor-specific Application Binary Interface
7090 (<ulink url='https://software.intel.com/en-us/node/628948'>x32 psABI</ulink>)
7091 is a native 32-bit processor-specific ABI for
7092 <trademark class='registered'>Intel</trademark> 64 (x86-64)
7093 architectures.
7094 An ABI defines the calling conventions between functions in a
7095 processing environment.
7096 The interface determines what registers are used and what the
7097 sizes are for various C data types.
7098 </para>
7099
7100 <para>
7101 Some processing environments prefer using 32-bit applications even
7102 when running on Intel 64-bit platforms.
7103 Consider the i386 psABI, which is a very old 32-bit ABI for Intel
7104 64-bit platforms.
7105 The i386 psABI does not provide efficient use and access of the
7106 Intel 64-bit processor resources, leaving the system underutilized.
7107 Now consider the x86_64 psABI.
7108 This ABI is newer and uses 64-bits for data sizes and program
7109 pointers.
7110 The extra bits increase the footprint size of the programs,
7111 libraries, and also increases the memory and file system size
7112 requirements.
7113 Executing under the x32 psABI enables user programs to utilize CPU
7114 and system resources more efficiently while keeping the memory
7115 footprint of the applications low.
7116 Extra bits are used for registers but not for addressing mechanisms.
7117 </para>
7118
7119 <para>
7120 The Yocto Project supports the final specifications of x32 psABI
7121 as follows:
7122 <itemizedlist>
7123 <listitem><para>
7124 You can create packages and images in x32 psABI format on
7125 x86_64 architecture targets.
7126 </para></listitem>
7127 <listitem><para>
7128 You can successfully build recipes with the x32 toolchain.
7129 </para></listitem>
7130 <listitem><para>
7131 You can create and boot
7132 <filename>core-image-minimal</filename> and
7133 <filename>core-image-sato</filename> images.
7134 </para></listitem>
7135 <listitem><para>
7136 RPM Package Manager (RPM) support exists for x32 binaries.
7137 </para></listitem>
7138 <listitem><para>
7139 Support for large images exists.
7140 </para></listitem>
7141 </itemizedlist>
7142 </para>
7143
7144 <para>
7145 To use the x32 psABI, you need to edit your
7146 <filename>conf/local.conf</filename> configuration file as
7147 follows:
7148 <literallayout class='monospaced'>
7149 MACHINE = "qemux86-64"
7150 DEFAULTTUNE = "x86-64-x32"
7151 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE') \
7152 or 'INVALID')) or 'lib'}"
7153 </literallayout>
7154 Once you have set up your configuration file, use BitBake to
7155 build an image that supports the x32 psABI.
7156 Here is an example:
7157 <literallayout class='monospaced'>
7158 $ bitbake core-image-sato
7159 </literallayout>
7160 </para>
7161 </section>
7162
7163 <section id='enabling-gobject-introspection-support'>
7164 <title>Enabling GObject Introspection Support</title>
7165
7166 <para>
7167 <ulink url='https://wiki.gnome.org/Projects/GObjectIntrospection'>GObject introspection</ulink>
7168 is the standard mechanism for accessing GObject-based software
7169 from runtime environments.
7170 GObject is a feature of the GLib library that provides an object
7171 framework for the GNOME desktop and related software.
7172 GObject Introspection adds information to GObject that allows
7173 objects created within it to be represented across different
7174 programming languages.
7175 If you want to construct GStreamer pipelines using Python, or
7176 control UPnP infrastructure using Javascript and GUPnP,
7177 GObject introspection is the only way to do it.
7178 </para>
7179
7180 <para>
7181 This section describes the Yocto Project support for generating
7182 and packaging GObject introspection data.
7183 GObject introspection data is a description of the
7184 API provided by libraries built on top of GLib framework,
7185 and, in particular, that framework's GObject mechanism.
7186 GObject Introspection Repository (GIR) files go to
7187 <filename>-dev</filename> packages,
7188 <filename>typelib</filename> files go to main packages as they
7189 are packaged together with libraries that are introspected.
7190 </para>
7191
7192 <para>
7193 The data is generated when building such a library, by linking
7194 the library with a small executable binary that asks the library
7195 to describe itself, and then executing the binary and
7196 processing its output.
7197 </para>
7198
7199 <para>
7200 Generating this data in a cross-compilation environment
7201 is difficult because the library is produced for the target
7202 architecture, but its code needs to be executed on the build host.
7203 This problem is solved with the OpenEmbedded build system by
7204 running the code through QEMU, which allows precisely that.
7205 Unfortunately, QEMU does not always work perfectly as mentioned
7206 in the
7207 "<link linkend='known-issues'>Known Issues</link>" section.
7208 </para>
7209
7210 <section id='enabling-the-generation-of-introspection-data'>
7211 <title>Enabling the Generation of Introspection Data</title>
7212
7213 <para>
7214 Enabling the generation of introspection data (GIR files)
7215 in your library package involves the following:
7216 <orderedlist>
7217 <listitem><para>
7218 Inherit the
7219 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-gobject-introspection'><filename>gobject-introspection</filename></ulink>
7220 class.
7221 </para></listitem>
7222 <listitem><para>
7223 Make sure introspection is not disabled anywhere in
7224 the recipe or from anything the recipe includes.
7225 Also, make sure that "gobject-introspection-data" is
7226 not in
7227 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
7228 and that "qemu-usermode" is not in
7229 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
7230 If either of these conditions exist, nothing will
7231 happen.
7232 </para></listitem>
7233 <listitem><para>
7234 Try to build the recipe.
7235 If you encounter build errors that look like
7236 something is unable to find
7237 <filename>.so</filename> libraries, check where these
7238 libraries are located in the source tree and add
7239 the following to the recipe:
7240 <literallayout class='monospaced'>
7241 GIR_EXTRA_LIBS_PATH = "${B}/<replaceable>something</replaceable>/.libs"
7242 </literallayout>
7243 <note>
7244 See recipes in the <filename>oe-core</filename>
7245 repository that use that
7246 <filename>GIR_EXTRA_LIBS_PATH</filename> variable
7247 as an example.
7248 </note>
7249 </para></listitem>
7250 <listitem><para>
7251 Look for any other errors, which probably mean that
7252 introspection support in a package is not entirely
7253 standard, and thus breaks down in a cross-compilation
7254 environment.
7255 For such cases, custom-made fixes are needed.
7256 A good place to ask and receive help in these cases
7257 is the
7258 <ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Yocto Project mailing lists</ulink>.
7259 </para></listitem>
7260 </orderedlist>
7261 <note>
7262 Using a library that no longer builds against the latest
7263 Yocto Project release and prints introspection related
7264 errors is a good candidate for the previous procedure.
7265 </note>
7266 </para>
7267 </section>
7268
7269 <section id='disabling-the-generation-of-introspection-data'>
7270 <title>Disabling the Generation of Introspection Data</title>
7271
7272 <para>
7273 You might find that you do not want to generate
7274 introspection data.
7275 Or, perhaps QEMU does not work on your build host and
7276 target architecture combination.
7277 If so, you can use either of the following methods to
7278 disable GIR file generations:
7279 <itemizedlist>
7280 <listitem><para>
7281 Add the following to your distro configuration:
7282 <literallayout class='monospaced'>
7283 DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
7284 </literallayout>
7285 Adding this statement disables generating
7286 introspection data using QEMU but will still enable
7287 building introspection tools and libraries
7288 (i.e. building them does not require the use of QEMU).
7289 </para></listitem>
7290 <listitem><para>
7291 Add the following to your machine configuration:
7292 <literallayout class='monospaced'>
7293 MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode"
7294 </literallayout>
7295 Adding this statement disables the use of QEMU
7296 when building packages for your machine.
7297 Currently, this feature is used only by introspection
7298 recipes and has the same effect as the previously
7299 described option.
7300 <note>
7301 Future releases of the Yocto Project might have
7302 other features affected by this option.
7303 </note>
7304 </para></listitem>
7305 </itemizedlist>
7306 If you disable introspection data, you can still
7307 obtain it through other means such as copying the data
7308 from a suitable sysroot, or by generating it on the
7309 target hardware.
7310 The OpenEmbedded build system does not currently
7311 provide specific support for these techniques.
7312 </para>
7313 </section>
7314
7315 <section id='testing-that-introspection-works-in-an-image'>
7316 <title>Testing that Introspection Works in an Image</title>
7317
7318 <para>
7319 Use the following procedure to test if generating
7320 introspection data is working in an image:
7321 <orderedlist>
7322 <listitem><para>
7323 Make sure that "gobject-introspection-data" is not in
7324 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
7325 and that "qemu-usermode" is not in
7326 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
7327 </para></listitem>
7328 <listitem><para>
7329 Build <filename>core-image-sato</filename>.
7330 </para></listitem>
7331 <listitem><para>
7332 Launch a Terminal and then start Python in the
7333 terminal.
7334 </para></listitem>
7335 <listitem><para>
7336 Enter the following in the terminal:
7337 <literallayout class='monospaced'>
7338 >>> from gi.repository import GLib
7339 >>> GLib.get_host_name()
7340 </literallayout>
7341 </para></listitem>
7342 <listitem><para>
7343 For something a little more advanced, enter the
7344 following:
7345 <literallayout class='monospaced'>
7346 http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html
7347 </literallayout>
7348 </para></listitem>
7349 </orderedlist>
7350 </para>
7351 </section>
7352
7353 <section id='known-issues'>
7354 <title>Known Issues</title>
7355
7356 <para>
7357 The following know issues exist for
7358 GObject Introspection Support:
7359 <itemizedlist>
7360 <listitem><para>
7361 <filename>qemu-ppc64</filename> immediately crashes.
7362 Consequently, you cannot build introspection data on
7363 that architecture.
7364 </para></listitem>
7365 <listitem><para>
7366 x32 is not supported by QEMU.
7367 Consequently, introspection data is disabled.
7368 </para></listitem>
7369 <listitem><para>
7370 musl causes transient GLib binaries to crash on
7371 assertion failures.
7372 Consequently, generating introspection data is
7373 disabled.
7374 </para></listitem>
7375 <listitem><para>
7376 Because QEMU is not able to run the binaries correctly,
7377 introspection is disabled for some specific packages
7378 under specific architectures (e.g.
7379 <filename>gcr</filename>,
7380 <filename>libsecret</filename>, and
7381 <filename>webkit</filename>).
7382 </para></listitem>
7383 <listitem><para>
7384 QEMU usermode might not work properly when running
7385 64-bit binaries under 32-bit host machines.
7386 In particular, "qemumips64" is known to not work under
7387 i686.
7388 </para></listitem>
7389 </itemizedlist>
7390 </para>
7391 </section>
7392 </section>
7393
7394 <section id='dev-optionally-using-an-external-toolchain'>
7395 <title>Optionally Using an External Toolchain</title>
7396
7397 <para>
7398 You might want to use an external toolchain as part of your
7399 development.
7400 If this is the case, the fundamental steps you need to accomplish
7401 are as follows:
7402 <itemizedlist>
7403 <listitem><para>
7404 Understand where the installed toolchain resides.
7405 For cases where you need to build the external toolchain,
7406 you would need to take separate steps to build and install
7407 the toolchain.
7408 </para></listitem>
7409 <listitem><para>
7410 Make sure you add the layer that contains the toolchain to
7411 your <filename>bblayers.conf</filename> file through the
7412 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink>
7413 variable.
7414 </para></listitem>
7415 <listitem><para>
7416 Set the <filename>EXTERNAL_TOOLCHAIN</filename>
7417 variable in your <filename>local.conf</filename> file
7418 to the location in which you installed the toolchain.
7419 </para></listitem>
7420 </itemizedlist>
7421 A good example of an external toolchain used with the Yocto Project
7422 is <trademark class='registered'>Mentor Graphics</trademark>
7423 Sourcery G++ Toolchain.
7424 You can see information on how to use that particular layer in the
7425 <filename>README</filename> file at
7426 <ulink url='http://github.com/MentorEmbedded/meta-sourcery/'></ulink>.
7427 You can find further information by reading about the
7428 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCMODE'><filename>TCMODE</filename></ulink>
7429 variable in the Yocto Project Reference Manual's variable glossary.
7430 </para>
7431 </section>
7432
7433 <section id='creating-partitioned-images-using-wic'>
7434 <title>Creating Partitioned Images Using Wic</title>
7435
7436 <para>
7437 Creating an image for a particular hardware target using the
7438 OpenEmbedded build system does not necessarily mean you can boot
7439 that image as is on your device.
7440 Physical devices accept and boot images in various ways depending
7441 on the specifics of the device.
7442 Usually, information about the hardware can tell you what image
7443 format the device requires.
7444 Should your device require multiple partitions on an SD card, flash,
7445 or an HDD, you can use the OpenEmbedded Image Creator,
7446 Wic, to create the properly partitioned image.
7447 </para>
7448
7449 <para>
7450 The <filename>wic</filename> command generates partitioned
7451 images from existing OpenEmbedded build artifacts.
7452 Image generation is driven by partitioning commands
7453 contained in an Openembedded kickstart file
7454 (<filename>.wks</filename>) specified either directly on
7455 the command line or as one of a selection of canned
7456 kickstart files as shown with the
7457 <filename>wic list images</filename> command in the
7458 "<link linkend='using-a-provided-kickstart-file'>Using an Existing Kickstart File</link>"
7459 section.
7460 When you apply the command to a given set of build
7461 artifacts, the result is an image or set of images that
7462 can be directly written onto media and used on a particular
7463 system.
7464 <note>
7465 For a kickstart file reference, see the
7466 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-kickstart'>OpenEmbedded Kickstart (<filename>.wks</filename>) Reference</ulink>"
7467 Chapter in the Yocto Project Reference Manual.
7468 </note>
7469 </para>
7470
7471 <para>
7472 The <filename>wic</filename> command and the infrastructure
7473 it is based on is by definition incomplete.
7474 The purpose of the command is to allow the generation of
7475 customized images, and as such, was designed to be
7476 completely extensible through a plugin interface.
7477 See the
7478 "<link linkend='wic-using-the-wic-plugin-interface'>Using the Wic PlugIn Interface</link>"
7479 section for information on these plugins.
7480 </para>
7481
7482 <para>
7483 This section provides some background information on Wic,
7484 describes what you need to have in
7485 place to run the tool, provides instruction on how to use
7486 the Wic utility, provides information on using the Wic plugins
7487 interface, and provides several examples that show how to use
7488 Wic.
7489 </para>
7490
7491 <section id='wic-background'>
7492 <title>Background</title>
7493
7494 <para>
7495 This section provides some background on the Wic utility.
7496 While none of this information is required to use
7497 Wic, you might find it interesting.
7498 <itemizedlist>
7499 <listitem><para>
7500 The name "Wic" is derived from OpenEmbedded
7501 Image Creator (oeic).
7502 The "oe" diphthong in "oeic" was promoted to the
7503 letter "w", because "oeic" is both difficult to
7504 remember and to pronounce.
7505 </para></listitem>
7506 <listitem><para>
7507 Wic is loosely based on the
7508 Meego Image Creator (<filename>mic</filename>)
7509 framework.
7510 The Wic implementation has been
7511 heavily modified to make direct use of OpenEmbedded
7512 build artifacts instead of package installation and
7513 configuration, which are already incorporated within
7514 the OpenEmbedded artifacts.
7515 </para></listitem>
7516 <listitem><para>
7517 Wic is a completely independent
7518 standalone utility that initially provides
7519 easier-to-use and more flexible replacements for an
7520 existing functionality in OE-Core's
7521 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image-live'><filename>image-live</filename></ulink>
7522 class.
7523 The difference between Wic and those examples is
7524 that with Wic the functionality of those scripts is
7525 implemented by a general-purpose partitioning language,
7526 which is based on Redhat kickstart syntax.
7527 </para></listitem>
7528 </itemizedlist>
7529 </para>
7530 </section>
7531
7532 <section id='wic-requirements'>
7533 <title>Requirements</title>
7534
7535 <para>
7536 In order to use the Wic utility with the OpenEmbedded Build
7537 system, your system needs to meet the following
7538 requirements:
7539 <itemizedlist>
7540 <listitem><para>
7541 The Linux distribution on your development host must
7542 support the Yocto Project.
7543 See the
7544 "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>"
7545 section in the Yocto Project Reference Manual for
7546 the list of distributions that support the
7547 Yocto Project.
7548 </para></listitem>
7549 <listitem><para>
7550 The standard system utilities, such as
7551 <filename>cp</filename>, must be installed on your
7552 development host system.
7553 </para></listitem>
7554 <listitem><para>
7555 You must have sourced the build environment
7556 setup script (i.e.
7557 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>)
7558 found in the
7559 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
7560 </para></listitem>
7561 <listitem><para>
7562 You need to have the build artifacts already
7563 available, which typically means that you must
7564 have already created an image using the
7565 Openembedded build system (e.g.
7566 <filename>core-image-minimal</filename>).
7567 While it might seem redundant to generate an image
7568 in order to create an image using
7569 Wic, the current version of
7570 Wic requires the artifacts
7571 in the form generated by the OpenEmbedded build
7572 system.
7573 </para></listitem>
7574 <listitem><para>
7575 You must build several native tools, which are
7576 built to run on the build system:
7577 <literallayout class='monospaced'>
7578 $ bitbake parted-native dosfstools-native mtools-native
7579 </literallayout>
7580 </para></listitem>
7581 <listitem><para>
7582 Include "wic" as part of the
7583 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink>
7584 variable.
7585 </para></listitem>
7586 <listitem><para>
7587 Include the name of the
7588 <ulink url='&YOCTO_DOCS_REF_URL;#openembedded-kickstart-wks-reference'>wic kickstart file</ulink>
7589 as part of the
7590 <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE'><filename>WKS_FILE</filename></ulink>
7591 variable
7592 </para></listitem>
7593 </itemizedlist>
7594 </para>
7595 </section>
7596
7597 <section id='wic-getting-help'>
7598 <title>Getting Help</title>
7599
7600 <para>
7601 You can get general help for the <filename>wic</filename>
7602 command by entering the <filename>wic</filename> command
7603 by itself or by entering the command with a help argument
7604 as follows:
7605 <literallayout class='monospaced'>
7606 $ wic -h
7607 $ wic --help
7608 $ wic help
7609 </literallayout>
7610 </para>
7611
7612 <para>
7613 Currently, Wic supports seven commands:
7614 <filename>cp</filename>, <filename>create</filename>,
7615 <filename>help</filename>, <filename>list</filename>,
7616 <filename>ls</filename>, <filename>rm</filename>, and
7617 <filename>write</filename>.
7618 You can get help for all these commands except "help" by
7619 using the following form:
7620 <literallayout class='monospaced'>
7621 $ wic help <replaceable>command</replaceable>
7622 </literallayout>
7623 For example, the following command returns help for the
7624 <filename>write</filename> command:
7625 <literallayout class='monospaced'>
7626 $ wic help write
7627 </literallayout>
7628 </para>
7629
7630 <para>
7631 Wic supports help for three topics:
7632 <filename>overview</filename>,
7633 <filename>plugins</filename>, and
7634 <filename>kickstart</filename>.
7635 You can get help for any topic using the following form:
7636 <literallayout class='monospaced'>
7637 $ wic help <replaceable>topic</replaceable>
7638 </literallayout>
7639 For example, the following returns overview help for Wic:
7640 <literallayout class='monospaced'>
7641 $ wic help overview
7642 </literallayout>
7643 </para>
7644
7645 <para>
7646 One additional level of help exists for Wic.
7647 You can get help on individual images through the
7648 <filename>list</filename> command.
7649 You can use the <filename>list</filename> command to return the
7650 available Wic images as follows:
7651 <literallayout class='monospaced'>
7652 $ wic list images
7653 genericx86 Create an EFI disk image for genericx86*
7654 beaglebone-yocto Create SD card image for Beaglebone
7655 edgerouter Create SD card image for Edgerouter
7656 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
7657 directdisk-gpt Create a 'pcbios' direct disk image
7658 mkefidisk Create an EFI disk image
7659 directdisk Create a 'pcbios' direct disk image
7660 systemd-bootdisk Create an EFI disk image with systemd-boot
7661 mkhybridiso Create a hybrid ISO image
7662 sdimage-bootpart Create SD card image with a boot partition
7663 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
7664 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
7665 </literallayout>
7666 Once you know the list of available Wic images, you can use
7667 <filename>help</filename> with the command to get help on a
7668 particular image.
7669 For example, the following command returns help on the
7670 "beaglebone-yocto" image:
7671 <literallayout class='monospaced'>
7672 $ wic list beaglebone-yocto help
7673
7674
7675 Creates a partitioned SD card image for Beaglebone.
7676 Boot files are located in the first vfat partition.
7677 </literallayout>
7678 </para>
7679 </section>
7680
7681 <section id='operational-modes'>
7682 <title>Operational Modes</title>
7683
7684 <para>
7685 You can use Wic in two different
7686 modes, depending on how much control you need for
7687 specifying the Openembedded build artifacts that are
7688 used for creating the image: Raw and Cooked:
7689 <itemizedlist>
7690 <listitem><para>
7691 <emphasis>Raw Mode:</emphasis>
7692 You explicitly specify build artifacts through
7693 Wic command-line arguments.
7694 </para></listitem>
7695 <listitem><para>
7696 <emphasis>Cooked Mode:</emphasis>
7697 The current
7698 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
7699 setting and image name are used to automatically
7700 locate and provide the build artifacts.
7701 You just supply a kickstart file and the name
7702 of the image from which to use artifacts.
7703 </para></listitem>
7704 </itemizedlist>
7705 </para>
7706
7707 <para>
7708 Regardless of the mode you use, you need to have the build
7709 artifacts ready and available.
7710 </para>
7711
7712 <section id='raw-mode'>
7713 <title>Raw Mode</title>
7714
7715 <para>
7716 Running Wic in raw mode allows you to specify all the
7717 partitions through the <filename>wic</filename>
7718 command line.
7719 The primary use for raw mode is if you have built
7720 your kernel outside of the Yocto Project
7721 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
7722 In other words, you can point to arbitrary kernel,
7723 root filesystem locations, and so forth.
7724 Contrast this behavior with cooked mode where Wic
7725 looks in the Build Directory (e.g.
7726 <filename>tmp/deploy/images/</filename><replaceable>machine</replaceable>).
7727 </para>
7728
7729 <para>
7730 The general form of the
7731 <filename>wic</filename> command in raw mode is:
7732 <literallayout class='monospaced'>
7733 $ wic create <replaceable>wks_file</replaceable> <replaceable>options</replaceable> ...
7734
7735 Where:
7736
7737 <replaceable>wks_file</replaceable>:
7738 An OpenEmbedded kickstart file. You can provide
7739 your own custom file or use a file from a set of
7740 existing files as described by further options.
7741
7742 optional arguments:
7743 -h, --help show this help message and exit
7744 -o <replaceable>OUTDIR</replaceable>, --outdir <replaceable>OUTDIR</replaceable>
7745 name of directory to create image in
7746 -e <replaceable>IMAGE_NAME</replaceable>, --image-name <replaceable>IMAGE_NAME</replaceable>
7747 name of the image to use the artifacts from e.g. core-
7748 image-sato
7749 -r <replaceable>ROOTFS_DIR</replaceable>, --rootfs-dir <replaceable>ROOTFS_DIR</replaceable>
7750 path to the /rootfs dir to use as the .wks rootfs
7751 source
7752 -b <replaceable>BOOTIMG_DIR</replaceable>, --bootimg-dir <replaceable>BOOTIMG_DIR</replaceable>
7753 path to the dir containing the boot artifacts (e.g.
7754 /EFI or /syslinux dirs) to use as the .wks bootimg
7755 source
7756 -k <replaceable>KERNEL_DIR</replaceable>, --kernel-dir <replaceable>KERNEL_DIR</replaceable>
7757 path to the dir containing the kernel to use in the
7758 .wks bootimg
7759 -n <replaceable>NATIVE_SYSROOT</replaceable>, --native-sysroot <replaceable>NATIVE_SYSROOT</replaceable>
7760 path to the native sysroot containing the tools to use
7761 to build the image
7762 -s, --skip-build-check
7763 skip the build check
7764 -f, --build-rootfs build rootfs
7765 -c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz}
7766 compress image with specified compressor
7767 -m, --bmap generate .bmap
7768 --no-fstab-update Do not change fstab file.
7769 -v <replaceable>VARS_DIR</replaceable>, --vars <replaceable>VARS_DIR</replaceable>
7770 directory with &lt;image&gt;.env files that store bitbake
7771 variables
7772 -D, --debug output debug information
7773 </literallayout>
7774 <note>
7775 You do not need root privileges to run
7776 Wic.
7777 In fact, you should not run as root when using the
7778 utility.
7779 </note>
7780 </para>
7781 </section>
7782
7783 <section id='cooked-mode'>
7784 <title>Cooked Mode</title>
7785
7786 <para>
7787 Running Wic in cooked mode leverages off artifacts in
7788 the Build Directory.
7789 In other words, you do not have to specify kernel or
7790 root filesystem locations as part of the command.
7791 All you need to provide is a kickstart file and the
7792 name of the image from which to use artifacts by using
7793 the "-e" option.
7794 Wic looks in the Build Directory (e.g.
7795 <filename>tmp/deploy/images/</filename><replaceable>machine</replaceable>)
7796 for artifacts.
7797 </para>
7798
7799 <para>
7800 The general form of the <filename>wic</filename>
7801 command using Cooked Mode is as follows:
7802 <literallayout class='monospaced'>
7803 $ wic create <replaceable>wks_file</replaceable> -e <replaceable>IMAGE_NAME</replaceable>
7804
7805 Where:
7806
7807 <replaceable>wks_file</replaceable>:
7808 An OpenEmbedded kickstart file. You can provide
7809 your own custom file or use a file from a set of
7810 existing files provided with the Yocto Project
7811 release.
7812
7813 required argument:
7814 -e <replaceable>IMAGE_NAME</replaceable>, --image-name <replaceable>IMAGE_NAME</replaceable>
7815 name of the image to use the artifacts from e.g. core-
7816 image-sato
7817 </literallayout>
7818 </para>
7819 </section>
7820 </section>
7821
7822 <section id='using-a-provided-kickstart-file'>
7823 <title>Using an Existing Kickstart File</title>
7824
7825 <para>
7826 If you do not want to create your own kickstart file, you
7827 can use an existing file provided by the Wic installation.
7828 As shipped, kickstart files can be found in the
7829 Yocto Project
7830 <ulink url='&YOCTO_DOCS_OM_URL;#source-repositories'>Source Repositories</ulink>
7831 in the following two locations:
7832 <literallayout class='monospaced'>
7833 poky/meta-yocto-bsp/wic
7834 poky/scripts/lib/wic/canned-wks
7835 </literallayout>
7836 Use the following command to list the available kickstart
7837 files:
7838 <literallayout class='monospaced'>
7839 $ wic list images
7840 genericx86 Create an EFI disk image for genericx86*
7841 beaglebone-yocto Create SD card image for Beaglebone
7842 edgerouter Create SD card image for Edgerouter
7843 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
7844 directdisk-gpt Create a 'pcbios' direct disk image
7845 mkefidisk Create an EFI disk image
7846 directdisk Create a 'pcbios' direct disk image
7847 systemd-bootdisk Create an EFI disk image with systemd-boot
7848 mkhybridiso Create a hybrid ISO image
7849 sdimage-bootpart Create SD card image with a boot partition
7850 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
7851 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
7852 </literallayout>
7853 When you use an existing file, you do not have to use the
7854 <filename>.wks</filename> extension.
7855 Here is an example in Raw Mode that uses the
7856 <filename>directdisk</filename> file:
7857 <literallayout class='monospaced'>
7858 $ wic create directdisk -r <replaceable>rootfs_dir</replaceable> -b <replaceable>bootimg_dir</replaceable> \
7859 -k <replaceable>kernel_dir</replaceable> -n <replaceable>native_sysroot</replaceable>
7860 </literallayout>
7861 </para>
7862
7863 <para>
7864 Here are the actual partition language commands
7865 used in the <filename>genericx86.wks</filename> file to
7866 generate an image:
7867 <literallayout class='monospaced'>
7868 # short-description: Create an EFI disk image for genericx86*
7869 # long-description: Creates a partitioned EFI disk image for genericx86* machines
7870 part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
7871 part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
7872 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
7873
7874 bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
7875 </literallayout>
7876 </para>
7877 </section>
7878
7879 <section id='wic-using-the-wic-plugin-interface'>
7880 <title>Using the Wic Plugin Interface</title>
7881
7882 <para>
7883 You can extend and specialize Wic functionality by using
7884 Wic plugins.
7885 This section explains the Wic plugin interface.
7886 <note>
7887 Wic plugins consist of "source" and "imager" plugins.
7888 Imager plugins are beyond the scope of this section.
7889 </note>
7890 </para>
7891
7892 <para>
7893 Source plugins provide a mechanism to customize partition
7894 content during the Wic image generation process.
7895 You can use source plugins to map values that you specify
7896 using <filename>--source</filename> commands in kickstart
7897 files (i.e. <filename>*.wks</filename>) to a plugin
7898 implementation used to populate a given partition.
7899 <note>
7900 If you use plugins that have build-time dependencies
7901 (e.g. native tools, bootloaders, and so forth)
7902 when building a Wic image, you need to specify those
7903 dependencies using the
7904 <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE_DEPENDS'><filename>WKS_FILE_DEPENDS</filename></ulink>
7905 variable.
7906 </note>
7907 </para>
7908
7909 <para>
7910 Source plugins are subclasses defined in plugin files.
7911 As shipped, the Yocto Project provides several plugin
7912 files.
7913 You can see the source plugin files that ship with the
7914 Yocto Project
7915 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source'>here</ulink>.
7916 Each of these plugin files contains source plugins that
7917 are designed to populate a specific Wic image partition.
7918 </para>
7919
7920 <para>
7921 Source plugins are subclasses of the
7922 <filename>SourcePlugin</filename> class, which is
7923 defined in the
7924 <filename>poky/scripts/lib/wic/pluginbase.py</filename>
7925 file.
7926 For example, the <filename>BootimgEFIPlugin</filename>
7927 source plugin found in the
7928 <filename>bootimg-efi.py</filename> file is a subclass of
7929 the <filename>SourcePlugin</filename> class, which is found
7930 in the <filename>pluginbase.py</filename> file.
7931 </para>
7932
7933 <para>
7934 You can also implement source plugins in a layer outside
7935 of the Source Repositories (external layer).
7936 To do so, be sure that your plugin files are located in
7937 a directory whose path is
7938 <filename>scripts/lib/wic/plugins/source/</filename>
7939 within your external layer.
7940 When the plugin files are located there, the source
7941 plugins they contain are made available to Wic.
7942 </para>
7943
7944 <para>
7945 When the Wic implementation needs to invoke a
7946 partition-specific implementation, it looks for the plugin
7947 with the same name as the <filename>--source</filename>
7948 parameter used in the kickstart file given to that
7949 partition.
7950 For example, if the partition is set up using the following
7951 command in a kickstart file:
7952 <literallayout class='monospaced'>
7953 part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
7954 </literallayout>
7955 The methods defined as class members of the matching
7956 source plugin (i.e. <filename>bootimg-pcbios</filename>)
7957 in the <filename>bootimg-pcbios.py</filename> plugin file
7958 are used.
7959 </para>
7960
7961 <para>
7962 To be more concrete, here is the corresponding plugin
7963 definition from the <filename>bootimg-pcbios.py</filename>
7964 file for the previous command along with an example
7965 method called by the Wic implementation when it needs to
7966 prepare a partition using an implementation-specific
7967 function:
7968 <literallayout class='monospaced'>
7969 .
7970 .
7971 .
7972 class BootimgPcbiosPlugin(SourcePlugin):
7973 """
7974 Create MBR boot partition and install syslinux on it.
7975 """
7976
7977 name = 'bootimg-pcbios'
7978 .
7979 .
7980 .
7981 @classmethod
7982 def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
7983 oe_builddir, bootimg_dir, kernel_dir,
7984 rootfs_dir, native_sysroot):
7985 """
7986 Called to do the actual content population for a partition i.e. it
7987 'prepares' the partition to be incorporated into the image.
7988 In this case, prepare content for legacy bios boot partition.
7989 """
7990 .
7991 .
7992 .
7993 </literallayout>
7994 If a subclass (plugin) itself does not implement a
7995 particular function, Wic locates and uses the default
7996 version in the superclass.
7997 It is for this reason that all source plugins are derived
7998 from the <filename>SourcePlugin</filename> class.
7999 </para>
8000
8001 <para>
8002 The <filename>SourcePlugin</filename> class defined in
8003 the <filename>pluginbase.py</filename> file defines
8004 a set of methods that source plugins can implement or
8005 override.
8006 Any plugins (subclass of
8007 <filename>SourcePlugin</filename>) that do not implement
8008 a particular method inherit the implementation of the
8009 method from the <filename>SourcePlugin</filename> class.
8010 For more information, see the
8011 <filename>SourcePlugin</filename> class in the
8012 <filename>pluginbase.py</filename> file for details:
8013 </para>
8014
8015 <para>
8016 The following list describes the methods implemented in the
8017 <filename>SourcePlugin</filename> class:
8018 <itemizedlist>
8019 <listitem><para>
8020 <emphasis><filename>do_prepare_partition()</filename>:</emphasis>
8021 Called to populate a partition with actual content.
8022 In other words, the method prepares the final
8023 partition image that is incorporated into the
8024 disk image.
8025 </para></listitem>
8026 <listitem><para>
8027 <emphasis><filename>do_configure_partition()</filename>:</emphasis>
8028 Called before
8029 <filename>do_prepare_partition()</filename> to
8030 create custom configuration files for a partition
8031 (e.g. syslinux or grub configuration files).
8032 </para></listitem>
8033 <listitem><para>
8034 <emphasis><filename>do_install_disk()</filename>:</emphasis>
8035 Called after all partitions have been prepared and
8036 assembled into a disk image.
8037 This method provides a hook to allow finalization
8038 of a disk image (e.g. writing an MBR).
8039 </para></listitem>
8040 <listitem><para>
8041 <emphasis><filename>do_stage_partition()</filename>:</emphasis>
8042 Special content-staging hook called before
8043 <filename>do_prepare_partition()</filename>.
8044 This method is normally empty.</para>
8045
8046 <para>Typically, a partition just uses the passed-in
8047 parameters (e.g. the unmodified value of
8048 <filename>bootimg_dir</filename>).
8049 However, in some cases, things might need to be
8050 more tailored.
8051 As an example, certain files might additionally
8052 need to be taken from
8053 <filename>bootimg_dir + /boot</filename>.
8054 This hook allows those files to be staged in a
8055 customized fashion.
8056 <note>
8057 <filename>get_bitbake_var()</filename>
8058 allows you to access non-standard variables
8059 that you might want to use for this
8060 behavior.
8061 </note>
8062 </para></listitem>
8063 </itemizedlist>
8064 </para>
8065
8066 <para>
8067 You can extend the source plugin mechanism.
8068 To add more hooks, create more source plugin methods
8069 within <filename>SourcePlugin</filename> and the
8070 corresponding derived subclasses.
8071 The code that calls the plugin methods uses the
8072 <filename>plugin.get_source_plugin_methods()</filename>
8073 function to find the method or methods needed by the call.
8074 Retrieval of those methods is accomplished by filling up
8075 a dict with keys that contain the method names of interest.
8076 On success, these will be filled in with the actual
8077 methods.
8078 See the Wic implementation for examples and details.
8079 </para>
8080 </section>
8081
8082 <section id='wic-usage-examples'>
8083 <title>Examples</title>
8084
8085 <para>
8086 This section provides several examples that show how to use
8087 the Wic utility.
8088 All the examples assume the list of requirements in the
8089 "<link linkend='wic-requirements'>Requirements</link>"
8090 section have been met.
8091 The examples assume the previously generated image is
8092 <filename>core-image-minimal</filename>.
8093 </para>
8094
8095 <section id='generate-an-image-using-a-provided-kickstart-file'>
8096 <title>Generate an Image using an Existing Kickstart File</title>
8097
8098 <para>
8099 This example runs in Cooked Mode and uses the
8100 <filename>mkefidisk</filename> kickstart file:
8101 <literallayout class='monospaced'>
8102 $ wic create mkefidisk -e core-image-minimal
8103 INFO: Building wic-tools...
8104 .
8105 .
8106 .
8107 INFO: The new image(s) can be found here:
8108 ./mkefidisk-201804191017-sda.direct
8109
8110 The following build artifacts were used to create the image(s):
8111 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
8112 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
8113 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
8114 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
8115
8116 INFO: The image(s) were created using OE kickstart file:
8117 /home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
8118 </literallayout>
8119 The previous example shows the easiest way to create
8120 an image by running in cooked mode and supplying
8121 a kickstart file and the "-e" option to point to the
8122 existing build artifacts.
8123 Your <filename>local.conf</filename> file needs to have
8124 the
8125 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
8126 variable set to the machine you are using, which is
8127 "qemux86" in this example.
8128 </para>
8129
8130 <para>
8131 Once the image builds, the output provides image
8132 location, artifact use, and kickstart file information.
8133 <note>
8134 You should always verify the details provided in the
8135 output to make sure that the image was indeed
8136 created exactly as expected.
8137 </note>
8138 </para>
8139
8140 <para>
8141 Continuing with the example, you can now write the
8142 image from the Build Directory onto a USB stick, or
8143 whatever media for which you built your image, and boot
8144 from the media.
8145 You can write the image by using
8146 <filename>bmaptool</filename> or
8147 <filename>dd</filename>:
8148 <literallayout class='monospaced'>
8149 $ oe-run-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sd<replaceable>X</replaceable>
8150 </literallayout>
8151 or
8152 <literallayout class='monospaced'>
8153 $ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sd<replaceable>X</replaceable>
8154 </literallayout>
8155 <note>
8156 For more information on how to use the
8157 <filename>bmaptool</filename> to flash a device
8158 with an image, see the
8159 "<link linkend='flashing-images-using-bmaptool'>Flashing Images Using <filename>bmaptool</filename></link>"
8160 section.
8161 </note>
8162 </para>
8163 </section>
8164
8165 <section id='using-a-modified-kickstart-file'>
8166 <title>Using a Modified Kickstart File</title>
8167
8168 <para>
8169 Because partitioned image creation is driven by the
8170 kickstart file, it is easy to affect image creation by
8171 changing the parameters in the file.
8172 This next example demonstrates that through modification
8173 of the <filename>directdisk-gpt</filename> kickstart
8174 file.
8175 </para>
8176
8177 <para>
8178 As mentioned earlier, you can use the command
8179 <filename>wic list images</filename> to show the list
8180 of existing kickstart files.
8181 The directory in which the
8182 <filename>directdisk-gpt.wks</filename> file resides is
8183 <filename>scripts/lib/image/canned-wks/</filename>,
8184 which is located in the
8185 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
8186 (e.g. <filename>poky</filename>).
8187 Because available files reside in this directory,
8188 you can create and add your own custom files to the
8189 directory.
8190 Subsequent use of the
8191 <filename>wic list images</filename> command would then
8192 include your kickstart files.
8193 </para>
8194
8195 <para>
8196 In this example, the existing
8197 <filename>directdisk-gpt</filename> file already does
8198 most of what is needed.
8199 However, for the hardware in this example, the image
8200 will need to boot from <filename>sdb</filename> instead
8201 of <filename>sda</filename>, which is what the
8202 <filename>directdisk-gpt</filename> kickstart file
8203 uses.
8204 </para>
8205
8206 <para>
8207 The example begins by making a copy of the
8208 <filename>directdisk-gpt.wks</filename> file in the
8209 <filename>scripts/lib/image/canned-wks</filename>
8210 directory and then by changing the lines that specify
8211 the target disk from which to boot.
8212 <literallayout class='monospaced'>
8213 $ cp /home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \
8214 /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
8215 </literallayout>
8216 Next, the example modifies the
8217 <filename>directdisksdb-gpt.wks</filename> file and
8218 changes all instances of
8219 "<filename>--ondisk sda</filename>" to
8220 "<filename>--ondisk sdb</filename>".
8221 The example changes the following two lines and leaves
8222 the remaining lines untouched:
8223 <literallayout class='monospaced'>
8224 part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024
8225 part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid
8226 </literallayout>
8227 Once the lines are changed, the example generates the
8228 <filename>directdisksdb-gpt</filename> image.
8229 The command points the process at the
8230 <filename>core-image-minimal</filename> artifacts for
8231 the Next Unit of Computing (nuc)
8232 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
8233 the <filename>local.conf</filename>.
8234 <literallayout class='monospaced'>
8235 $ wic create directdisksdb-gpt -e core-image-minimal
8236 INFO: Building wic-tools...
8237 .
8238 .
8239 .
8240 Initialising tasks: 100% |#######################################| Time: 0:00:01
8241 NOTE: Executing SetScene Tasks
8242 NOTE: Executing RunQueue Tasks
8243 NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded.
8244 INFO: Creating image(s)...
8245
8246 INFO: The new image(s) can be found here:
8247 ./directdisksdb-gpt-201710090938-sdb.direct
8248
8249 The following build artifacts were used to create the image(s):
8250 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
8251 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
8252 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
8253 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
8254
8255 INFO: The image(s) were created using OE kickstart file:
8256 /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
8257 </literallayout>
8258 Continuing with the example, you can now directly
8259 <filename>dd</filename> the image to a USB stick, or
8260 whatever media for which you built your image,
8261 and boot the resulting media:
8262 <literallayout class='monospaced'>
8263 $ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb
8264 140966+0 records in
8265 140966+0 records out
8266 72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s
8267 $ sudo eject /dev/sdb
8268 </literallayout>
8269 </para>
8270 </section>
8271
8272 <section id='using-a-modified-kickstart-file-and-running-in-raw-mode'>
8273 <title>Using a Modified Kickstart File and Running in Raw Mode</title>
8274
8275 <para>
8276 This next example manually specifies each build artifact
8277 (runs in Raw Mode) and uses a modified kickstart file.
8278 The example also uses the <filename>-o</filename> option
8279 to cause Wic to create the output
8280 somewhere other than the default output directory,
8281 which is the current directory:
8282 <literallayout class='monospaced'>
8283 $ wic create /home/stephano/my_yocto/test.wks -o /home/stephano/testwic \
8284 --rootfs-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \
8285 --bootimg-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \
8286 --kernel-dir /home/stephano/build/master/build/tmp/deploy/images/qemux86 \
8287 --native-sysroot /home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
8288
8289 INFO: Creating image(s)...
8290
8291 INFO: The new image(s) can be found here:
8292 /home/stephano/testwic/test-201710091445-sdb.direct
8293
8294 The following build artifacts were used to create the image(s):
8295 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
8296 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
8297 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
8298 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
8299
8300 INFO: The image(s) were created using OE kickstart file:
8301 /home/stephano/my_yocto/test.wks
8302 </literallayout>
8303 For this example,
8304 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
8305 did not have to be specified in the
8306 <filename>local.conf</filename> file since the
8307 artifact is manually specified.
8308 </para>
8309 </section>
8310
8311 <section id='using-wic-to-manipulate-an-image'>
8312 <title>Using Wic to Manipulate an Image</title>
8313
8314 <para>
8315 Wic image manipulation allows you to shorten turnaround
8316 time during image development.
8317 For example, you can use Wic to delete the kernel partition
8318 of a Wic image and then insert a newly built kernel.
8319 This saves you time from having to rebuild the entire image
8320 each time you modify the kernel.
8321 <note>
8322 In order to use Wic to manipulate a Wic image as in
8323 this example, your development machine must have the
8324 <filename>mtools</filename> package installed.
8325 </note>
8326 </para>
8327
8328 <para>
8329 The following example examines the contents of the Wic
8330 image, deletes the existing kernel, and then inserts a
8331 new kernel:
8332 <orderedlist>
8333 <listitem><para>
8334 <emphasis>List the Partitions:</emphasis>
8335 Use the <filename>wic ls</filename> command to list
8336 all the partitions in the Wic image:
8337 <literallayout class='monospaced'>
8338 $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic
8339 Num Start End Size Fstype
8340 1 1048576 25041919 23993344 fat16
8341 2 25165824 72157183 46991360 ext4
8342 </literallayout>
8343 The previous output shows two partitions in the
8344 <filename>core-image-minimal-qemux86.wic</filename>
8345 image.
8346 </para></listitem>
8347 <listitem><para>
8348 <emphasis>Examine a Particular Partition:</emphasis>
8349 Use the <filename>wic ls</filename> command again
8350 but in a different form to examine a particular
8351 partition.
8352 <note>
8353 You can get command usage on any Wic command
8354 using the following form:
8355 <literallayout class='monospaced'>
8356 $ wic help <replaceable>command</replaceable>
8357 </literallayout>
8358 For example, the following command shows you
8359 the various ways to use the
8360 <filename>wic ls</filename> command:
8361 <literallayout class='monospaced'>
8362 $ wic help ls
8363 </literallayout>
8364 </note>
8365 The following command shows what is in Partition
8366 one:
8367 <literallayout class='monospaced'>
8368 $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1
8369 Volume in drive : is boot
8370 Volume Serial Number is E894-1809
8371 Directory for ::/
8372
8373 libcom32 c32 186500 2017-10-09 16:06
8374 libutil c32 24148 2017-10-09 16:06
8375 syslinux cfg 220 2017-10-09 16:06
8376 vesamenu c32 27104 2017-10-09 16:06
8377 vmlinuz 6904608 2017-10-09 16:06
8378 5 files 7 142 580 bytes
8379 16 582 656 bytes free
8380 </literallayout>
8381 The previous output shows five files, with the
8382 <filename>vmlinuz</filename> being the kernel.
8383 <note>
8384 If you see the following error, you need to
8385 update or create a
8386 <filename>~/.mtoolsrc</filename> file and
8387 be sure to have the line "mtools_skip_check=1"
8388 in the file.
8389 Then, run the Wic command again:
8390 <literallayout class='monospaced'>
8391 ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
8392 output: Total number of sectors (47824) not a multiple of sectors per track (32)!
8393 Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
8394 </literallayout>
8395 </note>
8396 </para></listitem>
8397 <listitem><para>
8398 <emphasis>Remove the Old Kernel:</emphasis>
8399 Use the <filename>wic rm</filename> command to
8400 remove the <filename>vmlinuz</filename> file
8401 (kernel):
8402 <literallayout class='monospaced'>
8403 $ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
8404 </literallayout>
8405 </para></listitem>
8406 <listitem><para>
8407 <emphasis>Add In the New Kernel:</emphasis>
8408 Use the <filename>wic cp</filename> command to
8409 add the updated kernel to the Wic image.
8410 Depending on how you built your kernel, it could
8411 be in different places.
8412 If you used <filename>devtool</filename> and
8413 an SDK to build your kernel, it resides in the
8414 <filename>tmp/work</filename> directory of the
8415 extensible SDK.
8416 If you used <filename>make</filename> to build the
8417 kernel, the kernel will be in the
8418 <filename>workspace/sources</filename> area.
8419 </para>
8420
8421 <para>The following example assumes
8422 <filename>devtool</filename> was used to build
8423 the kernel:
8424 <literallayout class='monospaced'>
8425 cp ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \
8426 ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
8427 </literallayout>
8428 Once the new kernel is added back into the image,
8429 you can use the <filename>dd</filename>
8430 command or
8431 <link linkend='flashing-images-using-bmaptool'><filename>bmaptool</filename></link>
8432 to flash your wic image onto an SD card
8433 or USB stick and test your target.
8434 <note>
8435 Using <filename>bmaptool</filename> is
8436 generally 10 to 20 times faster than using
8437 <filename>dd</filename>.
8438 </note>
8439 </para></listitem>
8440 </orderedlist>
8441 </para>
8442 </section>
8443 </section>
8444 </section>
8445
8446 <section id='flashing-images-using-bmaptool'>
8447 <title>Flashing Images Using <filename>bmaptool</filename></title>
8448
8449 <para>
8450 A fast and easy way to flash an image to a bootable device
8451 is to use Bmaptool, which is integrated into the OpenEmbedded
8452 build system.
8453 Bmaptool is a generic tool that creates a file's block map (bmap)
8454 and then uses that map to copy the file.
8455 As compared to traditional tools such as dd or cp, Bmaptool
8456 can copy (or flash) large files like raw system image files
8457 much faster.
8458 <note><title>Notes</title>
8459 <itemizedlist>
8460 <listitem><para>
8461 If you are using Ubuntu or Debian distributions, you
8462 can install the <filename>bmap-tools</filename> package
8463 using the following command and then use the tool
8464 without specifying <filename>PATH</filename> even from
8465 the root account:
8466 <literallayout class='monospaced'>
8467 $ sudo apt-get install bmap-tools
8468 </literallayout>
8469 </para></listitem>
8470 <listitem><para>
8471 If you are unable to install the
8472 <filename>bmap-tools</filename> package, you will
8473 need to build Bmaptool before using it.
8474 Use the following command:
8475 <literallayout class='monospaced'>
8476 $ bitbake bmap-tools-native
8477 </literallayout>
8478 </para></listitem>
8479 </itemizedlist>
8480 </note>
8481 </para>
8482
8483 <para>
8484 Following, is an example that shows how to flash a Wic image.
8485 Realize that while this example uses a Wic image, you can use
8486 Bmaptool to flash any type of image.
8487 Use these steps to flash an image using Bmaptool:
8488 <orderedlist>
8489 <listitem><para>
8490 <emphasis>Update your <filename>local.conf</filename> File:</emphasis>
8491 You need to have the following set in your
8492 <filename>local.conf</filename> file before building
8493 your image:
8494 <literallayout class='monospaced'>
8495 IMAGE_FSTYPES += "wic wic.bmap"
8496 </literallayout>
8497 </para></listitem>
8498 <listitem><para>
8499 <emphasis>Get Your Image:</emphasis>
8500 Either have your image ready (pre-built with the
8501 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink>
8502 setting previously mentioned) or take the step to build
8503 the image:
8504 <literallayout class='monospaced'>
8505 $ bitbake <replaceable>image</replaceable>
8506 </literallayout>
8507 </para></listitem>
8508 <listitem><para>
8509 <emphasis>Flash the Device:</emphasis>
8510 Flash the device with the image by using Bmaptool
8511 depending on your particular setup.
8512 The following commands assume the image resides in the
8513 Build Directory's <filename>deploy/images/</filename>
8514 area:
8515 <itemizedlist>
8516 <listitem><para>
8517 If you have write access to the media, use this
8518 command form:
8519 <literallayout class='monospaced'>
8520 $ oe-run-native bmap-tools-native bmaptool copy <replaceable>build-directory</replaceable>/tmp/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.wic /dev/sd<replaceable>X</replaceable>
8521 </literallayout>
8522 </para></listitem>
8523 <listitem><para>
8524 If you do not have write access to the media, set
8525 your permissions first and then use the same
8526 command form:
8527 <literallayout class='monospaced'>
8528 $ sudo chmod 666 /dev/sd<replaceable>X</replaceable>
8529 $ oe-run-native bmap-tools-native bmaptool copy <replaceable>build-directory</replaceable>/tmp/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.wic /dev/sd<replaceable>X</replaceable>
8530 </literallayout>
8531 </para></listitem>
8532 </itemizedlist>
8533 </para></listitem>
8534 </orderedlist>
8535 </para>
8536
8537 <para>
8538 For help on the <filename>bmaptool</filename> command, use the
8539 following command:
8540 <literallayout class='monospaced'>
8541 $ bmaptool --help
8542 </literallayout>
8543 </para>
8544 </section>
8545
8546 <section id='making-images-more-secure'>
8547 <title>Making Images More Secure</title>
8548
8549 <para>
8550 Security is of increasing concern for embedded devices.
8551 Consider the issues and problems discussed in just this
8552 sampling of work found across the Internet:
8553 <itemizedlist>
8554 <listitem><para><emphasis>
8555 "<ulink url='https://www.schneier.com/blog/archives/2014/01/security_risks_9.html'>Security Risks of Embedded Systems</ulink>"</emphasis>
8556 by Bruce Schneier
8557 </para></listitem>
8558 <listitem><para><emphasis>
8559 "<ulink url='http://census2012.sourceforge.net/paper.html'>Internet Census 2012</ulink>"</emphasis>
8560 by Carna Botnet</para></listitem>
8561 <listitem><para><emphasis>
8562 "<ulink url='http://elinux.org/images/6/6f/Security-issues.pdf'>Security Issues for Embedded Devices</ulink>"</emphasis>
8563 by Jake Edge
8564 </para></listitem>
8565 </itemizedlist>
8566 </para>
8567
8568 <para>
8569 When securing your image is of concern, there are steps, tools,
8570 and variables that you can consider to help you reach the
8571 security goals you need for your particular device.
8572 Not all situations are identical when it comes to making an
8573 image secure.
8574 Consequently, this section provides some guidance and suggestions
8575 for consideration when you want to make your image more secure.
8576 <note>
8577 Because the security requirements and risks are
8578 different for every type of device, this section cannot
8579 provide a complete reference on securing your custom OS.
8580 It is strongly recommended that you also consult other sources
8581 of information on embedded Linux system hardening and on
8582 security.
8583 </note>
8584 </para>
8585
8586 <section id='general-considerations'>
8587 <title>General Considerations</title>
8588
8589 <para>
8590 General considerations exist that help you create more
8591 secure images.
8592 You should consider the following suggestions to help
8593 make your device more secure:
8594 <itemizedlist>
8595 <listitem><para>
8596 Scan additional code you are adding to the system
8597 (e.g. application code) by using static analysis
8598 tools.
8599 Look for buffer overflows and other potential
8600 security problems.
8601 </para></listitem>
8602 <listitem><para>
8603 Pay particular attention to the security for
8604 any web-based administration interface.
8605 </para>
8606 <para>Web interfaces typically need to perform
8607 administrative functions and tend to need to run with
8608 elevated privileges.
8609 Thus, the consequences resulting from the interface's
8610 security becoming compromised can be serious.
8611 Look for common web vulnerabilities such as
8612 cross-site-scripting (XSS), unvalidated inputs,
8613 and so forth.</para>
8614 <para>As with system passwords, the default credentials
8615 for accessing a web-based interface should not be the
8616 same across all devices.
8617 This is particularly true if the interface is enabled
8618 by default as it can be assumed that many end-users
8619 will not change the credentials.
8620 </para></listitem>
8621 <listitem><para>
8622 Ensure you can update the software on the device to
8623 mitigate vulnerabilities discovered in the future.
8624 This consideration especially applies when your
8625 device is network-enabled.
8626 </para></listitem>
8627 <listitem><para>
8628 Ensure you remove or disable debugging functionality
8629 before producing the final image.
8630 For information on how to do this, see the
8631 "<link linkend='considerations-specific-to-the-openembedded-build-system'>Considerations Specific to the OpenEmbedded Build System</link>"
8632 section.
8633 </para></listitem>
8634 <listitem><para>
8635 Ensure you have no network services listening that
8636 are not needed.
8637 </para></listitem>
8638 <listitem><para>
8639 Remove any software from the image that is not needed.
8640 </para></listitem>
8641 <listitem><para>
8642 Enable hardware support for secure boot functionality
8643 when your device supports this functionality.
8644 </para></listitem>
8645 </itemizedlist>
8646 </para>
8647 </section>
8648
8649 <section id='security-flags'>
8650 <title>Security Flags</title>
8651
8652 <para>
8653 The Yocto Project has security flags that you can enable that
8654 help make your build output more secure.
8655 The security flags are in the
8656 <filename>meta/conf/distro/include/security_flags.inc</filename>
8657 file in your
8658 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
8659 (e.g. <filename>poky</filename>).
8660 <note>
8661 Depending on the recipe, certain security flags are enabled
8662 and disabled by default.
8663 </note>
8664 </para>
8665
8666 <para>
8667<!--
8668 The GCC/LD flags in <filename>security_flags.inc</filename>
8669 enable more secure code generation.
8670 By including the <filename>security_flags.inc</filename>
8671 file, you enable flags to the compiler and linker that cause
8672 them to generate more secure code.
8673 <note>
8674 The GCC/LD flags are enabled by default in the
8675 <filename>poky-lsb</filename> distribution.
8676 </note>
8677-->
8678 Use the following line in your
8679 <filename>local.conf</filename> file or in your custom
8680 distribution configuration file to enable the security
8681 compiler and linker flags for your build:
8682 <literallayout class='monospaced'>
8683 require conf/distro/include/security_flags.inc
8684 </literallayout>
8685 </para>
8686 </section>
8687
8688 <section id='considerations-specific-to-the-openembedded-build-system'>
8689 <title>Considerations Specific to the OpenEmbedded Build System</title>
8690
8691 <para>
8692 You can take some steps that are specific to the
8693 OpenEmbedded build system to make your images more secure:
8694 <itemizedlist>
8695 <listitem><para>
8696 Ensure "debug-tweaks" is not one of your selected
8697 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>.
8698 When creating a new project, the default is to provide you
8699 with an initial <filename>local.conf</filename> file that
8700 enables this feature using the
8701 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> variable with the line:
8702 <literallayout class='monospaced'>
8703 EXTRA_IMAGE_FEATURES = "debug-tweaks"
8704 </literallayout>
8705 To disable that feature, simply comment out that line in your
8706 <filename>local.conf</filename> file, or
8707 make sure <filename>IMAGE_FEATURES</filename> does not contain
8708 "debug-tweaks" before producing your final image.
8709 Among other things, leaving this in place sets the
8710 root password as blank, which makes logging in for
8711 debugging or inspection easy during
8712 development but also means anyone can easily log in
8713 during production.
8714 </para></listitem>
8715 <listitem><para>
8716 It is possible to set a root password for the image
8717 and also to set passwords for any extra users you might
8718 add (e.g. administrative or service type users).
8719 When you set up passwords for multiple images or
8720 users, you should not duplicate passwords.
8721 </para>
8722 <para>
8723 To set up passwords, use the
8724 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers</filename></ulink>
8725 class, which is the preferred method.
8726 For an example on how to set up both root and user
8727 passwords, see the
8728 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers.bbclass</filename></ulink>"
8729 section.
8730 <note>
8731 When adding extra user accounts or setting a
8732 root password, be cautious about setting the
8733 same password on every device.
8734 If you do this, and the password you have set
8735 is exposed, then every device is now potentially
8736 compromised.
8737 If you need this access but want to ensure
8738 security, consider setting a different,
8739 random password for each device.
8740 Typically, you do this as a separate step after
8741 you deploy the image onto the device.
8742 </note>
8743 </para></listitem>
8744 <listitem><para>
8745 Consider enabling a Mandatory Access Control (MAC)
8746 framework such as SMACK or SELinux and tuning it
8747 appropriately for your device's usage.
8748 You can find more information in the
8749 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/'><filename>meta-selinux</filename></ulink>
8750 layer.
8751 </para></listitem>
8752 </itemizedlist>
8753 </para>
8754
8755 <para>
8756 </para>
8757 </section>
8758
8759 <section id='tools-for-hardening-your-image'>
8760 <title>Tools for Hardening Your Image</title>
8761
8762 <para>
8763 The Yocto Project provides tools for making your image
8764 more secure.
8765 You can find these tools in the
8766 <filename>meta-security</filename> layer of the
8767 <ulink url='&YOCTO_GIT_URL;'>Yocto Project Source Repositories</ulink>.
8768 </para>
8769 </section>
8770 </section>
8771
8772 <section id='creating-your-own-distribution'>
8773 <title>Creating Your Own Distribution</title>
8774
8775 <para>
8776 When you build an image using the Yocto Project and
8777 do not alter any distribution
8778 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>,
8779 you are creating a Poky distribution.
8780 If you wish to gain more control over package alternative
8781 selections, compile-time options, and other low-level
8782 configurations, you can create your own distribution.
8783 </para>
8784
8785 <para>
8786 To create your own distribution, the basic steps consist of
8787 creating your own distribution layer, creating your own
8788 distribution configuration file, and then adding any needed
8789 code and Metadata to the layer.
8790 The following steps provide some more detail:
8791 <itemizedlist>
8792 <listitem><para><emphasis>Create a layer for your new distro:</emphasis>
8793 Create your distribution layer so that you can keep your
8794 Metadata and code for the distribution separate.
8795 It is strongly recommended that you create and use your own
8796 layer for configuration and code.
8797 Using your own layer as compared to just placing
8798 configurations in a <filename>local.conf</filename>
8799 configuration file makes it easier to reproduce the same
8800 build configuration when using multiple build machines.
8801 See the
8802 "<link linkend='creating-a-general-layer-using-the-bitbake-layers-script'>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</link>"
8803 section for information on how to quickly set up a layer.
8804 </para></listitem>
8805 <listitem><para><emphasis>Create the distribution configuration file:</emphasis>
8806 The distribution configuration file needs to be created in
8807 the <filename>conf/distro</filename> directory of your
8808 layer.
8809 You need to name it using your distribution name
8810 (e.g. <filename>mydistro.conf</filename>).
8811 <note>
8812 The
8813 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
8814 variable in your
8815 <filename>local.conf</filename> file determines the
8816 name of your distribution.
8817 </note></para>
8818 <para>You can split out parts of your configuration file
8819 into include files and then "require" them from within
8820 your distribution configuration file.
8821 Be sure to place the include files in the
8822 <filename>conf/distro/include</filename> directory of
8823 your layer.
8824 A common example usage of include files would be to
8825 separate out the selection of desired version and revisions
8826 for individual recipes.
8827</para>
8828 <para>Your configuration file needs to set the following
8829 required variables:
8830 <literallayout class='monospaced'>
8831 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink>
8832 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink>
8833 </literallayout>
8834 These following variables are optional and you typically
8835 set them from the distribution configuration file:
8836 <literallayout class='monospaced'>
8837 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
8838 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink>
8839 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink>
8840 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink>
8841 </literallayout>
8842 <tip>
8843 If you want to base your distribution configuration file
8844 on the very basic configuration from OE-Core, you
8845 can use
8846 <filename>conf/distro/defaultsetup.conf</filename> as
8847 a reference and just include variables that differ
8848 as compared to <filename>defaultsetup.conf</filename>.
8849 Alternatively, you can create a distribution
8850 configuration file from scratch using the
8851 <filename>defaultsetup.conf</filename> file
8852 or configuration files from other distributions
8853 such as Poky or Angstrom as references.
8854 </tip></para></listitem>
8855 <listitem><para><emphasis>Provide miscellaneous variables:</emphasis>
8856 Be sure to define any other variables for which you want to
8857 create a default or enforce as part of the distribution
8858 configuration.
8859 You can include nearly any variable from the
8860 <filename>local.conf</filename> file.
8861 The variables you use are not limited to the list in the
8862 previous bulleted item.</para></listitem>
8863 <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis>
8864 In your <filename>local.conf</filename> file in the
8865 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>,
8866 set your
8867 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
8868 variable to point to your distribution's configuration file.
8869 For example, if your distribution's configuration file is
8870 named <filename>mydistro.conf</filename>, then you point
8871 to it as follows:
8872 <literallayout class='monospaced'>
8873 DISTRO = "mydistro"
8874 </literallayout></para></listitem>
8875 <listitem><para><emphasis>Add more to the layer if necessary:</emphasis>
8876 Use your layer to hold other information needed for the
8877 distribution:
8878 <itemizedlist>
8879 <listitem><para>Add recipes for installing
8880 distro-specific configuration files that are not
8881 already installed by another recipe.
8882 If you have distro-specific configuration files
8883 that are included by an existing recipe, you should
8884 add an append file (<filename>.bbappend</filename>)
8885 for those.
8886 For general information and recommendations
8887 on how to add recipes to your layer, see the
8888 "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>"
8889 and
8890 "<link linkend='best-practices-to-follow-when-creating-layers'>Following Best Practices When Creating Layers</link>"
8891 sections.</para></listitem>
8892 <listitem><para>Add any image recipes that are specific
8893 to your distribution.</para></listitem>
8894 <listitem><para>Add a <filename>psplash</filename>
8895 append file for a branded splash screen.
8896 For information on append files, see the
8897 "<link linkend='using-bbappend-files'>Using .bbappend Files in Your Layer</link>"
8898 section.</para></listitem>
8899 <listitem><para>Add any other append files to make
8900 custom changes that are specific to individual
8901 recipes.</para></listitem>
8902 </itemizedlist></para></listitem>
8903 </itemizedlist>
8904 </para>
8905 </section>
8906
8907 <section id='creating-a-custom-template-configuration-directory'>
8908 <title>Creating a Custom Template Configuration Directory</title>
8909
8910 <para>
8911 If you are producing your own customized version
8912 of the build system for use by other users, you might
8913 want to customize the message shown by the setup script or
8914 you might want to change the template configuration files (i.e.
8915 <filename>local.conf</filename> and
8916 <filename>bblayers.conf</filename>) that are created in
8917 a new build directory.
8918 </para>
8919
8920 <para>
8921 The OpenEmbedded build system uses the environment variable
8922 <filename>TEMPLATECONF</filename> to locate the directory
8923 from which it gathers configuration information that ultimately
8924 ends up in the
8925 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
8926 <filename>conf</filename> directory.
8927 By default, <filename>TEMPLATECONF</filename> is set as
8928 follows in the <filename>poky</filename> repository:
8929 <literallayout class='monospaced'>
8930 TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf}
8931 </literallayout>
8932 This is the directory used by the build system to find templates
8933 from which to build some key configuration files.
8934 If you look at this directory, you will see the
8935 <filename>bblayers.conf.sample</filename>,
8936 <filename>local.conf.sample</filename>, and
8937 <filename>conf-notes.txt</filename> files.
8938 The build system uses these files to form the respective
8939 <filename>bblayers.conf</filename> file,
8940 <filename>local.conf</filename> file, and display the list of
8941 BitBake targets when running the setup script.
8942 </para>
8943
8944 <para>
8945 To override these default configuration files with
8946 configurations you want used within every new
8947 Build Directory, simply set the
8948 <filename>TEMPLATECONF</filename> variable to your directory.
8949 The <filename>TEMPLATECONF</filename> variable is set in the
8950 <filename>.templateconf</filename> file, which is in the
8951 top-level
8952 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
8953 folder (e.g. <filename>poky</filename>).
8954 Edit the <filename>.templateconf</filename> so that it can locate
8955 your directory.
8956 </para>
8957
8958 <para>
8959 Best practices dictate that you should keep your
8960 template configuration directory in your custom distribution layer.
8961 For example, suppose you have a layer named
8962 <filename>meta-mylayer</filename> located in your home directory
8963 and you want your template configuration directory named
8964 <filename>myconf</filename>.
8965 Changing the <filename>.templateconf</filename> as follows
8966 causes the OpenEmbedded build system to look in your directory
8967 and base its configuration files on the
8968 <filename>*.sample</filename> configuration files it finds.
8969 The final configuration files (i.e.
8970 <filename>local.conf</filename> and
8971 <filename>bblayers.conf</filename> ultimately still end up in
8972 your Build Directory, but they are based on your
8973 <filename>*.sample</filename> files.
8974 <literallayout class='monospaced'>
8975 TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf}
8976 </literallayout>
8977 </para>
8978
8979 <para>
8980 Aside from the <filename>*.sample</filename> configuration files,
8981 the <filename>conf-notes.txt</filename> also resides in the
8982 default <filename>meta-poky/conf</filename> directory.
8983 The script that sets up the build environment
8984 (i.e.
8985 <ulink url="&YOCTO_DOCS_REF_URL;#structure-core-script"><filename>&OE_INIT_FILE;</filename></ulink>)
8986 uses this file to display BitBake targets as part of the script
8987 output.
8988 Customizing this <filename>conf-notes.txt</filename> file is a
8989 good way to make sure your list of custom targets appears
8990 as part of the script's output.
8991 </para>
8992
8993 <para>
8994 Here is the default list of targets displayed as a result of
8995 running either of the setup scripts:
8996 <literallayout class='monospaced'>
8997 You can now run 'bitbake &lt;target&gt;'
8998
8999 Common targets are:
9000 core-image-minimal
9001 core-image-sato
9002 meta-toolchain
9003 meta-ide-support
9004 </literallayout>
9005 </para>
9006
9007 <para>
9008 Changing the listed common targets is as easy as editing your
9009 version of <filename>conf-notes.txt</filename> in your
9010 custom template configuration directory and making sure you
9011 have <filename>TEMPLATECONF</filename> set to your directory.
9012 </para>
9013 </section>
9014
9015 <section id='dev-saving-memory-during-a-build'>
9016 <title>Conserving Disk Space During Builds</title>
9017
9018 <para>
9019 To help conserve disk space during builds, you can add the
9020 following statement to your project's
9021 <filename>local.conf</filename> configuration file found in the
9022 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
9023 <literallayout class='monospaced'>
9024 INHERIT += "rm_work"
9025 </literallayout>
9026 Adding this statement deletes the work directory used for building
9027 a recipe once the recipe is built.
9028 For more information on "rm_work", see the
9029 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink>
9030 class in the Yocto Project Reference Manual.
9031 </para>
9032 </section>
9033
9034 <section id='working-with-packages'>
9035 <title>Working with Packages</title>
9036
9037 <para>
9038 This section describes a few tasks that involve packages:
9039 <itemizedlist>
9040 <listitem><para>
9041 <link linkend='excluding-packages-from-an-image'>Excluding packages from an image</link>
9042 </para></listitem>
9043 <listitem><para>
9044 <link linkend='incrementing-a-binary-package-version'>Incrementing a binary package version</link>
9045 </para></listitem>
9046 <listitem><para>
9047 <link linkend='handling-optional-module-packaging'>Handling optional module packaging</link>
9048 </para></listitem>
9049 <listitem><para>
9050 <link linkend='using-runtime-package-management'>Using runtime package management</link>
9051 </para></listitem>
9052 <listitem><para>
9053 <link linkend='generating-and-using-signed-packages'>Generating and using signed packages</link>
9054 </para></listitem>
9055 <listitem><para>
9056 <link linkend='testing-packages-with-ptest'>Setting up and running package test (ptest)</link>
9057 </para></listitem>
9058 <listitem><para>
9059 <link linkend='creating-node-package-manager-npm-packages'>Creating node package manager (NPM) packages</link>
9060 </para></listitem>
9061 <listitem><para>
9062 <link linkend='adding-custom-metadata-to-packages'>Adding custom metadata to packages</link>
9063 </para></listitem>
9064 </itemizedlist>
9065 </para>
9066
9067 <section id='excluding-packages-from-an-image'>
9068 <title>Excluding Packages from an Image</title>
9069
9070 <para>
9071 You might find it necessary to prevent specific packages
9072 from being installed into an image.
9073 If so, you can use several variables to direct the build
9074 system to essentially ignore installing recommended packages
9075 or to not install a package at all.
9076 </para>
9077
9078 <para>
9079 The following list introduces variables you can use to
9080 prevent packages from being installed into your image.
9081 Each of these variables only works with IPK and RPM
9082 package types.
9083 Support for Debian packages does not exist.
9084 Also, you can use these variables from your
9085 <filename>local.conf</filename> file or attach them to a
9086 specific image recipe by using a recipe name override.
9087 For more detail on the variables, see the descriptions in the
9088 Yocto Project Reference Manual's glossary chapter.
9089 <itemizedlist>
9090 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>:
9091 Use this variable to specify "recommended-only"
9092 packages that you do not want installed.
9093 </para></listitem>
9094 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>:
9095 Use this variable to prevent all "recommended-only"
9096 packages from being installed.
9097 </para></listitem>
9098 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>:
9099 Use this variable to prevent specific packages from
9100 being installed regardless of whether they are
9101 "recommended-only" or not.
9102 You need to realize that the build process could
9103 fail with an error when you
9104 prevent the installation of a package whose presence
9105 is required by an installed package.
9106 </para></listitem>
9107 </itemizedlist>
9108 </para>
9109 </section>
9110
9111 <section id='incrementing-a-binary-package-version'>
9112 <title>Incrementing a Package Version</title>
9113
9114 <para>
9115 This section provides some background on how binary package
9116 versioning is accomplished and presents some of the services,
9117 variables, and terminology involved.
9118 </para>
9119
9120 <para>
9121 In order to understand binary package versioning, you need
9122 to consider the following:
9123 <itemizedlist>
9124 <listitem><para>
9125 Binary Package: The binary package that is eventually
9126 built and installed into an image.
9127 </para></listitem>
9128 <listitem><para>
9129 Binary Package Version: The binary package version
9130 is composed of two components - a version and a
9131 revision.
9132 <note>
9133 Technically, a third component, the "epoch" (i.e.
9134 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>)
9135 is involved but this discussion for the most part
9136 ignores <filename>PE</filename>.
9137 </note>
9138 The version and revision are taken from the
9139 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
9140 and
9141 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
9142 variables, respectively.
9143 </para></listitem>
9144 <listitem><para>
9145 <filename>PV</filename>: The recipe version.
9146 <filename>PV</filename> represents the version of the
9147 software being packaged.
9148 Do not confuse <filename>PV</filename> with the
9149 binary package version.
9150 </para></listitem>
9151 <listitem><para>
9152 <filename>PR</filename>: The recipe revision.
9153 </para></listitem>
9154 <listitem><para>
9155 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>:
9156 The OpenEmbedded build system uses this string
9157 to help define the value of <filename>PV</filename>
9158 when the source code revision needs to be included
9159 in it.
9160 </para></listitem>
9161 <listitem><para>
9162 <ulink url='https://wiki.yoctoproject.org/wiki/PR_Service'>PR Service</ulink>:
9163 A network-based service that helps automate keeping
9164 package feeds compatible with existing package
9165 manager applications such as RPM, APT, and OPKG.
9166 </para></listitem>
9167 </itemizedlist>
9168 </para>
9169
9170 <para>
9171 Whenever the binary package content changes, the binary package
9172 version must change.
9173 Changing the binary package version is accomplished by changing
9174 or "bumping" the <filename>PR</filename> and/or
9175 <filename>PV</filename> values.
9176 Increasing these values occurs one of two ways:
9177 <itemizedlist>
9178 <listitem><para>Automatically using a Package Revision
9179 Service (PR Service).
9180 </para></listitem>
9181 <listitem><para>Manually incrementing the
9182 <filename>PR</filename> and/or
9183 <filename>PV</filename> variables.
9184 </para></listitem>
9185 </itemizedlist>
9186 </para>
9187
9188 <para>
9189 Given a primary challenge of any build system and its users
9190 is how to maintain a package feed that is compatible with
9191 existing package manager applications such as RPM, APT, and
9192 OPKG, using an automated system is much preferred over a
9193 manual system.
9194 In either system, the main requirement is that binary package
9195 version numbering increases in a linear fashion and that a
9196 number of version components exist that support that linear
9197 progression.
9198 For information on how to ensure package revisioning remains
9199 linear, see the
9200 "<link linkend='automatically-incrementing-a-binary-package-revision-number'>Automatically Incrementing a Binary Package Revision Number</link>"
9201 section.
9202 </para>
9203
9204 <para>
9205 The following three sections provide related information on the
9206 PR Service, the manual method for "bumping"
9207 <filename>PR</filename> and/or <filename>PV</filename>, and
9208 on how to ensure binary package revisioning remains linear.
9209 </para>
9210
9211 <section id='working-with-a-pr-service'>
9212 <title>Working With a PR Service</title>
9213
9214 <para>
9215 As mentioned, attempting to maintain revision numbers in the
9216 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>
9217 is error prone, inaccurate, and causes problems for people
9218 submitting recipes.
9219 Conversely, the PR Service automatically generates
9220 increasing numbers, particularly the revision field,
9221 which removes the human element.
9222 <note>
9223 For additional information on using a PR Service, you
9224 can see the
9225 <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink>
9226 wiki page.
9227 </note>
9228 </para>
9229
9230 <para>
9231 The Yocto Project uses variables in order of
9232 decreasing priority to facilitate revision numbering (i.e.
9233 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>,
9234 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
9235 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
9236 for epoch, version, and revision, respectively).
9237 The values are highly dependent on the policies and
9238 procedures of a given distribution and package feed.
9239 </para>
9240
9241 <para>
9242 Because the OpenEmbedded build system uses
9243 "<ulink url='&YOCTO_DOCS_OM_URL;#overview-checksums'>signatures</ulink>",
9244 which are unique to a given build, the build system
9245 knows when to rebuild packages.
9246 All the inputs into a given task are represented by a
9247 signature, which can trigger a rebuild when different.
9248 Thus, the build system itself does not rely on the
9249 <filename>PR</filename>, <filename>PV</filename>, and
9250 <filename>PE</filename> numbers to trigger a rebuild.
9251 The signatures, however, can be used to generate
9252 these values.
9253 </para>
9254
9255 <para>
9256 The PR Service works with both
9257 <filename>OEBasic</filename> and
9258 <filename>OEBasicHash</filename> generators.
9259 The value of <filename>PR</filename> bumps when the
9260 checksum changes and the different generator mechanisms
9261 change signatures under different circumstances.
9262 </para>
9263
9264 <para>
9265 As implemented, the build system includes values from
9266 the PR Service into the <filename>PR</filename> field as
9267 an addition using the form "<filename>.x</filename>" so
9268 <filename>r0</filename> becomes <filename>r0.1</filename>,
9269 <filename>r0.2</filename> and so forth.
9270 This scheme allows existing <filename>PR</filename> values
9271 to be used for whatever reasons, which include manual
9272 <filename>PR</filename> bumps, should it be necessary.
9273 </para>
9274
9275 <para>
9276 By default, the PR Service is not enabled or running.
9277 Thus, the packages generated are just "self consistent".
9278 The build system adds and removes packages and
9279 there are no guarantees about upgrade paths but images
9280 will be consistent and correct with the latest changes.
9281 </para>
9282
9283 <para>
9284 The simplest form for a PR Service is for it to exist
9285 for a single host development system that builds the
9286 package feed (building system).
9287 For this scenario, you can enable a local PR Service by
9288 setting
9289 <ulink url='&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST'><filename>PRSERV_HOST</filename></ulink>
9290 in your <filename>local.conf</filename> file in the
9291 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
9292 <literallayout class='monospaced'>
9293 PRSERV_HOST = "localhost:0"
9294 </literallayout>
9295 Once the service is started, packages will automatically
9296 get increasing <filename>PR</filename> values and
9297 BitBake takes care of starting and stopping the server.
9298 </para>
9299
9300 <para>
9301 If you have a more complex setup where multiple host
9302 development systems work against a common, shared package
9303 feed, you have a single PR Service running and it is
9304 connected to each building system.
9305 For this scenario, you need to start the PR Service using
9306 the <filename>bitbake-prserv</filename> command:
9307 <literallayout class='monospaced'>
9308 bitbake-prserv --host <replaceable>ip</replaceable> --port <replaceable>port</replaceable> --start
9309 </literallayout>
9310 In addition to hand-starting the service, you need to
9311 update the <filename>local.conf</filename> file of each
9312 building system as described earlier so each system
9313 points to the server and port.
9314 </para>
9315
9316 <para>
9317 It is also recommended you use build history, which adds
9318 some sanity checks to binary package versions, in
9319 conjunction with the server that is running the PR Service.
9320 To enable build history, add the following to each building
9321 system's <filename>local.conf</filename> file:
9322 <literallayout class='monospaced'>
9323 # It is recommended to activate "buildhistory" for testing the PR service
9324 INHERIT += "buildhistory"
9325 BUILDHISTORY_COMMIT = "1"
9326 </literallayout>
9327 For information on build history, see the
9328 "<link linkend='maintaining-build-output-quality'>Maintaining Build Output Quality</link>"
9329 section.
9330 </para>
9331
9332 <note>
9333 <para>
9334 The OpenEmbedded build system does not maintain
9335 <filename>PR</filename> information as part of the
9336 shared state (sstate) packages.
9337 If you maintain an sstate feed, its expected that either
9338 all your building systems that contribute to the sstate
9339 feed use a shared PR Service, or you do not run a PR
9340 Service on any of your building systems.
9341 Having some systems use a PR Service while others do
9342 not leads to obvious problems.
9343 </para>
9344
9345 <para>
9346 For more information on shared state, see the
9347 "<ulink url='&YOCTO_DOCS_OM_URL;#shared-state-cache'>Shared State Cache</ulink>"
9348 section in the Yocto Project Overview and Concepts
9349 Manual.
9350 </para>
9351 </note>
9352 </section>
9353
9354 <section id='manually-bumping-pr'>
9355 <title>Manually Bumping PR</title>
9356
9357 <para>
9358 The alternative to setting up a PR Service is to manually
9359 "bump" the
9360 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
9361 variable.
9362 </para>
9363
9364 <para>
9365 If a committed change results in changing the package
9366 output, then the value of the PR variable needs to be
9367 increased (or "bumped") as part of that commit.
9368 For new recipes you should add the <filename>PR</filename>
9369 variable and set its initial value equal to "r0", which is
9370 the default.
9371 Even though the default value is "r0", the practice of
9372 adding it to a new recipe makes it harder to forget to bump
9373 the variable when you make changes to the recipe in future.
9374 </para>
9375
9376 <para>
9377 If you are sharing a common <filename>.inc</filename> file
9378 with multiple recipes, you can also use the
9379 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename>
9380 variable to ensure that the recipes sharing the
9381 <filename>.inc</filename> file are rebuilt when the
9382 <filename>.inc</filename> file itself is changed.
9383 The <filename>.inc</filename> file must set
9384 <filename>INC_PR</filename> (initially to "r0"), and all
9385 recipes referring to it should set <filename>PR</filename>
9386 to "${INC_PR}.0" initially, incrementing the last number
9387 when the recipe is changed.
9388 If the <filename>.inc</filename> file is changed then its
9389 <filename>INC_PR</filename> should be incremented.
9390 </para>
9391
9392 <para>
9393 When upgrading the version of a binary package, assuming the
9394 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename>
9395 changes, the <filename>PR</filename> variable should be
9396 reset to "r0" (or "${INC_PR}.0" if you are using
9397 <filename>INC_PR</filename>).
9398 </para>
9399
9400 <para>
9401 Usually, version increases occur only to binary packages.
9402 However, if for some reason <filename>PV</filename> changes
9403 but does not increase, you can increase the
9404 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename>
9405 variable (Package Epoch).
9406 The <filename>PE</filename> variable defaults to "0".
9407 </para>
9408
9409 <para>
9410 Binary package version numbering strives to follow the
9411 <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'>
9412 Debian Version Field Policy Guidelines</ulink>.
9413 These guidelines define how versions are compared and what
9414 "increasing" a version means.
9415 </para>
9416 </section>
9417
9418 <section id='automatically-incrementing-a-binary-package-revision-number'>
9419 <title>Automatically Incrementing a Package Version Number</title>
9420
9421 <para>
9422 When fetching a repository, BitBake uses the
9423 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
9424 variable to determine the specific source code revision
9425 from which to build.
9426 You set the <filename>SRCREV</filename> variable to
9427 <ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink>
9428 to cause the OpenEmbedded build system to automatically use the
9429 latest revision of the software:
9430 <literallayout class='monospaced'>
9431 SRCREV = "${AUTOREV}"
9432 </literallayout>
9433 </para>
9434
9435 <para>
9436 Furthermore, you need to reference <filename>SRCPV</filename>
9437 in <filename>PV</filename> in order to automatically update
9438 the version whenever the revision of the source code
9439 changes.
9440 Here is an example:
9441 <literallayout class='monospaced'>
9442 PV = "1.0+git${SRCPV}"
9443 </literallayout>
9444 The OpenEmbedded build system substitutes
9445 <filename>SRCPV</filename> with the following:
9446 <literallayout class='monospaced'>
9447 AUTOINC+<replaceable>source_code_revision</replaceable>
9448 </literallayout>
9449 The build system replaces the <filename>AUTOINC</filename> with
9450 a number.
9451 The number used depends on the state of the PR Service:
9452 <itemizedlist>
9453 <listitem><para>
9454 If PR Service is enabled, the build system increments
9455 the number, which is similar to the behavior of
9456 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>.
9457 This behavior results in linearly increasing package
9458 versions, which is desirable.
9459 Here is an example:
9460 <literallayout class='monospaced'>
9461 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
9462 hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk
9463 </literallayout>
9464 </para></listitem>
9465 <listitem><para>
9466 If PR Service is not enabled, the build system
9467 replaces the <filename>AUTOINC</filename>
9468 placeholder with zero (i.e. "0").
9469 This results in changing the package version since
9470 the source revision is included.
9471 However, package versions are not increased linearly.
9472 Here is an example:
9473 <literallayout class='monospaced'>
9474 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
9475 hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk
9476 </literallayout>
9477 </para></listitem>
9478 </itemizedlist>
9479 </para>
9480
9481 <para>
9482 In summary, the OpenEmbedded build system does not track the
9483 history of binary package versions for this purpose.
9484 <filename>AUTOINC</filename>, in this case, is comparable to
9485 <filename>PR</filename>.
9486 If PR server is not enabled, <filename>AUTOINC</filename>
9487 in the package version is simply replaced by "0".
9488 If PR server is enabled, the build system keeps track of the
9489 package versions and bumps the number when the package
9490 revision changes.
9491 </para>
9492 </section>
9493 </section>
9494
9495 <section id='handling-optional-module-packaging'>
9496 <title>Handling Optional Module Packaging</title>
9497
9498 <para>
9499 Many pieces of software split functionality into optional
9500 modules (or plugins) and the plugins that are built
9501 might depend on configuration options.
9502 To avoid having to duplicate the logic that determines what
9503 modules are available in your recipe or to avoid having
9504 to package each module by hand, the OpenEmbedded build system
9505 provides functionality to handle module packaging dynamically.
9506 </para>
9507
9508 <para>
9509 To handle optional module packaging, you need to do two things:
9510 <itemizedlist>
9511 <listitem><para>Ensure the module packaging is actually
9512 done.</para></listitem>
9513 <listitem><para>Ensure that any dependencies on optional
9514 modules from other recipes are satisfied by your recipe.
9515 </para></listitem>
9516 </itemizedlist>
9517 </para>
9518
9519 <section id='making-sure-the-packaging-is-done'>
9520 <title>Making Sure the Packaging is Done</title>
9521
9522 <para>
9523 To ensure the module packaging actually gets done, you use
9524 the <filename>do_split_packages</filename> function within
9525 the <filename>populate_packages</filename> Python function
9526 in your recipe.
9527 The <filename>do_split_packages</filename> function
9528 searches for a pattern of files or directories under a
9529 specified path and creates a package for each one it finds
9530 by appending to the
9531 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
9532 variable and setting the appropriate values for
9533 <filename>FILES_packagename</filename>,
9534 <filename>RDEPENDS_packagename</filename>,
9535 <filename>DESCRIPTION_packagename</filename>, and so forth.
9536 Here is an example from the <filename>lighttpd</filename>
9537 recipe:
9538 <literallayout class='monospaced'>
9539 python populate_packages_prepend () {
9540 lighttpd_libdir = d.expand('${libdir}')
9541 do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
9542 'lighttpd-module-%s', 'Lighttpd module for %s',
9543 extra_depends='')
9544 }
9545 </literallayout>
9546 The previous example specifies a number of things in the
9547 call to <filename>do_split_packages</filename>.
9548 <itemizedlist>
9549 <listitem><para>A directory within the files installed
9550 by your recipe through <filename>do_install</filename>
9551 in which to search.</para></listitem>
9552 <listitem><para>A regular expression used to match module
9553 files in that directory.
9554 In the example, note the parentheses () that mark
9555 the part of the expression from which the module
9556 name should be derived.</para></listitem>
9557 <listitem><para>A pattern to use for the package names.
9558 </para></listitem>
9559 <listitem><para>A description for each package.
9560 </para></listitem>
9561 <listitem><para>An empty string for
9562 <filename>extra_depends</filename>, which disables
9563 the default dependency on the main
9564 <filename>lighttpd</filename> package.
9565 Thus, if a file in <filename>${libdir}</filename>
9566 called <filename>mod_alias.so</filename> is found,
9567 a package called <filename>lighttpd-module-alias</filename>
9568 is created for it and the
9569 <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink>
9570 is set to "Lighttpd module for alias".</para></listitem>
9571 </itemizedlist>
9572 </para>
9573
9574 <para>
9575 Often, packaging modules is as simple as the previous
9576 example.
9577 However, more advanced options exist that you can use
9578 within <filename>do_split_packages</filename> to modify its
9579 behavior.
9580 And, if you need to, you can add more logic by specifying
9581 a hook function that is called for each package.
9582 It is also perfectly acceptable to call
9583 <filename>do_split_packages</filename> multiple times if
9584 you have more than one set of modules to package.
9585 </para>
9586
9587 <para>
9588 For more examples that show how to use
9589 <filename>do_split_packages</filename>, see the
9590 <filename>connman.inc</filename> file in the
9591 <filename>meta/recipes-connectivity/connman/</filename>
9592 directory of the <filename>poky</filename>
9593 <ulink url='&YOCTO_DOCS_OM_URL;#yocto-project-repositories'>source repository</ulink>.
9594 You can also find examples in
9595 <filename>meta/classes/kernel.bbclass</filename>.
9596 </para>
9597
9598 <para>
9599 Following is a reference that shows
9600 <filename>do_split_packages</filename> mandatory and
9601 optional arguments:
9602 <literallayout class='monospaced'>
9603 Mandatory arguments
9604
9605 root
9606 The path in which to search
9607 file_regex
9608 Regular expression to match searched files.
9609 Use parentheses () to mark the part of this
9610 expression that should be used to derive the
9611 module name (to be substituted where %s is
9612 used in other function arguments as noted below)
9613 output_pattern
9614 Pattern to use for the package names. Must
9615 include %s.
9616 description
9617 Description to set for each package. Must
9618 include %s.
9619
9620 Optional arguments
9621
9622 postinst
9623 Postinstall script to use for all packages
9624 (as a string)
9625 recursive
9626 True to perform a recursive search - default
9627 False
9628 hook
9629 A hook function to be called for every match.
9630 The function will be called with the following
9631 arguments (in the order listed):
9632
9633 f
9634 Full path to the file/directory match
9635 pkg
9636 The package name
9637 file_regex
9638 As above
9639 output_pattern
9640 As above
9641 modulename
9642 The module name derived using file_regex
9643
9644 extra_depends
9645 Extra runtime dependencies (RDEPENDS) to be
9646 set for all packages. The default value of None
9647 causes a dependency on the main package
9648 (${PN}) - if you do not want this, pass empty
9649 string '' for this parameter.
9650 aux_files_pattern
9651 Extra item(s) to be added to FILES for each
9652 package. Can be a single string item or a list
9653 of strings for multiple items. Must include %s.
9654 postrm
9655 postrm script to use for all packages (as a
9656 string)
9657 allow_dirs
9658 True to allow directories to be matched -
9659 default False
9660 prepend
9661 If True, prepend created packages to PACKAGES
9662 instead of the default False which appends them
9663 match_path
9664 match file_regex on the whole relative path to
9665 the root rather than just the file name
9666 aux_files_pattern_verbatim
9667 Extra item(s) to be added to FILES for each
9668 package, using the actual derived module name
9669 rather than converting it to something legal
9670 for a package name. Can be a single string item
9671 or a list of strings for multiple items. Must
9672 include %s.
9673 allow_links
9674 True to allow symlinks to be matched - default
9675 False
9676 summary
9677 Summary to set for each package. Must include %s;
9678 defaults to description if not set.
9679 </literallayout>
9680 </para>
9681 </section>
9682
9683 <section id='satisfying-dependencies'>
9684 <title>Satisfying Dependencies</title>
9685
9686 <para>
9687 The second part for handling optional module packaging
9688 is to ensure that any dependencies on optional modules
9689 from other recipes are satisfied by your recipe.
9690 You can be sure these dependencies are satisfied by
9691 using the
9692 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable.
9693 Here is an example that continues with the
9694 <filename>lighttpd</filename> recipe shown earlier:
9695 <literallayout class='monospaced'>
9696 PACKAGES_DYNAMIC = "lighttpd-module-.*"
9697 </literallayout>
9698 The name specified in the regular expression can of
9699 course be anything.
9700 In this example, it is <filename>lighttpd-module-</filename>
9701 and is specified as the prefix to ensure that any
9702 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
9703 and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>
9704 on a package name starting with the prefix are satisfied
9705 during build time.
9706 If you are using <filename>do_split_packages</filename>
9707 as described in the previous section, the value you put in
9708 <filename>PACKAGES_DYNAMIC</filename> should correspond to
9709 the name pattern specified in the call to
9710 <filename>do_split_packages</filename>.
9711 </para>
9712 </section>
9713 </section>
9714
9715 <section id='using-runtime-package-management'>
9716 <title>Using Runtime Package Management</title>
9717
9718 <para>
9719 During a build, BitBake always transforms a recipe into one or
9720 more packages.
9721 For example, BitBake takes the <filename>bash</filename> recipe
9722 and produces a number of packages (e.g.
9723 <filename>bash</filename>, <filename>bash-bashbug</filename>,
9724 <filename>bash-completion</filename>,
9725 <filename>bash-completion-dbg</filename>,
9726 <filename>bash-completion-dev</filename>,
9727 <filename>bash-completion-extra</filename>,
9728 <filename>bash-dbg</filename>, and so forth).
9729 Not all generated packages are included in an image.
9730 </para>
9731
9732 <para>
9733 In several situations, you might need to update, add, remove,
9734 or query the packages on a target device at runtime
9735 (i.e. without having to generate a new image).
9736 Examples of such situations include:
9737 <itemizedlist>
9738 <listitem><para>
9739 You want to provide in-the-field updates to deployed
9740 devices (e.g. security updates).
9741 </para></listitem>
9742 <listitem><para>
9743 You want to have a fast turn-around development cycle
9744 for one or more applications that run on your device.
9745 </para></listitem>
9746 <listitem><para>
9747 You want to temporarily install the "debug" packages
9748 of various applications on your device so that
9749 debugging can be greatly improved by allowing
9750 access to symbols and source debugging.
9751 </para></listitem>
9752 <listitem><para>
9753 You want to deploy a more minimal package selection of
9754 your device but allow in-the-field updates to add a
9755 larger selection for customization.
9756 </para></listitem>
9757 </itemizedlist>
9758 </para>
9759
9760 <para>
9761 In all these situations, you have something similar to a more
9762 traditional Linux distribution in that in-field devices
9763 are able to receive pre-compiled packages from a server for
9764 installation or update.
9765 Being able to install these packages on a running,
9766 in-field device is what is termed "runtime package
9767 management".
9768 </para>
9769
9770 <para>
9771 In order to use runtime package management, you
9772 need a host or server machine that serves up the pre-compiled
9773 packages plus the required metadata.
9774 You also need package manipulation tools on the target.
9775 The build machine is a likely candidate to act as the server.
9776 However, that machine does not necessarily have to be the
9777 package server.
9778 The build machine could push its artifacts to another machine
9779 that acts as the server (e.g. Internet-facing).
9780 In fact, doing so is advantageous for a production
9781 environment as getting the packages away from the
9782 development system's build directory prevents accidental
9783 overwrites.
9784 </para>
9785
9786 <para>
9787 A simple build that targets just one device produces
9788 more than one package database.
9789 In other words, the packages produced by a build are separated
9790 out into a couple of different package groupings based on
9791 criteria such as the target's CPU architecture, the target
9792 board, or the C library used on the target.
9793 For example, a build targeting the <filename>qemux86</filename>
9794 device produces the following three package databases:
9795 <filename>noarch</filename>, <filename>i586</filename>, and
9796 <filename>qemux86</filename>.
9797 If you wanted your <filename>qemux86</filename> device to be
9798 aware of all the packages that were available to it,
9799 you would need to point it to each of these databases
9800 individually.
9801 In a similar way, a traditional Linux distribution usually is
9802 configured to be aware of a number of software repositories
9803 from which it retrieves packages.
9804 </para>
9805
9806 <para>
9807 Using runtime package management is completely optional and
9808 not required for a successful build or deployment in any
9809 way.
9810 But if you want to make use of runtime package management,
9811 you need to do a couple things above and beyond the basics.
9812 The remainder of this section describes what you need to do.
9813 </para>
9814
9815 <section id='runtime-package-management-build'>
9816 <title>Build Considerations</title>
9817
9818 <para>
9819 This section describes build considerations of which you
9820 need to be aware in order to provide support for runtime
9821 package management.
9822 </para>
9823
9824 <para>
9825 When BitBake generates packages, it needs to know
9826 what format or formats to use.
9827 In your configuration, you use the
9828 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
9829 variable to specify the format:
9830 <orderedlist>
9831 <listitem><para>
9832 Open the <filename>local.conf</filename> file
9833 inside your
9834 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
9835 (e.g. <filename>~/poky/build/conf/local.conf</filename>).
9836 </para></listitem>
9837 <listitem><para>
9838 Select the desired package format as follows:
9839 <literallayout class='monospaced'>
9840 PACKAGE_CLASSES ?= "package_<replaceable>packageformat</replaceable>"
9841 </literallayout>
9842 where <replaceable>packageformat</replaceable>
9843 can be "ipk", "rpm", "deb", or "tar" which are the
9844 supported package formats.
9845 <note>
9846 Because the Yocto Project supports four
9847 different package formats, you can set the
9848 variable with more than one argument.
9849 However, the OpenEmbedded build system only
9850 uses the first argument when creating an image
9851 or Software Development Kit (SDK).
9852 </note>
9853 </para></listitem>
9854 </orderedlist>
9855 </para>
9856
9857 <para>
9858 If you would like your image to start off with a basic
9859 package database containing the packages in your current
9860 build as well as to have the relevant tools available on the
9861 target for runtime package management, you can include
9862 "package-management" in the
9863 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
9864 variable.
9865 Including "package-management" in this configuration
9866 variable ensures that when the image is assembled for your
9867 target, the image includes the currently-known package
9868 databases as well as the target-specific tools required
9869 for runtime package management to be performed on the
9870 target.
9871 However, this is not strictly necessary.
9872 You could start your image off without any databases
9873 but only include the required on-target package
9874 tool(s).
9875 As an example, you could include "opkg" in your
9876 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
9877 variable if you are using the IPK package format.
9878 You can then initialize your target's package database(s)
9879 later once your image is up and running.
9880 </para>
9881
9882 <para>
9883 Whenever you perform any sort of build step that can
9884 potentially generate a package or modify existing
9885 package, it is always a good idea to re-generate the
9886 package index after the build by using the following
9887 command:
9888 <literallayout class='monospaced'>
9889 $ bitbake package-index
9890 </literallayout>
9891 It might be tempting to build the package and the
9892 package index at the same time with a command such as
9893 the following:
9894 <literallayout class='monospaced'>
9895 $ bitbake <replaceable>some-package</replaceable> package-index
9896 </literallayout>
9897 Do not do this as BitBake does not schedule the package
9898 index for after the completion of the package you are
9899 building.
9900 Consequently, you cannot be sure of the package index
9901 including information for the package you just built.
9902 Thus, be sure to run the package update step separately
9903 after building any packages.
9904 </para>
9905
9906 <para>
9907 You can use the
9908 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
9909 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>,
9910 and
9911 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
9912 variables to pre-configure target images to use a package
9913 feed.
9914 If you do not define these variables, then manual steps
9915 as described in the subsequent sections are necessary to
9916 configure the target.
9917 You should set these variables before building the image
9918 in order to produce a correctly configured image.
9919 </para>
9920
9921 <para>
9922 When your build is complete, your packages reside in the
9923 <filename>${TMPDIR}/deploy/<replaceable>packageformat</replaceable></filename>
9924 directory.
9925 For example, if
9926 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink><filename>}</filename>
9927 is <filename>tmp</filename> and your selected package type
9928 is RPM, then your RPM packages are available in
9929 <filename>tmp/deploy/rpm</filename>.
9930 </para>
9931 </section>
9932
9933 <section id='runtime-package-management-server'>
9934 <title>Host or Server Machine Setup</title>
9935
9936 <para>
9937 Although other protocols are possible, a server using HTTP
9938 typically serves packages.
9939 If you want to use HTTP, then set up and configure a
9940 web server such as Apache 2, lighttpd, or
9941 SimpleHTTPServer on the machine serving the packages.
9942 </para>
9943
9944 <para>
9945 To keep things simple, this section describes how to set
9946 up a SimpleHTTPServer web server to share package feeds
9947 from the developer's machine.
9948 Although this server might not be the best for a production
9949 environment, the setup is simple and straight forward.
9950 Should you want to use a different server more suited for
9951 production (e.g. Apache 2, Lighttpd, or Nginx), take the
9952 appropriate steps to do so.
9953 </para>
9954
9955 <para>
9956 From within the build directory where you have built an
9957 image based on your packaging choice (i.e. the
9958 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
9959 setting), simply start the server.
9960 The following example assumes a build directory of
9961 <filename>~/poky/build/tmp/deploy/rpm</filename> and a
9962 <filename>PACKAGE_CLASSES</filename> setting of
9963 "package_rpm":
9964 <literallayout class='monospaced'>
9965 $ cd ~/poky/build/tmp/deploy/rpm
9966 $ python -m SimpleHTTPServer
9967 </literallayout>
9968 </para>
9969 </section>
9970
9971 <section id='runtime-package-management-target'>
9972 <title>Target Setup</title>
9973
9974 <para>
9975 Setting up the target differs depending on the
9976 package management system.
9977 This section provides information for RPM, IPK, and DEB.
9978 </para>
9979
9980 <section id='runtime-package-management-target-rpm'>
9981 <title>Using RPM</title>
9982
9983 <para>
9984 The
9985 <ulink url='https://en.wikipedia.org/wiki/DNF_(software)'>Dandified Packaging Tool</ulink>
9986 (DNF) performs runtime package management of RPM
9987 packages.
9988 In order to use DNF for runtime package management,
9989 you must perform an initial setup on the target
9990 machine for cases where the
9991 <filename>PACKAGE_FEED_*</filename> variables were not
9992 set as part of the image that is running on the
9993 target.
9994 This means if you built your image and did not not use
9995 these variables as part of the build and your image is
9996 now running on the target, you need to perform the
9997 steps in this section if you want to use runtime
9998 package management.
9999 <note>
10000 For information on the
10001 <filename>PACKAGE_FEED_*</filename> variables, see
10002 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
10003 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>,
10004 and
10005 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
10006 in the Yocto Project Reference Manual variables
10007 glossary.
10008 </note>
10009 </para>
10010
10011 <para>
10012 On the target, you must inform DNF that package
10013 databases are available.
10014 You do this by creating a file named
10015 <filename>/etc/yum.repos.d/oe-packages.repo</filename>
10016 and defining the <filename>oe-packages</filename>.
10017 </para>
10018
10019 <para>
10020 As an example, assume the target is able to use the
10021 following package databases:
10022 <filename>all</filename>, <filename>i586</filename>,
10023 and <filename>qemux86</filename> from a server named
10024 <filename>my.server</filename>.
10025 The specifics for setting up the web server are up to
10026 you.
10027 The critical requirement is that the URIs in the
10028 target repository configuration point to the
10029 correct remote location for the feeds.
10030 <note><title>Tip</title>
10031 For development purposes, you can point the web
10032 server to the build system's
10033 <filename>deploy</filename> directory.
10034 However, for production use, it is better to copy
10035 the package directories to a location outside of
10036 the build area and use that location.
10037 Doing so avoids situations where the build system
10038 overwrites or changes the
10039 <filename>deploy</filename> directory.
10040 </note>
10041 </para>
10042
10043 <para>
10044 When telling DNF where to look for the package
10045 databases, you must declare individual locations
10046 per architecture or a single location used for all
10047 architectures.
10048 You cannot do both:
10049 <itemizedlist>
10050 <listitem><para>
10051 <emphasis>Create an Explicit List of Architectures:</emphasis>
10052 Define individual base URLs to identify where
10053 each package database is located:
10054 <literallayout class='monospaced'>
10055 [oe-packages]
10056 baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all
10057 </literallayout>
10058 This example informs DNF about individual
10059 package databases for all three architectures.
10060 </para></listitem>
10061 <listitem><para>
10062 <emphasis>Create a Single (Full) Package Index:</emphasis>
10063 Define a single base URL that identifies where
10064 a full package database is located:
10065 <literallayout class='monospaced'>
10066 [oe-packages]
10067 baseurl=http://my.server/rpm
10068 </literallayout>
10069 This example informs DNF about a single package
10070 database that contains all the package index
10071 information for all supported architectures.
10072 </para></listitem>
10073 </itemizedlist>
10074 </para>
10075
10076 <para>
10077 Once you have informed DNF where to find the package
10078 databases, you need to fetch them:
10079 <literallayout class='monospaced'>
10080 # dnf makecache
10081 </literallayout>
10082 DNF is now able to find, install, and upgrade packages
10083 from the specified repository or repositories.
10084 <note>
10085 See the
10086 <ulink url='http://dnf.readthedocs.io/en/latest/'>DNF documentation</ulink>
10087 for additional information.
10088 </note>
10089 </para>
10090 </section>
10091
10092 <section id='runtime-package-management-target-ipk'>
10093 <title>Using IPK</title>
10094
10095 <para>
10096 The <filename>opkg</filename> application performs
10097 runtime package management of IPK packages.
10098 You must perform an initial setup for
10099 <filename>opkg</filename> on the target machine
10100 if the
10101 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
10102 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and
10103 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
10104 variables have not been set or the target image was
10105 built before the variables were set.
10106 </para>
10107
10108 <para>
10109 The <filename>opkg</filename> application uses
10110 configuration files to find available package
10111 databases.
10112 Thus, you need to create a configuration file inside
10113 the <filename>/etc/opkg/</filename> direction, which
10114 informs <filename>opkg</filename> of any repository
10115 you want to use.
10116 </para>
10117
10118 <para>
10119 As an example, suppose you are serving packages from a
10120 <filename>ipk/</filename> directory containing the
10121 <filename>i586</filename>,
10122 <filename>all</filename>, and
10123 <filename>qemux86</filename> databases through an
10124 HTTP server named <filename>my.server</filename>.
10125 On the target, create a configuration file
10126 (e.g. <filename>my_repo.conf</filename>) inside the
10127 <filename>/etc/opkg/</filename> directory containing
10128 the following:
10129 <literallayout class='monospaced'>
10130 src/gz all http://my.server/ipk/all
10131 src/gz i586 http://my.server/ipk/i586
10132 src/gz qemux86 http://my.server/ipk/qemux86
10133 </literallayout>
10134 Next, instruct <filename>opkg</filename> to fetch
10135 the repository information:
10136 <literallayout class='monospaced'>
10137 # opkg update
10138 </literallayout>
10139 The <filename>opkg</filename> application is now able
10140 to find, install, and upgrade packages from the
10141 specified repository.
10142 </para>
10143 </section>
10144
10145 <section id='runtime-package-management-target-deb'>
10146 <title>Using DEB</title>
10147
10148 <para>
10149 The <filename>apt</filename> application performs
10150 runtime package management of DEB packages.
10151 This application uses a source list file to find
10152 available package databases.
10153 You must perform an initial setup for
10154 <filename>apt</filename> on the target machine
10155 if the
10156 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
10157 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and
10158 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
10159 variables have not been set or the target image was
10160 built before the variables were set.
10161 </para>
10162
10163 <para>
10164 To inform <filename>apt</filename> of the repository
10165 you want to use, you might create a list file (e.g.
10166 <filename>my_repo.list</filename>) inside the
10167 <filename>/etc/apt/sources.list.d/</filename>
10168 directory.
10169 As an example, suppose you are serving packages from a
10170 <filename>deb/</filename> directory containing the
10171 <filename>i586</filename>,
10172 <filename>all</filename>, and
10173 <filename>qemux86</filename> databases through an
10174 HTTP server named <filename>my.server</filename>.
10175 The list file should contain:
10176 <literallayout class='monospaced'>
10177 deb http://my.server/deb/all ./
10178 deb http://my.server/deb/i586 ./
10179 deb http://my.server/deb/qemux86 ./
10180 </literallayout>
10181 Next, instruct the <filename>apt</filename>
10182 application to fetch the repository information:
10183 <literallayout class='monospaced'>
10184 # apt-get update
10185 </literallayout>
10186 After this step, <filename>apt</filename> is able
10187 to find, install, and upgrade packages from the
10188 specified repository.
10189 </para>
10190 </section>
10191 </section>
10192 </section>
10193
10194 <section id='generating-and-using-signed-packages'>
10195 <title>Generating and Using Signed Packages</title>
10196 <para>
10197 In order to add security to RPM packages used during a build,
10198 you can take steps to securely sign them.
10199 Once a signature is verified, the OpenEmbedded build system
10200 can use the package in the build.
10201 If security fails for a signed package, the build system
10202 aborts the build.
10203 </para>
10204
10205 <para>
10206 This section describes how to sign RPM packages during a build
10207 and how to use signed package feeds (repositories) when
10208 doing a build.
10209 </para>
10210
10211 <section id='signing-rpm-packages'>
10212 <title>Signing RPM Packages</title>
10213
10214 <para>
10215 To enable signing RPM packages, you must set up the
10216 following configurations in either your
10217 <filename>local.config</filename> or
10218 <filename>distro.config</filename> file:
10219 <literallayout class='monospaced'>
10220 # Inherit sign_rpm.bbclass to enable signing functionality
10221 INHERIT += " sign_rpm"
10222 # Define the GPG key that will be used for signing.
10223 RPM_GPG_NAME = "<replaceable>key_name</replaceable>"
10224 # Provide passphrase for the key
10225 RPM_GPG_PASSPHRASE = "<replaceable>passphrase</replaceable>"
10226 </literallayout>
10227 <note>
10228 Be sure to supply appropriate values for both
10229 <replaceable>key_name</replaceable> and
10230 <replaceable>passphrase</replaceable>
10231 </note>
10232 Aside from the
10233 <filename>RPM_GPG_NAME</filename> and
10234 <filename>RPM_GPG_PASSPHRASE</filename> variables in the
10235 previous example, two optional variables related to signing
10236 exist:
10237 <itemizedlist>
10238 <listitem><para>
10239 <emphasis><filename>GPG_BIN</filename>:</emphasis>
10240 Specifies a <filename>gpg</filename> binary/wrapper
10241 that is executed when the package is signed.
10242 </para></listitem>
10243 <listitem><para>
10244 <emphasis><filename>GPG_PATH</filename>:</emphasis>
10245 Specifies the <filename>gpg</filename> home
10246 directory used when the package is signed.
10247 </para></listitem>
10248 </itemizedlist>
10249 </para>
10250 </section>
10251
10252 <section id='processing-package-feeds'>
10253 <title>Processing Package Feeds</title>
10254
10255 <para>
10256 In addition to being able to sign RPM packages, you can
10257 also enable signed package feeds for IPK and RPM packages.
10258 </para>
10259
10260 <para>
10261 The steps you need to take to enable signed package feed
10262 use are similar to the steps used to sign RPM packages.
10263 You must define the following in your
10264 <filename>local.config</filename> or
10265 <filename>distro.config</filename> file:
10266 <literallayout class='monospaced'>
10267 INHERIT += "sign_package_feed"
10268 PACKAGE_FEED_GPG_NAME = "<replaceable>key_name</replaceable>"
10269 PACKAGE_FEED_GPG_PASSPHRASE_FILE = "<replaceable>path_to_file_containing_passphrase</replaceable>"
10270 </literallayout>
10271 For signed package feeds, the passphrase must exist in a
10272 separate file, which is pointed to by the
10273 <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename>
10274 variable.
10275 Regarding security, keeping a plain text passphrase out of
10276 the configuration is more secure.
10277 </para>
10278
10279 <para>
10280 Aside from the
10281 <filename>PACKAGE_FEED_GPG_NAME</filename> and
10282 <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename>
10283 variables, three optional variables related to signed
10284 package feeds exist:
10285 <itemizedlist>
10286 <listitem><para>
10287 <emphasis><filename>GPG_BIN</filename>:</emphasis>
10288 Specifies a <filename>gpg</filename> binary/wrapper
10289 that is executed when the package is signed.
10290 </para></listitem>
10291 <listitem><para>
10292 <emphasis><filename>GPG_PATH</filename>:</emphasis>
10293 Specifies the <filename>gpg</filename> home
10294 directory used when the package is signed.
10295 </para></listitem>
10296 <listitem><para>
10297 <emphasis><filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename>:</emphasis>
10298 Specifies the type of <filename>gpg</filename>
10299 signature.
10300 This variable applies only to RPM and IPK package
10301 feeds.
10302 Allowable values for the
10303 <filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename>
10304 are "ASC", which is the default and specifies ascii
10305 armored, and "BIN", which specifies binary.
10306 </para></listitem>
10307 </itemizedlist>
10308 </para>
10309 </section>
10310 </section>
10311
10312 <section id='testing-packages-with-ptest'>
10313 <title>Testing Packages With ptest</title>
10314
10315 <para>
10316 A Package Test (ptest) runs tests against packages built
10317 by the OpenEmbedded build system on the target machine.
10318 A ptest contains at least two items: the actual test, and
10319 a shell script (<filename>run-ptest</filename>) that starts
10320 the test.
10321 The shell script that starts the test must not contain
10322 the actual test - the script only starts the test.
10323 On the other hand, the test can be anything from a simple
10324 shell script that runs a binary and checks the output to
10325 an elaborate system of test binaries and data files.
10326 </para>
10327
10328 <para>
10329 The test generates output in the format used by
10330 Automake:
10331 <literallayout class='monospaced'>
10332 <replaceable>result</replaceable>: <replaceable>testname</replaceable>
10333 </literallayout>
10334 where the result can be <filename>PASS</filename>,
10335 <filename>FAIL</filename>, or <filename>SKIP</filename>,
10336 and the testname can be any identifying string.
10337 </para>
10338
10339 <para>
10340 For a list of Yocto Project recipes that are already
10341 enabled with ptest, see the
10342 <ulink url='https://wiki.yoctoproject.org/wiki/Ptest'>Ptest</ulink>
10343 wiki page.
10344 <note>
10345 A recipe is "ptest-enabled" if it inherits the
10346 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink>
10347 class.
10348 </note>
10349 </para>
10350
10351 <section id='adding-ptest-to-your-build'>
10352 <title>Adding ptest to Your Build</title>
10353
10354 <para>
10355 To add package testing to your build, add the
10356 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
10357 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
10358 variables to your <filename>local.conf</filename> file,
10359 which is found in the
10360 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
10361 <literallayout class='monospaced'>
10362 DISTRO_FEATURES_append = " ptest"
10363 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
10364 </literallayout>
10365 Once your build is complete, the ptest files are installed
10366 into the
10367 <filename>/usr/lib/<replaceable>package</replaceable>/ptest</filename>
10368 directory within the image, where
10369 <filename><replaceable>package</replaceable></filename>
10370 is the name of the package.
10371 </para>
10372 </section>
10373
10374 <section id='running-ptest'>
10375 <title>Running ptest</title>
10376
10377 <para>
10378 The <filename>ptest-runner</filename> package installs a
10379 shell script that loops through all installed ptest test
10380 suites and runs them in sequence.
10381 Consequently, you might want to add this package to
10382 your image.
10383 </para>
10384 </section>
10385
10386 <section id='getting-your-package-ready'>
10387 <title>Getting Your Package Ready</title>
10388
10389 <para>
10390 In order to enable a recipe to run installed ptests
10391 on target hardware,
10392 you need to prepare the recipes that build the packages
10393 you want to test.
10394 Here is what you have to do for each recipe:
10395 <itemizedlist>
10396 <listitem><para><emphasis>Be sure the recipe
10397 inherits the
10398 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink>
10399 class:</emphasis>
10400 Include the following line in each recipe:
10401 <literallayout class='monospaced'>
10402 inherit ptest
10403 </literallayout>
10404 </para></listitem>
10405 <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis>
10406 This script starts your test.
10407 Locate the script where you will refer to it
10408 using
10409 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
10410 Here is an example that starts a test for
10411 <filename>dbus</filename>:
10412 <literallayout class='monospaced'>
10413 #!/bin/sh
10414 cd test
10415 make -k runtest-TESTS
10416 </literallayout>
10417 </para></listitem>
10418 <listitem><para><emphasis>Ensure dependencies are
10419 met:</emphasis>
10420 If the test adds build or runtime dependencies
10421 that normally do not exist for the package
10422 (such as requiring "make" to run the test suite),
10423 use the
10424 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
10425 and
10426 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
10427 variables in your recipe in order for the package
10428 to meet the dependencies.
10429 Here is an example where the package has a runtime
10430 dependency on "make":
10431 <literallayout class='monospaced'>
10432 RDEPENDS_${PN}-ptest += "make"
10433 </literallayout>
10434 </para></listitem>
10435 <listitem><para><emphasis>Add a function to build the
10436 test suite:</emphasis>
10437 Not many packages support cross-compilation of
10438 their test suites.
10439 Consequently, you usually need to add a
10440 cross-compilation function to the package.
10441 </para>
10442
10443 <para>Many packages based on Automake compile and
10444 run the test suite by using a single command
10445 such as <filename>make check</filename>.
10446 However, the host <filename>make check</filename>
10447 builds and runs on the same computer, while
10448 cross-compiling requires that the package is built
10449 on the host but executed for the target
10450 architecture (though often, as in the case for
10451 ptest, the execution occurs on the host).
10452 The built version of Automake that ships with the
10453 Yocto Project includes a patch that separates
10454 building and execution.
10455 Consequently, packages that use the unaltered,
10456 patched version of <filename>make check</filename>
10457 automatically cross-compiles.</para>
10458 <para>Regardless, you still must add a
10459 <filename>do_compile_ptest</filename> function to
10460 build the test suite.
10461 Add a function similar to the following to your
10462 recipe:
10463 <literallayout class='monospaced'>
10464 do_compile_ptest() {
10465 oe_runmake buildtest-TESTS
10466 }
10467 </literallayout>
10468 </para></listitem>
10469 <listitem><para><emphasis>Ensure special configurations
10470 are set:</emphasis>
10471 If the package requires special configurations
10472 prior to compiling the test code, you must
10473 insert a <filename>do_configure_ptest</filename>
10474 function into the recipe.
10475 </para></listitem>
10476 <listitem><para><emphasis>Install the test
10477 suite:</emphasis>
10478 The <filename>ptest</filename> class
10479 automatically copies the file
10480 <filename>run-ptest</filename> to the target and
10481 then runs make <filename>install-ptest</filename>
10482 to run the tests.
10483 If this is not enough, you need to create a
10484 <filename>do_install_ptest</filename> function and
10485 make sure it gets called after the
10486 "make install-ptest" completes.
10487 </para></listitem>
10488 </itemizedlist>
10489 </para>
10490 </section>
10491 </section>
10492
10493 <section id='creating-node-package-manager-npm-packages'>
10494 <title>Creating Node Package Manager (NPM) Packages</title>
10495
10496 <para>
10497 <ulink url='https://en.wikipedia.org/wiki/Npm_(software)'>NPM</ulink>
10498 is a package manager for the JavaScript programming
10499 language.
10500 The Yocto Project supports the NPM
10501 <ulink url='&YOCTO_DOCS_BB_URL;#bb-fetchers'>fetcher</ulink>.
10502 You can use this fetcher in combination with
10503 <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool</filename></ulink>
10504 to create recipes that produce NPM packages.
10505 </para>
10506
10507 <para>
10508 Two workflows exist that allow you to create NPM packages
10509 using <filename>devtool</filename>: the NPM registry modules
10510 method and the NPM project code method.
10511 <note>
10512 While it is possible to create NPM recipes manually,
10513 using <filename>devtool</filename> is far simpler.
10514 </note>
10515 Additionally, some requirements and caveats exist.
10516 </para>
10517
10518 <section id='npm-package-creation-requirements'>
10519 <title>Requirements and Caveats</title>
10520
10521 <para>
10522 You need to be aware of the following before using
10523 <filename>devtool</filename> to create NPM packages:
10524 <itemizedlist>
10525 <listitem><para>
10526 Of the two methods that you can use
10527 <filename>devtool</filename> to create NPM
10528 packages, the registry approach is slightly
10529 simpler.
10530 However, you might consider the project
10531 approach because you do not have to publish
10532 your module in the NPM registry
10533 (<ulink url='https://docs.npmjs.com/misc/registry'><filename>npm-registry</filename></ulink>),
10534 which is NPM's public registry.
10535 </para></listitem>
10536 <listitem><para>
10537 Be familiar with
10538 <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool</filename></ulink>.
10539 </para></listitem>
10540 <listitem><para>
10541 The NPM host tools need the native
10542 <filename>nodejs-npm</filename> package, which
10543 is part of the OpenEmbedded environment.
10544 You need to get the package by cloning the
10545 <ulink url='https://github.com/openembedded/meta-openembedded'></ulink>
10546 repository out of GitHub.
10547 Be sure to add the path to your local copy to
10548 your <filename>bblayers.conf</filename> file.
10549 </para></listitem>
10550 <listitem><para>
10551 <filename>devtool</filename> cannot detect
10552 native libraries in module dependencies.
10553 Consequently, you must manually add packages
10554 to your recipe.
10555 </para></listitem>
10556 <listitem><para>
10557 While deploying NPM packages,
10558 <filename>devtool</filename> cannot determine
10559 which dependent packages are missing on the
10560 target (e.g. the node runtime
10561 <filename>nodejs</filename>).
10562 Consequently, you need to find out what
10563 files are missing and be sure they are on the
10564 target.
10565 </para></listitem>
10566 <listitem><para>
10567 Although you might not need NPM to run your
10568 node package, it is useful to have NPM on your
10569 target.
10570 The NPM package name is
10571 <filename>nodejs-npm</filename>.
10572 </para></listitem>
10573 </itemizedlist>
10574 </para>
10575 </section>
10576
10577 <section id='npm-using-the-registry-modules-method'>
10578 <title>Using the Registry Modules Method</title>
10579
10580 <para>
10581 This section presents an example that uses the
10582 <filename>cute-files</filename> module, which is a
10583 file browser web application.
10584 <note>
10585 You must know the <filename>cute-files</filename>
10586 module version.
10587 </note>
10588 </para>
10589
10590 <para>
10591 The first thing you need to do is use
10592 <filename>devtool</filename> and the NPM fetcher to
10593 create the recipe:
10594 <literallayout class='monospaced'>
10595 $ devtool add "npm://registry.npmjs.org;package=cute-files;version=1.0.2"
10596 </literallayout>
10597 The <filename>devtool add</filename> command runs
10598 <filename>recipetool create</filename> and uses the
10599 same fetch URI to download each dependency and capture
10600 license details where possible.
10601 The result is a generated recipe.
10602 </para>
10603
10604 <para>
10605 The recipe file is fairly simple and contains every
10606 license that <filename>recipetool</filename> finds
10607 and includes the licenses in the recipe's
10608 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
10609 variables.
10610 You need to examine the variables and look for those
10611 with "unknown" in the
10612 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
10613 field.
10614 You need to track down the license information for
10615 "unknown" modules and manually add the information to the
10616 recipe.
10617 </para>
10618
10619 <para>
10620 <filename>recipetool</filename> creates a "shrinkwrap" file
10621 for your recipe.
10622 Shrinkwrap files capture the version of all dependent
10623 modules.
10624 Many packages do not provide shrinkwrap files.
10625 <filename>recipetool</filename> create a shrinkwrap
10626 file as it runs.
10627 <note>
10628 A package is created for each sub-module.
10629 This policy is the only practical way to have the
10630 licenses for all of the dependencies represented
10631 in the license manifest of the image.
10632 </note>
10633 </para>
10634
10635 <para>
10636 The <filename>devtool edit-recipe</filename> command
10637 lets you take a look at the recipe:
10638 <literallayout class='monospaced'>
10639 $ devtool edit-recipe cute-files
10640 SUMMARY = "Turn any folder on your computer into a cute file browser, available on the local network."
10641 LICENSE = "MIT &amp; ISC &amp; Unknown"
10642 LIC_FILES_CHKSUM = "file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \
10643 file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502 \
10644 file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99 \
10645 ...
10646
10647 SRC_URI = " \
10648 npm://registry.npmjs.org/;package=cute-files;version=${PV} \
10649 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
10650 "
10651
10652 S = "${WORKDIR}/npm"
10653
10654 inherit npm
10655
10656 LICENSE_${PN} = "MIT"
10657 LICENSE_${PN}-accepts = "MIT"
10658 LICENSE_${PN}-array-flatten = "MIT"
10659 ...
10660 LICENSE_${PN}-vary = "MIT"
10661 </literallayout>
10662 Three key points exist in the previous example:
10663 <itemizedlist>
10664 <listitem><para>
10665 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
10666 uses the NPM scheme so that the NPM fetcher
10667 is used.
10668 </para></listitem>
10669 <listitem><para>
10670 <filename>recipetool</filename> collects all
10671 the license information.
10672 If a sub-module's license is unavailable,
10673 the sub-module's name appears in the comments.
10674 </para></listitem>
10675 <listitem><para>
10676 The <filename>inherit npm</filename> statement
10677 causes the
10678 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-npm'><filename>npm</filename></ulink>
10679 class to package up all the modules.
10680 </para></listitem>
10681 </itemizedlist>
10682 </para>
10683
10684 <para>
10685 You can run the following command to build the
10686 <filename>cute-files</filename> package:
10687 <literallayout class='monospaced'>
10688 $ devtool build cute-files
10689 </literallayout>
10690 Remember that <filename>nodejs</filename> must be
10691 installed on the target before your package.
10692 </para>
10693
10694 <para>
10695 Assuming 192.168.7.2 for the target's IP address, use
10696 the following command to deploy your package:
10697 <literallayout class='monospaced'>
10698 $ devtool deploy-target -s cute-files root@192.168.7.2
10699 </literallayout>
10700 Once the package is installed on the target, you can
10701 test the application:
10702 <note>
10703 Because of a know issue, you cannot simply run
10704 <filename>cute-files</filename> as you would if you
10705 had run <filename>npm install</filename>.
10706 </note>
10707 <literallayout class='monospaced'>
10708 $ cd /usr/lib/node_modules/cute-files
10709 $ node cute-files.js
10710 </literallayout>
10711 On a browser, go to
10712 <filename>http://192.168.7.2:3000</filename> and you
10713 see the following:
10714 <imagedata fileref="figures/cute-files-npm-example.png" align="center" width="6in" depth="4in" />
10715 </para>
10716
10717 <para>
10718 You can find the recipe in
10719 <filename>workspace/recipes/cute-files</filename>.
10720 You can use the recipe in any layer you choose.
10721 </para>
10722 </section>
10723
10724 <section id='npm-using-the-npm-projects-method'>
10725 <title>Using the NPM Projects Code Method</title>
10726
10727 <para>
10728 Although it is useful to package modules already in the
10729 NPM registry, adding <filename>node.js</filename> projects
10730 under development is a more common developer use case.
10731 </para>
10732
10733 <para>
10734 This section covers the NPM projects code method, which is
10735 very similar to the "registry" approach described in the
10736 previous section.
10737 In the NPM projects method, you provide
10738 <filename>devtool</filename> with an URL that points to the
10739 source files.
10740 </para>
10741
10742 <para>
10743 Replicating the same example, (i.e.
10744 <filename>cute-files</filename>) use the following command:
10745 <literallayout class='monospaced'>
10746 $ devtool add https://github.com/martinaglv/cute-files.git
10747 </literallayout>
10748 The recipe this command generates is very similar to the
10749 recipe created in the previous section.
10750 However, the <filename>SRC_URI</filename> looks like the
10751 following:
10752 <literallayout class='monospaced'>
10753 SRC_URI = " \
10754 git://github.com/martinaglv/cute-files.git;protocol=https \
10755 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
10756 "
10757 </literallayout>
10758 In this example, the main module is taken from the Git
10759 repository and dependents are taken from the NPM registry.
10760 Other than those differences, the recipe is basically the
10761 same between the two methods.
10762 You can build and deploy the package exactly as described
10763 in the previous section that uses the registry modules
10764 method.
10765 </para>
10766 </section>
10767 </section>
10768
10769 <section id='adding-custom-metadata-to-packages'>
10770 <title>Adding custom metadata to packages</title>
10771
10772 <para>
10773 The variable <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ADD_METADATA'><filename>PACKAGE_ADD_METADATA</filename></ulink>
10774 can be used to add additional metadata to packages. This is
10775 reflected in the package control/spec file. To take the ipk
10776 format for example, the CONTROL file stored inside would
10777 contain the additional metadata as additional lines.
10778 </para>
10779
10780 <para>
10781 The variable can be used in multiple ways, including using
10782 suffixes to set it for a specific package type and/or package.
10783 Note that the order of precedence is the same as this list:
10784 <itemizedlist>
10785 <listitem><para>
10786 <filename>PACKAGE_ADD_METADATA_&lt;PKGTYPE&gt;_&lt;PN&gt;</filename>
10787 </para></listitem>
10788 <listitem><para>
10789 <filename>PACKAGE_ADD_METADATA_&lt;PKGTYPE&gt;</filename>
10790 </para></listitem>
10791 <listitem><para>
10792 <filename>PACKAGE_ADD_METADATA_&lt;PN&gt;</filename>
10793 </para></listitem>
10794 <listitem><para>
10795 <filename>PACKAGE_ADD_METADATA</filename>
10796 </para></listitem>
10797 </itemizedlist>
10798 &lt;PKGTYPE&gt; is a parameter and expected to be a
10799 distinct name of specific package type:
10800 <itemizedlist>
10801 <listitem><para>IPK for .ipk packages</para></listitem>
10802 <listitem><para>DEB for .deb packages</para></listitem>
10803 <listitem><para>RPM for .rpm packages</para></listitem>
10804 </itemizedlist>
10805 &lt;PN&gt; is a parameter and expected to be a package name.
10806 </para>
10807
10808 <para>
10809 The variable can contain multiple [one-line] metadata fields
10810 separated by the literal sequence '\n'. The separator can be
10811 redefined using the variable flag <filename>separator</filename>.
10812 </para>
10813
10814 <para>
10815 The following is an example that adds two custom fields for
10816 ipk packages:
10817 <literallayout class='monospaced'>
10818 PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup: Applications/Spreadsheets"
10819 </literallayout>
10820 </para>
10821 </section>
10822
10823 </section>
10824
10825 <section id='efficiently-fetching-source-files-during-a-build'>
10826 <title>Efficiently Fetching Source Files During a Build</title>
10827
10828 <para>
10829 The OpenEmbedded build system works with source files located
10830 through the
10831 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
10832 variable.
10833 When you build something using BitBake, a big part of the operation
10834 is locating and downloading all the source tarballs.
10835 For images, downloading all the source for various packages can
10836 take a significant amount of time.
10837 </para>
10838
10839 <para>
10840 This section shows you how you can use mirrors to speed up
10841 fetching source files and how you can pre-fetch files all of which
10842 leads to more efficient use of resources and time.
10843 </para>
10844
10845 <section id='setting-up-effective-mirrors'>
10846 <title>Setting up Effective Mirrors</title>
10847
10848 <para>
10849 A good deal that goes into a Yocto Project
10850 build is simply downloading all of the source tarballs.
10851 Maybe you have been working with another build system
10852 (OpenEmbedded or Angstrom) for which you have built up a
10853 sizable directory of source tarballs.
10854 Or, perhaps someone else has such a directory for which you
10855 have read access.
10856 If so, you can save time by adding statements to your
10857 configuration file so that the build process checks local
10858 directories first for existing tarballs before checking the
10859 Internet.
10860 </para>
10861
10862 <para>
10863 Here is an efficient way to set it up in your
10864 <filename>local.conf</filename> file:
10865 <literallayout class='monospaced'>
10866 SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/"
10867 INHERIT += "own-mirrors"
10868 BB_GENERATE_MIRROR_TARBALLS = "1"
10869 # BB_NO_NETWORK = "1"
10870 </literallayout>
10871 </para>
10872
10873 <para>
10874 In the previous example, the
10875 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink>
10876 variable causes the OpenEmbedded build system to generate
10877 tarballs of the Git repositories and store them in the
10878 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
10879 directory.
10880 Due to performance reasons, generating and storing these
10881 tarballs is not the build system's default behavior.
10882 </para>
10883
10884 <para>
10885 You can also use the
10886 <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink>
10887 variable.
10888 For an example, see the variable's glossary entry in the
10889 Yocto Project Reference Manual.
10890 </para>
10891 </section>
10892
10893 <section id='getting-source-files-and-suppressing-the-build'>
10894 <title>Getting Source Files and Suppressing the Build</title>
10895
10896 <para>
10897 Another technique you can use to ready yourself for a
10898 successive string of build operations, is to pre-fetch
10899 all the source files without actually starting a build.
10900 This technique lets you work through any download issues
10901 and ultimately gathers all the source files into your
10902 download directory
10903 <ulink url='&YOCTO_DOCS_REF_URL;#structure-build-downloads'><filename>build/downloads</filename></ulink>,
10904 which is located with
10905 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>.
10906 </para>
10907
10908 <para>
10909 Use the following BitBake command form to fetch all the
10910 necessary sources without starting the build:
10911 <literallayout class='monospaced'>
10912 $ bitbake <replaceable>target</replaceable> --runall=fetch
10913 </literallayout>
10914 This variation of the BitBake command guarantees that you
10915 have all the sources for that BitBake target should you
10916 disconnect from the Internet and want to do the build
10917 later offline.
10918 </para>
10919 </section>
10920 </section>
10921
10922 <section id="selecting-an-initialization-manager">
10923 <title>Selecting an Initialization Manager</title>
10924
10925 <para>
10926 By default, the Yocto Project uses SysVinit as the initialization
10927 manager.
10928 However, support also exists for systemd,
10929 which is a full replacement for init with
10930 parallel starting of services, reduced shell overhead and other
10931 features that are used by many distributions.
10932 </para>
10933
10934 <para>
10935 Within the system, SysVinit treats system components as services.
10936 These services are maintained as shell scripts stored in the
10937 <filename>/etc/init.d/</filename> directory.
10938 Services organize into different run levels.
10939 This organization is maintained by putting links to the services
10940 in the <filename>/etc/rcN.d/</filename> directories, where
10941 <replaceable>N/</replaceable> is one of the following options:
10942 "S", "0", "1", "2", "3", "4", "5", or "6".
10943 <note>
10944 Each runlevel has a dependency on the previous runlevel.
10945 This dependency allows the services to work properly.
10946 </note>
10947 </para>
10948
10949 <para>
10950 In comparison, systemd treats components as units.
10951 Using units is a broader concept as compared to using a service.
10952 A unit includes several different types of entities.
10953 Service is one of the types of entities.
10954 The runlevel concept in SysVinit corresponds to the concept of a
10955 target in systemd, where target is also a type of supported unit.
10956 </para>
10957
10958 <para>
10959 In a SysVinit-based system, services load sequentially (i.e. one
10960 by one) during and parallelization is not supported.
10961 With systemd, services start in parallel.
10962 Needless to say, the method can have an impact on system startup
10963 performance.
10964 </para>
10965
10966 <para>
10967 If you want to use SysVinit, you do
10968 not have to do anything.
10969 But, if you want to use systemd, you must
10970 take some steps as described in the following sections.
10971 </para>
10972
10973 <section id='using-systemd-exclusively'>
10974 <title>Using systemd Exclusively</title>
10975
10976 <para>
10977 Set these variables in your distribution configuration
10978 file as follows:
10979 <literallayout class='monospaced'>
10980 DISTRO_FEATURES_append = " systemd"
10981 VIRTUAL-RUNTIME_init_manager = "systemd"
10982 </literallayout>
10983 You can also prevent the SysVinit
10984 distribution feature from
10985 being automatically enabled as follows:
10986 <literallayout class='monospaced'>
10987 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
10988 </literallayout>
10989 Doing so removes any redundant SysVinit scripts.
10990 </para>
10991
10992 <para>
10993 To remove initscripts from your image altogether,
10994 set this variable also:
10995 <literallayout class='monospaced'>
10996 VIRTUAL-RUNTIME_initscripts = ""
10997 </literallayout>
10998 </para>
10999
11000 <para>
11001 For information on the backfill variable, see
11002 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
11003 </para>
11004 </section>
11005
11006 <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'>
11007 <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title>
11008
11009 <para>
11010 Set these variables in your distribution configuration
11011 file as follows:
11012 <literallayout class='monospaced'>
11013 DISTRO_FEATURES_append = " systemd"
11014 VIRTUAL-RUNTIME_init_manager = "systemd"
11015 </literallayout>
11016 Doing so causes your main image to use the
11017 <filename>packagegroup-core-boot.bb</filename> recipe and
11018 systemd.
11019 The rescue/minimal image cannot use this package group.
11020 However, it can install SysVinit
11021 and the appropriate packages will have support for both
11022 systemd and SysVinit.
11023 </para>
11024 </section>
11025 </section>
11026
11027 <section id="selecting-dev-manager">
11028 <title>Selecting a Device Manager</title>
11029
11030 <para>
11031 The Yocto Project provides multiple ways to manage the device
11032 manager (<filename>/dev</filename>):
11033 <itemizedlist>
11034 <listitem><para><emphasis>Persistent and Pre-Populated<filename>/dev</filename>:</emphasis>
11035 For this case, the <filename>/dev</filename> directory
11036 is persistent and the required device nodes are created
11037 during the build.
11038 </para></listitem>
11039 <listitem><para><emphasis>Use <filename>devtmpfs</filename> with a Device Manager:</emphasis>
11040 For this case, the <filename>/dev</filename> directory
11041 is provided by the kernel as an in-memory file system and
11042 is automatically populated by the kernel at runtime.
11043 Additional configuration of device nodes is done in user
11044 space by a device manager like
11045 <filename>udev</filename> or
11046 <filename>busybox-mdev</filename>.
11047 </para></listitem>
11048 </itemizedlist>
11049 </para>
11050
11051 <section id="static-dev-management">
11052 <title>Using Persistent and Pre-Populated<filename>/dev</filename></title>
11053
11054 <para>
11055 To use the static method for device population, you need to
11056 set the
11057 <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink>
11058 variable to "0" as follows:
11059 <literallayout class='monospaced'>
11060 USE_DEVFS = "0"
11061 </literallayout>
11062 </para>
11063
11064 <para>
11065 The content of the resulting <filename>/dev</filename>
11066 directory is defined in a Device Table file.
11067 The
11068 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES'><filename>IMAGE_DEVICE_TABLES</filename></ulink>
11069 variable defines the Device Table to use and should be set
11070 in the machine or distro configuration file.
11071 Alternatively, you can set this variable in your
11072 <filename>local.conf</filename> configuration file.
11073 </para>
11074
11075 <para>
11076 If you do not define the
11077 <filename>IMAGE_DEVICE_TABLES</filename> variable, the default
11078 <filename>device_table-minimal.txt</filename> is used:
11079 <literallayout class='monospaced'>
11080 IMAGE_DEVICE_TABLES = "device_table-mymachine.txt"
11081 </literallayout>
11082 </para>
11083
11084 <para>
11085 The population is handled by the <filename>makedevs</filename>
11086 utility during image creation:
11087 </para>
11088 </section>
11089
11090 <section id="devtmpfs-dev-management">
11091 <title>Using <filename>devtmpfs</filename> and a Device Manager</title>
11092
11093 <para>
11094 To use the dynamic method for device population, you need to
11095 use (or be sure to set) the
11096 <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink>
11097 variable to "1", which is the default:
11098 <literallayout class='monospaced'>
11099 USE_DEVFS = "1"
11100 </literallayout>
11101 With this setting, the resulting <filename>/dev</filename>
11102 directory is populated by the kernel using
11103 <filename>devtmpfs</filename>.
11104 Make sure the corresponding kernel configuration variable
11105 <filename>CONFIG_DEVTMPFS</filename> is set when building
11106 you build a Linux kernel.
11107 </para>
11108
11109 <para>
11110 All devices created by <filename>devtmpfs</filename> will be
11111 owned by <filename>root</filename> and have permissions
11112 <filename>0600</filename>.
11113 </para>
11114
11115 <para>
11116 To have more control over the device nodes, you can use a
11117 device manager like <filename>udev</filename> or
11118 <filename>busybox-mdev</filename>.
11119 You choose the device manager by defining the
11120 <filename>VIRTUAL-RUNTIME_dev_manager</filename> variable
11121 in your machine or distro configuration file.
11122 Alternatively, you can set this variable in your
11123 <filename>local.conf</filename> configuration file:
11124 <literallayout class='monospaced'>
11125 VIRTUAL-RUNTIME_dev_manager = "udev"
11126
11127 # Some alternative values
11128 # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev"
11129 # VIRTUAL-RUNTIME_dev_manager = "systemd"
11130 </literallayout>
11131 </para>
11132 </section>
11133 </section>
11134
11135 <section id="platdev-appdev-srcrev">
11136 <title>Using an External SCM</title>
11137
11138 <para>
11139 If you're working on a recipe that pulls from an external Source
11140 Code Manager (SCM), it is possible to have the OpenEmbedded build
11141 system notice new recipe changes added to the SCM and then build
11142 the resulting packages that depend on the new recipes by using
11143 the latest versions.
11144 This only works for SCMs from which it is possible to get a
11145 sensible revision number for changes.
11146 Currently, you can do this with Apache Subversion (SVN), Git, and
11147 Bazaar (BZR) repositories.
11148 </para>
11149
11150 <para>
11151 To enable this behavior, the
11152 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
11153 of the recipe needs to reference
11154 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>.
11155 Here is an example:
11156 <literallayout class='monospaced'>
11157 PV = "1.2.3+git${SRCPV}"
11158 </literallayout>
11159 Then, you can add the following to your
11160 <filename>local.conf</filename>:
11161 <literallayout class='monospaced'>
11162 SRCREV_pn-<replaceable>PN</replaceable> = "${AUTOREV}"
11163 </literallayout>
11164 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>
11165 is the name of the recipe for which you want to enable automatic source
11166 revision updating.
11167 </para>
11168
11169 <para>
11170 If you do not want to update your local configuration file, you can
11171 add the following directly to the recipe to finish enabling
11172 the feature:
11173 <literallayout class='monospaced'>
11174 SRCREV = "${AUTOREV}"
11175 </literallayout>
11176 </para>
11177
11178 <para>
11179 The Yocto Project provides a distribution named
11180 <filename>poky-bleeding</filename>, whose configuration
11181 file contains the line:
11182 <literallayout class='monospaced'>
11183 require conf/distro/include/poky-floating-revisions.inc
11184 </literallayout>
11185 This line pulls in the listed include file that contains
11186 numerous lines of exactly that form:
11187 <literallayout class='monospaced'>
11188 #SRCREV_pn-opkg-native ?= "${AUTOREV}"
11189 #SRCREV_pn-opkg-sdk ?= "${AUTOREV}"
11190 #SRCREV_pn-opkg ?= "${AUTOREV}"
11191 #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}"
11192 #SRCREV_pn-opkg-utils ?= "${AUTOREV}"
11193 SRCREV_pn-gconf-dbus ?= "${AUTOREV}"
11194 SRCREV_pn-matchbox-common ?= "${AUTOREV}"
11195 SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}"
11196 SRCREV_pn-matchbox-desktop ?= "${AUTOREV}"
11197 SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}"
11198 SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}"
11199 SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}"
11200 SRCREV_pn-matchbox-terminal ?= "${AUTOREV}"
11201 SRCREV_pn-matchbox-wm ?= "${AUTOREV}"
11202 SRCREV_pn-settings-daemon ?= "${AUTOREV}"
11203 SRCREV_pn-screenshot ?= "${AUTOREV}"
11204 .
11205 .
11206 .
11207 </literallayout>
11208 These lines allow you to experiment with building a
11209 distribution that tracks the latest development source
11210 for numerous packages.
11211 <note><title>Caution</title>
11212 The <filename>poky-bleeding</filename> distribution
11213 is not tested on a regular basis.
11214 Keep this in mind if you use it.
11215 </note>
11216 </para>
11217 </section>
11218
11219 <section id='creating-a-read-only-root-filesystem'>
11220 <title>Creating a Read-Only Root Filesystem</title>
11221
11222 <para>
11223 Suppose, for security reasons, you need to disable
11224 your target device's root filesystem's write permissions
11225 (i.e. you need a read-only root filesystem).
11226 Or, perhaps you are running the device's operating system
11227 from a read-only storage device.
11228 For either case, you can customize your image for
11229 that behavior.
11230 </para>
11231
11232 <note>
11233 Supporting a read-only root filesystem requires that the system and
11234 applications do not try to write to the root filesystem.
11235 You must configure all parts of the target system to write
11236 elsewhere, or to gracefully fail in the event of attempting to
11237 write to the root filesystem.
11238 </note>
11239
11240 <section id='creating-the-root-filesystem'>
11241 <title>Creating the Root Filesystem</title>
11242
11243 <para>
11244 To create the read-only root filesystem, simply add the
11245 "read-only-rootfs" feature to your image, normally in one of two ways.
11246 The first way is to add the "read-only-rootfs" image feature
11247 in the image's recipe file via the
11248 <filename>IMAGE_FEATURES</filename> variable:
11249 <literallayout class='monospaced'>
11250 IMAGE_FEATURES += "read-only-rootfs"
11251 </literallayout>
11252 As an alternative, you can add the same feature from within your
11253 build directory's <filename>local.conf</filename> file with the
11254 associated <filename>EXTRA_IMAGE_FEATURES</filename> variable, as in:
11255 <literallayout class='monospaced'>
11256 EXTRA_IMAGE_FEATURES = "read-only-rootfs"
11257 </literallayout>
11258 </para>
11259
11260 <para>
11261 For more information on how to use these variables, see the
11262 "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>"
11263 section.
11264 For information on the variables, see
11265 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
11266 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>.
11267 </para>
11268 </section>
11269
11270 <section id='post-installation-scripts'>
11271 <title>Post-Installation Scripts</title>
11272
11273 <para>
11274 It is very important that you make sure all
11275 post-Installation (<filename>pkg_postinst</filename>) scripts
11276 for packages that are installed into the image can be run
11277 at the time when the root filesystem is created during the
11278 build on the host system.
11279 These scripts cannot attempt to run during first-boot on the
11280 target device.
11281 With the "read-only-rootfs" feature enabled,
11282 the build system checks during root filesystem creation to make
11283 sure all post-installation scripts succeed.
11284 If any of these scripts still need to be run after the root
11285 filesystem is created, the build immediately fails.
11286 These build-time checks ensure that the build fails
11287 rather than the target device fails later during its
11288 initial boot operation.
11289 </para>
11290
11291 <para>
11292 Most of the common post-installation scripts generated by the
11293 build system for the out-of-the-box Yocto Project are engineered
11294 so that they can run during root filesystem creation
11295 (e.g. post-installation scripts for caching fonts).
11296 However, if you create and add custom scripts, you need
11297 to be sure they can be run during this file system creation.
11298 </para>
11299
11300 <para>
11301 Here are some common problems that prevent
11302 post-installation scripts from running during root filesystem
11303 creation:
11304 <itemizedlist>
11305 <listitem><para>
11306 <emphasis>Not using $D in front of absolute
11307 paths:</emphasis>
11308 The build system defines
11309 <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
11310 when the root filesystem is created.
11311 Furthermore, <filename>$D</filename> is blank when the
11312 script is run on the target device.
11313 This implies two purposes for <filename>$D</filename>:
11314 ensuring paths are valid in both the host and target
11315 environments, and checking to determine which
11316 environment is being used as a method for taking
11317 appropriate actions.
11318 </para></listitem>
11319 <listitem><para>
11320 <emphasis>Attempting to run processes that are
11321 specific to or dependent on the target
11322 architecture:</emphasis>
11323 You can work around these attempts by using native
11324 tools, which run on the host system,
11325 to accomplish the same tasks, or
11326 by alternatively running the processes under QEMU,
11327 which has the <filename>qemu_run_binary</filename>
11328 function.
11329 For more information, see the
11330 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-qemu'><filename>qemu</filename></ulink>
11331 class.</para></listitem>
11332 </itemizedlist>
11333 </para>
11334 </section>
11335
11336 <section id='areas-with-write-access'>
11337 <title>Areas With Write Access</title>
11338
11339 <para>
11340 With the "read-only-rootfs" feature enabled,
11341 any attempt by the target to write to the root filesystem at
11342 runtime fails.
11343 Consequently, you must make sure that you configure processes
11344 and applications that attempt these types of writes do so
11345 to directories with write access (e.g.
11346 <filename>/tmp</filename> or <filename>/var/run</filename>).
11347 </para>
11348 </section>
11349 </section>
11350
11351
11352
11353
11354 <section id='maintaining-build-output-quality'>
11355 <title>Maintaining Build Output Quality</title>
11356
11357 <para>
11358 Many factors can influence the quality of a build.
11359 For example, if you upgrade a recipe to use a new version of an
11360 upstream software package or you experiment with some new
11361 configuration options, subtle changes can occur that you might
11362 not detect until later.
11363 Consider the case where your recipe is using a newer version of
11364 an upstream package.
11365 In this case, a new version of a piece of software might
11366 introduce an optional dependency on another library, which is
11367 auto-detected.
11368 If that library has already been built when the software is
11369 building, the software will link to the built library and that
11370 library will be pulled into your image along with the new
11371 software even if you did not want the library.
11372 </para>
11373
11374 <para>
11375 The
11376 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-buildhistory'><filename>buildhistory</filename></ulink>
11377 class exists to help you maintain the quality of your build
11378 output.
11379 You can use the class to highlight unexpected and possibly
11380 unwanted changes in the build output.
11381 When you enable build history, it records information about the
11382 contents of each package and image and then commits that
11383 information to a local Git repository where you can examine
11384 the information.
11385 </para>
11386
11387 <para>
11388 The remainder of this section describes the following:
11389 <itemizedlist>
11390 <listitem><para>
11391 How you can enable and disable build history
11392 </para></listitem>
11393 <listitem><para>
11394 How to understand what the build history contains
11395 </para></listitem>
11396 <listitem><para>
11397 How to limit the information used for build history
11398 </para></listitem>
11399 <listitem><para>
11400 How to examine the build history from both a
11401 command-line and web interface
11402 </para></listitem>
11403 </itemizedlist>
11404 </para>
11405
11406 <section id='enabling-and-disabling-build-history'>
11407 <title>Enabling and Disabling Build History</title>
11408
11409 <para>
11410 Build history is disabled by default.
11411 To enable it, add the following <filename>INHERIT</filename>
11412 statement and set the
11413 <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></ulink>
11414 variable to "1" at the end of your
11415 <filename>conf/local.conf</filename> file found in the
11416 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
11417 <literallayout class='monospaced'>
11418 INHERIT += "buildhistory"
11419 BUILDHISTORY_COMMIT = "1"
11420 </literallayout>
11421 Enabling build history as previously described causes the
11422 OpenEmbedded build system to collect build output information
11423 and commit it as a single commit to a local
11424 <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>
11425 repository.
11426 <note>
11427 Enabling build history increases your build times slightly,
11428 particularly for images, and increases the amount of disk
11429 space used during the build.
11430 </note>
11431 </para>
11432
11433 <para>
11434 You can disable build history by removing the previous
11435 statements from your <filename>conf/local.conf</filename>
11436 file.
11437 </para>
11438 </section>
11439
11440 <section id='understanding-what-the-build-history-contains'>
11441 <title>Understanding What the Build History Contains</title>
11442
11443 <para>
11444 Build history information is kept in
11445 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink><filename>}/buildhistory</filename>
11446 in the Build Directory as defined by the
11447 <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></ulink>
11448 variable.
11449 The following is an example abbreviated listing:
11450 <imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" />
11451 </para>
11452
11453 <para>
11454 At the top level, a <filename>metadata-revs</filename>
11455 file exists that lists the revisions of the repositories for
11456 the enabled layers when the build was produced.
11457 The rest of the data splits into separate
11458 <filename>packages</filename>, <filename>images</filename>
11459 and <filename>sdk</filename> directories, the contents of
11460 which are described as follows.
11461 </para>
11462
11463 <section id='build-history-package-information'>
11464 <title>Build History Package Information</title>
11465
11466 <para>
11467 The history for each package contains a text file that has
11468 name-value pairs with information about the package.
11469 For example,
11470 <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename>
11471 contains the following:
11472 <literallayout class='monospaced'>
11473 PV = 1.22.1
11474 PR = r32
11475 RPROVIDES =
11476 RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
11477 RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
11478 PKGSIZE = 540168
11479 FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
11480 /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
11481 /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
11482 /usr/share/pixmaps /usr/share/applications /usr/share/idl \
11483 /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
11484 FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
11485 /etc/busybox.links.nosuid /etc/busybox.links.suid
11486 </literallayout>
11487 Most of these name-value pairs correspond to variables
11488 used to produce the package.
11489 The exceptions are <filename>FILELIST</filename>, which
11490 is the actual list of files in the package, and
11491 <filename>PKGSIZE</filename>, which is the total size of
11492 files in the package in bytes.
11493 </para>
11494
11495 <para>
11496 A file also exists that corresponds to the recipe from
11497 which the package came (e.g.
11498 <filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>):
11499 <literallayout class='monospaced'>
11500 PV = 1.22.1
11501 PR = r32
11502 DEPENDS = initscripts kern-tools-native update-rc.d-native \
11503 virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
11504 virtual/libc virtual/update-alternatives
11505 PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
11506 busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
11507 busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
11508 </literallayout>
11509 </para>
11510
11511 <para>
11512 Finally, for those recipes fetched from a version control
11513 system (e.g., Git), a file exists that lists source
11514 revisions that are specified in the recipe and lists
11515 the actual revisions used during the build.
11516 Listed and actual revisions might differ when
11517 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
11518 is set to
11519 ${<ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink>}.
11520 Here is an example assuming
11521 <filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>):
11522 <literallayout class='monospaced'>
11523 # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
11524 SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
11525 # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
11526 SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
11527 </literallayout>
11528 You can use the
11529 <filename>buildhistory-collect-srcrevs</filename>
11530 command with the <filename>-a</filename> option to
11531 collect the stored <filename>SRCREV</filename> values
11532 from build history and report them in a format suitable for
11533 use in global configuration (e.g.,
11534 <filename>local.conf</filename> or a distro include file)
11535 to override floating <filename>AUTOREV</filename> values
11536 to a fixed set of revisions.
11537 Here is some example output from this command:
11538 <literallayout class='monospaced'>
11539 $ buildhistory-collect-srcrevs -a
11540 # i586-poky-linux
11541 SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072"
11542 SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072"
11543 SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a"
11544 SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
11545 # x86_64-linux
11546 SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa"
11547 SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf"
11548 SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11"
11549 SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072"
11550 SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3"
11551 SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca"
11552 SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a"
11553 SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff"
11554 SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
11555 # qemux86-poky-linux
11556 SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
11557 SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f"
11558 # all-poky-linux
11559 SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
11560 </literallayout>
11561 <note>
11562 Here are some notes on using the
11563 <filename>buildhistory-collect-srcrevs</filename>
11564 command:
11565 <itemizedlist>
11566 <listitem><para>
11567 By default, only values where the
11568 <filename>SRCREV</filename> was not hardcoded
11569 (usually when <filename>AUTOREV</filename>
11570 is used) are reported.
11571 Use the <filename>-a</filename> option to
11572 see all <filename>SRCREV</filename> values.
11573 </para></listitem>
11574 <listitem><para>
11575 The output statements might not have any effect
11576 if overrides are applied elsewhere in the
11577 build system configuration.
11578 Use the <filename>-f</filename> option to add
11579 the <filename>forcevariable</filename> override
11580 to each output line if you need to work around
11581 this restriction.
11582 </para></listitem>
11583 <listitem><para>
11584 The script does apply special handling when
11585 building for multiple machines.
11586 However, the script does place a comment before
11587 each set of values that specifies which
11588 triplet to which they belong as previously
11589 shown (e.g.,
11590 <filename>i586-poky-linux</filename>).
11591 </para></listitem>
11592 </itemizedlist>
11593 </note>
11594 </para>
11595 </section>
11596
11597 <section id='build-history-image-information'>
11598 <title>Build History Image Information</title>
11599
11600 <para>
11601 The files produced for each image are as follows:
11602 <itemizedlist>
11603 <listitem><para>
11604 <filename>image-files:</filename>
11605 A directory containing selected files from the root
11606 filesystem.
11607 The files are defined by
11608 <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></ulink>.
11609 </para></listitem>
11610 <listitem><para>
11611 <filename>build-id.txt:</filename>
11612 Human-readable information about the build
11613 configuration and metadata source revisions.
11614 This file contains the full build header as printed
11615 by BitBake.
11616 </para></listitem>
11617 <listitem><para>
11618 <filename>*.dot:</filename>
11619 Dependency graphs for the image that are
11620 compatible with <filename>graphviz</filename>.
11621 </para></listitem>
11622 <listitem><para>
11623 <filename>files-in-image.txt:</filename>
11624 A list of files in the image with permissions,
11625 owner, group, size, and symlink information.
11626 </para></listitem>
11627 <listitem><para>
11628 <filename>image-info.txt:</filename>
11629 A text file containing name-value pairs with
11630 information about the image.
11631 See the following listing example for more
11632 information.
11633 </para></listitem>
11634 <listitem><para>
11635 <filename>installed-package-names.txt:</filename>
11636 A list of installed packages by name only.
11637 </para></listitem>
11638 <listitem><para>
11639 <filename>installed-package-sizes.txt:</filename>
11640 A list of installed packages ordered by size.
11641 </para></listitem>
11642 <listitem><para>
11643 <filename>installed-packages.txt:</filename>
11644 A list of installed packages with full package
11645 filenames.
11646 </para></listitem>
11647 </itemizedlist>
11648 <note>
11649 Installed package information is able to be gathered
11650 and produced even if package management is disabled
11651 for the final image.
11652 </note>
11653 </para>
11654
11655 <para>
11656 Here is an example of <filename>image-info.txt</filename>:
11657 <literallayout class='monospaced'>
11658 DISTRO = poky
11659 DISTRO_VERSION = 1.7
11660 USER_CLASSES = buildstats image-mklibs image-prelink
11661 IMAGE_CLASSES = image_types
11662 IMAGE_FEATURES = debug-tweaks
11663 IMAGE_LINGUAS =
11664 IMAGE_INSTALL = packagegroup-core-boot run-postinsts
11665 BAD_RECOMMENDATIONS =
11666 NO_RECOMMENDATIONS =
11667 PACKAGE_EXCLUDE =
11668 ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \
11669 write_image_manifest ; buildhistory_list_installed_image ; \
11670 buildhistory_get_image_installed ; ssh_allow_empty_password; \
11671 postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ;
11672 IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
11673 IMAGESIZE = 6900
11674 </literallayout>
11675 Other than <filename>IMAGESIZE</filename>, which is the
11676 total size of the files in the image in Kbytes, the
11677 name-value pairs are variables that may have influenced the
11678 content of the image.
11679 This information is often useful when you are trying to
11680 determine why a change in the package or file
11681 listings has occurred.
11682 </para>
11683 </section>
11684
11685 <section id='using-build-history-to-gather-image-information-only'>
11686 <title>Using Build History to Gather Image Information Only</title>
11687
11688 <para>
11689 As you can see, build history produces image information,
11690 including dependency graphs, so you can see why something
11691 was pulled into the image.
11692 If you are just interested in this information and not
11693 interested in collecting specific package or SDK
11694 information, you can enable writing only image information
11695 without any history by adding the following to your
11696 <filename>conf/local.conf</filename> file found in the
11697 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
11698 <literallayout class='monospaced'>
11699 INHERIT += "buildhistory"
11700 BUILDHISTORY_COMMIT = "0"
11701 BUILDHISTORY_FEATURES = "image"
11702 </literallayout>
11703 Here, you set the
11704 <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></ulink>
11705 variable to use the image feature only.
11706 </para>
11707 </section>
11708
11709 <section id='build-history-sdk-information'>
11710 <title>Build History SDK Information</title>
11711
11712 <para>
11713 Build history collects similar information on the contents
11714 of SDKs
11715 (e.g. <filename>bitbake -c populate_sdk imagename</filename>)
11716 as compared to information it collects for images.
11717 Furthermore, this information differs depending on whether
11718 an extensible or standard SDK is being produced.
11719 </para>
11720
11721 <para>
11722 The following list shows the files produced for SDKs:
11723 <itemizedlist>
11724 <listitem><para>
11725 <filename>files-in-sdk.txt:</filename>
11726 A list of files in the SDK with permissions,
11727 owner, group, size, and symlink information.
11728 This list includes both the host and target parts
11729 of the SDK.
11730 </para></listitem>
11731 <listitem><para>
11732 <filename>sdk-info.txt:</filename>
11733 A text file containing name-value pairs with
11734 information about the SDK.
11735 See the following listing example for more
11736 information.
11737 </para></listitem>
11738 <listitem><para>
11739 <filename>sstate-task-sizes.txt:</filename>
11740 A text file containing name-value pairs with
11741 information about task group sizes
11742 (e.g. <filename>do_populate_sysroot</filename>
11743 tasks have a total size).
11744 The <filename>sstate-task-sizes.txt</filename> file
11745 exists only when an extensible SDK is created.
11746 </para></listitem>
11747 <listitem><para>
11748 <filename>sstate-package-sizes.txt:</filename>
11749 A text file containing name-value pairs with
11750 information for the shared-state packages and
11751 sizes in the SDK.
11752 The <filename>sstate-package-sizes.txt</filename>
11753 file exists only when an extensible SDK is created.
11754 </para></listitem>
11755 <listitem><para>
11756 <filename>sdk-files:</filename>
11757 A folder that contains copies of the files
11758 mentioned in
11759 <filename>BUILDHISTORY_SDK_FILES</filename> if the
11760 files are present in the output.
11761 Additionally, the default value of
11762 <filename>BUILDHISTORY_SDK_FILES</filename> is
11763 specific to the extensible SDK although you can
11764 set it differently if you would like to pull in
11765 specific files from the standard SDK.</para>
11766
11767 <para>The default files are
11768 <filename>conf/local.conf</filename>,
11769 <filename>conf/bblayers.conf</filename>,
11770 <filename>conf/auto.conf</filename>,
11771 <filename>conf/locked-sigs.inc</filename>, and
11772 <filename>conf/devtool.conf</filename>.
11773 Thus, for an extensible SDK, these files get
11774 copied into the <filename>sdk-files</filename>
11775 directory.
11776 </para></listitem>
11777 <listitem><para>
11778 The following information appears under
11779 each of the <filename>host</filename>
11780 and <filename>target</filename> directories
11781 for the portions of the SDK that run on the host
11782 and on the target, respectively:
11783 <note>
11784 The following files for the most part are empty
11785 when producing an extensible SDK because this
11786 type of SDK is not constructed from packages
11787 as is the standard SDK.
11788 </note>
11789 <itemizedlist>
11790 <listitem><para>
11791 <filename>depends.dot:</filename>
11792 Dependency graph for the SDK that is
11793 compatible with
11794 <filename>graphviz</filename>.
11795 </para></listitem>
11796 <listitem><para>
11797 <filename>installed-package-names.txt:</filename>
11798 A list of installed packages by name only.
11799 </para></listitem>
11800 <listitem><para>
11801 <filename>installed-package-sizes.txt:</filename>
11802 A list of installed packages ordered by size.
11803 </para></listitem>
11804 <listitem><para>
11805 <filename>installed-packages.txt:</filename>
11806 A list of installed packages with full
11807 package filenames.
11808 </para></listitem>
11809 </itemizedlist>
11810 </para></listitem>
11811 </itemizedlist>
11812 </para>
11813
11814 <para>
11815 Here is an example of <filename>sdk-info.txt</filename>:
11816 <literallayout class='monospaced'>
11817 DISTRO = poky
11818 DISTRO_VERSION = 1.3+snapshot-20130327
11819 SDK_NAME = poky-glibc-i686-arm
11820 SDK_VERSION = 1.3+snapshot
11821 SDKMACHINE =
11822 SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
11823 BAD_RECOMMENDATIONS =
11824 SDKSIZE = 352712
11825 </literallayout>
11826 Other than <filename>SDKSIZE</filename>, which is the
11827 total size of the files in the SDK in Kbytes, the
11828 name-value pairs are variables that might have influenced
11829 the content of the SDK.
11830 This information is often useful when you are trying to
11831 determine why a change in the package or file listings
11832 has occurred.
11833 </para>
11834 </section>
11835
11836 <section id='examining-build-history-information'>
11837 <title>Examining Build History Information</title>
11838
11839 <para>
11840 You can examine build history output from the command
11841 line or from a web interface.
11842 </para>
11843
11844 <para>
11845 To see any changes that have occurred (assuming you have
11846 <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></ulink><filename>&nbsp;= "1"</filename>),
11847 you can simply use any Git command that allows you to
11848 view the history of a repository.
11849 Here is one method:
11850 <literallayout class='monospaced'>
11851 $ git log -p
11852 </literallayout>
11853 You need to realize, however, that this method does show
11854 changes that are not significant (e.g. a package's size
11855 changing by a few bytes).
11856 </para>
11857
11858 <para>
11859 A command-line tool called
11860 <filename>buildhistory-diff</filename> does exist, though,
11861 that queries the Git repository and prints just the
11862 differences that might be significant in human-readable
11863 form.
11864 Here is an example:
11865 <literallayout class='monospaced'>
11866 $ ~/poky/poky/scripts/buildhistory-diff . HEAD^
11867 Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
11868 /etc/anotherpkg.conf was added
11869 /sbin/anotherpkg was added
11870 * (installed-package-names.txt):
11871 * anotherpkg was added
11872 Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
11873 anotherpkg was added
11874 packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
11875 * PR changed from "r0" to "r1"
11876 * PV changed from "0.1.10" to "0.1.12"
11877 packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
11878 * PR changed from "r0" to "r1"
11879 * PV changed from "0.1.10" to "0.1.12"
11880 </literallayout>
11881 <note>
11882 The <filename>buildhistory-diff</filename> tool
11883 requires the <filename>GitPython</filename> package.
11884 Be sure to install it using Pip3 as follows:
11885 <literallayout class='monospaced'>
11886 $ pip3 install GitPython --user
11887 </literallayout>
11888 Alternatively, you can install
11889 <filename>python3-git</filename> using the appropriate
11890 distribution package manager (e.g.
11891 <filename>apt-get</filename>, <filename>dnf</filename>,
11892 or <filename>zipper</filename>).
11893 </note>
11894 </para>
11895
11896 <para>
11897 To see changes to the build history using a web interface,
11898 follow the instruction in the <filename>README</filename>
11899 file here.
11900 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>.
11901 </para>
11902
11903 <para>
11904 Here is a sample screenshot of the interface:
11905 <imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" />
11906 </para>
11907 </section>
11908 </section>
11909 </section>
11910
11911 <section id="performing-automated-runtime-testing">
11912 <title>Performing Automated Runtime Testing</title>
11913
11914 <para>
11915 The OpenEmbedded build system makes available a series of automated
11916 tests for images to verify runtime functionality.
11917 You can run these tests on either QEMU or actual target hardware.
11918 Tests are written in Python making use of the
11919 <filename>unittest</filename> module, and the majority of them
11920 run commands on the target system over SSH.
11921 This section describes how you set up the environment to use these
11922 tests, run available tests, and write and add your own tests.
11923 </para>
11924
11925 <para>
11926 For information on the test and QA infrastructure available
11927 within the Yocto Project, see the
11928 "<ulink url='&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance'>Testing and Quality Assurance</ulink>"
11929 section in the Yocto Project Reference Manual.
11930 </para>
11931
11932 <section id='enabling-tests'>
11933 <title>Enabling Tests</title>
11934
11935 <para>
11936 Depending on whether you are planning to run tests using
11937 QEMU or on the hardware, you have to take
11938 different steps to enable the tests.
11939 See the following subsections for information on how to
11940 enable both types of tests.
11941 </para>
11942
11943 <section id='qemu-image-enabling-tests'>
11944 <title>Enabling Runtime Tests on QEMU</title>
11945
11946 <para>
11947 In order to run tests, you need to do the following:
11948 <itemizedlist>
11949 <listitem><para><emphasis>Set up to avoid interaction
11950 with <filename>sudo</filename> for networking:</emphasis>
11951 To accomplish this, you must do one of the
11952 following:
11953 <itemizedlist>
11954 <listitem><para>Add
11955 <filename>NOPASSWD</filename> for your user
11956 in <filename>/etc/sudoers</filename> either for
11957 all commands or just for
11958 <filename>runqemu-ifup</filename>.
11959 You must provide the full path as that can
11960 change if you are using multiple clones of the
11961 source repository.
11962 <note>
11963 On some distributions, you also need to
11964 comment out "Defaults requiretty" in
11965 <filename>/etc/sudoers</filename>.
11966 </note></para></listitem>
11967 <listitem><para>Manually configure a tap interface
11968 for your system.</para></listitem>
11969 <listitem><para>Run as root the script in
11970 <filename>scripts/runqemu-gen-tapdevs</filename>,
11971 which should generate a list of tap devices.
11972 This is the option typically chosen for
11973 Autobuilder-type environments.
11974 <note><title>Notes</title>
11975 <itemizedlist>
11976 <listitem><para>
11977 Be sure to use an absolute path
11978 when calling this script
11979 with sudo.
11980 </para></listitem>
11981 <listitem><para>
11982 The package recipe
11983 <filename>qemu-helper-native</filename>
11984 is required to run this script.
11985 Build the package using the
11986 following command:
11987 <literallayout class='monospaced'>
11988 $ bitbake qemu-helper-native
11989 </literallayout>
11990 </para></listitem>
11991 </itemizedlist>
11992 </note>
11993 </para></listitem>
11994 </itemizedlist></para></listitem>
11995 <listitem><para><emphasis>Set the
11996 <filename>DISPLAY</filename> variable:</emphasis>
11997 You need to set this variable so that you have an X
11998 server available (e.g. start
11999 <filename>vncserver</filename> for a headless machine).
12000 </para></listitem>
12001 <listitem><para><emphasis>Be sure your host's firewall
12002 accepts incoming connections from
12003 192.168.7.0/24:</emphasis>
12004 Some of the tests (in particular DNF tests) start
12005 an HTTP server on a random high number port,
12006 which is used to serve files to the target.
12007 The DNF module serves
12008 <filename>${WORKDIR}/oe-rootfs-repo</filename>
12009 so it can run DNF channel commands.
12010 That means your host's firewall
12011 must accept incoming connections from 192.168.7.0/24,
12012 which is the default IP range used for tap devices
12013 by <filename>runqemu</filename>.</para></listitem>
12014 <listitem><para><emphasis>Be sure your host has the
12015 correct packages installed:</emphasis>
12016 Depending your host's distribution, you need
12017 to have the following packages installed:
12018 <itemizedlist>
12019 <listitem><para>Ubuntu and Debian:
12020 <filename>sysstat</filename> and
12021 <filename>iproute2</filename>
12022 </para></listitem>
12023 <listitem><para>OpenSUSE:
12024 <filename>sysstat</filename> and
12025 <filename>iproute2</filename>
12026 </para></listitem>
12027 <listitem><para>Fedora:
12028 <filename>sysstat</filename> and
12029 <filename>iproute</filename>
12030 </para></listitem>
12031 <listitem><para>CentOS:
12032 <filename>sysstat</filename> and
12033 <filename>iproute</filename>
12034 </para></listitem>
12035 </itemizedlist>
12036 </para></listitem>
12037 </itemizedlist>
12038 </para>
12039
12040 <para>
12041 Once you start running the tests, the following happens:
12042 <orderedlist>
12043 <listitem><para>A copy of the root filesystem is written
12044 to <filename>${WORKDIR}/testimage</filename>.
12045 </para></listitem>
12046 <listitem><para>The image is booted under QEMU using the
12047 standard <filename>runqemu</filename> script.
12048 </para></listitem>
12049 <listitem><para>A default timeout of 500 seconds occurs
12050 to allow for the boot process to reach the login prompt.
12051 You can change the timeout period by setting
12052 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink>
12053 in the <filename>local.conf</filename> file.
12054 </para></listitem>
12055 <listitem><para>Once the boot process is reached and the
12056 login prompt appears, the tests run.
12057 The full boot log is written to
12058 <filename>${WORKDIR}/testimage/qemu_boot_log</filename>.
12059 </para></listitem>
12060 <listitem><para>Each test module loads in the order found
12061 in <filename>TEST_SUITES</filename>.
12062 You can find the full output of the commands run over
12063 SSH in
12064 <filename>${WORKDIR}/testimgage/ssh_target_log</filename>.
12065 </para></listitem>
12066 <listitem><para>If no failures occur, the task running the
12067 tests ends successfully.
12068 You can find the output from the
12069 <filename>unittest</filename> in the task log at
12070 <filename>${WORKDIR}/temp/log.do_testimage</filename>.
12071 </para></listitem>
12072 </orderedlist>
12073 </para>
12074 </section>
12075
12076 <section id='hardware-image-enabling-tests'>
12077 <title>Enabling Runtime Tests on Hardware</title>
12078
12079 <para>
12080 The OpenEmbedded build system can run tests on real
12081 hardware, and for certain devices it can also deploy
12082 the image to be tested onto the device beforehand.
12083 </para>
12084
12085 <para>
12086 For automated deployment, a "master image" is installed
12087 onto the hardware once as part of setup.
12088 Then, each time tests are to be run, the following
12089 occurs:
12090 <orderedlist>
12091 <listitem><para>The master image is booted into and
12092 used to write the image to be tested to
12093 a second partition.
12094 </para></listitem>
12095 <listitem><para>The device is then rebooted using an
12096 external script that you need to provide.
12097 </para></listitem>
12098 <listitem><para>The device boots into the image to be
12099 tested.
12100 </para></listitem>
12101 </orderedlist>
12102 </para>
12103
12104 <para>
12105 When running tests (independent of whether the image
12106 has been deployed automatically or not), the device is
12107 expected to be connected to a network on a
12108 pre-determined IP address.
12109 You can either use static IP addresses written into
12110 the image, or set the image to use DHCP and have your
12111 DHCP server on the test network assign a known IP address
12112 based on the MAC address of the device.
12113 </para>
12114
12115 <para>
12116 In order to run tests on hardware, you need to set
12117 <filename>TEST_TARGET</filename> to an appropriate value.
12118 For QEMU, you do not have to change anything, the default
12119 value is "qemu".
12120 For running tests on hardware, the following options exist:
12121 <itemizedlist>
12122 <listitem><para><emphasis>"simpleremote":</emphasis>
12123 Choose "simpleremote" if you are going to
12124 run tests on a target system that is already
12125 running the image to be tested and is available
12126 on the network.
12127 You can use "simpleremote" in conjunction
12128 with either real hardware or an image running
12129 within a separately started QEMU or any
12130 other virtual machine manager.
12131 </para></listitem>
12132 <listitem><para><emphasis>"SystemdbootTarget":</emphasis>
12133 Choose "SystemdbootTarget" if your hardware is
12134 an EFI-based machine with
12135 <filename>systemd-boot</filename> as bootloader and
12136 <filename>core-image-testmaster</filename>
12137 (or something similar) is installed.
12138 Also, your hardware under test must be in a
12139 DHCP-enabled network that gives it the same IP
12140 address for each reboot.</para>
12141 <para>If you choose "SystemdbootTarget", there are
12142 additional requirements and considerations.
12143 See the
12144 "<link linkend='selecting-systemdboottarget'>Selecting SystemdbootTarget</link>"
12145 section, which follows, for more information.
12146 </para></listitem>
12147 <listitem><para><emphasis>"BeagleBoneTarget":</emphasis>
12148 Choose "BeagleBoneTarget" if you are deploying
12149 images and running tests on the BeagleBone
12150 "Black" or original "White" hardware.
12151 For information on how to use these tests, see the
12152 comments at the top of the BeagleBoneTarget
12153 <filename>meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py</filename>
12154 file.
12155 </para></listitem>
12156 <listitem><para><emphasis>"EdgeRouterTarget":</emphasis>
12157 Choose "EdgeRouterTarget" is you are deploying
12158 images and running tests on the Ubiquiti Networks
12159 EdgeRouter Lite.
12160 For information on how to use these tests, see the
12161 comments at the top of the EdgeRouterTarget
12162 <filename>meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py</filename>
12163 file.
12164 </para></listitem>
12165 <listitem><para><emphasis>"GrubTarget":</emphasis>
12166 Choose the "supports deploying images and running
12167 tests on any generic PC that boots using GRUB.
12168 For information on how to use these tests, see the
12169 comments at the top of the GrubTarget
12170 <filename>meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py</filename>
12171 file.
12172 </para></listitem>
12173 <listitem><para><emphasis>"<replaceable>your-target</replaceable>":</emphasis>
12174 Create your own custom target if you want to run
12175 tests when you are deploying images and running
12176 tests on a custom machine within your BSP layer.
12177 To do this, you need to add a Python unit that
12178 defines the target class under
12179 <filename>lib/oeqa/controllers/</filename> within
12180 your layer.
12181 You must also provide an empty
12182 <filename>__init__.py</filename>.
12183 For examples, see files in
12184 <filename>meta-yocto-bsp/lib/oeqa/controllers/</filename>.
12185 </para></listitem>
12186 </itemizedlist>
12187 </para>
12188 </section>
12189
12190 <section id='selecting-systemdboottarget'>
12191 <title>Selecting SystemdbootTarget</title>
12192
12193 <para>
12194 If you did not set <filename>TEST_TARGET</filename> to
12195 "SystemdbootTarget", then you do not need any information
12196 in this section.
12197 You can skip down to the
12198 "<link linkend='qemu-image-running-tests'>Running Tests</link>"
12199 section.
12200 </para>
12201
12202 <para>
12203 If you did set <filename>TEST_TARGET</filename> to
12204 "SystemdbootTarget", you also need to perform a one-time
12205 setup of your master image by doing the following:
12206 <orderedlist>
12207 <listitem><para><emphasis>Set <filename>EFI_PROVIDER</filename>:</emphasis>
12208 Be sure that <filename>EFI_PROVIDER</filename>
12209 is as follows:
12210 <literallayout class='monospaced'>
12211 EFI_PROVIDER = "systemd-boot"
12212 </literallayout>
12213 </para></listitem>
12214 <listitem><para><emphasis>Build the master image:</emphasis>
12215 Build the <filename>core-image-testmaster</filename>
12216 image.
12217 The <filename>core-image-testmaster</filename>
12218 recipe is provided as an example for a
12219 "master" image and you can customize the image
12220 recipe as you would any other recipe.
12221 </para>
12222 <para>Here are the image recipe requirements:
12223 <itemizedlist>
12224 <listitem><para>Inherits
12225 <filename>core-image</filename>
12226 so that kernel modules are installed.
12227 </para></listitem>
12228 <listitem><para>Installs normal linux utilities
12229 not busybox ones (e.g.
12230 <filename>bash</filename>,
12231 <filename>coreutils</filename>,
12232 <filename>tar</filename>,
12233 <filename>gzip</filename>, and
12234 <filename>kmod</filename>).
12235 </para></listitem>
12236 <listitem><para>Uses a custom
12237 Initial RAM Disk (initramfs) image with a
12238 custom installer.
12239 A normal image that you can install usually
12240 creates a single rootfs partition.
12241 This image uses another installer that
12242 creates a specific partition layout.
12243 Not all Board Support Packages (BSPs)
12244 can use an installer.
12245 For such cases, you need to manually create
12246 the following partition layout on the
12247 target:
12248 <itemizedlist>
12249 <listitem><para>First partition mounted
12250 under <filename>/boot</filename>,
12251 labeled "boot".
12252 </para></listitem>
12253 <listitem><para>The main rootfs
12254 partition where this image gets
12255 installed, which is mounted under
12256 <filename>/</filename>.
12257 </para></listitem>
12258 <listitem><para>Another partition
12259 labeled "testrootfs" where test
12260 images get deployed.
12261 </para></listitem>
12262 </itemizedlist>
12263 </para></listitem>
12264 </itemizedlist>
12265 </para></listitem>
12266 <listitem><para><emphasis>Install image:</emphasis>
12267 Install the image that you just built on the target
12268 system.
12269 </para></listitem>
12270 </orderedlist>
12271 </para>
12272
12273 <para>
12274 The final thing you need to do when setting
12275 <filename>TEST_TARGET</filename> to "SystemdbootTarget" is
12276 to set up the test image:
12277 <orderedlist>
12278 <listitem><para><emphasis>Set up your <filename>local.conf</filename> file:</emphasis>
12279 Make sure you have the following statements in
12280 your <filename>local.conf</filename> file:
12281 <literallayout class='monospaced'>
12282 IMAGE_FSTYPES += "tar.gz"
12283 INHERIT += "testimage"
12284 TEST_TARGET = "SystemdbootTarget"
12285 TEST_TARGET_IP = "192.168.2.3"
12286 </literallayout>
12287 </para></listitem>
12288 <listitem><para><emphasis>Build your test image:</emphasis>
12289 Use BitBake to build the image:
12290 <literallayout class='monospaced'>
12291 $ bitbake core-image-sato
12292 </literallayout>
12293 </para></listitem>
12294 </orderedlist>
12295 </para>
12296 </section>
12297
12298 <section id='power-control'>
12299 <title>Power Control</title>
12300
12301 <para>
12302 For most hardware targets other than "simpleremote",
12303 you can control power:
12304 <itemizedlist>
12305 <listitem><para>
12306 You can use
12307 <filename>TEST_POWERCONTROL_CMD</filename>
12308 together with
12309 <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename>
12310 as a command that runs on the host and does power
12311 cycling.
12312 The test code passes one argument to that command:
12313 off, on or cycle (off then on).
12314 Here is an example that could appear in your
12315 <filename>local.conf</filename> file:
12316 <literallayout class='monospaced'>
12317 TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1"
12318 </literallayout>
12319 In this example, the expect script does the
12320 following:
12321 <literallayout class='monospaced'>
12322 ssh test@10.11.12.1 "pyctl nuc1 <replaceable>arg</replaceable>"
12323 </literallayout>
12324 It then runs a Python script that controls power
12325 for a label called <filename>nuc1</filename>.
12326 <note>
12327 You need to customize
12328 <filename>TEST_POWERCONTROL_CMD</filename>
12329 and
12330 <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename>
12331 for your own setup.
12332 The one requirement is that it accepts
12333 "on", "off", and "cycle" as the last argument.
12334 </note>
12335 </para></listitem>
12336 <listitem><para>
12337 When no command is defined, it connects to the
12338 device over SSH and uses the classic reboot command
12339 to reboot the device.
12340 Classic reboot is fine as long as the machine
12341 actually reboots (i.e. the SSH test has not
12342 failed).
12343 It is useful for scenarios where you have a simple
12344 setup, typically with a single board, and where
12345 some manual interaction is okay from time to time.
12346 </para></listitem>
12347 </itemizedlist>
12348 If you have no hardware to automatically perform power
12349 control but still wish to experiment with automated
12350 hardware testing, you can use the dialog-power-control
12351 script that shows a dialog prompting you to perform the
12352 required power action.
12353 This script requires either KDialog or Zenity to be
12354 installed.
12355 To use this script, set the
12356 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD'><filename>TEST_POWERCONTROL_CMD</filename></ulink>
12357 variable as follows:
12358 <literallayout class='monospaced'>
12359 TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control"
12360 </literallayout>
12361 </para>
12362 </section>
12363
12364 <section id='serial-console-connection'>
12365 <title>Serial Console Connection</title>
12366
12367 <para>
12368 For test target classes requiring a serial console
12369 to interact with the bootloader (e.g. BeagleBoneTarget,
12370 EdgeRouterTarget, and GrubTarget), you need to
12371 specify a command to use to connect to the serial console
12372 of the target machine by using the
12373 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD'><filename>TEST_SERIALCONTROL_CMD</filename></ulink>
12374 variable and optionally the
12375 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS'><filename>TEST_SERIALCONTROL_EXTRA_ARGS</filename></ulink>
12376 variable.
12377 </para>
12378
12379 <para>
12380 These cases could be a serial terminal program if the
12381 machine is connected to a local serial port, or a
12382 <filename>telnet</filename> or
12383 <filename>ssh</filename> command connecting to a remote
12384 console server.
12385 Regardless of the case, the command simply needs to
12386 connect to the serial console and forward that connection
12387 to standard input and output as any normal terminal
12388 program does.
12389 For example, to use the picocom terminal program on
12390 serial device <filename>/dev/ttyUSB0</filename>
12391 at 115200bps, you would set the variable as follows:
12392 <literallayout class='monospaced'>
12393 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
12394 </literallayout>
12395 For local devices where the serial port device disappears
12396 when the device reboots, an additional "serdevtry" wrapper
12397 script is provided.
12398 To use this wrapper, simply prefix the terminal command
12399 with
12400 <filename>${COREBASE}/scripts/contrib/serdevtry</filename>:
12401 <literallayout class='monospaced'>
12402 TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b
12403115200 /dev/ttyUSB0"
12404 </literallayout>
12405 </para>
12406 </section>
12407 </section>
12408
12409 <section id="qemu-image-running-tests">
12410 <title>Running Tests</title>
12411
12412 <para>
12413 You can start the tests automatically or manually:
12414 <itemizedlist>
12415 <listitem><para><emphasis>Automatically running tests:</emphasis>
12416 To run the tests automatically after the
12417 OpenEmbedded build system successfully creates an image,
12418 first set the
12419 <ulink url='&YOCTO_DOCS_REF_URL;#var-TESTIMAGE_AUTO'><filename>TESTIMAGE_AUTO</filename></ulink>
12420 variable to "1" in your <filename>local.conf</filename>
12421 file in the
12422 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
12423 <literallayout class='monospaced'>
12424 TESTIMAGE_AUTO = "1"
12425 </literallayout>
12426 Next, build your image.
12427 If the image successfully builds, the tests run:
12428 <literallayout class='monospaced'>
12429 bitbake core-image-sato
12430 </literallayout></para></listitem>
12431 <listitem><para><emphasis>Manually running tests:</emphasis>
12432 To manually run the tests, first globally inherit the
12433 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink>
12434 class by editing your <filename>local.conf</filename>
12435 file:
12436 <literallayout class='monospaced'>
12437 INHERIT += "testimage"
12438 </literallayout>
12439 Next, use BitBake to run the tests:
12440 <literallayout class='monospaced'>
12441 bitbake -c testimage <replaceable>image</replaceable>
12442 </literallayout></para></listitem>
12443 </itemizedlist>
12444 </para>
12445
12446 <para>
12447 All test files reside in
12448 <filename>meta/lib/oeqa/runtime</filename> in the
12449 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
12450 A test name maps directly to a Python module.
12451 Each test module may contain a number of individual tests.
12452 Tests are usually grouped together by the area
12453 tested (e.g tests for systemd reside in
12454 <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
12455 </para>
12456
12457 <para>
12458 You can add tests to any layer provided you place them in the
12459 proper area and you extend
12460 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
12461 in the <filename>local.conf</filename> file as normal.
12462 Be sure that tests reside in
12463 <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>.
12464 <note>
12465 Be sure that module names do not collide with module names
12466 used in the default set of test modules in
12467 <filename>meta/lib/oeqa/runtime</filename>.
12468 </note>
12469 </para>
12470
12471 <para>
12472 You can change the set of tests run by appending or overriding
12473 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>
12474 variable in <filename>local.conf</filename>.
12475 Each name in <filename>TEST_SUITES</filename> represents a
12476 required test for the image.
12477 Test modules named within <filename>TEST_SUITES</filename>
12478 cannot be skipped even if a test is not suitable for an image
12479 (e.g. running the RPM tests on an image without
12480 <filename>rpm</filename>).
12481 Appending "auto" to <filename>TEST_SUITES</filename> causes the
12482 build system to try to run all tests that are suitable for the
12483 image (i.e. each test module may elect to skip itself).
12484 </para>
12485
12486 <para>
12487 The order you list tests in <filename>TEST_SUITES</filename>
12488 is important and influences test dependencies.
12489 Consequently, tests that depend on other tests should be added
12490 after the test on which they depend.
12491 For example, since the <filename>ssh</filename> test
12492 depends on the
12493 <filename>ping</filename> test, "ssh" needs to come after
12494 "ping" in the list.
12495 The test class provides no re-ordering or dependency handling.
12496 <note>
12497 Each module can have multiple classes with multiple test
12498 methods.
12499 And, Python <filename>unittest</filename> rules apply.
12500 </note>
12501 </para>
12502
12503 <para>
12504 Here are some things to keep in mind when running tests:
12505 <itemizedlist>
12506 <listitem><para>The default tests for the image are defined
12507 as:
12508 <literallayout class='monospaced'>
12509 DEFAULT_TEST_SUITES_pn-<replaceable>image</replaceable> = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg"
12510 </literallayout></para></listitem>
12511 <listitem><para>Add your own test to the list of the
12512 by using the following:
12513 <literallayout class='monospaced'>
12514 TEST_SUITES_append = " mytest"
12515 </literallayout></para></listitem>
12516 <listitem><para>Run a specific list of tests as follows:
12517 <literallayout class='monospaced'>
12518 TEST_SUITES = "test1 test2 test3"
12519 </literallayout>
12520 Remember, order is important.
12521 Be sure to place a test that is dependent on another test
12522 later in the order.</para></listitem>
12523 </itemizedlist>
12524 </para>
12525 </section>
12526
12527 <section id="exporting-tests">
12528 <title>Exporting Tests</title>
12529
12530 <para>
12531 You can export tests so that they can run independently of
12532 the build system.
12533 Exporting tests is required if you want to be able to hand
12534 the test execution off to a scheduler.
12535 You can only export tests that are defined in
12536 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>.
12537 </para>
12538
12539 <para>
12540 If your image is already built, make sure the following are set
12541 in your <filename>local.conf</filename> file:
12542 <literallayout class='monospaced'>
12543 INHERIT +="testexport"
12544 TEST_TARGET_IP = "<replaceable>IP-address-for-the-test-target</replaceable>"
12545 TEST_SERVER_IP = "<replaceable>IP-address-for-the-test-server</replaceable>"
12546 </literallayout>
12547 You can then export the tests with the following BitBake
12548 command form:
12549 <literallayout class='monospaced'>
12550 $ bitbake <replaceable>image</replaceable> -c testexport
12551 </literallayout>
12552 Exporting the tests places them in the
12553 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
12554 in
12555 <filename>tmp/testexport/</filename><replaceable>image</replaceable>,
12556 which is controlled by the
12557 <filename>TEST_EXPORT_DIR</filename> variable.
12558 </para>
12559
12560 <para>
12561 You can now run the tests outside of the build environment:
12562 <literallayout class='monospaced'>
12563 $ cd tmp/testexport/<replaceable>image</replaceable>
12564 $ ./runexported.py testdata.json
12565 </literallayout>
12566 </para>
12567
12568 <para>
12569 Here is a complete example that shows IP addresses and uses
12570 the <filename>core-image-sato</filename> image:
12571 <literallayout class='monospaced'>
12572 INHERIT +="testexport"
12573 TEST_TARGET_IP = "192.168.7.2"
12574 TEST_SERVER_IP = "192.168.7.1"
12575 </literallayout>
12576 Use BitBake to export the tests:
12577 <literallayout class='monospaced'>
12578 $ bitbake core-image-sato -c testexport
12579 </literallayout>
12580 Run the tests outside of the build environment using the
12581 following:
12582 <literallayout class='monospaced'>
12583 $ cd tmp/testexport/core-image-sato
12584 $ ./runexported.py testdata.json
12585 </literallayout>
12586 </para>
12587 </section>
12588
12589 <section id="qemu-image-writing-new-tests">
12590 <title>Writing New Tests</title>
12591
12592 <para>
12593 As mentioned previously, all new test files need to be in the
12594 proper place for the build system to find them.
12595 New tests for additional functionality outside of the core
12596 should be added to the layer that adds the functionality, in
12597 <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>
12598 (as long as
12599 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
12600 is extended in the layer's
12601 <filename>layer.conf</filename> file as normal).
12602 Just remember the following:
12603 <itemizedlist>
12604 <listitem><para>Filenames need to map directly to test
12605 (module) names.
12606 </para></listitem>
12607 <listitem><para>Do not use module names that
12608 collide with existing core tests.
12609 </para></listitem>
12610 <listitem><para>Minimally, an empty
12611 <filename>__init__.py</filename> file must exist
12612 in the runtime directory.
12613 </para></listitem>
12614 </itemizedlist>
12615 </para>
12616
12617 <para>
12618 To create a new test, start by copying an existing module
12619 (e.g. <filename>syslog.py</filename> or
12620 <filename>gcc.py</filename> are good ones to use).
12621 Test modules can use code from
12622 <filename>meta/lib/oeqa/utils</filename>, which are helper
12623 classes.
12624 </para>
12625
12626 <note>
12627 Structure shell commands such that you rely on them and they
12628 return a single code for success.
12629 Be aware that sometimes you will need to parse the output.
12630 See the <filename>df.py</filename> and
12631 <filename>date.py</filename> modules for examples.
12632 </note>
12633
12634 <para>
12635 You will notice that all test classes inherit
12636 <filename>oeRuntimeTest</filename>, which is found in
12637 <filename>meta/lib/oetest.py</filename>.
12638 This base class offers some helper attributes, which are
12639 described in the following sections:
12640 </para>
12641
12642 <section id='qemu-image-writing-tests-class-methods'>
12643 <title>Class Methods</title>
12644
12645 <para>
12646 Class methods are as follows:
12647 <itemizedlist>
12648 <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis>
12649 Returns "True" if <filename>pkg</filename> is in the
12650 installed package list of the image, which is based
12651 on the manifest file that is generated during the
12652 <filename>do_rootfs</filename> task.
12653 </para></listitem>
12654 <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis>
12655 Returns "True" if the feature is in
12656 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
12657 or
12658 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>.
12659 </para></listitem>
12660 </itemizedlist>
12661 </para>
12662 </section>
12663
12664 <section id='qemu-image-writing-tests-class-attributes'>
12665 <title>Class Attributes</title>
12666
12667 <para>
12668 Class attributes are as follows:
12669 <itemizedlist>
12670 <listitem><para><emphasis><filename>pscmd</filename>:</emphasis>
12671 Equals "ps -ef" if <filename>procps</filename> is
12672 installed in the image.
12673 Otherwise, <filename>pscmd</filename> equals
12674 "ps" (busybox).
12675 </para></listitem>
12676 <listitem><para><emphasis><filename>tc</filename>:</emphasis>
12677 The called test context, which gives access to the
12678 following attributes:
12679 <itemizedlist>
12680 <listitem><para><emphasis><filename>d</filename>:</emphasis>
12681 The BitBake datastore, which allows you to
12682 use stuff such as
12683 <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>.
12684 </para></listitem>
12685 <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis>
12686 Used internally.
12687 The tests do not need these.
12688 </para></listitem>
12689 <listitem><para><emphasis><filename>filesdir</filename>:</emphasis>
12690 The absolute path to
12691 <filename>meta/lib/oeqa/runtime/files</filename>,
12692 which contains helper files for tests meant
12693 for copying on the target such as small
12694 files written in C for compilation.
12695 </para></listitem>
12696 <listitem><para><emphasis><filename>target</filename>:</emphasis>
12697 The target controller object used to deploy
12698 and start an image on a particular target
12699 (e.g. Qemu, SimpleRemote, and
12700 SystemdbootTarget).
12701 Tests usually use the following:
12702 <itemizedlist>
12703 <listitem><para><emphasis><filename>ip</filename>:</emphasis>
12704 The target's IP address.
12705 </para></listitem>
12706 <listitem><para><emphasis><filename>server_ip</filename>:</emphasis>
12707 The host's IP address, which is
12708 usually used by the DNF test
12709 suite.
12710 </para></listitem>
12711 <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis>
12712 The single, most used method.
12713 This command is a wrapper for:
12714 <filename>ssh root@host "cmd"</filename>.
12715 The command returns a tuple:
12716 (status, output), which are what
12717 their names imply - the return code
12718 of "cmd" and whatever output
12719 it produces.
12720 The optional timeout argument
12721 represents the number of seconds the
12722 test should wait for "cmd" to
12723 return.
12724 If the argument is "None", the
12725 test uses the default instance's
12726 timeout period, which is 300
12727 seconds.
12728 If the argument is "0", the test
12729 runs until the command returns.
12730 </para></listitem>
12731 <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis>
12732 <filename>scp localpath root@ip:remotepath</filename>.
12733 </para></listitem>
12734 <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis>
12735 <filename>scp root@host:remotepath localpath</filename>.
12736 </para></listitem>
12737 </itemizedlist></para></listitem>
12738 </itemizedlist></para></listitem>
12739 </itemizedlist>
12740 </para>
12741 </section>
12742
12743 <section id='qemu-image-writing-tests-instance-attributes'>
12744 <title>Instance Attributes</title>
12745
12746 <para>
12747 A single instance attribute exists, which is
12748 <filename>target</filename>.
12749 The <filename>target</filename> instance attribute is
12750 identical to the class attribute of the same name, which
12751 is described in the previous section.
12752 This attribute exists as both an instance and class
12753 attribute so tests can use
12754 <filename>self.target.run(cmd)</filename> in instance
12755 methods instead of
12756 <filename>oeRuntimeTest.tc.target.run(cmd)</filename>.
12757 </para>
12758 </section>
12759 </section>
12760
12761 <section id='installing-packages-in-the-dut-without-the-package-manager'>
12762 <title>Installing Packages in the DUT Without the Package Manager</title>
12763
12764 <para>
12765 When a test requires a package built by BitBake, it is possible
12766 to install that package.
12767 Installing the package does not require a package manager be
12768 installed in the device under test (DUT).
12769 It does, however, require an SSH connection and the target must
12770 be using the <filename>sshcontrol</filename> class.
12771 <note>
12772 This method uses <filename>scp</filename> to copy files
12773 from the host to the target, which causes permissions and
12774 special attributes to be lost.
12775 </note>
12776 </para>
12777
12778 <para>
12779 A JSON file is used to define the packages needed by a test.
12780 This file must be in the same path as the file used to define
12781 the tests.
12782 Furthermore, the filename must map directly to the test
12783 module name with a <filename>.json</filename> extension.
12784 </para>
12785
12786 <para>
12787 The JSON file must include an object with the test name as
12788 keys of an object or an array.
12789 This object (or array of objects) uses the following data:
12790 <itemizedlist>
12791 <listitem><para>"pkg" - A mandatory string that is the
12792 name of the package to be installed.
12793 </para></listitem>
12794 <listitem><para>"rm" - An optional boolean, which defaults
12795 to "false", that specifies to remove the package after
12796 the test.
12797 </para></listitem>
12798 <listitem><para>"extract" - An optional boolean, which
12799 defaults to "false", that specifies if the package must
12800 be extracted from the package format.
12801 When set to "true", the package is not automatically
12802 installed into the DUT.
12803 </para></listitem>
12804 </itemizedlist>
12805 </para>
12806
12807 <para>
12808 Following is an example JSON file that handles test "foo"
12809 installing package "bar" and test "foobar" installing
12810 packages "foo" and "bar".
12811 Once the test is complete, the packages are removed from the
12812 DUT.
12813 <literallayout class='monospaced'>
12814 {
12815 "foo": {
12816 "pkg": "bar"
12817 },
12818 "foobar": [
12819 {
12820 "pkg": "foo",
12821 "rm": true
12822 },
12823 {
12824 "pkg": "bar",
12825 "rm": true
12826 }
12827 ]
12828 }
12829 </literallayout>
12830 </para>
12831 </section>
12832 </section>
12833
12834 <section id='usingpoky-debugging-tools-and-techniques'>
12835 <title>Debugging Tools and Techniques</title>
12836
12837 <para>
12838 The exact method for debugging build failures depends on the nature
12839 of the problem and on the system's area from which the bug
12840 originates.
12841 Standard debugging practices such as comparison against the last
12842 known working version with examination of the changes and the
12843 re-application of steps to identify the one causing the problem are
12844 valid for the Yocto Project just as they are for any other system.
12845 Even though it is impossible to detail every possible potential
12846 failure, this section provides some general tips to aid in
12847 debugging given a variety of situations.
12848 <note><title>Tip</title>
12849 A useful feature for debugging is the error reporting tool.
12850 Configuring the Yocto Project to use this tool causes the
12851 OpenEmbedded build system to produce error reporting commands as
12852 part of the console output.
12853 You can enter the commands after the build completes to log
12854 error information into a common database, that can help you
12855 figure out what might be going wrong.
12856 For information on how to enable and use this feature, see the
12857 "<link linkend='using-the-error-reporting-tool'>Using the Error Reporting Tool</link>"
12858 section.
12859 </note>
12860 </para>
12861
12862 <para>
12863 The following list shows the debugging topics in the remainder of
12864 this section:
12865 <itemizedlist>
12866 <listitem><para>
12867 "<link linkend='dev-debugging-viewing-logs-from-failed-tasks'>Viewing Logs from Failed Tasks</link>"
12868 describes how to find and view logs from tasks that
12869 failed during the build process.
12870 </para></listitem>
12871 <listitem><para>
12872 "<link linkend='dev-debugging-viewing-variable-values'>Viewing Variable Values</link>"
12873 describes how to use the BitBake <filename>-e</filename>
12874 option to examine variable values after a recipe has been
12875 parsed.
12876 </para></listitem>
12877 <listitem><para>
12878 "<link linkend='viewing-package-information-with-oe-pkgdata-util'>Viewing Package Information with <filename>oe-pkgdata-util</filename></link>"
12879 describes how to use the
12880 <filename>oe-pkgdata-util</filename> utility to query
12881 <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink>
12882 and display package-related information for built
12883 packages.
12884 </para></listitem>
12885 <listitem><para>
12886 "<link linkend='dev-viewing-dependencies-between-recipes-and-tasks'>Viewing Dependencies Between Recipes and Tasks</link>"
12887 describes how to use the BitBake <filename>-g</filename>
12888 option to display recipe dependency information used
12889 during the build.
12890 </para></listitem>
12891 <listitem><para>
12892 "<link linkend='dev-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>"
12893 describes how to use the
12894 <filename>bitbake-dumpsig</filename> command in
12895 conjunction with key subdirectories in the
12896 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
12897 to determine variable dependencies.
12898 </para></listitem>
12899 <listitem><para>
12900 "<link linkend='dev-debugging-taskrunning'>Running Specific Tasks</link>"
12901 describes how to use several BitBake options (e.g.
12902 <filename>-c</filename>, <filename>-C</filename>, and
12903 <filename>-f</filename>) to run specific tasks in the
12904 build chain.
12905 It can be useful to run tasks "out-of-order" when trying
12906 isolate build issues.
12907 </para></listitem>
12908 <listitem><para>
12909 "<link linkend='dev-debugging-bitbake'>General BitBake Problems</link>"
12910 describes how to use BitBake's <filename>-D</filename>
12911 debug output option to reveal more about what BitBake is
12912 doing during the build.
12913 </para></listitem>
12914 <listitem><para>
12915 "<link linkend='dev-debugging-buildfile'>Building with No Dependencies</link>"
12916 describes how to use the BitBake <filename>-b</filename>
12917 option to build a recipe while ignoring dependencies.
12918 </para></listitem>
12919 <listitem><para>
12920 "<link linkend='recipe-logging-mechanisms'>Recipe Logging Mechanisms</link>"
12921 describes how to use the many recipe logging functions
12922 to produce debugging output and report errors and warnings.
12923 </para></listitem>
12924 <listitem><para>
12925 "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>"
12926 describes how to debug situations where the build consists
12927 of several parts that are run simultaneously and when the
12928 output or result of one part is not ready for use with a
12929 different part of the build that depends on that output.
12930 </para></listitem>
12931 <listitem><para>
12932 "<link linkend='platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</link>"
12933 describes how to use GDB to allow you to examine running
12934 programs, which can help you fix problems.
12935 </para></listitem>
12936 <listitem><para>
12937 "<link linkend='debugging-with-the-gnu-project-debugger-gdb-on-the-target'>Debugging with the GNU Project Debugger (GDB) on the Target</link>"
12938 describes how to use GDB directly on target hardware for
12939 debugging.
12940 </para></listitem>
12941 <listitem><para>
12942 "<link linkend='dev-other-debugging-others'>Other Debugging Tips</link>"
12943 describes miscellaneous debugging tips that can be useful.
12944 </para></listitem>
12945 </itemizedlist>
12946 </para>
12947
12948 <section id='dev-debugging-viewing-logs-from-failed-tasks'>
12949 <title>Viewing Logs from Failed Tasks</title>
12950
12951 <para>
12952 You can find the log for a task in the file
12953 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/temp/log.do_</filename><replaceable>taskname</replaceable>.
12954 For example, the log for the
12955 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
12956 task of the QEMU minimal image for the x86 machine
12957 (<filename>qemux86</filename>) might be in
12958 <filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile</filename>.
12959 To see the commands
12960 <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
12961 ran to generate a log, look at the corresponding
12962 <filename>run.do_</filename><replaceable>taskname</replaceable>
12963 file in the same directory.
12964 </para>
12965
12966 <para>
12967 <filename>log.do_</filename><replaceable>taskname</replaceable>
12968 and
12969 <filename>run.do_</filename><replaceable>taskname</replaceable>
12970 are actually symbolic links to
12971 <filename>log.do_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>
12972 and
12973 <filename>log.run_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>,
12974 where <replaceable>pid</replaceable> is the PID the task had
12975 when it ran.
12976 The symlinks always point to the files corresponding to the most
12977 recent run.
12978 </para>
12979 </section>
12980
12981 <section id='dev-debugging-viewing-variable-values'>
12982 <title>Viewing Variable Values</title>
12983
12984 <para>
12985 Sometimes you need to know the value of a variable as a
12986 result of BitBake's parsing step.
12987 This could be because some unexpected behavior occurred
12988 in your project.
12989 Perhaps an attempt to
12990 <ulink url='&YOCTO_DOCS_BB_URL;#modifying-existing-variables'>modify a variable</ulink>
12991 did not work out as expected.
12992 </para>
12993
12994 <para>
12995 BitBake's <filename>-e</filename> option is used to display
12996 variable values after parsing.
12997 The following command displays the variable values after the
12998 configuration files (i.e. <filename>local.conf</filename>,
12999 <filename>bblayers.conf</filename>,
13000 <filename>bitbake.conf</filename> and so forth) have been
13001 parsed:
13002 <literallayout class='monospaced'>
13003 $ bitbake -e
13004 </literallayout>
13005 The following command displays variable values after a specific
13006 recipe has been parsed.
13007 The variables include those from the configuration as well:
13008 <literallayout class='monospaced'>
13009 $ bitbake -e recipename
13010 </literallayout>
13011 <note><para>
13012 Each recipe has its own private set of variables
13013 (datastore).
13014 Internally, after parsing the configuration, a copy of the
13015 resulting datastore is made prior to parsing each recipe.
13016 This copying implies that variables set in one recipe will
13017 not be visible to other recipes.</para>
13018
13019 <para>Likewise, each task within a recipe gets a private
13020 datastore based on the recipe datastore, which means that
13021 variables set within one task will not be visible to
13022 other tasks.</para>
13023 </note>
13024 </para>
13025
13026 <para>
13027 In the output of <filename>bitbake -e</filename>, each
13028 variable is preceded by a description of how the variable
13029 got its value, including temporary values that were later
13030 overriden.
13031 This description also includes variable flags (varflags) set on
13032 the variable.
13033 The output can be very helpful during debugging.
13034 </para>
13035
13036 <para>
13037 Variables that are exported to the environment are preceded by
13038 <filename>export</filename> in the output of
13039 <filename>bitbake -e</filename>.
13040 See the following example:
13041 <literallayout class='monospaced'>
13042 export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86"
13043 </literallayout>
13044 </para>
13045
13046 <para>
13047 In addition to variable values, the output of the
13048 <filename>bitbake -e</filename> and
13049 <filename>bitbake -e</filename>&nbsp;<replaceable>recipe</replaceable>
13050 commands includes the following information:
13051 <itemizedlist>
13052 <listitem><para>
13053 The output starts with a tree listing all configuration
13054 files and classes included globally, recursively listing
13055 the files they include or inherit in turn.
13056 Much of the behavior of the OpenEmbedded build system
13057 (including the behavior of the
13058 <ulink url='&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks'>normal recipe build tasks</ulink>)
13059 is implemented in the
13060 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-base'><filename>base</filename></ulink>
13061 class and the classes it inherits, rather than being
13062 built into BitBake itself.
13063 </para></listitem>
13064 <listitem><para>
13065 After the variable values, all functions appear in the
13066 output.
13067 For shell functions, variables referenced within the
13068 function body are expanded.
13069 If a function has been modified using overrides or
13070 using override-style operators like
13071 <filename>_append</filename> and
13072 <filename>_prepend</filename>, then the final assembled
13073 function body appears in the output.
13074 </para></listitem>
13075 </itemizedlist>
13076 </para>
13077 </section>
13078
13079 <section id='viewing-package-information-with-oe-pkgdata-util'>
13080 <title>Viewing Package Information with <filename>oe-pkgdata-util</filename></title>
13081
13082 <para>
13083 You can use the <filename>oe-pkgdata-util</filename>
13084 command-line utility to query
13085 <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink>
13086 and display various package-related information.
13087 When you use the utility, you must use it to view information
13088 on packages that have already been built.
13089 </para>
13090
13091 <para>
13092 Following are a few of the available
13093 <filename>oe-pkgdata-util</filename> subcommands.
13094 <note>
13095 You can use the standard * and ? globbing wildcards as part
13096 of package names and paths.
13097 </note>
13098 <itemizedlist>
13099 <listitem><para>
13100 <filename>oe-pkgdata-util list-pkgs [</filename><replaceable>pattern</replaceable><filename>]</filename>:
13101 Lists all packages that have been built, optionally
13102 limiting the match to packages that match
13103 <replaceable>pattern</replaceable>.
13104 </para></listitem>
13105 <listitem><para>
13106 <filename>oe-pkgdata-util list-pkg-files&nbsp;</filename><replaceable>package</replaceable><filename>&nbsp;...</filename>:
13107 Lists the files and directories contained in the given
13108 packages.
13109 <note>
13110 <para>
13111 A different way to view the contents of a package is
13112 to look at the
13113 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename>
13114 directory of the recipe that generates the
13115 package.
13116 This directory is created by the
13117 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
13118 task and has one subdirectory for each package the
13119 recipe generates, which contains the files stored in
13120 that package.</para>
13121 <para>
13122 If you want to inspect the
13123 <filename>${WORKDIR}/packages-split</filename>
13124 directory, make sure that
13125 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink>
13126 is not enabled when you build the recipe.
13127 </para>
13128 </note>
13129 </para></listitem>
13130 <listitem><para>
13131 <filename>oe-pkgdata-util find-path&nbsp;</filename><replaceable>path</replaceable><filename>&nbsp;...</filename>:
13132 Lists the names of the packages that contain the given
13133 paths.
13134 For example, the following tells us that
13135 <filename>/usr/share/man/man1/make.1</filename>
13136 is contained in the <filename>make-doc</filename>
13137 package:
13138 <literallayout class='monospaced'>
13139 $ oe-pkgdata-util find-path /usr/share/man/man1/make.1
13140 make-doc: /usr/share/man/man1/make.1
13141 </literallayout>
13142 </para></listitem>
13143 <listitem><para>
13144 <filename>oe-pkgdata-util lookup-recipe&nbsp;</filename><replaceable>package</replaceable><filename>&nbsp;...</filename>:
13145 Lists the name of the recipes that
13146 produce the given packages.
13147 </para></listitem>
13148 </itemizedlist>
13149 </para>
13150
13151 <para>
13152 For more information on the <filename>oe-pkgdata-util</filename>
13153 command, use the help facility:
13154 <literallayout class='monospaced'>
13155 $ oe-pkgdata-util &dash;&dash;help
13156 $ oe-pkgdata-util <replaceable>subcommand</replaceable> --help
13157 </literallayout>
13158 </para>
13159 </section>
13160
13161 <section id='dev-viewing-dependencies-between-recipes-and-tasks'>
13162 <title>Viewing Dependencies Between Recipes and Tasks</title>
13163
13164 <para>
13165 Sometimes it can be hard to see why BitBake wants to build other
13166 recipes before the one you have specified.
13167 Dependency information can help you understand why a recipe is
13168 built.
13169 </para>
13170
13171 <para>
13172 To generate dependency information for a recipe, run the
13173 following command:
13174 <literallayout class='monospaced'>
13175 $ bitbake -g <replaceable>recipename</replaceable>
13176 </literallayout>
13177 This command writes the following files in the current
13178 directory:
13179 <itemizedlist>
13180 <listitem><para>
13181 <filename>pn-buildlist</filename>: A list of
13182 recipes/targets involved in building
13183 <replaceable>recipename</replaceable>.
13184 "Involved" here means that at least one task from the
13185 recipe needs to run when building
13186 <replaceable>recipename</replaceable> from scratch.
13187 Targets that are in
13188 <ulink url='&YOCTO_DOCS_REF_URL;#var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></ulink>
13189 are not listed.
13190 </para></listitem>
13191 <listitem><para>
13192 <filename>task-depends.dot</filename>: A graph showing
13193 dependencies between tasks.
13194 </para></listitem>
13195 </itemizedlist>
13196 </para>
13197
13198 <para>
13199 The graphs are in
13200 <ulink url='https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29'>DOT</ulink>
13201 format and can be converted to images (e.g. using the
13202 <filename>dot</filename> tool from
13203 <ulink url='http://www.graphviz.org/'>Graphviz</ulink>).
13204 <note><title>Notes</title>
13205 <itemizedlist>
13206 <listitem><para>
13207 DOT files use a plain text format.
13208 The graphs generated using the
13209 <filename>bitbake -g</filename> command are often so
13210 large as to be difficult to read without special
13211 pruning (e.g. with Bitbake's
13212 <filename>-I</filename> option) and processing.
13213 Despite the form and size of the graphs, the
13214 corresponding <filename>.dot</filename> files can
13215 still be possible to read and provide useful
13216 information.
13217 </para>
13218
13219 <para>As an example, the
13220 <filename>task-depends.dot</filename> file contains
13221 lines such as the following:
13222 <literallayout class='monospaced'>
13223 "libxslt.do_configure" -> "libxml2.do_populate_sysroot"
13224 </literallayout>
13225 The above example line reveals that the
13226 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
13227 task in <filename>libxslt</filename> depends on the
13228 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
13229 task in <filename>libxml2</filename>, which is a
13230 normal
13231 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
13232 dependency between the two recipes.
13233 </para></listitem>
13234 <listitem><para>
13235 For an example of how <filename>.dot</filename>
13236 files can be processed, see the
13237 <filename>scripts/contrib/graph-tool</filename>
13238 Python script, which finds and displays paths
13239 between graph nodes.
13240 </para></listitem>
13241 </itemizedlist>
13242 </note>
13243 </para>
13244
13245 <para>
13246 You can use a different method to view dependency information
13247 by using the following command:
13248 <literallayout class='monospaced'>
13249 $ bitbake -g -u taskexp <replaceable>recipename</replaceable>
13250 </literallayout>
13251 This command displays a GUI window from which you can view
13252 build-time and runtime dependencies for the recipes involved in
13253 building <replaceable>recipename</replaceable>.
13254 </para>
13255 </section>
13256
13257 <section id='dev-viewing-task-variable-dependencies'>
13258 <title>Viewing Task Variable Dependencies</title>
13259
13260 <para>
13261 As mentioned in the
13262 "<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>"
13263 section of the BitBake User Manual, BitBake tries to
13264 automatically determine what variables a task depends on so
13265 that it can rerun the task if any values of the variables
13266 change.
13267 This determination is usually reliable.
13268 However, if you do things like construct variable names at
13269 runtime, then you might have to manually declare dependencies
13270 on those variables using <filename>vardeps</filename> as
13271 described in the
13272 "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>"
13273 section of the BitBake User Manual.
13274 </para>
13275
13276 <para>
13277 If you are unsure whether a variable dependency is being
13278 picked up automatically for a given task, you can list the
13279 variable dependencies BitBake has determined by doing the
13280 following:
13281 <orderedlist>
13282 <listitem><para>
13283 Build the recipe containing the task:
13284 <literallayout class='monospaced'>
13285 $ bitbake <replaceable>recipename</replaceable>
13286 </literallayout>
13287 </para></listitem>
13288 <listitem><para>
13289 Inside the
13290 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR'><filename>STAMPS_DIR</filename></ulink>
13291 directory, find the signature data
13292 (<filename>sigdata</filename>) file that corresponds
13293 to the task.
13294 The <filename>sigdata</filename> files contain a pickled
13295 Python database of all the metadata that went into
13296 creating the input checksum for the task.
13297 As an example, for the
13298 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>
13299 task of the <filename>db</filename> recipe, the
13300 <filename>sigdata</filename> file might be found in the
13301 following location:
13302 <literallayout class='monospaced'>
13303 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
13304 </literallayout>
13305 For tasks that are accelerated through the shared state
13306 (<ulink url='&YOCTO_DOCS_OM_URL;#shared-state-cache'>sstate</ulink>)
13307 cache, an additional <filename>siginfo</filename> file
13308 is written into
13309 <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>
13310 along with the cached task output.
13311 The <filename>siginfo</filename> files contain exactly
13312 the same information as <filename>sigdata</filename>
13313 files.
13314 </para></listitem>
13315 <listitem><para>
13316 Run <filename>bitbake-dumpsig</filename> on the
13317 <filename>sigdata</filename> or
13318 <filename>siginfo</filename> file.
13319 Here is an example:
13320 <literallayout class='monospaced'>
13321 $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
13322 </literallayout>
13323 In the output of the above command, you will find a
13324 line like the following, which lists all the (inferred)
13325 variable dependencies for the task.
13326 This list also includes indirect dependencies from
13327 variables depending on other variables, recursively.
13328 <literallayout class='monospaced'>
13329 Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
13330 </literallayout>
13331 <note>
13332 Functions (e.g. <filename>base_do_fetch</filename>)
13333 also count as variable dependencies.
13334 These functions in turn depend on the variables they
13335 reference.
13336 </note>
13337 The output of <filename>bitbake-dumpsig</filename> also
13338 includes the value each variable had, a list of
13339 dependencies for each variable, and
13340 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink>
13341 information.
13342 </para></listitem>
13343 </orderedlist>
13344 </para>
13345
13346 <para>
13347 There is also a <filename>bitbake-diffsigs</filename> command
13348 for comparing two <filename>siginfo</filename> or
13349 <filename>sigdata</filename> files.
13350 This command can be helpful when trying to figure out what
13351 changed between two versions of a task.
13352 If you call <filename>bitbake-diffsigs</filename> with just one
13353 file, the command behaves like
13354 <filename>bitbake-dumpsig</filename>.
13355 </para>
13356
13357 <para>
13358 You can also use BitBake to dump out the signature construction
13359 information without executing tasks by using either of the
13360 following BitBake command-line options:
13361 <literallayout class='monospaced'>
13362 &dash;&dash;dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable>
13363 -S <replaceable>SIGNATURE_HANDLER</replaceable>
13364 </literallayout>
13365 <note>
13366 Two common values for
13367 <replaceable>SIGNATURE_HANDLER</replaceable> are "none" and
13368 "printdiff", which dump only the signature or compare the
13369 dumped signature with the cached one, respectively.
13370 </note>
13371 Using BitBake with either of these options causes BitBake to
13372 dump out <filename>sigdata</filename> files in the
13373 <filename>stamps</filename> directory for every task it would
13374 have executed instead of building the specified target package.
13375 </para>
13376 </section>
13377
13378 <section id='dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task'>
13379 <title>Viewing Metadata Used to Create the Input Signature of a Shared State Task</title>
13380
13381 <para>
13382 Seeing what metadata went into creating the input signature
13383 of a shared state (sstate) task can be a useful debugging
13384 aid.
13385 This information is available in signature information
13386 (<filename>siginfo</filename>) files in
13387 <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>.
13388 For information on how to view and interpret information in
13389 <filename>siginfo</filename> files, see the
13390 "<link linkend='dev-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>"
13391 section.
13392 </para>
13393
13394 <para>
13395 For conceptual information on shared state, see the
13396 "<ulink url='&YOCTO_DOCS_OM_URL;#shared-state'>Shared State</ulink>"
13397 section in the Yocto Project Overview and Concepts Manual.
13398 </para>
13399 </section>
13400
13401 <section id='dev-invalidating-shared-state-to-force-a-task-to-run'>
13402 <title>Invalidating Shared State to Force a Task to Run</title>
13403
13404 <para>
13405 The OpenEmbedded build system uses
13406 <ulink url='&YOCTO_DOCS_OM_URL;#overview-checksums'>checksums</ulink>
13407 and
13408 <ulink url='&YOCTO_DOCS_OM_URL;#shared-state'>shared state</ulink>
13409 cache to avoid unnecessarily rebuilding tasks.
13410 Collectively, this scheme is known as "shared state code."
13411 </para>
13412
13413 <para>
13414 As with all schemes, this one has some drawbacks.
13415 It is possible that you could make implicit changes to your
13416 code that the checksum calculations do not take into
13417 account.
13418 These implicit changes affect a task's output but do not
13419 trigger the shared state code into rebuilding a recipe.
13420 Consider an example during which a tool changes its output.
13421 Assume that the output of <filename>rpmdeps</filename>
13422 changes.
13423 The result of the change should be that all the
13424 <filename>package</filename> and
13425 <filename>package_write_rpm</filename> shared state cache
13426 items become invalid.
13427 However, because the change to the output is
13428 external to the code and therefore implicit,
13429 the associated shared state cache items do not become
13430 invalidated.
13431 In this case, the build process uses the cached items
13432 rather than running the task again.
13433 Obviously, these types of implicit changes can cause
13434 problems.
13435 </para>
13436
13437 <para>
13438 To avoid these problems during the build, you need to
13439 understand the effects of any changes you make.
13440 Realize that changes you make directly to a function
13441 are automatically factored into the checksum calculation.
13442 Thus, these explicit changes invalidate the associated
13443 area of shared state cache.
13444 However, you need to be aware of any implicit changes that
13445 are not obvious changes to the code and could affect
13446 the output of a given task.
13447 </para>
13448
13449 <para>
13450 When you identify an implicit change, you can easily
13451 take steps to invalidate the cache and force the tasks
13452 to run.
13453 The steps you can take are as simple as changing a
13454 function's comments in the source code.
13455 For example, to invalidate package shared state files,
13456 change the comment statements of
13457 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
13458 or the comments of one of the functions it calls.
13459 Even though the change is purely cosmetic, it causes the
13460 checksum to be recalculated and forces the build system to
13461 run the task again.
13462 <note>
13463 For an example of a commit that makes a cosmetic
13464 change to invalidate shared state, see this
13465 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
13466 </note>
13467 </para>
13468 </section>
13469
13470 <section id='dev-debugging-taskrunning'>
13471 <title>Running Specific Tasks</title>
13472
13473 <para>
13474 Any given recipe consists of a set of tasks.
13475 The standard BitBake behavior in most cases is:
13476 <filename>do_fetch</filename>,
13477 <filename>do_unpack</filename>,
13478 <filename>do_patch</filename>,
13479 <filename>do_configure</filename>,
13480 <filename>do_compile</filename>,
13481 <filename>do_install</filename>,
13482 <filename>do_package</filename>,
13483 <filename>do_package_write_*</filename>, and
13484 <filename>do_build</filename>.
13485 The default task is <filename>do_build</filename> and any tasks
13486 on which it depends build first.
13487 Some tasks, such as <filename>do_devshell</filename>, are not
13488 part of the default build chain.
13489 If you wish to run a task that is not part of the default build
13490 chain, you can use the <filename>-c</filename> option in
13491 BitBake.
13492 Here is an example:
13493 <literallayout class='monospaced'>
13494 $ bitbake matchbox-desktop -c devshell
13495 </literallayout>
13496 </para>
13497
13498 <para>
13499 The <filename>-c</filename> option respects task dependencies,
13500 which means that all other tasks (including tasks from other
13501 recipes) that the specified task depends on will be run before
13502 the task.
13503 Even when you manually specify a task to run with
13504 <filename>-c</filename>, BitBake will only run the task if it
13505 considers it "out of date".
13506 See the
13507 "<ulink url='&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks'>Stamp Files and the Rerunning of Tasks</ulink>"
13508 section in the Yocto Project Overview and Concepts Manual for
13509 how BitBake determines whether a task is "out of date".
13510 </para>
13511
13512 <para>
13513 If you want to force an up-to-date task to be rerun (e.g.
13514 because you made manual modifications to the recipe's
13515 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
13516 that you want to try out), then you can use the
13517 <filename>-f</filename> option.
13518 <note>
13519 The reason <filename>-f</filename> is never required when
13520 running the
13521 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-devshell'><filename>do_devshell</filename></ulink>
13522 task is because the
13523 <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink><filename>]</filename>
13524 variable flag is already set for the task.
13525 </note>
13526 The following example shows one way you can use the
13527 <filename>-f</filename> option:
13528 <literallayout class='monospaced'>
13529 $ bitbake matchbox-desktop
13530 .
13531 .
13532 make some changes to the source code in the work directory
13533 .
13534 .
13535 $ bitbake matchbox-desktop -c compile -f
13536 $ bitbake matchbox-desktop
13537 </literallayout>
13538 </para>
13539
13540 <para>
13541 This sequence first builds and then recompiles
13542 <filename>matchbox-desktop</filename>.
13543 The last command reruns all tasks (basically the packaging
13544 tasks) after the compile.
13545 BitBake recognizes that the <filename>do_compile</filename>
13546 task was rerun and therefore understands that the other tasks
13547 also need to be run again.
13548 </para>
13549
13550 <para>
13551 Another, shorter way to rerun a task and all
13552 <ulink url='&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks'>normal recipe build tasks</ulink>
13553 that depend on it is to use the <filename>-C</filename>
13554 option.
13555 <note>
13556 This option is upper-cased and is separate from the
13557 <filename>-c</filename> option, which is lower-cased.
13558 </note>
13559 Using this option invalidates the given task and then runs the
13560 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-build'><filename>do_build</filename></ulink>
13561 task, which is the default task if no task is given, and the
13562 tasks on which it depends.
13563 You could replace the final two commands in the previous example
13564 with the following single command:
13565 <literallayout class='monospaced'>
13566 $ bitbake matchbox-desktop -C compile
13567 </literallayout>
13568 Internally, the <filename>-f</filename> and
13569 <filename>-C</filename> options work by tainting (modifying) the
13570 input checksum of the specified task.
13571 This tainting indirectly causes the task and its
13572 dependent tasks to be rerun through the normal task dependency
13573 mechanisms.
13574 <note>
13575 BitBake explicitly keeps track of which tasks have been
13576 tainted in this fashion, and will print warnings such as the
13577 following for builds involving such tasks:
13578 <literallayout class='monospaced'>
13579 WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run
13580 </literallayout>
13581 The purpose of the warning is to let you know that the work
13582 directory and build output might not be in the clean state
13583 they would be in for a "normal" build, depending on what
13584 actions you took.
13585 To get rid of such warnings, you can remove the work
13586 directory and rebuild the recipe, as follows:
13587 <literallayout class='monospaced'>
13588 $ bitbake matchbox-desktop -c clean
13589 $ bitbake matchbox-desktop
13590 </literallayout>
13591 </note>
13592 </para>
13593
13594 <para>
13595 You can view a list of tasks in a given package by running the
13596 <filename>do_listtasks</filename> task as follows:
13597 <literallayout class='monospaced'>
13598 $ bitbake matchbox-desktop -c listtasks
13599 </literallayout>
13600 The results appear as output to the console and are also in the
13601 file <filename>${WORKDIR}/temp/log.do_listtasks</filename>.
13602 </para>
13603 </section>
13604
13605 <section id='dev-debugging-bitbake'>
13606 <title>General BitBake Problems</title>
13607
13608 <para>
13609 You can see debug output from BitBake by using the
13610 <filename>-D</filename> option.
13611 The debug output gives more information about what BitBake
13612 is doing and the reason behind it.
13613 Each <filename>-D</filename> option you use increases the
13614 logging level.
13615 The most common usage is <filename>-DDD</filename>.
13616 </para>
13617
13618 <para>
13619 The output from
13620 <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable>
13621 can reveal why BitBake chose a certain version of a package or
13622 why BitBake picked a certain provider.
13623 This command could also help you in a situation where you think
13624 BitBake did something unexpected.
13625 </para>
13626 </section>
13627
13628 <section id='dev-debugging-buildfile'>
13629 <title>Building with No Dependencies</title>
13630
13631 <para>
13632 To build a specific recipe (<filename>.bb</filename> file),
13633 you can use the following command form:
13634 <literallayout class='monospaced'>
13635 $ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb
13636 </literallayout>
13637 This command form does not check for dependencies.
13638 Consequently, you should use it only when you know existing
13639 dependencies have been met.
13640 <note>
13641 You can also specify fragments of the filename.
13642 In this case, BitBake checks for a unique match.
13643 </note>
13644 </para>
13645 </section>
13646
13647 <section id='recipe-logging-mechanisms'>
13648 <title>Recipe Logging Mechanisms</title>
13649
13650 <para>
13651 The Yocto Project provides several logging functions for
13652 producing debugging output and reporting errors and warnings.
13653 For Python functions, the following logging functions exist.
13654 All of these functions log to
13655 <filename>${T}/log.do_</filename><replaceable>task</replaceable>,
13656 and can also log to standard output (stdout) with the right
13657 settings:
13658 <itemizedlist>
13659 <listitem><para>
13660 <filename>bb.plain(</filename><replaceable>msg</replaceable><filename>)</filename>:
13661 Writes <replaceable>msg</replaceable> as is to the
13662 log while also logging to stdout.
13663 </para></listitem>
13664 <listitem><para>
13665 <filename>bb.note(</filename><replaceable>msg</replaceable><filename>)</filename>:
13666 Writes "NOTE: <replaceable>msg</replaceable>" to the
13667 log.
13668 Also logs to stdout if BitBake is called with "-v".
13669 </para></listitem>
13670 <listitem><para>
13671 <filename>bb.debug(</filename><replaceable>level</replaceable><filename>,&nbsp;</filename><replaceable>msg</replaceable><filename>)</filename>:
13672 Writes "DEBUG: <replaceable>msg</replaceable>" to the
13673 log.
13674 Also logs to stdout if the log level is greater than or
13675 equal to <replaceable>level</replaceable>.
13676 See the
13677 "<ulink url='&YOCTO_DOCS_BB_URL;#usage-and-syntax'>-D</ulink>"
13678 option in the BitBake User Manual for more information.
13679 </para></listitem>
13680 <listitem><para>
13681 <filename>bb.warn(</filename><replaceable>msg</replaceable><filename>)</filename>:
13682 Writes "WARNING: <replaceable>msg</replaceable>" to the
13683 log while also logging to stdout.
13684 </para></listitem>
13685 <listitem><para>
13686 <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>:
13687 Writes "ERROR: <replaceable>msg</replaceable>" to the
13688 log while also logging to standard out (stdout).
13689 <note>
13690 Calling this function does not cause the task to fail.
13691 </note>
13692 </para></listitem>
13693 <listitem><para>
13694 <filename>bb.fatal(</filename><replaceable>msg</replaceable><filename>)</filename>:
13695 This logging function is similar to
13696 <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>
13697 but also causes the calling task to fail.
13698 <note>
13699 <filename>bb.fatal()</filename> raises an exception,
13700 which means you do not need to put a "return"
13701 statement after the function.
13702 </note>
13703 </para></listitem>
13704 </itemizedlist>
13705 </para>
13706
13707 <para>
13708 The same logging functions are also available in shell
13709 functions, under the names
13710 <filename>bbplain</filename>, <filename>bbnote</filename>,
13711 <filename>bbdebug</filename>, <filename>bbwarn</filename>,
13712 <filename>bberror</filename>, and <filename>bbfatal</filename>.
13713 The
13714 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-logging'><filename>logging</filename></ulink>
13715 class implements these functions.
13716 See that class in the
13717 <filename>meta/classes</filename> folder of the
13718 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
13719 for information.
13720 </para>
13721
13722 <section id='logging-with-python'>
13723 <title>Logging With Python</title>
13724
13725 <para>
13726 When creating recipes using Python and inserting code that
13727 handles build logs, keep in mind the goal is to have
13728 informative logs while keeping the console as "silent" as
13729 possible.
13730 Also, if you want status messages in the log, use the
13731 "debug" loglevel.
13732 </para>
13733
13734 <para>
13735 Following is an example written in Python.
13736 The code handles logging for a function that determines the
13737 number of tasks needed to be run.
13738 See the
13739 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-listtasks'><filename>do_listtasks</filename></ulink>"
13740 section for additional information:
13741 <literallayout class='monospaced'>
13742 python do_listtasks() {
13743 bb.debug(2, "Starting to figure out the task list")
13744 if noteworthy_condition:
13745 bb.note("There are 47 tasks to run")
13746 bb.debug(2, "Got to point xyz")
13747 if warning_trigger:
13748 bb.warn("Detected warning_trigger, this might be a problem later.")
13749 if recoverable_error:
13750 bb.error("Hit recoverable_error, you really need to fix this!")
13751 if fatal_error:
13752 bb.fatal("fatal_error detected, unable to print the task list")
13753 bb.plain("The tasks present are abc")
13754 bb.debug(2, "Finished figuring out the tasklist")
13755 }
13756 </literallayout>
13757 </para>
13758 </section>
13759
13760 <section id='logging-with-bash'>
13761 <title>Logging With Bash</title>
13762
13763 <para>
13764 When creating recipes using Bash and inserting code that
13765 handles build logs, you have the same goals - informative
13766 with minimal console output.
13767 The syntax you use for recipes written in Bash is similar
13768 to that of recipes written in Python described in the
13769 previous section.
13770 </para>
13771
13772 <para>
13773 Following is an example written in Bash.
13774 The code logs the progress of the <filename>do_my_function</filename> function.
13775 <literallayout class='monospaced'>
13776 do_my_function() {
13777 bbdebug 2 "Running do_my_function"
13778 if [ exceptional_condition ]; then
13779 bbnote "Hit exceptional_condition"
13780 fi
13781 bbdebug 2 "Got to point xyz"
13782 if [ warning_trigger ]; then
13783 bbwarn "Detected warning_trigger, this might cause a problem later."
13784 fi
13785 if [ recoverable_error ]; then
13786 bberror "Hit recoverable_error, correcting"
13787 fi
13788 if [ fatal_error ]; then
13789 bbfatal "fatal_error detected"
13790 fi
13791 bbdebug 2 "Completed do_my_function"
13792 }
13793 </literallayout>
13794 </para>
13795 </section>
13796 </section>
13797
13798 <section id='debugging-parallel-make-races'>
13799 <title>Debugging Parallel Make Races</title>
13800
13801 <para>
13802 A parallel <filename>make</filename> race occurs when the build
13803 consists of several parts that are run simultaneously and
13804 a situation occurs when the output or result of one
13805 part is not ready for use with a different part of the build
13806 that depends on that output.
13807 Parallel make races are annoying and can sometimes be difficult
13808 to reproduce and fix.
13809 However, some simple tips and tricks exist that can help
13810 you debug and fix them.
13811 This section presents a real-world example of an error
13812 encountered on the Yocto Project autobuilder and the process
13813 used to fix it.
13814 <note>
13815 If you cannot properly fix a <filename>make</filename> race
13816 condition, you can work around it by clearing either the
13817 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
13818 or
13819 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink>
13820 variables.
13821 </note>
13822 </para>
13823
13824 <section id='the-failure'>
13825 <title>The Failure</title>
13826
13827 <para>
13828 For this example, assume that you are building an image that
13829 depends on the "neard" package.
13830 And, during the build, BitBake runs into problems and
13831 creates the following output.
13832 <note>
13833 This example log file has longer lines artificially
13834 broken to make the listing easier to read.
13835 </note>
13836 If you examine the output or the log file, you see the
13837 failure during <filename>make</filename>:
13838 <literallayout class='monospaced'>
13839 | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
13840 | DEBUG: Executing shell function do_compile
13841 | NOTE: make -j 16
13842 | make --no-print-directory all-am
13843 | /bin/mkdir -p include/near
13844 | /bin/mkdir -p include/near
13845 | /bin/mkdir -p include/near
13846 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13847 0.14-r0/neard-0.14/include/types.h include/near/types.h
13848 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13849 0.14-r0/neard-0.14/include/log.h include/near/log.h
13850 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13851 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h
13852 | /bin/mkdir -p include/near
13853 | /bin/mkdir -p include/near
13854 | /bin/mkdir -p include/near
13855 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13856 0.14-r0/neard-0.14/include/tag.h include/near/tag.h
13857 | /bin/mkdir -p include/near
13858 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13859 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h
13860 | /bin/mkdir -p include/near
13861 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13862 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h
13863 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13864 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h
13865 | /bin/mkdir -p include/near
13866 | /bin/mkdir -p include/near
13867 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13868 0.14-r0/neard-0.14/include/setting.h include/near/setting.h
13869 | /bin/mkdir -p include/near
13870 | /bin/mkdir -p include/near
13871 | /bin/mkdir -p include/near
13872 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13873 0.14-r0/neard-0.14/include/device.h include/near/device.h
13874 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13875 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h
13876 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13877 0.14-r0/neard-0.14/include/snep.h include/near/snep.h
13878 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13879 0.14-r0/neard-0.14/include/version.h include/near/version.h
13880 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
13881 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h
13882 | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h
13883 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/
13884 build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/
13885 yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0
13886 -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/
13887 lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/
13888 tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/
13889 nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/
13890 yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3
13891 -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\"
13892 -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c
13893 -o tools/snep-send.o tools/snep-send.c
13894 | In file included from tools/snep-send.c:16:0:
13895 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
13896 | #include &lt;near/dbus.h&gt;
13897 | ^
13898 | compilation terminated.
13899 | make[1]: *** [tools/snep-send.o] Error 1
13900 | make[1]: *** Waiting for unfinished jobs....
13901 | make: *** [all] Error 2
13902 | ERROR: oe_runmake failed
13903 </literallayout>
13904 </para>
13905 </section>
13906
13907 <section id='reproducing-the-error'>
13908 <title>Reproducing the Error</title>
13909
13910 <para>
13911 Because race conditions are intermittent, they do not
13912 manifest themselves every time you do the build.
13913 In fact, most times the build will complete without problems
13914 even though the potential race condition exists.
13915 Thus, once the error surfaces, you need a way to reproduce
13916 it.
13917 </para>
13918
13919 <para>
13920 In this example, compiling the "neard" package is causing
13921 the problem.
13922 So the first thing to do is build "neard" locally.
13923 Before you start the build, set the
13924 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
13925 variable in your <filename>local.conf</filename> file to
13926 a high number (e.g. "-j 20").
13927 Using a high value for <filename>PARALLEL_MAKE</filename>
13928 increases the chances of the race condition showing up:
13929 <literallayout class='monospaced'>
13930 $ bitbake neard
13931 </literallayout>
13932 </para>
13933
13934 <para>
13935 Once the local build for "neard" completes, start a
13936 <filename>devshell</filename> build:
13937 <literallayout class='monospaced'>
13938 $ bitbake neard -c devshell
13939 </literallayout>
13940 For information on how to use a
13941 <filename>devshell</filename>, see the
13942 "<link linkend='platdev-appdev-devshell'>Using a Development Shell</link>"
13943 section.
13944 </para>
13945
13946 <para>
13947 In the <filename>devshell</filename>, do the following:
13948 <literallayout class='monospaced'>
13949 $ make clean
13950 $ make tools/snep-send.o
13951 </literallayout>
13952 The <filename>devshell</filename> commands cause the failure
13953 to clearly be visible.
13954 In this case, a missing dependency exists for the "neard"
13955 Makefile target.
13956 Here is some abbreviated, sample output with the
13957 missing dependency clearly visible at the end:
13958 <literallayout class='monospaced'>
13959 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/......
13960 .
13961 .
13962 .
13963 tools/snep-send.c
13964 In file included from tools/snep-send.c:16:0:
13965 tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
13966 #include &lt;near/dbus.h&gt;
13967 ^
13968 compilation terminated.
13969 make: *** [tools/snep-send.o] Error 1
13970 $
13971 </literallayout>
13972 </para>
13973 </section>
13974
13975 <section id='creating-a-patch-for-the-fix'>
13976 <title>Creating a Patch for the Fix</title>
13977
13978 <para>
13979 Because there is a missing dependency for the Makefile
13980 target, you need to patch the
13981 <filename>Makefile.am</filename> file, which is generated
13982 from <filename>Makefile.in</filename>.
13983 You can use Quilt to create the patch:
13984 <literallayout class='monospaced'>
13985 $ quilt new parallelmake.patch
13986 Patch patches/parallelmake.patch is now on top
13987 $ quilt add Makefile.am
13988 File Makefile.am added to patch patches/parallelmake.patch
13989 </literallayout>
13990 For more information on using Quilt, see the
13991 "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>"
13992 section.
13993 </para>
13994
13995 <para>
13996 At this point you need to make the edits to
13997 <filename>Makefile.am</filename> to add the missing
13998 dependency.
13999 For our example, you have to add the following line
14000 to the file:
14001 <literallayout class='monospaced'>
14002 tools/snep-send.$(OBJEXT): include/near/dbus.h
14003 </literallayout>
14004 </para>
14005
14006 <para>
14007 Once you have edited the file, use the
14008 <filename>refresh</filename> command to create the patch:
14009 <literallayout class='monospaced'>
14010 $ quilt refresh
14011 Refreshed patch patches/parallelmake.patch
14012 </literallayout>
14013 Once the patch file exists, you need to add it back to the
14014 originating recipe folder.
14015 Here is an example assuming a top-level
14016 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
14017 named <filename>poky</filename>:
14018 <literallayout class='monospaced'>
14019 $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard
14020 </literallayout>
14021 The final thing you need to do to implement the fix in the
14022 build is to update the "neard" recipe (i.e.
14023 <filename>neard-0.14.bb</filename>) so that the
14024 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
14025 statement includes the patch file.
14026 The recipe file is in the folder above the patch.
14027 Here is what the edited <filename>SRC_URI</filename>
14028 statement would look like:
14029 <literallayout class='monospaced'>
14030 SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \
14031 file://neard.in \
14032 file://neard.service.in \
14033 file://parallelmake.patch \
14034 "
14035 </literallayout>
14036 </para>
14037
14038 <para>
14039 With the patch complete and moved to the correct folder and
14040 the <filename>SRC_URI</filename> statement updated, you can
14041 exit the <filename>devshell</filename>:
14042 <literallayout class='monospaced'>
14043 $ exit
14044 </literallayout>
14045 </para>
14046 </section>
14047
14048 <section id='testing-the-build'>
14049 <title>Testing the Build</title>
14050
14051 <para>
14052 With everything in place, you can get back to trying the
14053 build again locally:
14054 <literallayout class='monospaced'>
14055 $ bitbake neard
14056 </literallayout>
14057 This build should succeed.
14058 </para>
14059
14060 <para>
14061 Now you can open up a <filename>devshell</filename> again
14062 and repeat the clean and make operations as follows:
14063 <literallayout class='monospaced'>
14064 $ bitbake neard -c devshell
14065 $ make clean
14066 $ make tools/snep-send.o
14067 </literallayout>
14068 The build should work without issue.
14069 </para>
14070
14071 <para>
14072 As with all solved problems, if they originated upstream,
14073 you need to submit the fix for the recipe in OE-Core and
14074 upstream so that the problem is taken care of at its
14075 source.
14076 See the
14077 "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>"
14078 section for more information.
14079 </para>
14080 </section>
14081 </section>
14082
14083 <section id="platdev-gdb-remotedebug">
14084 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title>
14085
14086 <para>
14087 GDB allows you to examine running programs, which in turn helps
14088 you to understand and fix problems.
14089 It also allows you to perform post-mortem style analysis of
14090 program crashes.
14091 GDB is available as a package within the Yocto Project and is
14092 installed in SDK images by default.
14093 See the
14094 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
14095 chapter in the Yocto Project Reference Manual for a description of
14096 these images.
14097 You can find information on GDB at
14098 <ulink url="http://sourceware.org/gdb/"/>.
14099 <note><title>Tip</title>
14100 For best results, install debug (<filename>-dbg</filename>)
14101 packages for the applications you are going to debug.
14102 Doing so makes extra debug symbols available that give you
14103 more meaningful output.
14104 </note>
14105 </para>
14106
14107 <para>
14108 Sometimes, due to memory or disk space constraints, it is not
14109 possible to use GDB directly on the remote target to debug
14110 applications.
14111 These constraints arise because GDB needs to load the debugging
14112 information and the binaries of the process being debugged.
14113 Additionally, GDB needs to perform many computations to locate
14114 information such as function names, variable names and values,
14115 stack traces and so forth - even before starting the debugging
14116 process.
14117 These extra computations place more load on the target system
14118 and can alter the characteristics of the program being debugged.
14119 </para>
14120
14121 <para>
14122 To help get past the previously mentioned constraints, you can
14123 use gdbserver, which runs on the remote target and does not
14124 load any debugging information from the debugged process.
14125 Instead, a GDB instance processes the debugging information that
14126 is run on a remote computer - the host GDB.
14127 The host GDB then sends control commands to gdbserver to make
14128 it stop or start the debugged program, as well as read or write
14129 memory regions of that debugged program.
14130 All the debugging information loaded and processed as well
14131 as all the heavy debugging is done by the host GDB.
14132 Offloading these processes gives the gdbserver running on the
14133 target a chance to remain small and fast.
14134 </para>
14135
14136 <para>
14137 Because the host GDB is responsible for loading the debugging
14138 information and for doing the necessary processing to make
14139 actual debugging happen, you have to make sure the host can
14140 access the unstripped binaries complete with their debugging
14141 information and also be sure the target is compiled with no
14142 optimizations.
14143 The host GDB must also have local access to all the libraries
14144 used by the debugged program.
14145 Because gdbserver does not need any local debugging information,
14146 the binaries on the remote target can remain stripped.
14147 However, the binaries must also be compiled without optimization
14148 so they match the host's binaries.
14149 </para>
14150
14151 <para>
14152 To remain consistent with GDB documentation and terminology,
14153 the binary being debugged on the remote target machine is
14154 referred to as the "inferior" binary.
14155 For documentation on GDB see the
14156 <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>.
14157 </para>
14158
14159 <para>
14160 The following steps show you how to debug using the GNU project
14161 debugger.
14162 <orderedlist>
14163 <listitem><para>
14164 <emphasis>Configure your build system to construct the
14165 companion debug filesystem:</emphasis></para>
14166
14167 <para>In your <filename>local.conf</filename> file, set
14168 the following:
14169 <literallayout class='monospaced'>
14170 IMAGE_GEN_DEBUGFS = "1"
14171 IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
14172 </literallayout>
14173 These options cause the OpenEmbedded build system
14174 to generate a special companion filesystem fragment,
14175 which contains the matching source and debug symbols to
14176 your deployable filesystem.
14177 The build system does this by looking at what is in the
14178 deployed filesystem, and pulling the corresponding
14179 <filename>-dbg</filename> packages.</para>
14180
14181 <para>The companion debug filesystem is not a complete
14182 filesystem, but only contains the debug fragments.
14183 This filesystem must be combined with the full filesystem
14184 for debugging.
14185 Subsequent steps in this procedure show how to combine
14186 the partial filesystem with the full filesystem.
14187 </para></listitem>
14188 <listitem><para>
14189 <emphasis>Configure the system to include gdbserver in
14190 the target filesystem:</emphasis></para>
14191
14192 <para>Make the following addition in either your
14193 <filename>local.conf</filename> file or in an image
14194 recipe:
14195 <literallayout class='monospaced'>
14196 IMAGE_INSTALL_append = " gdbserver"
14197 </literallayout>
14198 The change makes sure the <filename>gdbserver</filename>
14199 package is included.
14200 </para></listitem>
14201 <listitem><para>
14202 <emphasis>Build the environment:</emphasis></para>
14203
14204 <para>Use the following command to construct the image
14205 and the companion Debug Filesystem:
14206 <literallayout class='monospaced'>
14207 $ bitbake <replaceable>image</replaceable>
14208 </literallayout>
14209 Build the cross GDB component and make it available
14210 for debugging.
14211 Build the SDK that matches the image.
14212 Building the SDK is best for a production build
14213 that can be used later for debugging, especially
14214 during long term maintenance:
14215 <literallayout class='monospaced'>
14216 $ bitbake -c populate_sdk <replaceable>image</replaceable>
14217 </literallayout></para>
14218
14219 <para>Alternatively, you can build the minimal
14220 toolchain components that match the target.
14221 Doing so creates a smaller than typical SDK and only
14222 contains a minimal set of components with which to
14223 build simple test applications, as well as run the
14224 debugger:
14225 <literallayout class='monospaced'>
14226 $ bitbake meta-toolchain
14227 </literallayout></para>
14228
14229 <para>A final method is to build Gdb itself within
14230 the build system:
14231 <literallayout class='monospaced'>
14232 $ bitbake gdb-cross-<replaceable>architecture</replaceable>
14233 </literallayout>
14234 Doing so produces a temporary copy of
14235 <filename>cross-gdb</filename> you can use for
14236 debugging during development.
14237 While this is the quickest approach, the two previous
14238 methods in this step are better when considering
14239 long-term maintenance strategies.
14240 <note>
14241 If you run
14242 <filename>bitbake gdb-cross</filename>, the
14243 OpenEmbedded build system suggests the actual
14244 image (e.g. <filename>gdb-cross-i586</filename>).
14245 The suggestion is usually the actual name you want
14246 to use.
14247 </note>
14248 </para></listitem>
14249 <listitem><para>
14250 <emphasis>Set up the</emphasis>&nbsp;<filename>debugfs</filename></para>
14251
14252 <para>Run the following commands to set up the
14253 <filename>debugfs</filename>:
14254 <literallayout class='monospaced'>
14255 $ mkdir debugfs
14256 $ cd debugfs
14257 $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.rootfs.tar.bz2
14258 $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>-dbg.rootfs.tar.bz2
14259 </literallayout>
14260 </para></listitem>
14261 <listitem><para>
14262 <emphasis>Set up GDB</emphasis></para>
14263
14264 <para>Install the SDK (if you built one) and then
14265 source the correct environment file.
14266 Sourcing the environment file puts the SDK in your
14267 <filename>PATH</filename> environment variable.</para>
14268
14269 <para>If you are using the build system, Gdb is
14270 located in
14271 <replaceable>build-dir</replaceable>/tmp/sysroots/<replaceable>host</replaceable>/usr/bin/<replaceable>architecture</replaceable>/<replaceable>architecture</replaceable>-gdb
14272 </para></listitem>
14273 <listitem><para>
14274 <emphasis>Boot the target:</emphasis></para>
14275
14276 <para>For information on how to run QEMU, see the
14277 <ulink url='http://wiki.qemu.org/Documentation/GettingStartedDevelopers'>QEMU Documentation</ulink>.
14278 <note>
14279 Be sure to verify that your host can access the
14280 target via TCP.
14281 </note>
14282 </para></listitem>
14283 <listitem><para>
14284 <emphasis>Debug a program:</emphasis></para>
14285
14286 <para>Debugging a program involves running gdbserver
14287 on the target and then running Gdb on the host.
14288 The example in this step debugs
14289 <filename>gzip</filename>:
14290 <literallayout class='monospaced'>
14291 root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help
14292 </literallayout>
14293 For additional gdbserver options, see the
14294 <ulink url='https://www.gnu.org/software/gdb/documentation/'>GDB Server Documentation</ulink>.
14295 </para>
14296
14297 <para>After running gdbserver on the target, you need
14298 to run Gdb on the host and configure it and connect to
14299 the target.
14300 Use these commands:
14301 <literallayout class='monospaced'>
14302 $ cd <replaceable>directory-holding-the-debugfs-directory</replaceable>
14303 $ <replaceable>arch</replaceable>-gdb
14304
14305 (gdb) set sysroot debugfs
14306 (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug
14307 (gdb) target remote <replaceable>IP-of-target</replaceable>:1234
14308 </literallayout>
14309 At this point, everything should automatically load
14310 (i.e. matching binaries, symbols and headers).
14311 <note>
14312 The Gdb <filename>set</filename> commands in the
14313 previous example can be placed into the users
14314 <filename>~/.gdbinit</filename> file.
14315 Upon starting, Gdb automatically runs whatever
14316 commands are in that file.
14317 </note>
14318 </para></listitem>
14319 <listitem><para>
14320 <emphasis>Deploying without a full image
14321 rebuild:</emphasis></para>
14322
14323 <para>In many cases, during development you want a
14324 quick method to deploy a new binary to the target and
14325 debug it, without waiting for a full image build.
14326 </para>
14327
14328 <para>One approach to solving this situation is to
14329 just build the component you want to debug.
14330 Once you have built the component, copy the
14331 executable directly to both the target and the
14332 host <filename>debugfs</filename>.</para>
14333
14334 <para>If the binary is processed through the debug
14335 splitting in OpenEmbedded, you should also
14336 copy the debug items (i.e. <filename>.debug</filename>
14337 contents and corresponding
14338 <filename>/usr/src/debug</filename> files)
14339 from the work directory.
14340 Here is an example:
14341 <literallayout class='monospaced'>
14342 $ bitbake bash
14343 $ bitbake -c devshell bash
14344 $ cd ..
14345 $ scp packages-split/bash/bin/bash <replaceable>target</replaceable>:/bin/bash
14346 $ cp -a packages-split/bash-dbg/* <replaceable>path</replaceable>/debugfs
14347 </literallayout>
14348 </para></listitem>
14349 </orderedlist>
14350 </para>
14351 </section>
14352
14353 <section id='debugging-with-the-gnu-project-debugger-gdb-on-the-target'>
14354 <title>Debugging with the GNU Project Debugger (GDB) on the Target</title>
14355
14356 <para>
14357 The previous section addressed using GDB remotely for debugging
14358 purposes, which is the most usual case due to the inherent
14359 hardware limitations on many embedded devices.
14360 However, debugging in the target hardware itself is also
14361 possible with more powerful devices.
14362 This section describes what you need to do in order to support
14363 using GDB to debug on the target hardware.
14364 </para>
14365
14366 <para>
14367 To support this kind of debugging, you need do the following:
14368 <itemizedlist>
14369 <listitem><para>
14370 Ensure that GDB is on the target.
14371 You can do this by adding "gdb" to
14372 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>:
14373 <literallayout class='monospaced'>
14374 IMAGE_INSTALL_append = " gdb"
14375 </literallayout>
14376 Alternatively, you can add "tools-debug" to
14377 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>:
14378 <literallayout class='monospaced'>
14379 IMAGE_FEATURES_append = " tools-debug"
14380 </literallayout>
14381 </para></listitem>
14382 <listitem><para>
14383 Ensure that debug symbols are present.
14384 You can make sure these symbols are present by
14385 installing <filename>-dbg</filename>:
14386 <literallayout class='monospaced'>
14387 IMAGE_INSTALL_append = " <replaceable>packagename</replaceable>-dbg"
14388 </literallayout>
14389 Alternatively, you can do the following to include all
14390 the debug symbols:
14391 <literallayout class='monospaced'>
14392 IMAGE_FEATURES_append = " dbg-pkgs"
14393 </literallayout>
14394 </para></listitem>
14395 </itemizedlist>
14396 <note>
14397 To improve the debug information accuracy, you can reduce
14398 the level of optimization used by the compiler.
14399 For example, when adding the following line to your
14400 <filename>local.conf</filename> file, you will reduce
14401 optimization from
14402 <ulink url='&YOCTO_DOCS_REF_URL;#var-FULL_OPTIMIZATION'><filename>FULL_OPTIMIZATION</filename></ulink>
14403 of "-O2" to
14404 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_OPTIMIZATION'><filename>DEBUG_OPTIMIZATION</filename></ulink>
14405 of "-O -fno-omit-frame-pointer":
14406 <literallayout class='monospaced'>
14407 DEBUG_BUILD = "1"
14408 </literallayout>
14409 Consider that this will reduce the application's performance
14410 and is recommended only for debugging purposes.
14411 </note>
14412 </para>
14413 </section>
14414
14415 <section id='dev-other-debugging-others'>
14416 <title>Other Debugging Tips</title>
14417
14418 <para>
14419 Here are some other tips that you might find useful:
14420 <itemizedlist>
14421 <listitem><para>
14422 When adding new packages, it is worth watching for
14423 undesirable items making their way into compiler command
14424 lines.
14425 For example, you do not want references to local system
14426 files like
14427 <filename>/usr/lib/</filename> or
14428 <filename>/usr/include/</filename>.
14429 </para></listitem>
14430 <listitem><para>
14431 If you want to remove the <filename>psplash</filename>
14432 boot splashscreen,
14433 add <filename>psplash=false</filename> to the kernel
14434 command line.
14435 Doing so prevents <filename>psplash</filename> from
14436 loading and thus allows you to see the console.
14437 It is also possible to switch out of the splashscreen by
14438 switching the virtual console (e.g. Fn+Left or Fn+Right
14439 on a Zaurus).
14440 </para></listitem>
14441 <listitem><para>
14442 Removing
14443 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
14444 (usually <filename>tmp/</filename>, within the
14445 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>)
14446 can often fix temporary build issues.
14447 Removing <filename>TMPDIR</filename> is usually a
14448 relatively cheap operation, because task output will be
14449 cached in
14450 <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>
14451 (usually <filename>sstate-cache/</filename>, which is
14452 also in the Build Directory).
14453 <note>
14454 Removing <filename>TMPDIR</filename> might be a
14455 workaround rather than a fix.
14456 Consequently, trying to determine the underlying
14457 cause of an issue before removing the directory is
14458 a good idea.
14459 </note>
14460 </para></listitem>
14461 <listitem><para>
14462 Understanding how a feature is used in practice within
14463 existing recipes can be very helpful.
14464 It is recommended that you configure some method that
14465 allows you to quickly search through files.</para>
14466
14467 <para>Using GNU Grep, you can use the following shell
14468 function to recursively search through common
14469 recipe-related files, skipping binary files,
14470 <filename>.git</filename> directories, and the
14471 Build Directory (assuming its name starts with
14472 "build"):
14473 <literallayout class='monospaced'>
14474 g() {
14475 grep -Ir \
14476 --exclude-dir=.git \
14477 --exclude-dir='build*' \
14478 --include='*.bb*' \
14479 --include='*.inc*' \
14480 --include='*.conf*' \
14481 --include='*.py*' \
14482 "$@"
14483 }
14484 </literallayout>
14485 Following are some usage examples:
14486 <literallayout class='monospaced'>
14487 $ g FOO # Search recursively for "FOO"
14488 $ g -i foo # Search recursively for "foo", ignoring case
14489 $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR"
14490 </literallayout>
14491 If figuring out how some feature works requires a lot of
14492 searching, it might indicate that the documentation
14493 should be extended or improved.
14494 In such cases, consider filing a documentation bug using
14495 the Yocto Project implementation of
14496 <ulink url='https://bugzilla.yoctoproject.org/'>Bugzilla</ulink>.
14497 For information on how to submit a bug against
14498 the Yocto Project, see the Yocto Project Bugzilla
14499 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink>
14500 and the
14501 "<link linkend='submitting-a-defect-against-the-yocto-project'>Submitting a Defect Against the Yocto Project</link>"
14502 section.
14503 <note>
14504 The manuals might not be the right place to document
14505 variables that are purely internal and have a
14506 limited scope (e.g. internal variables used to
14507 implement a single <filename>.bbclass</filename>
14508 file).
14509 </note>
14510 </para></listitem>
14511 </itemizedlist>
14512 </para>
14513 </section>
14514 </section>
14515
14516 <section id='making-changes-to-the-yocto-project'>
14517 <title>Making Changes to the Yocto Project</title>
14518
14519 <para>
14520 Because the Yocto Project is an open-source, community-based
14521 project, you can effect changes to the project.
14522 This section presents procedures that show you how to submit
14523 a defect against the project and how to submit a change.
14524 </para>
14525
14526 <section id='submitting-a-defect-against-the-yocto-project'>
14527 <title>Submitting a Defect Against the Yocto Project</title>
14528
14529 <para>
14530 Use the Yocto Project implementation of
14531 <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink>
14532 to submit a defect (bug) against the Yocto Project.
14533 For additional information on this implementation of Bugzilla see the
14534 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-bugtracker'>Yocto Project Bugzilla</ulink>"
14535 section in the Yocto Project Reference Manual.
14536 For more detail on any of the following steps, see the Yocto Project
14537 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>Bugzilla wiki page</ulink>.
14538 </para>
14539
14540 <para>
14541 Use the following general steps to submit a bug"
14542
14543 <orderedlist>
14544 <listitem><para>
14545 Open the Yocto Project implementation of
14546 <ulink url='&YOCTO_BUGZILLA_URL;'>Bugzilla</ulink>.
14547 </para></listitem>
14548 <listitem><para>
14549 Click "File a Bug" to enter a new bug.
14550 </para></listitem>
14551 <listitem><para>
14552 Choose the appropriate "Classification", "Product", and
14553 "Component" for which the bug was found.
14554 Bugs for the Yocto Project fall into one of several
14555 classifications, which in turn break down into several
14556 products and components.
14557 For example, for a bug against the
14558 <filename>meta-intel</filename> layer, you would choose
14559 "Build System, Metadata &amp; Runtime", "BSPs", and
14560 "bsps-meta-intel", respectively.
14561 </para></listitem>
14562 <listitem><para>
14563 Choose the "Version" of the Yocto Project for which you found
14564 the bug (e.g. &DISTRO;).
14565 </para></listitem>
14566 <listitem><para>
14567 Determine and select the "Severity" of the bug.
14568 The severity indicates how the bug impacted your work.
14569 </para></listitem>
14570 <listitem><para>
14571 Choose the "Hardware" that the bug impacts.
14572 </para></listitem>
14573 <listitem><para>
14574 Choose the "Architecture" that the bug impacts.
14575 </para></listitem>
14576 <listitem><para>
14577 Choose a "Documentation change" item for the bug.
14578 Fixing a bug might or might not affect the Yocto Project
14579 documentation.
14580 If you are unsure of the impact to the documentation, select
14581 "Don't Know".
14582 </para></listitem>
14583 <listitem><para>
14584 Provide a brief "Summary" of the bug.
14585 Try to limit your summary to just a line or two and be sure
14586 to capture the essence of the bug.
14587 </para></listitem>
14588 <listitem><para>
14589 Provide a detailed "Description" of the bug.
14590 You should provide as much detail as you can about the context,
14591 behavior, output, and so forth that surrounds the bug.
14592 You can even attach supporting files for output from logs by
14593 using the "Add an attachment" button.
14594 </para></listitem>
14595 <listitem><para>
14596 Click the "Submit Bug" button submit the bug.
14597 A new Bugzilla number is assigned to the bug and the defect
14598 is logged in the bug tracking system.
14599 </para></listitem>
14600 </orderedlist>
14601 Once you file a bug, the bug is processed by the Yocto Project Bug
14602 Triage Team and further details concerning the bug are assigned
14603 (e.g. priority and owner).
14604 You are the "Submitter" of the bug and any further categorization,
14605 progress, or comments on the bug result in Bugzilla sending you an
14606 automated email concerning the particular change or progress to the
14607 bug.
14608 </para>
14609 </section>
14610
14611 <section id='how-to-submit-a-change'>
14612 <title>Submitting a Change to the Yocto Project</title>
14613
14614 <para>
14615 Contributions to the Yocto Project and OpenEmbedded are very welcome.
14616 Because the system is extremely configurable and flexible, we recognize
14617 that developers will want to extend, configure or optimize it for
14618 their specific uses.
14619 </para>
14620
14621 <para>
14622 The Yocto Project uses a mailing list and a patch-based workflow
14623 that is similar to the Linux kernel but contains important
14624 differences.
14625 In general, a mailing list exists through which you can submit
14626 patches.
14627 You should send patches to the appropriate mailing list so that they
14628 can be reviewed and merged by the appropriate maintainer.
14629 The specific mailing list you need to use depends on the
14630 location of the code you are changing.
14631 Each component (e.g. layer) should have a
14632 <filename>README</filename> file that indicates where to send
14633 the changes and which process to follow.
14634 </para>
14635
14636 <para>
14637 You can send the patch to the mailing list using whichever approach
14638 you feel comfortable with to generate the patch.
14639 Once sent, the patch is usually reviewed by the community at large.
14640 If somebody has concerns with the patch, they will usually voice
14641 their concern over the mailing list.
14642 If a patch does not receive any negative reviews, the maintainer of
14643 the affected layer typically takes the patch, tests it, and then
14644 based on successful testing, merges the patch.
14645 </para>
14646
14647 <para id='figuring-out-the-mailing-list-to-use'>
14648 The "poky" repository, which is the Yocto Project's reference build
14649 environment, is a hybrid repository that contains several
14650 individual pieces (e.g. BitBake, Metadata, documentation,
14651 and so forth) built using the combo-layer tool.
14652 The upstream location used for submitting changes varies by
14653 component:
14654 <itemizedlist>
14655 <listitem><para>
14656 <emphasis>Core Metadata:</emphasis>
14657 Send your patch to the
14658 <ulink url='http://lists.openembedded.org/mailman/listinfo/openembedded-core'>openembedded-core</ulink>
14659 mailing list. For example, a change to anything under
14660 the <filename>meta</filename> or
14661 <filename>scripts</filename> directories should be sent
14662 to this mailing list.
14663 </para></listitem>
14664 <listitem><para>
14665 <emphasis>BitBake:</emphasis>
14666 For changes to BitBake (i.e. anything under the
14667 <filename>bitbake</filename> directory), send your patch
14668 to the
14669 <ulink url='http://lists.openembedded.org/mailman/listinfo/bitbake-devel'>bitbake-devel</ulink>
14670 mailing list.
14671 </para></listitem>
14672 <listitem><para>
14673 <emphasis>"meta-*" trees:</emphasis>
14674 These trees contain Metadata.
14675 Use the
14676 <ulink url='https://lists.yoctoproject.org/listinfo/poky'>poky</ulink>
14677 mailing list.
14678 </para></listitem>
14679 </itemizedlist>
14680 </para>
14681
14682 <para>
14683 For changes to other layers hosted in the Yocto Project source
14684 repositories (i.e. <filename>yoctoproject.org</filename>), tools,
14685 and the Yocto Project documentation, use the
14686 <ulink url='https://lists.yoctoproject.org/listinfo/yocto'>Yocto Project</ulink>
14687 general mailing list.
14688 <note>
14689 Sometimes a layer's documentation specifies to use a
14690 particular mailing list.
14691 If so, use that list.
14692 </note>
14693 For additional recipes that do not fit into the core Metadata, you
14694 should determine which layer the recipe should go into and submit
14695 the change in the manner recommended by the documentation (e.g.
14696 the <filename>README</filename> file) supplied with the layer.
14697 If in doubt, please ask on the Yocto general mailing list or on
14698 the openembedded-devel mailing list.
14699 </para>
14700
14701 <para>
14702 You can also push a change upstream and request a maintainer to
14703 pull the change into the component's upstream repository.
14704 You do this by pushing to a contribution repository that is upstream.
14705 See the
14706 "<ulink url='&YOCTO_DOCS_OM_URL;#gs-git-workflows-and-the-yocto-project'>Git Workflows and the Yocto Project</ulink>"
14707 section in the Yocto Project Overview and Concepts Manual for additional
14708 concepts on working in the Yocto Project development environment.
14709 </para>
14710
14711 <para>
14712 Two commonly used testing repositories exist for
14713 OpenEmbedded-Core:
14714 <itemizedlist>
14715 <listitem><para>
14716 <emphasis>"ross/mut" branch:</emphasis>
14717 The "mut" (master-under-test) tree
14718 exists in the <filename>poky-contrib</filename> repository
14719 in the
14720 <ulink url='&YOCTO_GIT_URL;'>Yocto Project source repositories</ulink>.
14721 </para></listitem>
14722 <listitem><para>
14723 <emphasis>"master-next" branch:</emphasis>
14724 This branch is part of the main
14725 "poky" repository in the Yocto Project source repositories.
14726 </para></listitem>
14727 </itemizedlist>
14728 Maintainers use these branches to test submissions prior to merging
14729 patches.
14730 Thus, you can get an idea of the status of a patch based on
14731 whether the patch has been merged into one of these branches.
14732 <note>
14733 This system is imperfect and changes can sometimes get lost in the
14734 flow.
14735 Asking about the status of a patch or change is reasonable if the
14736 change has been idle for a while with no feedback.
14737 The Yocto Project does have plans to use
14738 <ulink url='https://en.wikipedia.org/wiki/Patchwork_(software)'>Patchwork</ulink>
14739 to track the status of patches and also to automatically preview
14740 patches.
14741 </note>
14742 </para>
14743
14744 <para>
14745 The following sections provide procedures for submitting a change.
14746 </para>
14747
14748 <section id='pushing-a-change-upstream'>
14749 <title>Using Scripts to Push a Change Upstream and Request a Pull</title>
14750
14751 <para>
14752 Follow this procedure to push a change to an upstream "contrib"
14753 Git repository:
14754 <note>
14755 You can find general Git information on how to push a change
14756 upstream in the
14757 <ulink url='http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows'>Git Community Book</ulink>.
14758 </note>
14759 <orderedlist>
14760 <listitem><para>
14761 <emphasis>Make Your Changes Locally:</emphasis>
14762 Make your changes in your local Git repository.
14763 You should make small, controlled, isolated changes.
14764 Keeping changes small and isolated aids review,
14765 makes merging/rebasing easier and keeps the change
14766 history clean should anyone need to refer to it in
14767 future.
14768 </para></listitem>
14769 <listitem><para>
14770 <emphasis>Stage Your Changes:</emphasis>
14771 Stage your changes by using the <filename>git add</filename>
14772 command on each file you changed.
14773 </para></listitem>
14774 <listitem><para id='making-sure-you-have-correct-commit-information'>
14775 <emphasis>Commit Your Changes:</emphasis>
14776 Commit the change by using the
14777 <filename>git commit</filename> command.
14778 Make sure your commit information follows standards by
14779 following these accepted conventions:
14780 <itemizedlist>
14781 <listitem><para>
14782 Be sure to include a "Signed-off-by:" line in the
14783 same style as required by the Linux kernel.
14784 Adding this line signifies that you, the submitter,
14785 have agreed to the Developer's Certificate of
14786 Origin 1.1 as follows:
14787 <literallayout class='monospaced'>
14788 Developer's Certificate of Origin 1.1
14789
14790 By making a contribution to this project, I certify that:
14791
14792 (a) The contribution was created in whole or in part by me and I
14793 have the right to submit it under the open source license
14794 indicated in the file; or
14795
14796 (b) The contribution is based upon previous work that, to the best
14797 of my knowledge, is covered under an appropriate open source
14798 license and I have the right under that license to submit that
14799 work with modifications, whether created in whole or in part
14800 by me, under the same open source license (unless I am
14801 permitted to submit under a different license), as indicated
14802 in the file; or
14803
14804 (c) The contribution was provided directly to me by some other
14805 person who certified (a), (b) or (c) and I have not modified
14806 it.
14807
14808 (d) I understand and agree that this project and the contribution
14809 are public and that a record of the contribution (including all
14810 personal information I submit with it, including my sign-off) is
14811 maintained indefinitely and may be redistributed consistent with
14812 this project or the open source license(s) involved.
14813 </literallayout>
14814 </para></listitem>
14815 <listitem><para>
14816 Provide a single-line summary of the change.
14817 and,
14818 if more explanation is needed, provide more
14819 detail in the body of the commit.
14820 This summary is typically viewable in the
14821 "shortlist" of changes.
14822 Thus, providing something short and descriptive
14823 that gives the reader a summary of the change is
14824 useful when viewing a list of many commits.
14825 You should prefix this short description with the
14826 recipe name (if changing a recipe), or else with
14827 the short form path to the file being changed.
14828 </para></listitem>
14829 <listitem><para>
14830 For the body of the commit message, provide
14831 detailed information that describes what you
14832 changed, why you made the change, and the approach
14833 you used.
14834 It might also be helpful if you mention how you
14835 tested the change.
14836 Provide as much detail as you can in the body of
14837 the commit message.
14838 <note>
14839 You do not need to provide a more detailed
14840 explanation of a change if the change is
14841 minor to the point of the single line
14842 summary providing all the information.
14843 </note>
14844 </para></listitem>
14845 <listitem><para>
14846 If the change addresses a specific bug or issue
14847 that is associated with a bug-tracking ID,
14848 include a reference to that ID in your detailed
14849 description.
14850 For example, the Yocto Project uses a specific
14851 convention for bug references - any commit that
14852 addresses a specific bug should use the following
14853 form for the detailed description.
14854 Be sure to use the actual bug-tracking ID from
14855 Bugzilla for
14856 <replaceable>bug-id</replaceable>:
14857 <literallayout class='monospaced'>
14858 Fixes [YOCTO #<replaceable>bug-id</replaceable>]
14859
14860 <replaceable>detailed description of change</replaceable>
14861 </literallayout>
14862 </para></listitem>
14863 </itemizedlist>
14864 </para></listitem>
14865 <listitem><para>
14866 <emphasis>Push Your Commits to a "Contrib" Upstream:</emphasis>
14867 If you have arranged for permissions to push to an
14868 upstream contrib repository, push the change to that
14869 repository:
14870 <literallayout class='monospaced'>
14871 $ git push <replaceable>upstream_remote_repo</replaceable> <replaceable>local_branch_name</replaceable>
14872 </literallayout>
14873 For example, suppose you have permissions to push into the
14874 upstream <filename>meta-intel-contrib</filename>
14875 repository and you are working in a local branch named
14876 <replaceable>your_name</replaceable><filename>/README</filename>.
14877 The following command pushes your local commits to the
14878 <filename>meta-intel-contrib</filename> upstream
14879 repository and puts the commit in a branch named
14880 <replaceable>your_name</replaceable><filename>/README</filename>:
14881 <literallayout class='monospaced'>
14882 $ git push meta-intel-contrib <replaceable>your_name</replaceable>/README
14883 </literallayout>
14884 </para></listitem>
14885 <listitem><para id='push-determine-who-to-notify'>
14886 <emphasis>Determine Who to Notify:</emphasis>
14887 Determine the maintainer or the mailing list
14888 that you need to notify for the change.</para>
14889
14890 <para>Before submitting any change, you need to be sure
14891 who the maintainer is or what mailing list that you need
14892 to notify.
14893 Use either these methods to find out:
14894 <itemizedlist>
14895 <listitem><para>
14896 <emphasis>Maintenance File:</emphasis>
14897 Examine the <filename>maintainers.inc</filename>
14898 file, which is located in the
14899 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
14900 at
14901 <filename>meta/conf/distro/include</filename>,
14902 to see who is responsible for code.
14903 </para></listitem>
14904 <listitem><para>
14905 <emphasis>Search by File:</emphasis>
14906 Using <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>,
14907 you can enter the following command to bring up a
14908 short list of all commits against a specific file:
14909 <literallayout class='monospaced'>
14910 git shortlog -- <replaceable>filename</replaceable>
14911 </literallayout>
14912 Just provide the name of the file for which you
14913 are interested.
14914 The information returned is not ordered by history
14915 but does include a list of everyone who has
14916 committed grouped by name.
14917 From the list, you can see who is responsible for
14918 the bulk of the changes against the file.
14919 </para></listitem>
14920 <listitem><para>
14921 <emphasis>Examine the List of Mailing Lists:</emphasis>
14922 For a list of the Yocto Project and related mailing
14923 lists, see the
14924 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>"
14925 section in the Yocto Project Reference Manual.
14926 </para></listitem>
14927 </itemizedlist>
14928 </para></listitem>
14929 <listitem><para>
14930 <emphasis>Make a Pull Request:</emphasis>
14931 Notify the maintainer or the mailing list that you have
14932 pushed a change by making a pull request.</para>
14933
14934 <para>The Yocto Project provides two scripts that
14935 conveniently let you generate and send pull requests to the
14936 Yocto Project.
14937 These scripts are <filename>create-pull-request</filename>
14938 and <filename>send-pull-request</filename>.
14939 You can find these scripts in the
14940 <filename>scripts</filename> directory within the
14941 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
14942 (e.g. <filename>~/poky/scripts</filename>).
14943 </para>
14944
14945 <para>Using these scripts correctly formats the requests
14946 without introducing any whitespace or HTML formatting.
14947 The maintainer that receives your patches either directly
14948 or through the mailing list needs to be able to save and
14949 apply them directly from your emails.
14950 Using these scripts is the preferred method for sending
14951 patches.</para>
14952
14953 <para>First, create the pull request.
14954 For example, the following command runs the script,
14955 specifies the upstream repository in the contrib directory
14956 into which you pushed the change, and provides a subject
14957 line in the created patch files:
14958 <literallayout class='monospaced'>
14959 $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README"
14960 </literallayout>
14961 Running this script forms
14962 <filename>*.patch</filename> files in a folder named
14963 <filename>pull-</filename><replaceable>PID</replaceable>
14964 in the current directory.
14965 One of the patch files is a cover letter.</para>
14966
14967 <para>Before running the
14968 <filename>send-pull-request</filename> script, you must
14969 edit the cover letter patch to insert information about
14970 your change.
14971 After editing the cover letter, send the pull request.
14972 For example, the following command runs the script and
14973 specifies the patch directory and email address.
14974 In this example, the email address is a mailing list:
14975 <literallayout class='monospaced'>
14976 $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org
14977 </literallayout>
14978 You need to follow the prompts as the script is
14979 interactive.
14980 <note>
14981 For help on using these scripts, simply provide the
14982 <filename>-h</filename> argument as follows:
14983 <literallayout class='monospaced'>
14984 $ poky/scripts/create-pull-request -h
14985 $ poky/scripts/send-pull-request -h
14986 </literallayout>
14987 </note>
14988 </para></listitem>
14989 </orderedlist>
14990 </para>
14991 </section>
14992
14993 <section id='submitting-a-patch'>
14994 <title>Using Email to Submit a Patch</title>
14995
14996 <para>
14997 You can submit patches without using the
14998 <filename>create-pull-request</filename> and
14999 <filename>send-pull-request</filename> scripts described in the
15000 previous section.
15001 However, keep in mind, the preferred method is to use the scripts.
15002 </para>
15003
15004 <para>
15005 Depending on the components changed, you need to submit the email
15006 to a specific mailing list.
15007 For some guidance on which mailing list to use, see the
15008 <link linkend='figuring-out-the-mailing-list-to-use'>list</link>
15009 at the beginning of this section.
15010 For a description of all the available mailing lists, see the
15011 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
15012 section in the Yocto Project Reference Manual.
15013 </para>
15014
15015 <para>
15016 Here is the general procedure on how to submit a patch through
15017 email without using the scripts:
15018 <orderedlist>
15019 <listitem><para>
15020 <emphasis>Make Your Changes Locally:</emphasis>
15021 Make your changes in your local Git repository.
15022 You should make small, controlled, isolated changes.
15023 Keeping changes small and isolated aids review,
15024 makes merging/rebasing easier and keeps the change
15025 history clean should anyone need to refer to it in
15026 future.
15027 </para></listitem>
15028 <listitem><para>
15029 <emphasis>Stage Your Changes:</emphasis>
15030 Stage your changes by using the <filename>git add</filename>
15031 command on each file you changed.
15032 </para></listitem>
15033 <listitem><para>
15034 <emphasis>Commit Your Changes:</emphasis>
15035 Commit the change by using the
15036 <filename>git commit --signoff</filename> command.
15037 Using the <filename>--signoff</filename> option identifies
15038 you as the person making the change and also satisfies
15039 the Developer's Certificate of Origin (DCO) shown earlier.
15040 </para>
15041
15042 <para>When you form a commit, you must follow certain
15043 standards established by the Yocto Project development
15044 team.
15045 See
15046 <link linkend='making-sure-you-have-correct-commit-information'>Step 3</link>
15047 in the previous section for information on how to
15048 provide commit information that meets Yocto Project
15049 commit message standards.
15050 </para></listitem>
15051 <listitem><para>
15052 <emphasis>Format the Commit:</emphasis>
15053 Format the commit into an email message.
15054 To format commits, use the
15055 <filename>git format-patch</filename> command.
15056 When you provide the command, you must include a revision
15057 list or a number of patches as part of the command.
15058 For example, either of these two commands takes your most
15059 recent single commit and formats it as an email message in
15060 the current directory:
15061 <literallayout class='monospaced'>
15062 $ git format-patch -1
15063 </literallayout>
15064 or
15065 <literallayout class='monospaced'>
15066 $ git format-patch HEAD~
15067 </literallayout></para>
15068
15069 <para>After the command is run, the current directory
15070 contains a numbered <filename>.patch</filename> file for
15071 the commit.</para>
15072
15073 <para>If you provide several commits as part of the
15074 command, the <filename>git format-patch</filename> command
15075 produces a series of numbered files in the current
15076 directory – one for each commit.
15077 If you have more than one patch, you should also use the
15078 <filename>--cover</filename> option with the command,
15079 which generates a cover letter as the first "patch" in
15080 the series.
15081 You can then edit the cover letter to provide a
15082 description for the series of patches.
15083 For information on the
15084 <filename>git format-patch</filename> command,
15085 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed
15086 using the <filename>man git-format-patch</filename>
15087 command.
15088 <note>
15089 If you are or will be a frequent contributor to the
15090 Yocto Project or to OpenEmbedded, you might consider
15091 requesting a contrib area and the necessary associated
15092 rights.
15093 </note>
15094 </para></listitem>
15095 <listitem><para>
15096 <emphasis>Import the Files Into Your Mail Client:</emphasis>
15097 Import the files into your mail client by using the
15098 <filename>git send-email</filename> command.
15099 <note>
15100 In order to use <filename>git send-email</filename>,
15101 you must have the proper Git packages installed on
15102 your host.
15103 For Ubuntu, Debian, and Fedora the package is
15104 <filename>git-email</filename>.
15105 </note></para>
15106
15107 <para>The <filename>git send-email</filename> command
15108 sends email by using a local or remote Mail Transport Agent
15109 (MTA) such as <filename>msmtp</filename>,
15110 <filename>sendmail</filename>, or through a direct
15111 <filename>smtp</filename> configuration in your Git
15112 <filename>~/.gitconfig</filename> file.
15113 If you are submitting patches through email only, it is
15114 very important that you submit them without any whitespace
15115 or HTML formatting that either you or your mailer
15116 introduces.
15117 The maintainer that receives your patches needs to be able
15118 to save and apply them directly from your emails.
15119 A good way to verify that what you are sending will be
15120 applicable by the maintainer is to do a dry run and send
15121 them to yourself and then save and apply them as the
15122 maintainer would.</para>
15123
15124 <para>The <filename>git send-email</filename> command is
15125 the preferred method for sending your patches using
15126 email since there is no risk of compromising whitespace
15127 in the body of the message, which can occur when you use
15128 your own mail client.
15129 The command also has several options that let you
15130 specify recipients and perform further editing of the
15131 email message.
15132 For information on how to use the
15133 <filename>git send-email</filename> command,
15134 see <filename>GIT-SEND-EMAIL(1)</filename> displayed using
15135 the <filename>man git-send-email</filename> command.
15136 </para></listitem>
15137 </orderedlist>
15138 </para>
15139 </section>
15140 </section>
15141 </section>
15142
15143 <section id='working-with-licenses'>
15144 <title>Working With Licenses</title>
15145
15146 <para>
15147 As mentioned in the
15148 "<ulink url='&YOCTO_DOCS_OM_URL;#licensing'>Licensing</ulink>"
15149 section in the Yocto Project Overview and Concepts Manual,
15150 open source projects are open to the public and they
15151 consequently have different licensing structures in place.
15152 This section describes the mechanism by which the
15153 <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
15154 tracks changes to licensing text and covers how to maintain open
15155 source license compliance during your project's lifecycle.
15156 The section also describes how to enable commercially licensed
15157 recipes, which by default are disabled.
15158 </para>
15159
15160 <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
15161 <title>Tracking License Changes</title>
15162
15163 <para>
15164 The license of an upstream project might change in the future.
15165 In order to prevent these changes going unnoticed, the
15166 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
15167 variable tracks changes to the license text. The checksums are
15168 validated at the end of the configure step, and if the
15169 checksums do not match, the build will fail.
15170 </para>
15171
15172 <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
15173 <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
15174
15175 <para>
15176 The <filename>LIC_FILES_CHKSUM</filename>
15177 variable contains checksums of the license text in the
15178 source code for the recipe.
15179 Following is an example of how to specify
15180 <filename>LIC_FILES_CHKSUM</filename>:
15181 <literallayout class='monospaced'>
15182 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
15183 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
15184 file://licfile2.txt;endline=50;md5=zzzz \
15185 ..."
15186 </literallayout>
15187 <note><title>Notes</title>
15188 <itemizedlist>
15189 <listitem><para>
15190 When using "beginline" and "endline", realize
15191 that line numbering begins with one and not
15192 zero.
15193 Also, the included lines are inclusive (i.e.
15194 lines five through and including 29 in the
15195 previous example for
15196 <filename>licfile1.txt</filename>).
15197 </para></listitem>
15198 <listitem><para>
15199 When a license check fails, the selected license
15200 text is included as part of the QA message.
15201 Using this output, you can determine the exact
15202 start and finish for the needed license text.
15203 </para></listitem>
15204 </itemizedlist>
15205 </note>
15206 </para>
15207
15208 <para>
15209 The build system uses the
15210 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
15211 variable as the default directory when searching files
15212 listed in <filename>LIC_FILES_CHKSUM</filename>.
15213 The previous example employs the default directory.
15214 </para>
15215
15216 <para>
15217 Consider this next example:
15218 <literallayout class='monospaced'>
15219 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
15220 md5=bb14ed3c4cda583abc85401304b5cd4e"
15221 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
15222 </literallayout>
15223 </para>
15224
15225 <para>
15226 The first line locates a file in
15227 <filename>${S}/src/ls.c</filename> and isolates lines five
15228 through 16 as license text.
15229 The second line refers to a file in
15230 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.
15231 </para>
15232
15233 <para>
15234 Note that <filename>LIC_FILES_CHKSUM</filename> variable is
15235 mandatory for all recipes, unless the
15236 <filename>LICENSE</filename> variable is set to "CLOSED".
15237 </para>
15238 </section>
15239
15240 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
15241 <title>Explanation of Syntax</title>
15242
15243 <para>
15244 As mentioned in the previous section, the
15245 <filename>LIC_FILES_CHKSUM</filename> variable lists all
15246 the important files that contain the license text for the
15247 source code.
15248 It is possible to specify a checksum for an entire file,
15249 or a specific section of a file (specified by beginning and
15250 ending line numbers with the "beginline" and "endline"
15251 parameters, respectively).
15252 The latter is useful for source files with a license
15253 notice header, README documents, and so forth.
15254 If you do not use the "beginline" parameter, then it is
15255 assumed that the text begins on the first line of the file.
15256 Similarly, if you do not use the "endline" parameter,
15257 it is assumed that the license text ends with the last
15258 line of the file.
15259 </para>
15260
15261 <para>
15262 The "md5" parameter stores the md5 checksum of the license
15263 text.
15264 If the license text changes in any way as compared to
15265 this parameter then a mismatch occurs.
15266 This mismatch triggers a build failure and notifies
15267 the developer.
15268 Notification allows the developer to review and address
15269 the license text changes.
15270 Also note that if a mismatch occurs during the build,
15271 the correct md5 checksum is placed in the build log and
15272 can be easily copied to the recipe.
15273 </para>
15274
15275 <para>
15276 There is no limit to how many files you can specify using
15277 the <filename>LIC_FILES_CHKSUM</filename> variable.
15278 Generally, however, every project requires a few
15279 specifications for license tracking.
15280 Many projects have a "COPYING" file that stores the
15281 license information for all the source code files.
15282 This practice allows you to just track the "COPYING"
15283 file as long as it is kept up to date.
15284 <note><title>Tips</title>
15285 <itemizedlist>
15286 <listitem><para>
15287 If you specify an empty or invalid "md5"
15288 parameter,
15289 <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
15290 returns an md5 mis-match
15291 error and displays the correct "md5" parameter
15292 value during the build.
15293 The correct parameter is also captured in
15294 the build log.
15295 </para></listitem>
15296 <listitem><para>
15297 If the whole file contains only license text,
15298 you do not need to use the "beginline" and
15299 "endline" parameters.
15300 </para></listitem>
15301 </itemizedlist>
15302 </note>
15303 </para>
15304 </section>
15305 </section>
15306
15307 <section id="enabling-commercially-licensed-recipes">
15308 <title>Enabling Commercially Licensed Recipes</title>
15309
15310 <para>
15311 By default, the OpenEmbedded build system disables
15312 components that have commercial or other special licensing
15313 requirements.
15314 Such requirements are defined on a
15315 recipe-by-recipe basis through the
15316 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></ulink>
15317 variable definition in the affected recipe.
15318 For instance, the
15319 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
15320 recipe contains the following statement:
15321 <literallayout class='monospaced'>
15322 LICENSE_FLAGS = "commercial"
15323 </literallayout>
15324 Here is a slightly more complicated example that contains both
15325 an explicit recipe name and version (after variable expansion):
15326 <literallayout class='monospaced'>
15327 LICENSE_FLAGS = "license_${PN}_${PV}"
15328 </literallayout>
15329 In order for a component restricted by a
15330 <filename>LICENSE_FLAGS</filename> definition to be enabled and
15331 included in an image, it needs to have a matching entry in the
15332 global
15333 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></ulink>
15334 variable, which is a variable typically defined in your
15335 <filename>local.conf</filename> file.
15336 For example, to enable the
15337 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
15338 package, you could add either the string
15339 "commercial_gst-plugins-ugly" or the more general string
15340 "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
15341 See the
15342 "<link linkend='license-flag-matching'>License Flag Matching</link>"
15343 section for a full
15344 explanation of how <filename>LICENSE_FLAGS</filename> matching
15345 works.
15346 Here is the example:
15347 <literallayout class='monospaced'>
15348 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
15349 </literallayout>
15350 Likewise, to additionally enable the package built from the
15351 recipe containing
15352 <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>,
15353 and assuming that the actual recipe name was
15354 <filename>emgd_1.10.bb</filename>, the following string would
15355 enable that package as well as the original
15356 <filename>gst-plugins-ugly</filename> package:
15357 <literallayout class='monospaced'>
15358 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
15359 </literallayout>
15360 As a convenience, you do not need to specify the complete
15361 license string in the whitelist for every package.
15362 You can use an abbreviated form, which consists
15363 of just the first portion or portions of the license
15364 string before the initial underscore character or characters.
15365 A partial string will match any license that contains the
15366 given string as the first portion of its license.
15367 For example, the following whitelist string will also match
15368 both of the packages previously mentioned as well as any other
15369 packages that have licenses starting with "commercial" or
15370 "license".
15371 <literallayout class='monospaced'>
15372 LICENSE_FLAGS_WHITELIST = "commercial license"
15373 </literallayout>
15374 </para>
15375
15376 <section id="license-flag-matching">
15377 <title>License Flag Matching</title>
15378
15379 <para>
15380 License flag matching allows you to control what recipes
15381 the OpenEmbedded build system includes in the build.
15382 Fundamentally, the build system attempts to match
15383 <filename>LICENSE_FLAGS</filename> strings found in recipes
15384 against <filename>LICENSE_FLAGS_WHITELIST</filename>
15385 strings found in the whitelist.
15386 A match causes the build system to include a recipe in the
15387 build, while failure to find a match causes the build
15388 system to exclude a recipe.
15389 </para>
15390
15391 <para>
15392 In general, license flag matching is simple.
15393 However, understanding some concepts will help you
15394 correctly and effectively use matching.
15395 </para>
15396
15397 <para>
15398 Before a flag
15399 defined by a particular recipe is tested against the
15400 contents of the whitelist, the expanded string
15401 <filename>_${PN}</filename> is appended to the flag.
15402 This expansion makes each
15403 <filename>LICENSE_FLAGS</filename> value recipe-specific.
15404 After expansion, the string is then matched against the
15405 whitelist.
15406 Thus, specifying
15407 <filename>LICENSE_FLAGS = "commercial"</filename>
15408 in recipe "foo", for example, results in the string
15409 <filename>"commercial_foo"</filename>.
15410 And, to create a match, that string must appear in the
15411 whitelist.
15412 </para>
15413
15414 <para>
15415 Judicious use of the <filename>LICENSE_FLAGS</filename>
15416 strings and the contents of the
15417 <filename>LICENSE_FLAGS_WHITELIST</filename> variable
15418 allows you a lot of flexibility for including or excluding
15419 recipes based on licensing.
15420 For example, you can broaden the matching capabilities by
15421 using license flags string subsets in the whitelist.
15422 <note>
15423 When using a string subset, be sure to use the part of
15424 the expanded string that precedes the appended
15425 underscore character (e.g.
15426 <filename>usethispart_1.3</filename>,
15427 <filename>usethispart_1.4</filename>, and so forth).
15428 </note>
15429 For example, simply specifying the string "commercial" in
15430 the whitelist matches any expanded
15431 <filename>LICENSE_FLAGS</filename> definition that starts
15432 with the string "commercial" such as "commercial_foo" and
15433 "commercial_bar", which are the strings the build system
15434 automatically generates for hypothetical recipes named
15435 "foo" and "bar" assuming those recipes simply specify the
15436 following:
15437 <literallayout class='monospaced'>
15438 LICENSE_FLAGS = "commercial"
15439 </literallayout>
15440 Thus, you can choose to exhaustively
15441 enumerate each license flag in the whitelist and
15442 allow only specific recipes into the image, or
15443 you can use a string subset that causes a broader range of
15444 matches to allow a range of recipes into the image.
15445 </para>
15446
15447 <para>
15448 This scheme works even if the
15449 <filename>LICENSE_FLAGS</filename> string already
15450 has <filename>_${PN}</filename> appended.
15451 For example, the build system turns the license flag
15452 "commercial_1.2_foo" into "commercial_1.2_foo_foo" and
15453 would match both the general "commercial" and the specific
15454 "commercial_1.2_foo" strings found in the whitelist, as
15455 expected.
15456 </para>
15457
15458 <para>
15459 Here are some other scenarios:
15460 <itemizedlist>
15461 <listitem><para>
15462 You can specify a versioned string in the recipe
15463 such as "commercial_foo_1.2" in a "foo" recipe.
15464 The build system expands this string to
15465 "commercial_foo_1.2_foo".
15466 Combine this license flag with a whitelist that has
15467 the string "commercial" and you match the flag
15468 along with any other flag that starts with the
15469 string "commercial".
15470 </para></listitem>
15471 <listitem><para>
15472 Under the same circumstances, you can use
15473 "commercial_foo" in the whitelist and the build
15474 system not only matches "commercial_foo_1.2" but
15475 also matches any license flag with the string
15476 "commercial_foo", regardless of the version.
15477 </para></listitem>
15478 <listitem><para>
15479 You can be very specific and use both the
15480 package and version parts in the whitelist (e.g.
15481 "commercial_foo_1.2") to specifically match a
15482 versioned recipe.
15483 </para></listitem>
15484 </itemizedlist>
15485 </para>
15486 </section>
15487
15488 <section id="other-variables-related-to-commercial-licenses">
15489 <title>Other Variables Related to Commercial Licenses</title>
15490
15491 <para>
15492 Other helpful variables related to commercial
15493 license handling exist and are defined in the
15494 <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
15495 <literallayout class='monospaced'>
15496 COMMERCIAL_AUDIO_PLUGINS ?= ""
15497 COMMERCIAL_VIDEO_PLUGINS ?= ""
15498 </literallayout>
15499 If you want to enable these components, you can do so by
15500 making sure you have statements similar to the following
15501 in your <filename>local.conf</filename> configuration file:
15502 <literallayout class='monospaced'>
15503 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
15504 gst-plugins-ugly-mpegaudioparse"
15505 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
15506 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
15507 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
15508 </literallayout>
15509 Of course, you could also create a matching whitelist
15510 for those components using the more general "commercial"
15511 in the whitelist, but that would also enable all the
15512 other packages with <filename>LICENSE_FLAGS</filename>
15513 containing "commercial", which you may or may not want:
15514 <literallayout class='monospaced'>
15515 LICENSE_FLAGS_WHITELIST = "commercial"
15516 </literallayout>
15517 </para>
15518
15519 <para>
15520 Specifying audio and video plugins as part of the
15521 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
15522 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
15523 (along with the enabling
15524 <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
15525 plugins or components into built images, thus adding
15526 support for media formats or components.
15527 </para>
15528 </section>
15529 </section>
15530
15531 <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'>
15532 <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title>
15533
15534 <para>
15535 One of the concerns for a development organization using open source
15536 software is how to maintain compliance with various open source
15537 licensing during the lifecycle of the product.
15538 While this section does not provide legal advice or
15539 comprehensively cover all scenarios, it does
15540 present methods that you can use to
15541 assist you in meeting the compliance requirements during a software
15542 release.
15543 </para>
15544
15545 <para>
15546 With hundreds of different open source licenses that the Yocto
15547 Project tracks, it is difficult to know the requirements of each
15548 and every license.
15549 However, the requirements of the major FLOSS licenses can begin
15550 to be covered by
15551 assuming that three main areas of concern exist:
15552 <itemizedlist>
15553 <listitem><para>Source code must be provided.</para></listitem>
15554 <listitem><para>License text for the software must be
15555 provided.</para></listitem>
15556 <listitem><para>Compilation scripts and modifications to the
15557 source code must be provided.
15558 </para></listitem>
15559 </itemizedlist>
15560 There are other requirements beyond the scope of these
15561 three and the methods described in this section
15562 (e.g. the mechanism through which source code is distributed).
15563 </para>
15564
15565 <para>
15566 As different organizations have different methods of complying with
15567 open source licensing, this section is not meant to imply that
15568 there is only one single way to meet your compliance obligations,
15569 but rather to describe one method of achieving compliance.
15570 The remainder of this section describes methods supported to meet the
15571 previously mentioned three requirements.
15572 Once you take steps to meet these requirements,
15573 and prior to releasing images, sources, and the build system,
15574 you should audit all artifacts to ensure completeness.
15575 <note>
15576 The Yocto Project generates a license manifest during
15577 image creation that is located
15578 in <filename>${DEPLOY_DIR}/licenses/<replaceable>image_name-datestamp</replaceable></filename>
15579 to assist with any audits.
15580 </note>
15581 </para>
15582
15583 <section id='providing-the-source-code'>
15584 <title>Providing the Source Code</title>
15585
15586 <para>
15587 Compliance activities should begin before you generate the
15588 final image.
15589 The first thing you should look at is the requirement that
15590 tops the list for most compliance groups - providing
15591 the source.
15592 The Yocto Project has a few ways of meeting this
15593 requirement.
15594 </para>
15595
15596 <para>
15597 One of the easiest ways to meet this requirement is
15598 to provide the entire
15599 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
15600 used by the build.
15601 This method, however, has a few issues.
15602 The most obvious is the size of the directory since it includes
15603 all sources used in the build and not just the source used in
15604 the released image.
15605 It will include toolchain source, and other artifacts, which
15606 you would not generally release.
15607 However, the more serious issue for most companies is accidental
15608 release of proprietary software.
15609 The Yocto Project provides an
15610 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'><filename>archiver</filename></ulink>
15611 class to help avoid some of these concerns.
15612 </para>
15613
15614 <para>
15615 Before you employ <filename>DL_DIR</filename> or the
15616 <filename>archiver</filename> class, you need to decide how
15617 you choose to provide source.
15618 The source <filename>archiver</filename> class can generate
15619 tarballs and SRPMs and can create them with various levels of
15620 compliance in mind.
15621 </para>
15622
15623 <para>
15624 One way of doing this (but certainly not the only way) is to
15625 release just the source as a tarball.
15626 You can do this by adding the following to the
15627 <filename>local.conf</filename> file found in the
15628 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>:
15629 <literallayout class='monospaced'>
15630 INHERIT += "archiver"
15631 ARCHIVER_MODE[src] = "original"
15632 </literallayout>
15633 During the creation of your image, the source from all
15634 recipes that deploy packages to the image is placed within
15635 subdirectories of
15636 <filename>DEPLOY_DIR/sources</filename> based on the
15637 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
15638 for each recipe.
15639 Releasing the entire directory enables you to comply with
15640 requirements concerning providing the unmodified source.
15641 It is important to note that the size of the directory can
15642 get large.
15643 </para>
15644
15645 <para>
15646 A way to help mitigate the size issue is to only release
15647 tarballs for licenses that require the release of
15648 source.
15649 Let us assume you are only concerned with GPL code as
15650 identified by running the following script:
15651 <literallayout class='monospaced'>
15652 # Script to archive a subset of packages matching specific license(s)
15653 # Source and license files are copied into sub folders of package folder
15654 # Must be run from build folder
15655 #!/bin/bash
15656 src_release_dir="source-release"
15657 mkdir -p $src_release_dir
15658 for a in tmp/deploy/sources/*; do
15659 for d in $a/*; do
15660 # Get package name from path
15661 p=`basename $d`
15662 p=${p%-*}
15663 p=${p%-*}
15664 # Only archive GPL packages (update *GPL* regex for your license check)
15665 numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
15666 if [ $numfiles -gt 1 ]; then
15667 echo Archiving $p
15668 mkdir -p $src_release_dir/$p/source
15669 cp $d/* $src_release_dir/$p/source 2> /dev/null
15670 mkdir -p $src_release_dir/$p/license
15671 cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
15672 fi
15673 done
15674 done
15675 </literallayout>
15676 At this point, you could create a tarball from the
15677 <filename>gpl_source_release</filename> directory and
15678 provide that to the end user.
15679 This method would be a step toward achieving compliance
15680 with section 3a of GPLv2 and with section 6 of GPLv3.
15681 </para>
15682 </section>
15683
15684 <section id='providing-license-text'>
15685 <title>Providing License Text</title>
15686
15687 <para>
15688 One requirement that is often overlooked is inclusion
15689 of license text.
15690 This requirement also needs to be dealt with prior to
15691 generating the final image.
15692 Some licenses require the license text to accompany
15693 the binary.
15694 You can achieve this by adding the following to your
15695 <filename>local.conf</filename> file:
15696 <literallayout class='monospaced'>
15697 COPY_LIC_MANIFEST = "1"
15698 COPY_LIC_DIRS = "1"
15699 LICENSE_CREATE_PACKAGE = "1"
15700 </literallayout>
15701 Adding these statements to the configuration file ensures
15702 that the licenses collected during package generation
15703 are included on your image.
15704 <note>
15705 <para>Setting all three variables to "1" results in the
15706 image having two copies of the same license file.
15707 One copy resides in
15708 <filename>/usr/share/common-licenses</filename> and
15709 the other resides in
15710 <filename>/usr/share/license</filename>.</para>
15711
15712 <para>The reason for this behavior is because
15713 <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS'><filename>COPY_LIC_DIRS</filename></ulink>
15714 and
15715 <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST'><filename>COPY_LIC_MANIFEST</filename></ulink>
15716 add a copy of the license when the image is built but do
15717 not offer a path for adding licenses for newly installed
15718 packages to an image.
15719 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE'><filename>LICENSE_CREATE_PACKAGE</filename></ulink>
15720 adds a separate package and an upgrade path for adding
15721 licenses to an image.</para>
15722 </note>
15723 </para>
15724
15725 <para>
15726 As the source <filename>archiver</filename> class has already
15727 archived the original
15728 unmodified source that contains the license files,
15729 you would have already met the requirements for inclusion
15730 of the license information with source as defined by the GPL
15731 and other open source licenses.
15732 </para>
15733 </section>
15734
15735 <section id='providing-compilation-scripts-and-source-code-modifications'>
15736 <title>Providing Compilation Scripts and Source Code Modifications</title>
15737
15738 <para>
15739 At this point, we have addressed all we need to
15740 prior to generating the image.
15741 The next two requirements are addressed during the final
15742 packaging of the release.
15743 </para>
15744
15745 <para>
15746 By releasing the version of the OpenEmbedded build system
15747 and the layers used during the build, you will be providing both
15748 compilation scripts and the source code modifications in one
15749 step.
15750 </para>
15751
15752 <para>
15753 If the deployment team has a
15754 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink>
15755 and a distro layer, and those those layers are used to patch,
15756 compile, package, or modify (in any way) any open source
15757 software included in your released images, you
15758 might be required to release those layers under section 3 of
15759 GPLv2 or section 1 of GPLv3.
15760 One way of doing that is with a clean
15761 checkout of the version of the Yocto Project and layers used
15762 during your build.
15763 Here is an example:
15764 <literallayout class='monospaced'>
15765 # We built using the &DISTRO_NAME_NO_CAP; branch of the poky repo
15766 $ git clone -b &DISTRO_NAME_NO_CAP; git://git.yoctoproject.org/poky
15767 $ cd poky
15768 # We built using the release_branch for our layers
15769 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
15770 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
15771 # clean up the .git repos
15772 $ find . -name ".git" -type d -exec rm -rf {} \;
15773 </literallayout>
15774 One thing a development organization might want to consider
15775 for end-user convenience is to modify
15776 <filename>meta-poky/conf/bblayers.conf.sample</filename> to
15777 ensure that when the end user utilizes the released build
15778 system to build an image, the development organization's
15779 layers are included in the <filename>bblayers.conf</filename>
15780 file automatically:
15781 <literallayout class='monospaced'>
15782 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
15783 # changes incompatibly
15784 POKY_BBLAYERS_CONF_VERSION = "2"
15785
15786 BBPATH = "${TOPDIR}"
15787 BBFILES ?= ""
15788
15789 BBLAYERS ?= " \
15790 ##OEROOT##/meta \
15791 ##OEROOT##/meta-poky \
15792 ##OEROOT##/meta-yocto-bsp \
15793 ##OEROOT##/meta-mylayer \
15794 "
15795 </literallayout>
15796 Creating and providing an archive of the
15797 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>
15798 layers (recipes, configuration files, and so forth)
15799 enables you to meet your
15800 requirements to include the scripts to control compilation
15801 as well as any modifications to the original source.
15802 </para>
15803 </section>
15804 </section>
15805
15806 <section id='copying-licenses-that-do-not-exist'>
15807 <title>Copying Licenses that Do Not Exist</title>
15808
15809 <para>
15810 Some packages, such as the linux-firmware package, have many
15811 licenses that are not in any way common.
15812 You can avoid adding a lot of these types of common license
15813 files, which are only applicable to a specific package, by using
15814 the
15815 <ulink url='&YOCTO_DOCS_REF_URL;#var-NO_GENERIC_LICENSE'><filename>NO_GENERIC_LICENSE</filename></ulink>
15816 variable.
15817 Using this variable also avoids QA errors when you use a
15818 non-common, non-CLOSED license in a recipe.
15819 </para>
15820
15821 <para>
15822 The following is an example that uses the
15823 <filename>LICENSE.Abilis.txt</filename>
15824 file as the license from the fetched source:
15825 <literallayout class='monospaced'>
15826 NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
15827 </literallayout>
15828 </para>
15829 </section>
15830 </section>
15831
15832 <section id='using-the-error-reporting-tool'>
15833 <title>Using the Error Reporting Tool</title>
15834
15835 <para>
15836 The error reporting tool allows you to
15837 submit errors encountered during builds to a central database.
15838 Outside of the build environment, you can use a web interface to
15839 browse errors, view statistics, and query for errors.
15840 The tool works using a client-server system where the client
15841 portion is integrated with the installed Yocto Project
15842 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
15843 (e.g. <filename>poky</filename>).
15844 The server receives the information collected and saves it in a
15845 database.
15846 </para>
15847
15848 <para>
15849 A live instance of the error reporting server exists at
15850 <ulink url='http://errors.yoctoproject.org'></ulink>.
15851 This server exists so that when you want to get help with
15852 build failures, you can submit all of the information on the
15853 failure easily and then point to the URL in your bug report
15854 or send an email to the mailing list.
15855 <note>
15856 If you send error reports to this server, the reports become
15857 publicly visible.
15858 </note>
15859 </para>
15860
15861 <section id='enabling-and-using-the-tool'>
15862 <title>Enabling and Using the Tool</title>
15863
15864 <para>
15865 By default, the error reporting tool is disabled.
15866 You can enable it by inheriting the
15867 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-report-error'><filename>report-error</filename></ulink>
15868 class by adding the following statement to the end of
15869 your <filename>local.conf</filename> file in your
15870 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
15871 <literallayout class='monospaced'>
15872 INHERIT += "report-error"
15873 </literallayout>
15874 </para>
15875
15876 <para>
15877 By default, the error reporting feature stores information in
15878 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LOG_DIR'><filename>LOG_DIR</filename></ulink><filename>}/error-report</filename>.
15879 However, you can specify a directory to use by adding the following
15880 to your <filename>local.conf</filename> file:
15881 <literallayout class='monospaced'>
15882 ERR_REPORT_DIR = "path"
15883 </literallayout>
15884 Enabling error reporting causes the build process to collect
15885 the errors and store them in a file as previously described.
15886 When the build system encounters an error, it includes a
15887 command as part of the console output.
15888 You can run the command to send the error file to the server.
15889 For example, the following command sends the errors to an
15890 upstream server:
15891 <literallayout class='monospaced'>
15892 $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
15893 </literallayout>
15894 In the previous example, the errors are sent to a public
15895 database available at
15896 <ulink url='http://errors.yoctoproject.org'></ulink>, which is
15897 used by the entire community.
15898 If you specify a particular server, you can send the errors
15899 to a different database.
15900 Use the following command for more information on available
15901 options:
15902 <literallayout class='monospaced'>
15903 $ send-error-report --help
15904 </literallayout>
15905 </para>
15906
15907 <para>
15908 When sending the error file, you are prompted to review the
15909 data being sent as well as to provide a name and optional
15910 email address.
15911 Once you satisfy these prompts, the command returns a link
15912 from the server that corresponds to your entry in the database.
15913 For example, here is a typical link:
15914 <literallayout class='monospaced'>
15915 http://errors.yoctoproject.org/Errors/Details/9522/
15916 </literallayout>
15917 Following the link takes you to a web interface where you can
15918 browse, query the errors, and view statistics.
15919 </para>
15920 </section>
15921
15922 <section id='disabling-the-tool'>
15923 <title>Disabling the Tool</title>
15924
15925 <para>
15926 To disable the error reporting feature, simply remove or comment
15927 out the following statement from the end of your
15928 <filename>local.conf</filename> file in your
15929 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
15930 <literallayout class='monospaced'>
15931 INHERIT += "report-error"
15932 </literallayout>
15933 </para>
15934 </section>
15935
15936 <section id='setting-up-your-own-error-reporting-server'>
15937 <title>Setting Up Your Own Error Reporting Server</title>
15938
15939 <para>
15940 If you want to set up your own error reporting server, you
15941 can obtain the code from the Git repository at
15942 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/'></ulink>.
15943 Instructions on how to set it up are in the README document.
15944 </para>
15945 </section>
15946 </section>
15947
15948 <section id="dev-using-wayland-and-weston">
15949 <title>Using Wayland and Weston</title>
15950
15951 <para>
15952 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
15953 is a computer display server protocol that
15954 provides a method for compositing window managers to communicate
15955 directly with applications and video hardware and expects them to
15956 communicate with input hardware using other libraries.
15957 Using Wayland with supporting targets can result in better control
15958 over graphics frame rendering than an application might otherwise
15959 achieve.
15960 </para>
15961
15962 <para>
15963 The Yocto Project provides the Wayland protocol libraries and the
15964 reference
15965 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
15966 compositor as part of its release.
15967 You can find the integrated packages in the
15968 <filename>meta</filename> layer of the
15969 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
15970 Specifically, you can find the recipes that build both Wayland
15971 and Weston at <filename>meta/recipes-graphics/wayland</filename>.
15972 </para>
15973
15974 <para>
15975 You can build both the Wayland and Weston packages for use only
15976 with targets that accept the
15977 <ulink url='https://en.wikipedia.org/wiki/Mesa_(computer_graphics)'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
15978 which is also known as Mesa DRI.
15979 This implies that you cannot build and use the packages if your
15980 target uses, for example, the
15981 <trademark class='registered'>Intel</trademark> Embedded Media
15982 and Graphics Driver
15983 (<trademark class='registered'>Intel</trademark> EMGD) that
15984 overrides Mesa DRI.
15985 <note>
15986 Due to lack of EGL support, Weston 1.0.3 will not run
15987 directly on the emulated QEMU hardware.
15988 However, this version of Weston will run under X emulation
15989 without issues.
15990 </note>
15991 </para>
15992
15993 <para>
15994 This section describes what you need to do to implement Wayland and
15995 use the Weston compositor when building an image for a supporting
15996 target.
15997 </para>
15998
15999 <section id="enabling-wayland-in-an-image">
16000 <title>Enabling Wayland in an Image</title>
16001
16002 <para>
16003 To enable Wayland, you need to enable it to be built and enable
16004 it to be included (installed) in the image.
16005 </para>
16006
16007 <section id="enable-building">
16008 <title>Building</title>
16009
16010 <para>
16011 To cause Mesa to build the <filename>wayland-egl</filename>
16012 platform and Weston to build Wayland with Kernel Mode
16013 Setting
16014 (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
16015 support, include the "wayland" flag in the
16016 <ulink url="&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></ulink>
16017 statement in your <filename>local.conf</filename> file:
16018 <literallayout class='monospaced'>
16019 DISTRO_FEATURES_append = " wayland"
16020 </literallayout>
16021 <note>
16022 If X11 has been enabled elsewhere, Weston will build
16023 Wayland with X11 support
16024 </note>
16025 </para>
16026 </section>
16027
16028 <section id="enable-installation-in-an-image">
16029 <title>Installing</title>
16030
16031 <para>
16032 To install the Wayland feature into an image, you must
16033 include the following
16034 <ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></ulink>
16035 statement in your <filename>local.conf</filename> file:
16036 <literallayout class='monospaced'>
16037 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
16038 </literallayout>
16039 </para>
16040 </section>
16041 </section>
16042
16043 <section id="running-weston">
16044 <title>Running Weston</title>
16045
16046 <para>
16047 To run Weston inside X11, enabling it as described earlier and
16048 building a Sato image is sufficient.
16049 If you are running your image under Sato, a Weston Launcher
16050 appears in the "Utility" category.
16051 </para>
16052
16053 <para>
16054 Alternatively, you can run Weston through the command-line
16055 interpretor (CLI), which is better suited for development work.
16056 To run Weston under the CLI, you need to do the following after
16057 your image is built:
16058 <orderedlist>
16059 <listitem><para>
16060 Run these commands to export
16061 <filename>XDG_RUNTIME_DIR</filename>:
16062 <literallayout class='monospaced'>
16063 mkdir -p /tmp/$USER-weston
16064 chmod 0700 /tmp/$USER-weston
16065 export XDG_RUNTIME_DIR=/tmp/$USER-weston
16066 </literallayout>
16067 </para></listitem>
16068 <listitem><para>
16069 Launch Weston in the shell:
16070 <literallayout class='monospaced'>
16071 weston
16072 </literallayout></para></listitem>
16073 </orderedlist>
16074 </para>
16075 </section>
16076 </section>
16077</chapter>
16078
16079<!--
16080vim: expandtab tw=80 ts=4
16081-->
diff --git a/documentation/dev-manual/dev-manual-customization.xsl b/documentation/dev-manual/dev-manual-customization.xsl
deleted file mode 100644
index 6b16bcabf9..0000000000
--- a/documentation/dev-manual/dev-manual-customization.xsl
+++ /dev/null
@@ -1,29 +0,0 @@
1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
5
6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
7
8<!--
9
10 <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
11
12 <xsl:import href="http://docbook.sourceforge.net/release/xsl/1.76.1/xhtml/docbook.xsl" />
13
14-->
15
16 <xsl:include href="../template/permalinks.xsl"/>
17 <xsl:include href="../template/section.title.xsl"/>
18 <xsl:include href="../template/component.title.xsl"/>
19 <xsl:include href="../template/division.title.xsl"/>
20 <xsl:include href="../template/formal.object.heading.xsl"/>
21
22 <xsl:param name="html.stylesheet" select="'dev-style.css'" />
23 <xsl:param name="chapter.autolabel" select="1" />
24 <xsl:param name="appendix.autolabel" select="A" />
25 <xsl:param name="section.autolabel" select="1" />
26 <xsl:param name="section.label.includes.component.label" select="1" />
27 <xsl:param name="generate.id.attributes" select="1" />
28
29</xsl:stylesheet>
diff --git a/documentation/dev-manual/dev-manual-intro.xml b/documentation/dev-manual/dev-manual-intro.xml
deleted file mode 100644
index 38de5e4f53..0000000000
--- a/documentation/dev-manual/dev-manual-intro.xml
+++ /dev/null
@@ -1,104 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='dev-manual-intro'>
7
8<title>The Yocto Project Development Tasks Manual</title>
9 <section id='dev-welcome'>
10 <title>Welcome</title>
11
12 <para>
13 Welcome to the Yocto Project Development Tasks Manual!
14 This manual provides relevant procedures necessary for developing
15 in the Yocto Project environment (i.e. developing embedded Linux
16 images and user-space applications that run on targeted devices).
17 The manual groups related procedures into higher-level sections.
18 Procedures can consist of high-level steps or low-level steps
19 depending on the topic.
20 </para>
21
22 <para>
23 This manual provides the following:
24 <itemizedlist>
25 <listitem><para>
26 Procedures that help you get going with the Yocto Project.
27 For example, procedures that show you how to set up
28 a build host and work with the Yocto Project
29 source repositories.
30 </para></listitem>
31 <listitem><para>
32 Procedures that show you how to submit changes to the
33 Yocto Project.
34 Changes can be improvements, new features, or bug
35 fixes.
36 </para></listitem>
37 <listitem><para>
38 Procedures related to "everyday" tasks you perform while
39 developing images and applications using the Yocto
40 Project.
41 For example, procedures to create a layer, customize an
42 image, write a new recipe, and so forth.
43 </para></listitem>
44 </itemizedlist>
45 </para>
46
47 <para>
48 This manual does not provide the following:
49 <itemizedlist>
50 <listitem><para>
51 Redundant Step-by-step Instructions:
52 For example, the
53 <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
54 manual contains detailed instructions on how to install an
55 SDK, which is used to develop applications for target
56 hardware.
57 </para></listitem>
58 <listitem><para>
59 Reference or Conceptual Material:
60 This type of material resides in an appropriate reference
61 manual.
62 For example, system variables are documented in the
63 <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>.
64 </para></listitem>
65 <listitem><para>
66 Detailed Public Information Not Specific to the
67 Yocto Project:
68 For example, exhaustive information on how to use the
69 Source Control Manager Git is better covered with Internet
70 searches and official Git Documentation than through the
71 Yocto Project documentation.
72 </para></listitem>
73 </itemizedlist>
74 </para>
75 </section>
76
77 <section id='other-information'>
78 <title>Other Information</title>
79
80 <para>
81 Because this manual presents information for many different
82 topics, supplemental information is recommended for full
83 comprehension.
84 For introductory information on the Yocto Project, see the
85 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>.
86 If you want to build an image with no knowledge of Yocto Project
87 as a way of quickly testing it out, see the
88 <ulink url='&YOCTO_DOCS_BRIEF_URL;'>Yocto Project Quick Build</ulink>
89 document.
90 </para>
91
92 <para>
93 For a comprehensive list of links and other documentation, see the
94 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-links-and-related-documentation'>Links and Related Documentation</ulink>"
95 section in the Yocto Project Reference Manual.
96 </para>
97
98 <para>
99 </para>
100 </section>
101</chapter>
102<!--
103vim: expandtab tw=80 ts=4
104-->
diff --git a/documentation/dev-manual/dev-manual-qemu.xml b/documentation/dev-manual/dev-manual-qemu.xml
deleted file mode 100644
index 1a526dd2f5..0000000000
--- a/documentation/dev-manual/dev-manual-qemu.xml
+++ /dev/null
@@ -1,691 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='dev-manual-qemu'>
7
8<title>Using the Quick EMUlator (QEMU)</title>
9
10 <para>
11 The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
12 Open Source project as part of the Yocto Project development "tool
13 set".
14 This chapter provides both procedures that show you how to use the
15 Quick EMUlator (QEMU) and other QEMU information helpful for
16 development purposes.
17 </para>
18
19 <section id='qemu-dev-overview'>
20 <title>Overview</title>
21
22 <para>
23 Within the context of the Yocto Project, QEMU is an
24 emulator and virtualization machine that allows you to run a
25 complete image you have built using the Yocto Project as just
26 another task on your build system.
27 QEMU is useful for running and testing images and applications on
28 supported Yocto Project architectures without having actual
29 hardware.
30 Among other things, the Yocto Project uses QEMU to run automated
31 Quality Assurance (QA) tests on final images shipped with each
32 release.
33 <note>
34 This implementation is not the same as QEMU in general.
35 </note>
36 This section provides a brief reference for the Yocto Project
37 implementation of QEMU.
38 </para>
39
40 <para>
41 For official information and documentation on QEMU in general, see
42 the following references:
43 <itemizedlist>
44 <listitem><para>
45 <emphasis><ulink url='http://wiki.qemu.org/Main_Page'>QEMU Website</ulink>:</emphasis>
46 The official website for the QEMU Open Source project.
47 </para></listitem>
48 <listitem><para>
49 <emphasis><ulink url='http://wiki.qemu.org/Manual'>Documentation</ulink>:</emphasis>
50 The QEMU user manual.
51 </para></listitem>
52 </itemizedlist>
53 </para>
54 </section>
55
56 <section id='qemu-running-qemu'>
57 <title>Running QEMU</title>
58
59 <para>
60 To use QEMU, you need to have QEMU installed and initialized as
61 well as have the proper artifacts (i.e. image files and root
62 filesystems) available.
63 Follow these general steps to run QEMU:
64 <orderedlist>
65 <listitem><para>
66 <emphasis>Install QEMU:</emphasis>
67 QEMU is made available with the Yocto Project a number of
68 ways.
69 One method is to install a Software Development Kit (SDK).
70 See
71 "<ulink url='&YOCTO_DOCS_SDK_URL;#the-qemu-emulator'>The QEMU Emulator</ulink>"
72 section in the Yocto Project Application Development and
73 the Extensible Software Development Kit (eSDK) manual
74 for information on how to install QEMU.
75 </para></listitem>
76 <listitem><para>
77 <emphasis>Setting Up the Environment:</emphasis>
78 How you set up the QEMU environment depends on how you
79 installed QEMU:
80 <itemizedlist>
81 <listitem><para>
82 If you cloned the <filename>poky</filename>
83 repository or you downloaded and unpacked a
84 Yocto Project release tarball, you can source
85 the build environment script (i.e.
86 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>):
87 <literallayout class='monospaced'>
88 $ cd ~/poky
89 $ source oe-init-build-env
90 </literallayout>
91 </para></listitem>
92 <listitem><para>
93 If you installed a cross-toolchain, you can
94 run the script that initializes the toolchain.
95 For example, the following commands run the
96 initialization script from the default
97 <filename>poky_sdk</filename> directory:
98 <literallayout class='monospaced'>
99 . ~/poky_sdk/environment-setup-core2-64-poky-linux
100 </literallayout>
101 </para></listitem>
102 </itemizedlist>
103 </para></listitem>
104 <listitem><para>
105 <emphasis>Ensure the Artifacts are in Place:</emphasis>
106 You need to be sure you have a pre-built kernel that
107 will boot in QEMU.
108 You also need the target root filesystem for your target
109 machine's architecture:
110 <itemizedlist>
111 <listitem><para>
112 If you have previously built an image for QEMU
113 (e.g. <filename>qemux86</filename>,
114 <filename>qemuarm</filename>, and so forth),
115 then the artifacts are in place in your
116 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
117 </para></listitem>
118 <listitem><para>
119 If you have not built an image, you can go to the
120 <ulink url='&YOCTO_MACHINES_DL_URL;'>machines/qemu</ulink>
121 area and download a pre-built image that matches
122 your architecture and can be run on QEMU.
123 </para></listitem>
124 </itemizedlist></para>
125
126 <para>See the
127 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-extracting-the-root-filesystem'>Extracting the Root Filesystem</ulink>"
128 section in the Yocto Project Application Development and
129 the Extensible Software Development Kit (eSDK) manual
130 for information on how to extract a root filesystem.
131 </para></listitem>
132 <listitem><para>
133 <emphasis>Run QEMU:</emphasis>
134 The basic <filename>runqemu</filename> command syntax is as
135 follows:
136 <literallayout class='monospaced'>
137 $ runqemu [<replaceable>option</replaceable> ] [...]
138 </literallayout>
139 Based on what you provide on the command line,
140 <filename>runqemu</filename> does a good job of figuring
141 out what you are trying to do.
142 For example, by default, QEMU looks for the most recently
143 built image according to the timestamp when it needs to
144 look for an image.
145 Minimally, through the use of options, you must provide
146 either a machine name, a virtual machine image
147 (<filename>*wic.vmdk</filename>), or a kernel image
148 (<filename>*.bin</filename>).</para>
149
150 <para>Here are some additional examples to help illustrate
151 further QEMU:
152 <itemizedlist>
153 <listitem><para>
154 This example starts QEMU with
155 <replaceable>MACHINE</replaceable> set to "qemux86-64".
156 Assuming a standard
157 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>,
158 <filename>runqemu</filename> automatically finds the
159 <filename>bzImage-qemux86-64.bin</filename> image file and
160 the
161 <filename>core-image-minimal-qemux86-64-20200218002850.rootfs.ext4</filename>
162 (assuming the current build created a
163 <filename>core-image-minimal</filename> image).
164 <note>
165 When more than one image with the same name exists, QEMU finds
166 and uses the most recently built image according to the
167 timestamp.
168 </note>
169 <literallayout class='monospaced'>
170 $ runqemu qemux86-64
171 </literallayout>
172 </para></listitem>
173 <listitem><para>
174 This example produces the exact same results as the
175 previous example.
176 This command, however, specifically provides the image
177 and root filesystem type.
178 <literallayout class='monospaced'>
179 $ runqemu qemux86-64 core-image-minimal ext4
180 </literallayout>
181 </para></listitem>
182 <listitem><para>
183 This example specifies to boot an initial RAM disk image
184 and to enable audio in QEMU.
185 For this case, <filename>runqemu</filename> set the
186 internal variable <filename>FSTYPE</filename> to
187 "cpio.gz".
188 Also, for audio to be enabled, an appropriate driver must
189 be installed (see the previous description for the
190 <filename>audio</filename> option for more information).
191 <literallayout class='monospaced'>
192 $ runqemu qemux86-64 ramfs audio
193 </literallayout>
194 </para></listitem>
195 <listitem><para>
196 This example does not provide enough information for
197 QEMU to launch.
198 While the command does provide a root filesystem type, it
199 must also minimally provide a
200 <replaceable>MACHINE</replaceable>,
201 <replaceable>KERNEL</replaceable>, or
202 <replaceable>VM</replaceable> option.
203 <literallayout class='monospaced'>
204 $ runqemu ext4
205 </literallayout>
206 </para></listitem>
207 <listitem><para>
208 This example specifies to boot a virtual machine
209 image (<filename>.wic.vmdk</filename> file).
210 From the <filename>.wic.vmdk</filename>,
211 <filename>runqemu</filename> determines the QEMU
212 architecture (<replaceable>MACHINE</replaceable>) to be
213 "qemux86-64" and the root filesystem type to be "vmdk".
214 <literallayout class='monospaced'>
215 $ runqemu /home/scott-lenovo/vm/core-image-minimal-qemux86-64.wic.vmdk
216 </literallayout>
217 </para></listitem>
218 </itemizedlist>
219 </para></listitem>
220 </orderedlist>
221 </para>
222 </section>
223
224 <section id='switching-between-consoles'>
225 <title>Switching Between Consoles</title>
226
227 <para>
228 When booting or running QEMU, you can switch between
229 supported consoles by using
230 Ctrl+Alt+<replaceable>number</replaceable>.
231 For example, Ctrl+Alt+3 switches you to the serial console
232 as long as that console is enabled.
233 Being able to switch consoles is helpful, for example, if
234 the main QEMU console breaks for some reason.
235 <note>
236 Usually, "2" gets you to the main console and "3"
237 gets you to the serial console.
238 </note>
239 </para>
240 </section>
241
242 <section id='removing-the-splash-screen'>
243 <title>Removing the Splash Screen</title>
244
245 <para>
246 You can remove the splash screen when QEMU is booting by
247 using Alt+left.
248 Removing the splash screen allows you to see what is
249 happening in the background.
250 </para>
251 </section>
252
253 <section id='disabling-the-cursor-grab'>
254 <title>Disabling the Cursor Grab</title>
255
256 <para>
257 The default QEMU integration captures the cursor within the
258 main window.
259 It does this since standard mouse devices only provide
260 relative input and not absolute coordinates.
261 You then have to break out of the grab using the "Ctrl+Alt"
262 key combination.
263 However, the Yocto Project's integration of QEMU enables
264 the wacom USB touch pad driver by default to allow input
265 of absolute coordinates.
266 This default means that the mouse can enter and leave the
267 main window without the grab taking effect leading to a
268 better user experience.
269 </para>
270 </section>
271
272 <section id='qemu-running-under-a-network-file-system-nfs-server'>
273 <title>Running Under a Network File System (NFS) Server</title>
274
275 <para>
276 One method for running QEMU is to run it on an NFS server.
277 This is useful when you need to access the same file system
278 from both the build and the emulated system at the same time.
279 It is also worth noting that the system does not need root
280 privileges to run.
281 It uses a user space NFS server to avoid that.
282 Follow these steps to set up for running QEMU using an NFS
283 server.
284 <orderedlist>
285 <listitem><para>
286 <emphasis>Extract a Root Filesystem:</emphasis>
287 Once you are able to run QEMU in your environment, you can
288 use the <filename>runqemu-extract-sdk</filename> script,
289 which is located in the <filename>scripts</filename>
290 directory along with the <filename>runqemu</filename>
291 script.</para>
292
293 <para>The <filename>runqemu-extract-sdk</filename> takes a
294 root filesystem tarball and extracts it into a location
295 that you specify.
296 Here is an example that takes a file system and
297 extracts it to a directory named
298 <filename>test-nfs</filename>:
299 <literallayout class='monospaced'>
300 runqemu-extract-sdk ./tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.tar.bz2 test-nfs
301 </literallayout>
302 </para></listitem>
303 <listitem><para>
304 <emphasis>Start QEMU:</emphasis>
305 Once you have extracted the file system, you can run
306 <filename>runqemu</filename> normally with the additional
307 location of the file system.
308 You can then also make changes to the files within
309 <filename>./test-nfs</filename> and see those changes
310 appear in the image in real time.
311 Here is an example using the <filename>qemux86</filename>
312 image:
313 <literallayout class='monospaced'>
314 runqemu qemux86-64 ./test-nfs
315 </literallayout>
316 </para></listitem>
317 </orderedlist>
318 <note>
319 <para>
320 Should you need to start, stop, or restart the NFS share,
321 you can use the following commands:
322 <itemizedlist>
323 <listitem><para>
324 The following command starts the NFS share:
325 <literallayout class='monospaced'>
326 runqemu-export-rootfs start <replaceable>file-system-location</replaceable>
327 </literallayout>
328 </para></listitem>
329 <listitem><para>
330 The following command stops the NFS share:
331 <literallayout class='monospaced'>
332 runqemu-export-rootfs stop <replaceable>file-system-location</replaceable>
333 </literallayout>
334 </para></listitem>
335 <listitem><para>
336 The following command restarts the NFS share:
337 <literallayout class='monospaced'>
338 runqemu-export-rootfs restart <replaceable>file-system-location</replaceable>
339 </literallayout>
340 </para></listitem>
341 </itemizedlist>
342 </para>
343 </note>
344 </para>
345 </section>
346
347 <section id='qemu-kvm-cpu-compatibility'>
348 <title>QEMU CPU Compatibility Under KVM</title>
349
350 <para>
351 By default, the QEMU build compiles for and targets 64-bit and x86
352 <trademark class='registered'>Intel</trademark> <trademark class='trademark'>Core</trademark>2
353 Duo processors and 32-bit x86
354 <trademark class='registered'>Intel</trademark> <trademark class='registered'>Pentium</trademark>
355 II processors.
356 QEMU builds for and targets these CPU types because they display
357 a broad range of CPU feature compatibility with many commonly
358 used CPUs.
359 </para>
360
361 <para>
362 Despite this broad range of compatibility, the CPUs could support
363 a feature that your host CPU does not support.
364 Although this situation is not a problem when QEMU uses software
365 emulation of the feature, it can be a problem when QEMU is
366 running with KVM enabled.
367 Specifically, software compiled with a certain CPU feature crashes
368 when run on a CPU under KVM that does not support that feature.
369 To work around this problem, you can override QEMU's runtime CPU
370 setting by changing the <filename>QB_CPU_KVM</filename>
371 variable in <filename>qemuboot.conf</filename> in the
372 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory's</ulink>
373 <filename>deploy/image</filename> directory.
374 This setting specifies a <filename>-cpu</filename> option
375 passed into QEMU in the <filename>runqemu</filename> script.
376 Running <filename>qemu -cpu help</filename> returns a list of
377 available supported CPU types.
378 </para>
379 </section>
380
381 <section id='qemu-dev-performance'>
382 <title>QEMU Performance</title>
383
384 <para>
385 Using QEMU to emulate your hardware can result in speed issues
386 depending on the target and host architecture mix.
387 For example, using the <filename>qemux86</filename> image in the
388 emulator on an Intel-based 32-bit (x86) host machine is fast
389 because the target and host architectures match.
390 On the other hand, using the <filename>qemuarm</filename> image
391 on the same Intel-based host can be slower.
392 But, you still achieve faithful emulation of ARM-specific issues.
393 </para>
394
395 <para>
396 To speed things up, the QEMU images support using
397 <filename>distcc</filename> to call a cross-compiler outside the
398 emulated system.
399 If you used <filename>runqemu</filename> to start QEMU, and the
400 <filename>distccd</filename> application is present on the host
401 system, any BitBake cross-compiling toolchain available from the
402 build system is automatically used from within QEMU simply by
403 calling <filename>distcc</filename>.
404 You can accomplish this by defining the cross-compiler variable
405 (e.g. <filename>export CC="distcc"</filename>).
406 Alternatively, if you are using a suitable SDK image or the
407 appropriate stand-alone toolchain is present, the toolchain is
408 also automatically used.
409 <note>
410 Several mechanisms exist that let you connect to the system
411 running on the QEMU emulator:
412 <itemizedlist>
413 <listitem><para>
414 QEMU provides a framebuffer interface that makes
415 standard consoles available.
416 </para></listitem>
417 <listitem><para>
418 Generally, headless embedded devices have a serial port.
419 If so, you can configure the operating system of the
420 running image to use that port to run a console.
421 The connection uses standard IP networking.
422 </para></listitem>
423 <listitem><para>
424 SSH servers exist in some QEMU images.
425 The <filename>core-image-sato</filename> QEMU image
426 has a Dropbear secure shell (SSH) server that runs
427 with the root password disabled.
428 The <filename>core-image-full-cmdline</filename> and
429 <filename>core-image-lsb</filename> QEMU images
430 have OpenSSH instead of Dropbear.
431 Including these SSH servers allow you to use standard
432 <filename>ssh</filename> and <filename>scp</filename>
433 commands.
434 The <filename>core-image-minimal</filename> QEMU image,
435 however, contains no SSH server.
436 </para></listitem>
437 <listitem><para>
438 You can use a provided, user-space NFS server to boot
439 the QEMU session using a local copy of the root
440 filesystem on the host.
441 In order to make this connection, you must extract a
442 root filesystem tarball by using the
443 <filename>runqemu-extract-sdk</filename> command.
444 After running the command, you must then point the
445 <filename>runqemu</filename>
446 script to the extracted directory instead of a root
447 filesystem image file.
448 See the
449 "<link linkend='qemu-running-under-a-network-file-system-nfs-server'>Running Under a Network File System (NFS) Server</link>"
450 section for more information.
451 </para></listitem>
452 </itemizedlist>
453 </note>
454 </para>
455 </section>
456
457 <section id='qemu-dev-command-line-syntax'>
458 <title>QEMU Command-Line Syntax</title>
459
460 <para>
461 The basic <filename>runqemu</filename> command syntax is as
462 follows:
463 <literallayout class='monospaced'>
464 $ runqemu [<replaceable>option</replaceable> ] [...]
465 </literallayout>
466 Based on what you provide on the command line,
467 <filename>runqemu</filename> does a good job of figuring out what
468 you are trying to do.
469 For example, by default, QEMU looks for the most recently built
470 image according to the timestamp when it needs to look for an
471 image.
472 Minimally, through the use of options, you must provide either
473 a machine name, a virtual machine image
474 (<filename>*wic.vmdk</filename>), or a kernel image
475 (<filename>*.bin</filename>).
476 </para>
477
478 <para>
479 Following is the command-line help output for the
480 <filename>runqemu</filename> command:
481 <literallayout class='monospaced'>
482 $ runqemu --help
483
484 Usage: you can run this script with any valid combination
485 of the following environment variables (in any order):
486 KERNEL - the kernel image file to use
487 ROOTFS - the rootfs image file or nfsroot directory to use
488 MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
489 Simplified QEMU command-line options can be passed with:
490 nographic - disable video console
491 serial - enable a serial console on /dev/ttyS0
492 slirp - enable user networking, no root privileges is required
493 kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
494 kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
495 publicvnc - enable a VNC server open to all hosts
496 audio - enable audio
497 [*/]ovmf* - OVMF firmware file or base name for booting with UEFI
498 tcpserial=&lt;port&gt; - specify tcp serial port number
499 biosdir=&lt;dir&gt; - specify custom bios dir
500 biosfilename=&lt;filename&gt; - specify bios filename
501 qemuparams=&lt;xyz&gt; - specify custom parameters to QEMU
502 bootparams=&lt;xyz&gt; - specify custom kernel parameters during boot
503 help, -h, --help: print this text
504
505 Examples:
506 runqemu
507 runqemu qemuarm
508 runqemu tmp/deploy/images/qemuarm
509 runqemu tmp/deploy/images/qemux86/&lt;qemuboot.conf&gt;
510 runqemu qemux86-64 core-image-sato ext4
511 runqemu qemux86-64 wic-image-minimal wic
512 runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
513 runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
514 runqemu qemux86 qemuparams="-m 256"
515 runqemu qemux86 bootparams="psplash=false"
516 runqemu path/to/&lt;image&gt;-&lt;machine&gt;.wic
517 runqemu path/to/&lt;image&gt;-&lt;machine&gt;.wic.vmdk
518 </literallayout>
519 </para>
520 </section>
521
522 <section id='qemu-dev-runqemu-command-line-options'>
523 <title><filename>runqemu</filename> Command-Line Options</title>
524
525 <para>
526 Following is a description of <filename>runqemu</filename>
527 options you can provide on the command line:
528 <note><title>Tip</title>
529 If you do provide some "illegal" option combination or perhaps
530 you do not provide enough in the way of options,
531 <filename>runqemu</filename> provides appropriate error
532 messaging to help you correct the problem.
533 </note>
534 <itemizedlist>
535 <listitem><para>
536 <replaceable>QEMUARCH</replaceable>:
537 The QEMU machine architecture, which must be "qemuarm",
538 "qemuarm64", "qemumips", "qemumips64", "qemuppc",
539 "qemux86", or "qemux86-64".
540 </para></listitem>
541 <listitem><para>
542 <filename><replaceable>VM</replaceable></filename>:
543 The virtual machine image, which must be a
544 <filename>.wic.vmdk</filename> file.
545 Use this option when you want to boot a
546 <filename>.wic.vmdk</filename> image.
547 The image filename you provide must contain one of the
548 following strings: "qemux86-64", "qemux86", "qemuarm",
549 "qemumips64", "qemumips", "qemuppc", or "qemush4".
550 </para></listitem>
551 <listitem><para>
552 <replaceable>ROOTFS</replaceable>:
553 A root filesystem that has one of the following
554 filetype extensions: "ext2", "ext3", "ext4", "jffs2",
555 "nfs", or "btrfs".
556 If the filename you provide for this option uses "nfs", it
557 must provide an explicit root filesystem path.
558 </para></listitem>
559 <listitem><para>
560 <replaceable>KERNEL</replaceable>:
561 A kernel image, which is a <filename>.bin</filename> file.
562 When you provide a <filename>.bin</filename> file,
563 <filename>runqemu</filename> detects it and assumes the
564 file is a kernel image.
565 </para></listitem>
566 <listitem><para>
567 <replaceable>MACHINE</replaceable>:
568 The architecture of the QEMU machine, which must be one
569 of the following: "qemux86", "qemux86-64", "qemuarm",
570 "qemuarm64", "qemumips", "qemumips64", or "qemuppc".
571 The <replaceable>MACHINE</replaceable> and
572 <replaceable>QEMUARCH</replaceable> options are basically
573 identical.
574 If you do not provide a <replaceable>MACHINE</replaceable>
575 option, <filename>runqemu</filename> tries to determine
576 it based on other options.
577 </para></listitem>
578 <listitem><para>
579 <filename>ramfs</filename>:
580 Indicates you are booting an initial RAM disk (initramfs)
581 image, which means the <filename>FSTYPE</filename> is
582 <filename>cpio.gz</filename>.
583 </para></listitem>
584 <listitem><para>
585 <filename>iso</filename>:
586 Indicates you are booting an ISO image, which means the
587 <filename>FSTYPE</filename> is
588 <filename>.iso</filename>.
589 </para></listitem>
590 <listitem><para>
591 <filename>nographic</filename>:
592 Disables the video console, which sets the console to
593 "ttys0".
594 This option is useful when you have logged into a server
595 and you do not want to disable forwarding from the
596 X Window System (X11) to your workstation or laptop.
597 </para></listitem>
598 <listitem><para>
599 <filename>serial</filename>:
600 Enables a serial console on
601 <filename>/dev/ttyS0</filename>.
602 </para></listitem>
603 <listitem><para>
604 <filename>biosdir</filename>:
605 Establishes a custom directory for BIOS, VGA BIOS and
606 keymaps.
607 </para></listitem>
608 <listitem><para>
609 <filename>biosfilename</filename>:
610 Establishes a custom BIOS name.
611 </para></listitem>
612 <listitem><para>
613 <filename>qemuparams=\"<replaceable>xyz</replaceable>\"</filename>:
614 Specifies custom QEMU parameters.
615 Use this option to pass options other than the simple
616 "kvm" and "serial" options.
617 </para></listitem>
618 <listitem><para><filename>bootparams=\"<replaceable>xyz</replaceable>\"</filename>:
619 Specifies custom boot parameters for the kernel.
620 </para></listitem>
621 <listitem><para>
622 <filename>audio</filename>:
623 Enables audio in QEMU.
624 The <replaceable>MACHINE</replaceable> option must be
625 either "qemux86" or "qemux86-64" in order for audio to be
626 enabled.
627 Additionally, the <filename>snd_intel8x0</filename>
628 or <filename>snd_ens1370</filename> driver must be
629 installed in linux guest.
630 </para></listitem>
631 <listitem><para>
632 <filename>slirp</filename>:
633 Enables "slirp" networking, which is a different way
634 of networking that does not need root access
635 but also is not as easy to use or comprehensive
636 as the default.
637 </para></listitem>
638 <listitem><para id='kvm-cond'>
639 <filename>kvm</filename>:
640 Enables KVM when running "qemux86" or "qemux86-64"
641 QEMU architectures.
642 For KVM to work, all the following conditions must be met:
643 <itemizedlist>
644 <listitem><para>
645 Your <replaceable>MACHINE</replaceable> must be either
646qemux86" or "qemux86-64".
647 </para></listitem>
648 <listitem><para>
649 Your build host has to have the KVM modules
650 installed, which are
651 <filename>/dev/kvm</filename>.
652 </para></listitem>
653 <listitem><para>
654 The build host <filename>/dev/kvm</filename>
655 directory has to be both writable and readable.
656 </para></listitem>
657 </itemizedlist>
658 </para></listitem>
659 <listitem><para>
660 <filename>kvm-vhost</filename>:
661 Enables KVM with VHOST support when running "qemux86"
662 or "qemux86-64" QEMU architectures.
663 For KVM with VHOST to work, the following conditions must
664 be met:
665 <itemizedlist>
666 <listitem><para>
667 <link linkend='kvm-cond'>kvm</link> option
668 conditions must be met.
669 </para></listitem>
670 <listitem><para>
671 Your build host has to have virtio net device, which
672 are <filename>/dev/vhost-net</filename>.
673 </para></listitem>
674 <listitem><para>
675 The build host <filename>/dev/vhost-net</filename>
676 directory has to be either readable or writable
677 and "slirp-enabled".
678 </para></listitem>
679 </itemizedlist>
680 </para></listitem>
681 <listitem><para>
682 <filename>publicvnc</filename>:
683 Enables a VNC server open to all hosts.
684 </para></listitem>
685 </itemizedlist>
686 </para>
687 </section>
688</chapter>
689<!--
690vim: expandtab tw=80 ts=4
691-->
diff --git a/documentation/dev-manual/dev-manual-start.xml b/documentation/dev-manual/dev-manual-start.xml
deleted file mode 100644
index 9ff9ac4c8f..0000000000
--- a/documentation/dev-manual/dev-manual-start.xml
+++ /dev/null
@@ -1,1288 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='dev-manual-start'>
7
8<title>Setting Up to Use the Yocto Project</title>
9
10<para>
11 This chapter provides guidance on how to prepare to use the
12 Yocto Project.
13 You can learn about creating a team environment that develops using the
14 Yocto Project, how to set up a
15 <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink>,
16 how to locate Yocto Project source repositories, and how to create local
17 Git repositories.
18</para>
19
20<section id="usingpoky-changes-collaborate">
21 <title>Creating a Team Development Environment</title>
22
23 <para>
24 It might not be immediately clear how you can use the Yocto
25 Project in a team development environment, or how to scale it for a
26 large team of developers.
27 You can adapt the Yocto Project to many different use cases and
28 scenarios;
29 however, this flexibility could cause difficulties if you are trying
30 to create a working setup that scales effectively.
31 </para>
32
33 <para>
34 To help you understand how to set up this type of environment,
35 this section presents a procedure that gives you information
36 that can help you get the results you want.
37 The procedure is high-level and presents some of the project's most
38 successful experiences, practices, solutions, and available
39 technologies that have proved to work well in the past;
40 however, keep in mind, the procedure here is simply a starting point.
41 You can build off these steps and customize the procedure to fit any
42 particular working environment and set of practices.
43 <orderedlist>
44 <listitem><para>
45 <emphasis>Determine Who is Going to be Developing:</emphasis>
46 You first need to understand who is going to be doing anything
47 related to the Yocto Project and determine their roles.
48 Making this determination is essential to completing
49 subsequent steps, which are to get your equipment together
50 and set up your development environment's hardware topology.
51 </para>
52
53 <para>The following roles exist:
54 <itemizedlist>
55 <listitem><para>
56 <emphasis>Application Developer:</emphasis>
57 This type of developer does application level work
58 on top of an existing software stack.
59 </para></listitem>
60 <listitem><para>
61 <emphasis>Core System Developer:</emphasis>
62 This type of developer works on the contents of the
63 operating system image itself.
64 </para></listitem>
65 <listitem><para>
66 <emphasis>Build Engineer:</emphasis>
67 This type of developer manages Autobuilders and
68 releases. Depending on the specifics of the environment,
69 not all situations might need a Build Engineer.
70 </para></listitem>
71 <listitem><para>
72 <emphasis>Test Engineer:</emphasis>
73 This type of developer creates and manages automated
74 tests that are used to ensure all application and
75 core system development meets desired quality
76 standards.
77 </para></listitem>
78 </itemizedlist>
79 </para></listitem>
80 <listitem><para>
81 <emphasis>Gather the Hardware:</emphasis>
82 Based on the size and make-up of the team, get the hardware
83 together.
84 Ideally, any development, build, or test engineer uses
85 a system that runs a supported Linux distribution.
86 These systems, in general, should be high performance
87 (e.g. dual, six-core Xeons with 24 Gbytes of RAM and plenty
88 of disk space).
89 You can help ensure efficiency by having any machines used
90 for testing or that run Autobuilders be as high performance
91 as possible.
92 <note>
93 Given sufficient processing power, you might also consider
94 building Yocto Project development containers to be run
95 under Docker, which is described later.
96 </note>
97 </para></listitem>
98 <listitem><para>
99 <emphasis>Understand the Hardware Topology of the Environment:</emphasis>
100 Once you understand the hardware involved and the make-up
101 of the team, you can understand the hardware topology of the
102 development environment.
103 You can get a visual idea of the machines and their roles
104 across the development environment.
105
106<!--
107 The following figure shows a moderately sized Yocto Project
108 development environment.
109
110 <para role="writernotes">
111 Need figure.</para>
112-->
113
114 </para></listitem>
115 <listitem><para>
116 <emphasis>Use Git as Your Source Control Manager (SCM):</emphasis>
117 Keeping your
118 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>
119 (i.e. recipes, configuration files, classes, and so forth)
120 and any software you are developing under the control of an SCM
121 system that is compatible with the OpenEmbedded build system
122 is advisable.
123 Of all of the SCMs supported by BitBake, the Yocto Project team strongly
124 recommends using
125 <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>.
126 Git is a distributed system that is easy to back up,
127 allows you to work remotely, and then connects back to the
128 infrastructure.
129 <note>
130 For information about BitBake, see the
131 <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
132 </note></para>
133
134 <para>It is relatively easy to set up Git services and create
135 infrastructure like
136 <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>,
137 which is based on server software called
138 <filename>gitolite</filename> with <filename>cgit</filename>
139 being used to generate the web interface that lets you view the
140 repositories.
141 The <filename>gitolite</filename> software identifies users
142 using SSH keys and allows branch-based access controls to
143 repositories that you can control as little or as much as
144 necessary.
145 <note>
146 The setup of these services is beyond the scope of this
147 manual.
148 However, sites such as the following exist that describe
149 how to perform setup:
150 <itemizedlist>
151 <listitem><para>
152 <ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>:
153 Describes how to install
154 <filename>gitolite</filename> on the server.
155 </para></listitem>
156 <listitem><para>
157 <ulink url='http://gitolite.com'>Gitolite</ulink>:
158 Information for <filename>gitolite</filename>.
159 </para></listitem>
160 <listitem><para>
161 <ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
162 Documentation on how to create interfaces and
163 frontends for Git.
164 </para></listitem>
165 </itemizedlist>
166 </note>
167 </para></listitem>
168 <listitem><para>
169 <emphasis>Set up the Application Development Machines:</emphasis>
170 As mentioned earlier, application developers are creating
171 applications on top of existing software stacks.
172 Following are some best practices for setting up machines
173 used for application development:
174 <itemizedlist>
175 <listitem><para>
176 Use a pre-built toolchain that contains the software
177 stack itself.
178 Then, develop the application code on top of the
179 stack.
180 This method works well for small numbers of relatively
181 isolated applications.
182 </para></listitem>
183 <listitem><para>
184 Keep your cross-development toolchains updated.
185 You can do this through provisioning either as new
186 toolchain downloads or as updates through a package
187 update mechanism using <filename>opkg</filename>
188 to provide updates to an existing toolchain.
189 The exact mechanics of how and when to do this depend
190 on local policy.
191 </para></listitem>
192 <listitem><para>
193 Use multiple toolchains installed locally into
194 different locations to allow development across
195 versions.
196 </para></listitem>
197 </itemizedlist>
198 </para></listitem>
199 <listitem><para>
200 <emphasis>Set up the Core Development Machines:</emphasis>
201 As mentioned earlier, core developers work on the contents of
202 the operating system itself.
203 Following are some best practices for setting up machines
204 used for developing images:
205 <itemizedlist>
206 <listitem><para>
207 Have the
208 <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
209 available on the developer workstations so developers
210 can run their own builds and directly rebuild the
211 software stack.
212 </para></listitem>
213 <listitem><para>
214 Keep the core system unchanged as much as
215 possible and do your work in layers on top of the
216 core system.
217 Doing so gives you a greater level of portability when
218 upgrading to new versions of the core system or Board
219 Support Packages (BSPs).
220 </para></listitem>
221 <listitem><para>
222 Share layers amongst the developers of a
223 particular project and contain the policy configuration
224 that defines the project.
225 </para></listitem>
226 </itemizedlist>
227 </para></listitem>
228 <listitem><para>
229 <emphasis>Set up an Autobuilder:</emphasis>
230 Autobuilders are often the core of the development
231 environment.
232 It is here that changes from individual developers are brought
233 together and centrally tested.
234 Based on this automated build and test environment, subsequent
235 decisions about releases can be made.
236 Autobuilders also allow for "continuous integration" style
237 testing of software components and regression identification
238 and tracking.</para>
239
240 <para>See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
241 for more information and links to buildbot.
242 The Yocto Project team has found this implementation
243 works well in this role.
244 A public example of this is the Yocto Project
245 Autobuilders, which the Yocto Project team uses to test the
246 overall health of the project.</para>
247
248 <para>The features of this system are:
249 <itemizedlist>
250 <listitem><para>
251 Highlights when commits break the build.
252 </para></listitem>
253 <listitem><para>
254 Populates an
255 <ulink url='&YOCTO_DOCS_OM_URL;#shared-state-cache'>sstate cache</ulink>
256 from which developers can pull rather than requiring
257 local builds.
258 </para></listitem>
259 <listitem><para>
260 Allows commit hook triggers, which trigger builds when
261 commits are made.
262 </para></listitem>
263 <listitem><para>
264 Allows triggering of automated image booting
265 and testing under the QuickEMUlator (QEMU).
266 </para></listitem>
267 <listitem><para>
268 Supports incremental build testing and
269 from-scratch builds.
270 </para></listitem>
271 <listitem><para>
272 Shares output that allows developer
273 testing and historical regression investigation.
274 </para></listitem>
275 <listitem><para>
276 Creates output that can be used for releases.
277 </para></listitem>
278 <listitem><para>
279 Allows scheduling of builds so that resources
280 can be used efficiently.
281 </para></listitem>
282 </itemizedlist>
283 </para></listitem>
284 <listitem><para>
285 <emphasis>Set up Test Machines:</emphasis>
286 Use a small number of shared, high performance systems
287 for testing purposes.
288 Developers can use these systems for wider, more
289 extensive testing while they continue to develop
290 locally using their primary development system.
291 </para></listitem>
292 <listitem><para>
293 <emphasis>Document Policies and Change Flow:</emphasis>
294 The Yocto Project uses a hierarchical structure and a
295 pull model.
296 Scripts exist to create and send pull requests
297 (i.e. <filename>create-pull-request</filename> and
298 <filename>send-pull-request</filename>).
299 This model is in line with other open source projects where
300 maintainers are responsible for specific areas of the project
301 and a single maintainer handles the final "top-of-tree" merges.
302 <note>
303 You can also use a more collective push model.
304 The <filename>gitolite</filename> software supports both the
305 push and pull models quite easily.
306 </note></para>
307
308 <para>As with any development environment, it is important
309 to document the policy used as well as any main project
310 guidelines so they are understood by everyone.
311 It is also a good idea to have well-structured
312 commit messages, which are usually a part of a project's
313 guidelines.
314 Good commit messages are essential when looking back in time and
315 trying to understand why changes were made.</para>
316
317 <para>If you discover that changes are needed to the core
318 layer of the project, it is worth sharing those with the
319 community as soon as possible.
320 Chances are if you have discovered the need for changes,
321 someone else in the community needs them also.
322 </para></listitem>
323 <listitem><para>
324 <emphasis>Development Environment Summary:</emphasis>
325 Aside from the previous steps, some best practices exist
326 within the Yocto Project development environment.
327 Consider the following:
328 <itemizedlist>
329 <listitem><para>
330 Use
331 <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>
332 as the source control system.
333 </para></listitem>
334 <listitem><para>
335 Maintain your Metadata in layers that make sense
336 for your situation.
337 See the
338 "<ulink url='&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model'>The Yocto Project Layer Model</ulink>"
339 section in the Yocto Project Overview and Concepts
340 Manual and the
341 "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>"
342 section for more information on layers.
343 </para></listitem>
344 <listitem><para>
345 Separate the project's Metadata and code by using
346 separate Git repositories.
347 See the
348 "<ulink url='&YOCTO_DOCS_OM_URL;#yocto-project-repositories'>Yocto Project Source Repositories</ulink>"
349 section in the Yocto Project Overview and Concepts
350 Manual for information on these repositories.
351 See the
352 "<link linkend='locating-yocto-project-source-files'>Locating Yocto Project Source Files</link>"
353 section for information on how to set up local Git
354 repositories for related upstream Yocto Project
355 Git repositories.
356 </para></listitem>
357 <listitem><para>
358 Set up the directory for the shared state cache
359 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
360 where it makes sense.
361 For example, set up the sstate cache on a system used
362 by developers in the same organization and share the
363 same source directories on their machines.
364 </para></listitem>
365 <listitem><para>
366 Set up an Autobuilder and have it populate the
367 sstate cache and source directories.
368 </para></listitem>
369 <listitem><para>
370 The Yocto Project community encourages you
371 to send patches to the project to fix bugs or add
372 features.
373 If you do submit patches, follow the project commit
374 guidelines for writing good commit messages.
375 See the "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>"
376 section.
377 </para></listitem>
378 <listitem><para>
379 Send changes to the core sooner than later
380 as others are likely to run into the same issues.
381 For some guidance on mailing lists to use, see the list
382 in the
383 "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>"
384 section.
385 For a description of the available mailing lists, see
386 the
387 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
388 section in the Yocto Project Reference Manual.
389 </para></listitem>
390 </itemizedlist>
391 </para></listitem>
392 </orderedlist>
393 </para>
394</section>
395
396<section id='dev-preparing-the-build-host'>
397 <title>Preparing the Build Host</title>
398
399 <para>
400 This section provides procedures to set up a system to be used as your
401 <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink>
402 for development using the Yocto Project.
403 Your build host can be a native Linux machine (recommended), it can
404 be a machine (Linux, Mac, or Windows) that uses
405 <ulink url='https://github.com/crops/poky-container'>CROPS</ulink>,
406 which leverages
407 <ulink url='https://www.docker.com/'>Docker Containers</ulink> or it can
408 be a Windows machine capable of running Windows Subsystem For Linux v2 (WSL).
409 <note>
410 The Yocto Project is not compatible with
411 <ulink url='https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux'>Windows Subsystem for Linux v1</ulink>.
412 It is compatible but not officially supported nor validated with WSLv2.
413 If you still decide to use WSL please upgrade to
414 <ulink url='https://docs.microsoft.com/en-us/windows/wsl/wsl2-install'>WSLv2</ulink>.
415 </note>
416 </para>
417
418 <para>
419 Once your build host is set up to use the Yocto Project,
420 further steps are necessary depending on what you want to
421 accomplish.
422 See the following references for information on how to prepare for
423 Board Support Package (BSP) development and kernel development:
424 <itemizedlist>
425 <listitem><para>
426 <emphasis>BSP Development:</emphasis>
427 See the
428 "<ulink url='&YOCTO_DOCS_BSP_URL;#preparing-your-build-host-to-work-with-bsp-layers'>Preparing Your Build Host to Work With BSP Layers</ulink>"
429 section in the Yocto Project Board Support Package (BSP)
430 Developer's Guide.
431 </para></listitem>
432 <listitem><para>
433 <emphasis>Kernel Development:</emphasis>
434 See the
435 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</ulink>"
436 section in the Yocto Project Linux Kernel Development Manual.
437 </para></listitem>
438 </itemizedlist>
439 </para>
440
441 <section id='setting-up-a-native-linux-host'>
442 <title>Setting Up a Native Linux Host</title>
443
444 <para>
445 Follow these steps to prepare a native Linux machine as your
446 Yocto Project Build Host:
447 <orderedlist>
448 <listitem><para>
449 <emphasis>Use a Supported Linux Distribution:</emphasis>
450 You should have a reasonably current Linux-based host
451 system.
452 You will have the best results with a recent release of
453 Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS as these
454 releases are frequently tested against the Yocto Project
455 and officially supported.
456 For a list of the distributions under validation and their
457 status, see the
458 "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>" section
459 in the Yocto Project Reference Manual and the wiki page at
460 <ulink url='&YOCTO_WIKI_URL;/wiki/Distribution_Support'>Distribution Support</ulink>.
461 </para></listitem>
462 <listitem><para>
463 <emphasis>Have Enough Free Memory:</emphasis>
464 Your system should have at least 50 Gbytes of free disk
465 space for building images.
466 </para></listitem>
467 <listitem><para>
468 <emphasis>Meet Minimal Version Requirements:</emphasis>
469 The OpenEmbedded build system should be able to run on any
470 modern distribution that has the following versions for
471 Git, tar, Python and gcc.
472 <itemizedlist>
473 <listitem><para>
474 Git 1.8.3.1 or greater
475 </para></listitem>
476 <listitem><para>
477 tar 1.28 or greater
478 </para></listitem>
479 <listitem><para>
480 Python 3.5.0 or greater.
481 </para></listitem>
482 <listitem><para>
483 gcc 5.0 or greater.
484 </para></listitem>
485 </itemizedlist>
486 If your build host does not meet any of these three listed
487 version requirements, you can take steps to prepare the
488 system so that you can still use the Yocto Project.
489 See the
490 "<ulink url='&YOCTO_DOCS_REF_URL;#required-git-tar-python-and-gcc-versions'>Required Git, tar, Python and gcc Versions</ulink>"
491 section in the Yocto Project Reference Manual for
492 information.
493 </para></listitem>
494 <listitem><para>
495 <emphasis>Install Development Host Packages:</emphasis>
496 Required development host packages vary depending on your
497 build host and what you want to do with the Yocto
498 Project.
499 Collectively, the number of required packages is large
500 if you want to be able to cover all cases.</para>
501
502 <para>For lists of required packages for all scenarios,
503 see the
504 "<ulink url='&YOCTO_DOCS_REF_URL;#required-packages-for-the-build-host'>Required Packages for the Build Host</ulink>"
505 section in the Yocto Project Reference Manual.
506 </para></listitem>
507 </orderedlist>
508 Once you have completed the previous steps, you are ready to
509 continue using a given development path on your native Linux
510 machine.
511 If you are going to use BitBake, see the
512 "<link linkend='cloning-the-poky-repository'>Cloning the <filename>poky</filename> Repository</link>"
513 section.
514 If you are going to use the Extensible SDK, see the
515 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-extensible'>Using the Extensible SDK</ulink>"
516 Chapter in the Yocto Project Application Development and the
517 Extensible Software Development Kit (eSDK) manual.
518 If you want to work on the kernel, see the
519 <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>.
520 If you are going to use Toaster, see the
521 "<ulink url='&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use'>Setting Up and Using Toaster</ulink>"
522 section in the Toaster User Manual.
523 </para>
524 </section>
525
526 <section id='setting-up-to-use-crops'>
527 <title>Setting Up to Use CROss PlatformS (CROPS)</title>
528
529 <para>
530 With
531 <ulink url='https://github.com/crops/poky-container'>CROPS</ulink>,
532 which leverages
533 <ulink url='https://www.docker.com/'>Docker Containers</ulink>,
534 you can create a Yocto Project development environment that
535 is operating system agnostic.
536 You can set up a container in which you can develop using the
537 Yocto Project on a Windows, Mac, or Linux machine.
538 </para>
539
540 <para>
541 Follow these general steps to prepare a Windows, Mac, or Linux
542 machine as your Yocto Project build host:
543 <orderedlist>
544 <listitem><para>
545 <emphasis>Determine What Your Build Host Needs:</emphasis>
546 <ulink url='https://www.docker.com/what-docker'>Docker</ulink>
547 is a software container platform that you need to install
548 on the build host.
549 Depending on your build host, you might have to install
550 different software to support Docker containers.
551 Go to the Docker installation page and read about the
552 platform requirements in
553 "<ulink url='https://docs.docker.com/install/#supported-platforms'>Supported Platforms</ulink>"
554 your build host needs to run containers.
555 </para></listitem>
556 <listitem><para>
557 <emphasis>Choose What To Install:</emphasis>
558 Depending on whether or not your build host meets system
559 requirements, you need to install "Docker CE Stable" or
560 the "Docker Toolbox".
561 Most situations call for Docker CE.
562 However, if you have a build host that does not meet
563 requirements (e.g. Pre-Windows 10 or Windows 10 "Home"
564 version), you must install Docker Toolbox instead.
565 </para></listitem>
566 <listitem><para>
567 <emphasis>Go to the Install Site for Your Platform:</emphasis>
568 Click the link for the Docker edition associated with
569 your build host's native software.
570 For example, if your build host is running Microsoft
571 Windows Version 10 and you want the Docker CE Stable
572 edition, click that link under "Supported Platforms".
573 </para></listitem>
574 <listitem><para>
575 <emphasis>Install the Software:</emphasis>
576 Once you have understood all the pre-requisites, you can
577 download and install the appropriate software.
578 Follow the instructions for your specific machine and
579 the type of the software you need to install:
580 <itemizedlist>
581 <listitem><para>
582 Install
583 <ulink url='https://docs.docker.com/docker-for-windows/install/#install-docker-for-windows-desktop-app'>Docker CE for Windows</ulink>
584 for Windows build hosts that meet requirements.
585 </para></listitem>
586 <listitem><para>
587 Install
588 <ulink url='https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-for-mac'>Docker CE for Macs</ulink>
589 for Mac build hosts that meet requirements.
590 </para></listitem>
591 <listitem><para>
592 Install
593 <ulink url='https://docs.docker.com/toolbox/toolbox_install_windows/'>Docker Toolbox for Windows</ulink>
594 for Windows build hosts that do not meet Docker
595 requirements.
596 </para></listitem>
597 <listitem><para>
598 Install
599 <ulink url='https://docs.docker.com/toolbox/toolbox_install_mac/'>Docker Toolbox for MacOS</ulink>
600 for Mac build hosts that do not meet Docker
601 requirements.
602 </para></listitem>
603 <listitem><para>
604 Install
605 <ulink url='https://docs.docker.com/install/linux/docker-ce/centos/'>Docker CE for CentOS</ulink>
606 for Linux build hosts running the CentOS
607 distribution.
608 </para></listitem>
609 <listitem><para>
610 Install
611 <ulink url='https://docs.docker.com/install/linux/docker-ce/debian/'>Docker CE for Debian</ulink>
612 for Linux build hosts running the Debian
613 distribution.
614 </para></listitem>
615 <listitem><para>
616 Install
617 <ulink url='https://docs.docker.com/install/linux/docker-ce/fedora/'>Docker CE for Fedora</ulink>
618 for Linux build hosts running the Fedora
619 distribution.
620 </para></listitem>
621 <listitem><para>
622 Install
623 <ulink url='https://docs.docker.com/install/linux/docker-ce/ubuntu/'>Docker CE for Ubuntu</ulink>
624 for Linux build hosts running the Ubuntu
625 distribution.
626 </para></listitem>
627 </itemizedlist>
628 </para></listitem>
629 <listitem><para>
630 <emphasis>Optionally Orient Yourself With Docker:</emphasis>
631 If you are unfamiliar with Docker and the container
632 concept, you can learn more here -
633 <ulink url='https://docs.docker.com/get-started/'></ulink>.
634 </para></listitem>
635 <listitem><para>
636 <emphasis>Launch Docker or Docker Toolbox:</emphasis>
637 You should be able to launch Docker or the Docker Toolbox
638 and have a terminal shell on your development host.
639 </para></listitem>
640 <listitem><para>
641 <emphasis>Set Up the Containers to Use the Yocto Project:</emphasis>
642 Go to
643 <ulink url='https://github.com/crops/docker-win-mac-docs/wiki'></ulink>
644 and follow the directions for your particular
645 build host (i.e. Linux, Mac, or Windows).</para>
646
647 <para>Once you complete the setup instructions for your
648 machine, you have the Poky, Extensible SDK, and Toaster
649 containers available.
650 You can click those links from the page and learn more
651 about using each of those containers.
652 </para></listitem>
653 </orderedlist>
654 Once you have a container set up, everything is in place to
655 develop just as if you were running on a native Linux machine.
656 If you are going to use the Poky container, see the
657 "<link linkend='cloning-the-poky-repository'>Cloning the <filename>poky</filename> Repository</link>"
658 section.
659 If you are going to use the Extensible SDK container, see the
660 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-extensible'>Using the Extensible SDK</ulink>"
661 Chapter in the Yocto Project Application Development and the
662 Extensible Software Development Kit (eSDK) manual.
663 If you are going to use the Toaster container, see the
664 "<ulink url='&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use'>Setting Up and Using Toaster</ulink>"
665 section in the Toaster User Manual.
666 </para>
667 </section>
668
669 <section id='setting-up-to-use-wsl'>
670 <title>Setting Up to Use Windows Subsystem For Linux (WSLv2)</title>
671
672 <para>
673 With <ulink url='https://docs.microsoft.com/en-us/windows/wsl/wsl2-about'>
674 Windows Subsystem for Linux (WSLv2)</ulink>, you can create a
675 Yocto Project development environment that allows you to build
676 on Windows. You can set up a Linux distribution inside Windows
677 in which you can develop using the Yocto Project.
678 </para>
679
680 <para>
681 Follow these general steps to prepare a Windows machine using WSLv2
682 as your Yocto Project build host:
683 <orderedlist>
684 <listitem><para>
685 <emphasis>Make sure your Windows 10 machine is capable of running WSLv2:</emphasis>
686
687 WSLv2 is only available for Windows 10 builds > 18917. To
688 check which build version you are running, you may open a
689 command prompt on Windows and execute the command "ver".
690 <literallayout class='monospaced'>
691 C:\Users\myuser> ver
692
693 Microsoft Windows [Version 10.0.19041.153]
694 </literallayout>
695 If your build is capable of running WSLv2 you may continue,
696 for more information on this subject or instructions on how
697 to upgrade to WSLv2 visit <ulink url='https://docs.microsoft.com/en-us/windows/wsl/wsl2-install'>Windows 10 WSLv2</ulink>
698 </para></listitem>
699 <listitem><para>
700 <emphasis>Install the Linux distribution of your choice inside Windows 10:</emphasis>
701 Once you know your version of Windows 10 supports WSLv2,
702 you can install the distribution of your choice from the
703 Microsoft Store.
704 Open the Microsoft Store and search for Linux. While there
705 are several Linux distributions available, the assumption
706 is that your pick will be one of the distributions supported
707 by the Yocto Project as stated on the instructions for
708 using a native Linux host.
709 After making your selection, simply click "Get" to download
710 and install the distribution.
711 </para></listitem>
712 <listitem><para>
713 <emphasis>Check your Linux distribution is using WSLv2:</emphasis>
714 Open a Windows PowerShell and run:
715 <literallayout class='monospaced'>
716 C:\WINDOWS\system32> wsl -l -v
717 NAME STATE VERSION
718 *Ubuntu Running 2
719 </literallayout>
720 Note the version column which says the WSL version being used by
721 your distribution, on compatible systems, this can be changed back
722 at any point in time.
723 </para></listitem>
724 <listitem><para>
725 <emphasis>Optionally Orient Yourself on WSL:</emphasis>
726 If you are unfamiliar with WSL, you can learn more here -
727 <ulink url='https://docs.microsoft.com/en-us/windows/wsl/wsl2-about'></ulink>.
728 </para></listitem>
729 <listitem><para>
730 <emphasis>Launch your WSL Distibution:</emphasis>
731 From the Windows start menu simply launch your WSL distribution
732 just like any other application.
733 </para></listitem>
734 <listitem><para>
735 <emphasis>Optimize your WSLv2 storage often:</emphasis>
736 Due to the way storage is handled on WSLv2, the storage
737 space used by the undelying Linux distribution is not
738 reflected immedately, and since bitbake heavily uses
739 storage, after several builds, you may be unaware you
740 are running out of space. WSLv2 uses a VHDX file for
741 storage, this issue can be easily avoided by manually
742 optimizing this file often, this can be done in the
743 following way:
744 <orderedlist>
745 <listitem><para>
746 <emphasis>Find the location of your VHDX file:</emphasis>
747 First you need to find the distro app package directory,
748 to achieve this open a Windows Powershell as Administrator
749 and run:
750 <literallayout class='monospaced'>
751 C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
752 PackageFamilyName
753 -----------------
754 CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
755 </literallayout>
756 You should now replace the <replaceable>PackageFamilyName</replaceable>
757 and your <replaceable>user</replaceable> on the following
758 path to find your VHDX file: <filename>C:\Users\user\AppData\Local\Packages\PackageFamilyName\LocalState\</filename>
759 For example:
760 <literallayout class='monospaced'>
761 ls C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
762 Mode LastWriteTime Length Name
763 -a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
764 </literallayout>
765 Your VHDX file path is: <filename>C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx</filename>
766 </para></listitem>
767 <listitem><para><emphasis>Optimize your VHDX file:</emphasis>
768 Open a Windows Powershell as Administrator to optimize
769 your VHDX file, shutting down WSL first:
770 <literallayout class='monospaced'>
771 C:\WINDOWS\system32> wsl --shutdown
772 C:\WINDOWS\system32> optimize-vhd -Path C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
773 </literallayout>
774 A progress bar should be shown while optimizing the VHDX file,
775 and storage should now be reflected correctly on the Windows
776 Explorer.
777 </para></listitem>
778 </orderedlist>
779 </para></listitem>
780 </orderedlist>
781 <note>
782 The current implementation of WSLv2 does not have out-of-the-box
783 access to external devices such as those connected through a
784 USB port, but it automatically mounts your <filename>C:</filename>
785 drive on <filename>/mnt/c/</filename> (and others), which
786 you can use to share deploy artifacts to be later flashed on
787 hardware through Windows, but your build directory should not
788 reside inside this mountpoint.
789 </note>
790 Once you have WSLv2 set up, everything is in place to
791 develop just as if you were running on a native Linux machine.
792 If you are going to use the Extensible SDK container, see the
793 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-extensible'>Using the Extensible SDK</ulink>"
794 Chapter in the Yocto Project Application Development and the
795 Extensible Software Development Kit (eSDK) manual.
796 If you are going to use the Toaster container, see the
797 "<ulink url='&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use'>Setting Up and Using Toaster</ulink>"
798 section in the Toaster User Manual.
799 </para>
800 </section>
801</section>
802
803<section id='locating-yocto-project-source-files'>
804 <title>Locating Yocto Project Source Files</title>
805
806 <para>
807 This section shows you how to locate, fetch and configure the source
808 files you'll need to work with the Yocto Project.
809 <note><title>Notes</title>
810 <itemizedlist>
811 <listitem><para>
812 For concepts and introductory information about Git as it
813 is used in the Yocto Project, see the
814 "<ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>"
815 section in the Yocto Project Overview and Concepts Manual.
816 </para></listitem>
817 <listitem><para>
818 For concepts on Yocto Project source repositories, see the
819 "<ulink url='&YOCTO_DOCS_OM_URL;#yocto-project-repositories'>Yocto Project Source Repositories</ulink>"
820 section in the Yocto Project Overview and Concepts Manual."
821 </para></listitem>
822 </itemizedlist>
823 </note>
824 </para>
825
826 <section id='accessing-source-repositories'>
827 <title>Accessing Source Repositories</title>
828
829 <para>
830 Working from a copy of the upstream Yocto Project
831 <ulink url='&YOCTO_DOCS_OM_URL;#source-repositories'>Source Repositories</ulink>
832 is the preferred method for obtaining and using a Yocto Project
833 release.
834 You can view the Yocto Project Source Repositories at
835 <ulink url='&YOCTO_GIT_URL;'></ulink>.
836 In particular, you can find the
837 <filename>poky</filename> repository at
838 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/poky/'></ulink>.
839 </para>
840
841 <para>
842 Use the following procedure to locate the latest upstream copy of
843 the <filename>poky</filename> Git repository:
844 <orderedlist>
845 <listitem><para>
846 <emphasis>Access Repositories:</emphasis>
847 Open a browser and go to
848 <ulink url='&YOCTO_GIT_URL;'></ulink> to access the
849 GUI-based interface into the Yocto Project source
850 repositories.
851 </para></listitem>
852 <listitem><para>
853 <emphasis>Select the Repository:</emphasis>
854 Click on the repository in which you are interested (e.g.
855 <filename>poky</filename>).
856 </para></listitem>
857 <listitem><para>
858 <emphasis>Find the URL Used to Clone the Repository:</emphasis>
859 At the bottom of the page, note the URL used to
860 <ulink url='&YOCTO_DOCS_OM_URL;#git-commands-clone'>clone</ulink>
861 that repository (e.g.
862 <filename>&YOCTO_GIT_URL;/poky</filename>).
863 <note>
864 For information on cloning a repository, see the
865 "<link linkend='cloning-the-poky-repository'>Cloning the <filename>poky</filename> Repository</link>"
866 section.
867 </note>
868 </para></listitem>
869 </orderedlist>
870 </para>
871 </section>
872
873 <section id='accessing-index-of-releases'>
874 <title>Accessing Index of Releases</title>
875
876 <para>
877 Yocto Project maintains an Index of Releases area that contains
878 related files that contribute to the Yocto Project.
879 Rather than Git repositories, these files are tarballs that
880 represent snapshots in time of a given component.
881 <note><title>Tip</title>
882 The recommended method for accessing Yocto Project
883 components is to use Git to clone the upstream repository and
884 work from within that locally cloned repository.
885 The procedure in this section exists should you desire a
886 tarball snapshot of any given component.
887 </note>
888 Follow these steps to locate and download a particular tarball:
889 <orderedlist>
890 <listitem><para>
891 <emphasis>Access the Index of Releases:</emphasis>
892 Open a browser and go to
893 <ulink url='&YOCTO_DL_URL;/releases'></ulink> to access the
894 Index of Releases.
895 The list represents released components (e.g.
896 <filename>bitbake</filename>,
897 <filename>sato</filename>, and so on).
898 <note>
899 The <filename>yocto</filename> directory contains the
900 full array of released Poky tarballs.
901 The <filename>poky</filename> directory in the
902 Index of Releases was historically used for very
903 early releases and exists now only for retroactive
904 completeness.
905 </note>
906 </para></listitem>
907 <listitem><para>
908 <emphasis>Select a Component:</emphasis>
909 Click on any released component in which you are interested
910 (e.g. <filename>yocto</filename>).
911 </para></listitem>
912 <listitem><para>
913 <emphasis>Find the Tarball:</emphasis>
914 Drill down to find the associated tarball.
915 For example, click on <filename>yocto-&DISTRO;</filename> to
916 view files associated with the Yocto Project &DISTRO;
917 release (e.g. <filename>poky-&DISTRO_NAME_NO_CAP;-&POKYVERSION;.tar.bz2</filename>,
918 which is the released Poky tarball).
919 </para></listitem>
920 <listitem><para>
921 <emphasis>Download the Tarball:</emphasis>
922 Click the tarball to download and save a snapshot of the
923 given component.
924 </para></listitem>
925 </orderedlist>
926 </para>
927 </section>
928
929 <section id='using-the-downloads-page'>
930 <title>Using the Downloads Page</title>
931
932 <para>
933 The
934 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>
935 uses a "DOWNLOADS" page from which you can locate and download
936 tarballs of any Yocto Project release.
937 Rather than Git repositories, these files represent snapshot
938 tarballs similar to the tarballs located in the Index of Releases
939 described in the
940 "<link linkend='accessing-index-of-releases'>Accessing Index of Releases</link>"
941 section.
942 <note><title>Tip</title>
943 The recommended method for accessing Yocto Project
944 components is to use Git to clone a repository and work from
945 within that local repository.
946 The procedure in this section exists should you desire a
947 tarball snapshot of any given component.
948 </note>
949 <orderedlist>
950 <listitem><para>
951 <emphasis>Go to the Yocto Project Website:</emphasis>
952 Open The
953 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>
954 in your browser.
955 </para></listitem>
956 <listitem><para>
957 <emphasis>Get to the Downloads Area:</emphasis>
958 Select the "DOWNLOADS" item from the pull-down
959 "SOFTWARE" tab menu near the top of the page.
960 </para></listitem>
961 <listitem><para>
962 <emphasis>Select a Yocto Project Release:</emphasis>
963 Use the menu next to "RELEASE" to display and choose
964 a recent or past supported Yocto Project release
965 (e.g. &DISTRO_NAME_NO_CAP;,
966 &DISTRO_NAME_NO_CAP_MINUS_ONE;, and so forth).
967 <note><title>Tip</title>
968 For a "map" of Yocto Project releases to version
969 numbers, see the
970 <ulink url='https://wiki.yoctoproject.org/wiki/Releases'>Releases</ulink>
971 wiki page.
972 </note>
973 You can use the "RELEASE ARCHIVE" link to reveal a menu of
974 all Yocto Project releases.
975 </para></listitem>
976 <listitem><para>
977 <emphasis>Download Tools or Board Support Packages (BSPs):</emphasis>
978 From the "DOWNLOADS" page, you can download tools or
979 BSPs as well.
980 Just scroll down the page and look for what you need.
981 </para></listitem>
982 </orderedlist>
983 </para>
984 </section>
985
986 <section id='accessing-nightly-builds'>
987 <title>Accessing Nightly Builds</title>
988
989 <para>
990 Yocto Project maintains an area for nightly builds that contains
991 tarball releases at <ulink url='&YOCTO_AB_NIGHTLY_URL;'/>.
992 These builds include Yocto Project releases ("poky"),
993 toolchains, and builds for supported machines.
994 </para>
995
996 <para>
997 Should you ever want to access a nightly build of a particular
998 Yocto Project component, use the following procedure:
999 <orderedlist>
1000 <listitem><para>
1001 <emphasis>Locate the Index of Nightly Builds:</emphasis>
1002 Open a browser and go to
1003 <ulink url='&YOCTO_AB_NIGHTLY_URL;'/> to access the
1004 Nightly Builds.
1005 </para></listitem>
1006 <listitem><para>
1007 <emphasis>Select a Date:</emphasis>
1008 Click on the date in which you are interested.
1009 If you want the latest builds, use "CURRENT".
1010 </para></listitem>
1011 <listitem><para>
1012 <emphasis>Select a Build:</emphasis>
1013 Choose the area in which you are interested.
1014 For example, if you are looking for the most recent
1015 toolchains, select the "toolchain" link.
1016 </para></listitem>
1017 <listitem><para>
1018 <emphasis>Find the Tarball:</emphasis>
1019 Drill down to find the associated tarball.
1020 </para></listitem>
1021 <listitem><para>
1022 <emphasis>Download the Tarball:</emphasis>
1023 Click the tarball to download and save a snapshot of the
1024 given component.
1025 </para></listitem>
1026 </orderedlist>
1027 </para>
1028 </section>
1029</section>
1030
1031<section id='cloning-and-checking-out-branches'>
1032 <title>Cloning and Checking Out Branches</title>
1033
1034 <para>
1035 To use the Yocto Project for development, you need a release locally
1036 installed on your development system.
1037 This locally installed set of files is referred to as the
1038 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
1039 in the Yocto Project documentation.
1040 </para>
1041
1042 <para>
1043 The preferred method of creating your Source Directory is by using
1044 <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink> to clone a local
1045 copy of the upstream <filename>poky</filename> repository.
1046 Working from a cloned copy of the upstream repository allows you
1047 to contribute back into the Yocto Project or to simply work with
1048 the latest software on a development branch.
1049 Because Git maintains and creates an upstream repository with
1050 a complete history of changes and you are working with a local
1051 clone of that repository, you have access to all the Yocto
1052 Project development branches and tag names used in the upstream
1053 repository.
1054 </para>
1055
1056 <section id='cloning-the-poky-repository'>
1057 <title>Cloning the <filename>poky</filename> Repository</title>
1058
1059 <para>
1060 Follow these steps to create a local version of the
1061 upstream
1062 <ulink url='&YOCTO_DOCS_REF_URL;#poky'><filename>poky</filename></ulink>
1063 Git repository.
1064 <orderedlist>
1065 <listitem><para>
1066 <emphasis>Set Your Directory:</emphasis>
1067 Change your working directory to where you want to
1068 create your local copy of
1069 <filename>poky</filename>.
1070 </para></listitem>
1071 <listitem><para>
1072 <emphasis>Clone the Repository:</emphasis>
1073 The following example command clones the
1074 <filename>poky</filename> repository and uses
1075 the default name "poky" for your local repository:
1076 <literallayout class='monospaced'>
1077 $ git clone git://git.yoctoproject.org/poky
1078 Cloning into 'poky'...
1079 remote: Counting objects: 432160, done.
1080 remote: Compressing objects: 100% (102056/102056), done.
1081 remote: Total 432160 (delta 323116), reused 432037 (delta 323000)
1082 Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
1083 Resolving deltas: 100% (323116/323116), done.
1084 Checking connectivity... done.
1085 </literallayout>
1086 Unless you specify a specific development branch or
1087 tag name, Git clones the "master" branch, which results
1088 in a snapshot of the latest development changes for
1089 "master".
1090 For information on how to check out a specific
1091 development branch or on how to check out a local
1092 branch based on a tag name, see the
1093 "<link linkend='checking-out-by-branch-in-poky'>Checking Out By Branch in Poky</link>"
1094 and
1095 <link linkend='checkout-out-by-tag-in-poky'>Checking Out By Tag in Poky</link>"
1096 sections, respectively.</para>
1097
1098 <para>Once the local repository is created, you can
1099 change to that directory and check its status.
1100 Here, the single "master" branch exists on your system
1101 and by default, it is checked out:
1102 <literallayout class='monospaced'>
1103 $ cd ~/poky
1104 $ git status
1105 On branch master
1106 Your branch is up-to-date with 'origin/master'.
1107 nothing to commit, working directory clean
1108 $ git branch
1109 * master
1110 </literallayout>
1111 Your local repository of poky is identical to the
1112 upstream poky repository at the time from which it was
1113 cloned.
1114 As you work with the local branch, you can periodically
1115 use the <filename>git pull &dash;&dash;rebase</filename>
1116 command to be sure you are up-to-date with the upstream
1117 branch.
1118 </para></listitem>
1119 </orderedlist>
1120 </para>
1121 </section>
1122
1123 <section id='checking-out-by-branch-in-poky'>
1124 <title>Checking Out by Branch in Poky</title>
1125
1126 <para>
1127 When you clone the upstream poky repository, you have access to
1128 all its development branches.
1129 Each development branch in a repository is unique as it forks
1130 off the "master" branch.
1131 To see and use the files of a particular development branch
1132 locally, you need to know the branch name and then specifically
1133 check out that development branch.
1134 <note>
1135 Checking out an active development branch by branch name
1136 gives you a snapshot of that particular branch at the time
1137 you check it out.
1138 Further development on top of the branch that occurs after
1139 check it out can occur.
1140 </note>
1141 <orderedlist>
1142 <listitem><para>
1143 <emphasis>Switch to the Poky Directory:</emphasis>
1144 If you have a local poky Git repository, switch to that
1145 directory.
1146 If you do not have the local copy of poky, see the
1147 "<link linkend='cloning-the-poky-repository'>Cloning the <filename>poky</filename> Repository</link>"
1148 section.
1149 </para></listitem>
1150 <listitem><para>
1151 <emphasis>Determine Existing Branch Names:</emphasis>
1152 <literallayout class='monospaced'>
1153 $ git branch -a
1154 * master
1155 remotes/origin/1.1_M1
1156 remotes/origin/1.1_M2
1157 remotes/origin/1.1_M3
1158 remotes/origin/1.1_M4
1159 remotes/origin/1.2_M1
1160 remotes/origin/1.2_M2
1161 remotes/origin/1.2_M3
1162 .
1163 .
1164 .
1165 remotes/origin/thud
1166 remotes/origin/thud-next
1167 remotes/origin/warrior
1168 remotes/origin/warrior-next
1169 remotes/origin/zeus
1170 remotes/origin/zeus-next
1171 ... and so on ...
1172 </literallayout>
1173 </para></listitem>
1174 <listitem><para>
1175 <emphasis>Check out the Branch:</emphasis>
1176 Check out the development branch in which you want to work.
1177 For example, to access the files for the Yocto Project
1178 &DISTRO; Release (&DISTRO_NAME;), use the following command:
1179 <literallayout class='monospaced'>
1180 $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
1181 Branch &DISTRO_NAME_NO_CAP; set up to track remote branch &DISTRO_NAME_NO_CAP; from origin.
1182 Switched to a new branch '&DISTRO_NAME_NO_CAP;'
1183 </literallayout>
1184 The previous command checks out the "&DISTRO_NAME_NO_CAP;"
1185 development branch and reports that the branch is tracking
1186 the upstream "origin/&DISTRO_NAME_NO_CAP;" branch.</para>
1187
1188 <para>The following command displays the branches
1189 that are now part of your local poky repository.
1190 The asterisk character indicates the branch that is
1191 currently checked out for work:
1192 <literallayout class='monospaced'>
1193 $ git branch
1194 master
1195 * &DISTRO_NAME_NO_CAP;
1196 </literallayout>
1197 </para></listitem>
1198 </orderedlist>
1199 </para>
1200 </section>
1201
1202 <section id='checkout-out-by-tag-in-poky'>
1203 <title>Checking Out by Tag in Poky</title>
1204
1205 <para>
1206 Similar to branches, the upstream repository uses tags
1207 to mark specific commits associated with significant points in
1208 a development branch (i.e. a release point or stage of a
1209 release).
1210 You might want to set up a local branch based on one of those
1211 points in the repository.
1212 The process is similar to checking out by branch name except you
1213 use tag names.
1214 <note>
1215 Checking out a branch based on a tag gives you a
1216 stable set of files not affected by development on the
1217 branch above the tag.
1218 </note>
1219 <orderedlist>
1220 <listitem><para>
1221 <emphasis>Switch to the Poky Directory:</emphasis>
1222 If you have a local poky Git repository, switch to that
1223 directory.
1224 If you do not have the local copy of poky, see the
1225 "<link linkend='cloning-the-poky-repository'>Cloning the <filename>poky</filename> Repository</link>"
1226 section.
1227 </para></listitem>
1228 <listitem><para>
1229 <emphasis>Fetch the Tag Names:</emphasis>
1230 To checkout the branch based on a tag name, you need to
1231 fetch the upstream tags into your local repository:
1232 <literallayout class='monospaced'>
1233 $ git fetch --tags
1234 $
1235 </literallayout>
1236 </para></listitem>
1237 <listitem><para>
1238 <emphasis>List the Tag Names:</emphasis>
1239 You can list the tag names now:
1240 <literallayout class='monospaced'>
1241 $ git tag
1242 1.1_M1.final
1243 1.1_M1.rc1
1244 1.1_M1.rc2
1245 1.1_M2.final
1246 1.1_M2.rc1
1247 .
1248 .
1249 .
1250 yocto-2.5
1251 yocto-2.5.1
1252 yocto-2.5.2
1253 yocto-2.5.3
1254 yocto-2.6
1255 yocto-2.6.1
1256 yocto-2.6.2
1257 yocto-2.7
1258 yocto_1.5_M5.rc8
1259 </literallayout>
1260 </para></listitem>
1261 <listitem><para>
1262 <emphasis>Check out the Branch:</emphasis>
1263 <literallayout class='monospaced'>
1264 $ git checkout tags/&DISTRO_REL_TAG; -b my_yocto_&DISTRO;
1265 Switched to a new branch 'my_yocto_&DISTRO;'
1266 $ git branch
1267 master
1268 * my_yocto_&DISTRO;
1269 </literallayout>
1270 The previous command creates and checks out a local
1271 branch named "my_yocto_&DISTRO;", which is based on
1272 the commit in the upstream poky repository that has
1273 the same tag.
1274 In this example, the files you have available locally
1275 as a result of the <filename>checkout</filename>
1276 command are a snapshot of the
1277 "&DISTRO_NAME_NO_CAP;" development branch at the point
1278 where Yocto Project &DISTRO; was released.
1279 </para></listitem>
1280 </orderedlist>
1281 </para>
1282 </section>
1283</section>
1284
1285</chapter>
1286<!--
1287vim: expandtab tw=80 ts=4
1288-->
diff --git a/documentation/dev-manual/dev-manual.xml b/documentation/dev-manual/dev-manual.xml
deleted file mode 100755
index 66439930e4..0000000000
--- a/documentation/dev-manual/dev-manual.xml
+++ /dev/null
@@ -1,195 +0,0 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<book id='dev-manual' lang='en'
7 xmlns:xi="http://www.w3.org/2003/XInclude"
8 xmlns="http://docbook.org/ns/docbook"
9 >
10 <bookinfo>
11
12 <mediaobject>
13 <imageobject>
14 <imagedata fileref='figures/dev-title.png'
15 format='SVG'
16 align='left' scalefit='1' width='100%'/>
17 </imageobject>
18 </mediaobject>
19
20 <title>
21 Yocto Project Development Tasks Manual
22 </title>
23
24 <authorgroup>
25 <author>
26 <affiliation>
27 <orgname>&ORGNAME;</orgname>
28 </affiliation>
29 <email>&ORGEMAIL;</email>
30 </author>
31 </authorgroup>
32
33 <revhistory>
34 <revision>
35 <revnumber>1.1</revnumber>
36 <date>October 2011</date>
37 <revremark>The initial document released with the Yocto Project 1.1 Release.</revremark>
38 </revision>
39 <revision>
40 <revnumber>1.2</revnumber>
41 <date>April 2012</date>
42 <revremark>Released with the Yocto Project 1.2 Release.</revremark>
43 </revision>
44 <revision>
45 <revnumber>1.3</revnumber>
46 <date>October 2012</date>
47 <revremark>Released with the Yocto Project 1.3 Release.</revremark>
48 </revision>
49 <revision>
50 <revnumber>1.4</revnumber>
51 <date>April 2013</date>
52 <revremark>Released with the Yocto Project 1.4 Release.</revremark>
53 </revision>
54 <revision>
55 <revnumber>1.5</revnumber>
56 <date>October 2013</date>
57 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
58 </revision>
59 <revision>
60 <revnumber>1.6</revnumber>
61 <date>April 2014</date>
62 <revremark>Released with the Yocto Project 1.6 Release.</revremark>
63 </revision>
64 <revision>
65 <revnumber>1.7</revnumber>
66 <date>October 2014</date>
67 <revremark>Released with the Yocto Project 1.7 Release.</revremark>
68 </revision>
69 <revision>
70 <revnumber>1.8</revnumber>
71 <date>April 2015</date>
72 <revremark>Released with the Yocto Project 1.8 Release.</revremark>
73 </revision>
74 <revision>
75 <revnumber>2.0</revnumber>
76 <date>October 2015</date>
77 <revremark>Released with the Yocto Project 2.0 Release.</revremark>
78 </revision>
79 <revision>
80 <revnumber>2.1</revnumber>
81 <date>April 2016</date>
82 <revremark>Released with the Yocto Project 2.1 Release.</revremark>
83 </revision>
84 <revision>
85 <revnumber>2.2</revnumber>
86 <date>October 2016</date>
87 <revremark>Released with the Yocto Project 2.2 Release.</revremark>
88 </revision>
89 <revision>
90 <revnumber>2.3</revnumber>
91 <date>May 2017</date>
92 <revremark>Released with the Yocto Project 2.3 Release.</revremark>
93 </revision>
94 <revision>
95 <revnumber>2.4</revnumber>
96 <date>October 2017</date>
97 <revremark>Released with the Yocto Project 2.4 Release.</revremark>
98 </revision>
99 <revision>
100 <revnumber>2.5</revnumber>
101 <date>May 2018</date>
102 <revremark>Released with the Yocto Project 2.5 Release.</revremark>
103 </revision>
104 <revision>
105 <revnumber>2.6</revnumber>
106 <date>November 2018</date>
107 <revremark>Released with the Yocto Project 2.6 Release.</revremark>
108 </revision>
109 <revision>
110 <revnumber>2.7</revnumber>
111 <date>May 2019</date>
112 <revremark>Released with the Yocto Project 2.7 Release.</revremark>
113 </revision>
114 <revision>
115 <revnumber>3.0</revnumber>
116 <date>October 2019</date>
117 <revremark>Released with the Yocto Project 3.0 Release.</revremark>
118 </revision>
119 <revision>
120 <revnumber>3.1</revnumber>
121 <date>&REL_MONTH_YEAR;</date>
122 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
123 </revision>
124 </revhistory>
125
126 <copyright>
127 <year>&COPYRIGHT_YEAR;</year>
128 <holder>Linux Foundation</holder>
129 </copyright>
130
131 <legalnotice>
132 <para>
133 Permission is granted to copy, distribute and/or modify this document under
134 the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">
135 Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by
136 Creative Commons.
137 </para>
138 <note><title>Manual Notes</title>
139 <itemizedlist>
140 <listitem><para>
141 This version of the
142 <emphasis>Yocto Project Development Tasks Manual</emphasis>
143 is for the &YOCTO_DOC_VERSION; release of the
144 Yocto Project.
145 To be sure you have the latest version of the manual
146 for this release, go to the
147 <ulink url='&YOCTO_DOCS_URL;'>Yocto Project documentation page</ulink>
148 and select the manual from that site.
149 Manuals from the site are more up-to-date than manuals
150 derived from the Yocto Project released TAR files.
151 </para></listitem>
152 <listitem><para>
153 If you located this manual through a web search, the
154 version of the manual might not be the one you want
155 (e.g. the search might have returned a manual much
156 older than the Yocto Project version with which you
157 are working).
158 You can see all Yocto Project major releases by
159 visiting the
160 <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Releases</ulink>
161 page.
162 If you need a version of this manual for a different
163 Yocto Project release, visit the
164 <ulink url='&YOCTO_DOCS_URL;'>Yocto Project documentation page</ulink>
165 and select the manual set by using the
166 "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
167 pull-down menus.
168 </para></listitem>
169 <listitem>
170 <para>
171 To report any inaccuracies or problems with this
172 (or any other Yocto Project) manual, send an email to
173 the Yocto Project documentation mailing list at
174 <filename>docs@lists.yoctoproject.org</filename> or
175 log into the freenode <filename>#yocto</filename> channel.
176 </para>
177 </listitem>
178 </itemizedlist>
179 </note>
180 </legalnotice>
181
182 </bookinfo>
183
184 <xi:include href="dev-manual-intro.xml"/>
185
186 <xi:include href="dev-manual-start.xml"/>
187
188 <xi:include href="dev-manual-common-tasks.xml"/>
189
190 <xi:include href="dev-manual-qemu.xml"/>
191
192</book>
193<!--
194vim: expandtab tw=80 ts=4
195-->
diff --git a/documentation/dev-manual/dev-style.css b/documentation/dev-manual/dev-style.css
deleted file mode 100644
index 331c7c54d4..0000000000
--- a/documentation/dev-manual/dev-style.css
+++ /dev/null
@@ -1,991 +0,0 @@
1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
5 Generic XHTML / DocBook XHTML CSS Stylesheet.
6
7 Browser wrangling and typographic design by
8 Oyvind Kolas / pippin@gimp.org
9
10 Customised for Poky by
11 Matthew Allum / mallum@o-hand.com
12
13 Thanks to:
14 Liam R. E. Quin
15 William Skaggs
16 Jakub Steiner
17
18 Structure
19 ---------
20
21 The stylesheet is divided into the following sections:
22
23 Positioning
24 Margins, paddings, width, font-size, clearing.
25 Decorations
26 Borders, style
27 Colors
28 Colors
29 Graphics
30 Graphical backgrounds
31 Nasty IE tweaks
32 Workarounds needed to make it work in internet explorer,
33 currently makes the stylesheet non validating, but up until
34 this point it is validating.
35 Mozilla extensions
36 Transparency for footer
37 Rounded corners on boxes
38
39*/
40
41
42 /*************** /
43 / Positioning /
44/ ***************/
45
46body {
47 font-family: Verdana, Sans, sans-serif;
48
49 min-width: 640px;
50 width: 80%;
51 margin: 0em auto;
52 padding: 2em 5em 5em 5em;
53 color: #333;
54}
55
56h1,h2,h3,h4,h5,h6,h7 {
57 font-family: Arial, Sans;
58 color: #00557D;
59 clear: both;
60}
61
62h1 {
63 font-size: 2em;
64 text-align: left;
65 padding: 0em 0em 0em 0em;
66 margin: 2em 0em 0em 0em;
67}
68
69h2.subtitle {
70 margin: 0.10em 0em 3.0em 0em;
71 padding: 0em 0em 0em 0em;
72 font-size: 1.8em;
73 padding-left: 20%;
74 font-weight: normal;
75 font-style: italic;
76}
77
78h2 {
79 margin: 2em 0em 0.66em 0em;
80 padding: 0.5em 0em 0em 0em;
81 font-size: 1.5em;
82 font-weight: bold;
83}
84
85h3.subtitle {
86 margin: 0em 0em 1em 0em;
87 padding: 0em 0em 0em 0em;
88 font-size: 142.14%;
89 text-align: right;
90}
91
92h3 {
93 margin: 1em 0em 0.5em 0em;
94 padding: 1em 0em 0em 0em;
95 font-size: 140%;
96 font-weight: bold;
97}
98
99h4 {
100 margin: 1em 0em 0.5em 0em;
101 padding: 1em 0em 0em 0em;
102 font-size: 120%;
103 font-weight: bold;
104}
105
106h5 {
107 margin: 1em 0em 0.5em 0em;
108 padding: 1em 0em 0em 0em;
109 font-size: 110%;
110 font-weight: bold;
111}
112
113h6 {
114 margin: 1em 0em 0em 0em;
115 padding: 1em 0em 0em 0em;
116 font-size: 110%;
117 font-weight: bold;
118}
119
120.authorgroup {
121 background-color: transparent;
122 background-repeat: no-repeat;
123 padding-top: 256px;
124 background-image: url("figures/dev-title.png");
125 background-position: left top;
126 margin-top: -256px;
127 padding-right: 50px;
128 margin-left: 0px;
129 text-align: right;
130 width: 740px;
131}
132
133h3.author {
134 margin: 0em 0me 0em 0em;
135 padding: 0em 0em 0em 0em;
136 font-weight: normal;
137 font-size: 100%;
138 color: #333;
139 clear: both;
140}
141
142.author tt.email {
143 font-size: 66%;
144}
145
146.titlepage hr {
147 width: 0em;
148 clear: both;
149}
150
151.revhistory {
152 padding-top: 2em;
153 clear: both;
154}
155
156.toc,
157.list-of-tables,
158.list-of-examples,
159.list-of-figures {
160 padding: 1.33em 0em 2.5em 0em;
161 color: #00557D;
162}
163
164.toc p,
165.list-of-tables p,
166.list-of-figures p,
167.list-of-examples p {
168 padding: 0em 0em 0em 0em;
169 padding: 0em 0em 0.3em;
170 margin: 1.5em 0em 0em 0em;
171}
172
173.toc p b,
174.list-of-tables p b,
175.list-of-figures p b,
176.list-of-examples p b{
177 font-size: 100.0%;
178 font-weight: bold;
179}
180
181.toc dl,
182.list-of-tables dl,
183.list-of-figures dl,
184.list-of-examples dl {
185 margin: 0em 0em 0.5em 0em;
186 padding: 0em 0em 0em 0em;
187}
188
189.toc dt {
190 margin: 0em 0em 0em 0em;
191 padding: 0em 0em 0em 0em;
192}
193
194.toc dd {
195 margin: 0em 0em 0em 2.6em;
196 padding: 0em 0em 0em 0em;
197}
198
199div.glossary dl,
200div.variablelist dl {
201}
202
203.glossary dl dt,
204.variablelist dl dt,
205.variablelist dl dt span.term {
206 font-weight: normal;
207 width: 20em;
208 text-align: right;
209}
210
211.variablelist dl dt {
212 margin-top: 0.5em;
213}
214
215.glossary dl dd,
216.variablelist dl dd {
217 margin-top: -1em;
218 margin-left: 25.5em;
219}
220
221.glossary dd p,
222.variablelist dd p {
223 margin-top: 0em;
224 margin-bottom: 1em;
225}
226
227
228div.calloutlist table td {
229 padding: 0em 0em 0em 0em;
230 margin: 0em 0em 0em 0em;
231}
232
233div.calloutlist table td p {
234 margin-top: 0em;
235 margin-bottom: 1em;
236}
237
238div p.copyright {
239 text-align: left;
240}
241
242div.legalnotice p.legalnotice-title {
243 margin-bottom: 0em;
244}
245
246p {
247 line-height: 1.5em;
248 margin-top: 0em;
249
250}
251
252dl {
253 padding-top: 0em;
254}
255
256hr {
257 border: solid 1px;
258}
259
260
261.mediaobject,
262.mediaobjectco {
263 text-align: center;
264}
265
266img {
267 border: none;
268}
269
270ul {
271 padding: 0em 0em 0em 1.5em;
272}
273
274ul li {
275 padding: 0em 0em 0em 0em;
276}
277
278ul li p {
279 text-align: left;
280}
281
282table {
283 width :100%;
284}
285
286th {
287 padding: 0.25em;
288 text-align: left;
289 font-weight: normal;
290 vertical-align: top;
291}
292
293td {
294 padding: 0.25em;
295 vertical-align: top;
296}
297
298p a[id] {
299 margin: 0px;
300 padding: 0px;
301 display: inline;
302 background-image: none;
303}
304
305a {
306 text-decoration: underline;
307 color: #444;
308}
309
310pre {
311 overflow: auto;
312}
313
314a:hover {
315 text-decoration: underline;
316 /*font-weight: bold;*/
317}
318
319/* This style defines how the permalink character
320 appears by itself and when hovered over with
321 the mouse. */
322
323[alt='Permalink'] { color: #eee; }
324[alt='Permalink']:hover { color: black; }
325
326
327div.informalfigure,
328div.informalexample,
329div.informaltable,
330div.figure,
331div.table,
332div.example {
333 margin: 1em 0em;
334 padding: 1em;
335 page-break-inside: avoid;
336}
337
338
339div.informalfigure p.title b,
340div.informalexample p.title b,
341div.informaltable p.title b,
342div.figure p.title b,
343div.example p.title b,
344div.table p.title b{
345 padding-top: 0em;
346 margin-top: 0em;
347 font-size: 100%;
348 font-weight: normal;
349}
350
351.mediaobject .caption,
352.mediaobject .caption p {
353 text-align: center;
354 font-size: 80%;
355 padding-top: 0.5em;
356 padding-bottom: 0.5em;
357}
358
359.epigraph {
360 padding-left: 55%;
361 margin-bottom: 1em;
362}
363
364.epigraph p {
365 text-align: left;
366}
367
368.epigraph .quote {
369 font-style: italic;
370}
371.epigraph .attribution {
372 font-style: normal;
373 text-align: right;
374}
375
376span.application {
377 font-style: italic;
378}
379
380.programlisting {
381 font-family: monospace;
382 font-size: 80%;
383 white-space: pre;
384 margin: 1.33em 0em;
385 padding: 1.33em;
386}
387
388.tip,
389.warning,
390.caution,
391.note {
392 margin-top: 1em;
393 margin-bottom: 1em;
394
395}
396
397/* force full width of table within div */
398.tip table,
399.warning table,
400.caution table,
401.note table {
402 border: none;
403 width: 100%;
404}
405
406
407.tip table th,
408.warning table th,
409.caution table th,
410.note table th {
411 padding: 0.8em 0.0em 0.0em 0.0em;
412 margin : 0em 0em 0em 0em;
413}
414
415.tip p,
416.warning p,
417.caution p,
418.note p {
419 margin-top: 0.5em;
420 margin-bottom: 0.5em;
421 padding-right: 1em;
422 text-align: left;
423}
424
425.acronym {
426 text-transform: uppercase;
427}
428
429b.keycap,
430.keycap {
431 padding: 0.09em 0.3em;
432 margin: 0em;
433}
434
435.itemizedlist li {
436 clear: none;
437}
438
439.filename {
440 font-size: medium;
441 font-family: Courier, monospace;
442}
443
444
445div.navheader, div.heading{
446 position: absolute;
447 left: 0em;
448 top: 0em;
449 width: 100%;
450 background-color: #cdf;
451 width: 100%;
452}
453
454div.navfooter, div.footing{
455 position: fixed;
456 left: 0em;
457 bottom: 0em;
458 background-color: #eee;
459 width: 100%;
460}
461
462
463div.navheader td,
464div.navfooter td {
465 font-size: 66%;
466}
467
468div.navheader table th {
469 /*font-family: Georgia, Times, serif;*/
470 /*font-size: x-large;*/
471 font-size: 80%;
472}
473
474div.navheader table {
475 border-left: 0em;
476 border-right: 0em;
477 border-top: 0em;
478 width: 100%;
479}
480
481div.navfooter table {
482 border-left: 0em;
483 border-right: 0em;
484 border-bottom: 0em;
485 width: 100%;
486}
487
488div.navheader table td a,
489div.navfooter table td a {
490 color: #777;
491 text-decoration: none;
492}
493
494/* normal text in the footer */
495div.navfooter table td {
496 color: black;
497}
498
499div.navheader table td a:visited,
500div.navfooter table td a:visited {
501 color: #444;
502}
503
504
505/* links in header and footer */
506div.navheader table td a:hover,
507div.navfooter table td a:hover {
508 text-decoration: underline;
509 background-color: transparent;
510 color: #33a;
511}
512
513div.navheader hr,
514div.navfooter hr {
515 display: none;
516}
517
518
519.qandaset tr.question td p {
520 margin: 0em 0em 1em 0em;
521 padding: 0em 0em 0em 0em;
522}
523
524.qandaset tr.answer td p {
525 margin: 0em 0em 1em 0em;
526 padding: 0em 0em 0em 0em;
527}
528.answer td {
529 padding-bottom: 1.5em;
530}
531
532.emphasis {
533 font-weight: bold;
534}
535
536
537 /************* /
538 / decorations /
539/ *************/
540
541.titlepage {
542}
543
544.part .title {
545}
546
547.subtitle {
548 border: none;
549}
550
551/*
552h1 {
553 border: none;
554}
555
556h2 {
557 border-top: solid 0.2em;
558 border-bottom: solid 0.06em;
559}
560
561h3 {
562 border-top: 0em;
563 border-bottom: solid 0.06em;
564}
565
566h4 {
567 border: 0em;
568 border-bottom: solid 0.06em;
569}
570
571h5 {
572 border: 0em;
573}
574*/
575
576.programlisting {
577 border: solid 1px;
578}
579
580div.figure,
581div.table,
582div.informalfigure,
583div.informaltable,
584div.informalexample,
585div.example {
586 border: 1px solid;
587}
588
589
590
591.tip,
592.warning,
593.caution,
594.note {
595 border: 1px solid;
596}
597
598.tip table th,
599.warning table th,
600.caution table th,
601.note table th {
602 border-bottom: 1px solid;
603}
604
605.question td {
606 border-top: 1px solid black;
607}
608
609.answer {
610}
611
612
613b.keycap,
614.keycap {
615 border: 1px solid;
616}
617
618
619div.navheader, div.heading{
620 border-bottom: 1px solid;
621}
622
623
624div.navfooter, div.footing{
625 border-top: 1px solid;
626}
627
628 /********* /
629 / colors /
630/ *********/
631
632body {
633 color: #333;
634 background: white;
635}
636
637a {
638 background: transparent;
639}
640
641a:hover {
642 background-color: #dedede;
643}
644
645
646h1,
647h2,
648h3,
649h4,
650h5,
651h6,
652h7,
653h8 {
654 background-color: transparent;
655}
656
657hr {
658 border-color: #aaa;
659}
660
661
662.tip, .warning, .caution, .note {
663 border-color: #fff;
664}
665
666
667.tip table th,
668.warning table th,
669.caution table th,
670.note table th {
671 border-bottom-color: #fff;
672}
673
674
675.warning {
676 background-color: #f0f0f2;
677}
678
679.caution {
680 background-color: #f0f0f2;
681}
682
683.tip {
684 background-color: #f0f0f2;
685}
686
687.note {
688 background-color: #f0f0f2;
689}
690
691.glossary dl dt,
692.variablelist dl dt,
693.variablelist dl dt span.term {
694 color: #044;
695}
696
697div.figure,
698div.table,
699div.example,
700div.informalfigure,
701div.informaltable,
702div.informalexample {
703 border-color: #aaa;
704}
705
706pre.programlisting {
707 color: black;
708 background-color: #fff;
709 border-color: #aaa;
710 border-width: 2px;
711}
712
713.guimenu,
714.guilabel,
715.guimenuitem {
716 background-color: #eee;
717}
718
719
720b.keycap,
721.keycap {
722 background-color: #eee;
723 border-color: #999;
724}
725
726
727div.navheader {
728 border-color: black;
729}
730
731
732div.navfooter {
733 border-color: black;
734}
735
736.writernotes {
737 color: red;
738}
739
740
741 /*********** /
742 / graphics /
743/ ***********/
744
745/*
746body {
747 background-image: url("images/body_bg.jpg");
748 background-attachment: fixed;
749}
750
751.navheader,
752.note,
753.tip {
754 background-image: url("images/note_bg.jpg");
755 background-attachment: fixed;
756}
757
758.warning,
759.caution {
760 background-image: url("images/warning_bg.jpg");
761 background-attachment: fixed;
762}
763
764.figure,
765.informalfigure,
766.example,
767.informalexample,
768.table,
769.informaltable {
770 background-image: url("images/figure_bg.jpg");
771 background-attachment: fixed;
772}
773
774*/
775h1,
776h2,
777h3,
778h4,
779h5,
780h6,
781h7{
782}
783
784/*
785Example of how to stick an image as part of the title.
786
787div.article .titlepage .title
788{
789 background-image: url("figures/white-on-black.png");
790 background-position: center;
791 background-repeat: repeat-x;
792}
793*/
794
795div.preface .titlepage .title,
796div.colophon .title,
797div.chapter .titlepage .title,
798div.article .titlepage .title
799{
800}
801
802div.section div.section .titlepage .title,
803div.sect2 .titlepage .title {
804 background: none;
805}
806
807
808h1.title {
809 background-color: transparent;
810 background-repeat: no-repeat;
811 height: 256px;
812 text-indent: -9000px;
813 overflow:hidden;
814}
815
816h2.subtitle {
817 background-color: transparent;
818 text-indent: -9000px;
819 overflow:hidden;
820 width: 0px;
821 display: none;
822}
823
824 /*************************************** /
825 / pippin.gimp.org specific alterations /
826/ ***************************************/
827
828/*
829div.heading, div.navheader {
830 color: #777;
831 font-size: 80%;
832 padding: 0;
833 margin: 0;
834 text-align: left;
835 position: absolute;
836 top: 0px;
837 left: 0px;
838 width: 100%;
839 height: 50px;
840 background: url('/gfx/heading_bg.png') transparent;
841 background-repeat: repeat-x;
842 background-attachment: fixed;
843 border: none;
844}
845
846div.heading a {
847 color: #444;
848}
849
850div.footing, div.navfooter {
851 border: none;
852 color: #ddd;
853 font-size: 80%;
854 text-align:right;
855
856 width: 100%;
857 padding-top: 10px;
858 position: absolute;
859 bottom: 0px;
860 left: 0px;
861
862 background: url('/gfx/footing_bg.png') transparent;
863}
864*/
865
866
867
868 /****************** /
869 / nasty ie tweaks /
870/ ******************/
871
872/*
873div.heading, div.navheader {
874 width:expression(document.body.clientWidth + "px");
875}
876
877div.footing, div.navfooter {
878 width:expression(document.body.clientWidth + "px");
879 margin-left:expression("-5em");
880}
881body {
882 padding:expression("4em 5em 0em 5em");
883}
884*/
885
886 /**************************************** /
887 / mozilla vendor specific css extensions /
888/ ****************************************/
889/*
890div.navfooter, div.footing{
891 -moz-opacity: 0.8em;
892}
893
894div.figure,
895div.table,
896div.informalfigure,
897div.informaltable,
898div.informalexample,
899div.example,
900.tip,
901.warning,
902.caution,
903.note {
904 -moz-border-radius: 0.5em;
905}
906
907b.keycap,
908.keycap {
909 -moz-border-radius: 0.3em;
910}
911*/
912
913table tr td table tr td {
914 display: none;
915}
916
917
918hr {
919 display: none;
920}
921
922table {
923 border: 0em;
924}
925
926 .photo {
927 float: right;
928 margin-left: 1.5em;
929 margin-bottom: 1.5em;
930 margin-top: 0em;
931 max-width: 17em;
932 border: 1px solid gray;
933 padding: 3px;
934 background: white;
935}
936 .seperator {
937 padding-top: 2em;
938 clear: both;
939 }
940
941 #validators {
942 margin-top: 5em;
943 text-align: right;
944 color: #777;
945 }
946 @media print {
947 body {
948 font-size: 8pt;
949 }
950 .noprint {
951 display: none;
952 }
953 }
954
955
956.tip,
957.note {
958 background: #f0f0f2;
959 color: #333;
960 padding: 20px;
961 margin: 20px;
962}
963
964.tip h3,
965.note h3 {
966 padding: 0em;
967 margin: 0em;
968 font-size: 2em;
969 font-weight: bold;
970 color: #333;
971}
972
973.tip a,
974.note a {
975 color: #333;
976 text-decoration: underline;
977}
978
979.footnote {
980 font-size: small;
981 color: #333;
982}
983
984/* Changes the announcement text */
985.tip h3,
986.warning h3,
987.caution h3,
988.note h3 {
989 font-size:large;
990 color: #00557D;
991}
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-01-23 17:40:16 +0000