summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual/dev-manual-common-tasks.xml
diff options
context:
space:
mode:
authorAdrian Dudau <adrian.dudau@enea.com>2013-12-12 13:46:05 +0100
committerAdrian Dudau <adrian.dudau@enea.com>2013-12-12 13:47:32 +0100
commit14b00ff23a912494edc7f146e668c40ca82b8508 (patch)
treef7f4e592ccb935bc312cfa0cfc3c0cbbe444970d /documentation/dev-manual/dev-manual-common-tasks.xml
downloadyocto-docs-master.tar.gz
initial commit of Enea Linux 3.1HEADmaster
Migrated from the internal git server on the dora-enea branch Signed-off-by: Adrian Dudau <adrian.dudau@enea.com>
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.xml')
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml6603
1 files changed, 6603 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
new file mode 100644
index 0000000..305dc31
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -0,0 +1,6603 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='extendpoky'>
6
7<title>Common Tasks</title>
8 <para>
9 This chapter describes fundamental procedures such as creating layers,
10 adding new software packages, extending or customizing images,
11 porting work to new hardware (adding a new machine), and so forth.
12 You will find the procedures documented here occur often in the
13 development cycle using the Yocto Project.
14 </para>
15
16 <section id="understanding-and-creating-layers">
17 <title>Understanding and Creating Layers</title>
18
19 <para>
20 The OpenEmbedded build system supports organizing
21 <link linkend='metadata'>Metadata</link> into multiple layers.
22 Layers allow you to isolate different types of customizations from
23 each other.
24 You might find it tempting to keep everything in one layer when
25 working on a single project.
26 However, the more modular you organize your Metadata, the easier
27 it is to cope with future changes.
28 </para>
29
30 <para>
31 To illustrate how layers are used to keep things modular, consider
32 machine customizations.
33 These types of customizations typically reside in a special layer,
34 rather than a general layer, called a Board Support Package (BSP)
35 Layer.
36 Furthermore, the machine customizations should be isolated from
37 recipes and Metadata that support a new GUI environment,
38 for example.
39 This situation gives you a couple of layers: one for the machine
40 configurations, and one for the GUI environment.
41 It is important to understand, however, that the BSP layer can
42 still make machine-specific additions to recipes within the GUI
43 environment layer without polluting the GUI layer itself
44 with those machine-specific changes.
45 You can accomplish this through a recipe that is a BitBake append
46 (<filename>.bbappend</filename>) file, which is described later
47 in this section.
48 </para>
49
50 <para>
51 </para>
52
53 <section id='yocto-project-layers'>
54 <title>Layers</title>
55
56 <para>
57 The <link linkend='source-directory'>Source Directory</link>
58 contains both general layers and BSP
59 layers right out of the box.
60 You can easily identify layers that ship with a
61 Yocto Project release in the Source Directory by their
62 folder names.
63 Folders that represent layers typically have names that begin with
64 the string <filename>meta-</filename>.
65 <note>
66 It is not a requirement that a layer name begin with the
67 prefix <filename>meta-</filename>, but it's a commonly accepted
68 standard in the Yocto Project community.
69 </note>
70 For example, when you set up the Source Directory structure,
71 you will see several layers:
72 <filename>meta</filename>, <filename>meta-hob</filename>,
73 <filename>meta-skeleton</filename>,
74 <filename>meta-yocto</filename>, and
75 <filename>meta-yocto-bsp</filename>.
76 Each of these folders represents a distinct layer.
77 </para>
78
79 <para>
80 Furthermore, if you set up a local copy of the
81 <filename>meta-intel</filename> Git repository
82 and then explore the folder of that general layer,
83 you will discover many Intel-specific BSP layers inside.
84 For more information on BSP layers, see the
85 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
86 section in the Yocto Project Board Support Package (BSP)
87 Developer's Guide.
88 </para>
89 </section>
90
91 <section id='creating-your-own-layer'>
92 <title>Creating Your Own Layer</title>
93
94 <para>
95 It is very easy to create your own layers to use with the
96 OpenEmbedded build system.
97 The Yocto Project ships with scripts that speed up creating
98 general layers and BSP layers.
99 This section describes the steps you perform by hand to create
100 a layer so that you can better understand them.
101 For information about the layer-creation scripts, see the
102 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
103 section in the Yocto Project Board Support Package (BSP)
104 Developer's Guide and the
105 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
106 section further down in this manual.
107 </para>
108
109 <para>
110 Follow these general steps to create your layer:
111 <orderedlist>
112 <listitem><para><emphasis>Check Existing Layers:</emphasis>
113 Before creating a new layer, you should be sure someone
114 has not already created a layer containing the Metadata
115 you need.
116 You can see the
117 <ulink url='http://layers.openembedded.org/layerindex/layers/'><filename>OpenEmbedded Metadata Index</filename></ulink>
118 for a list of layers from the OpenEmbedded community
119 that can be used in the Yocto Project.
120 </para></listitem>
121 <listitem><para><emphasis>Create a Directory:</emphasis>
122 Create the directory for your layer.
123 While not strictly required, prepend the name of the
124 folder with the string <filename>meta-</filename>.
125 For example:
126 <literallayout class='monospaced'>
127 meta-mylayer
128 meta-GUI_xyz
129 meta-mymachine
130 </literallayout>
131 </para></listitem>
132 <listitem><para><emphasis>Create a Layer Configuration
133 File:</emphasis>
134 Inside your new layer folder, you need to create a
135 <filename>conf/layer.conf</filename> file.
136 It is easiest to take an existing layer configuration
137 file and copy that to your layer's
138 <filename>conf</filename> directory and then modify the
139 file as needed.</para>
140 <para>The
141 <filename>meta-yocto-bsp/conf/layer.conf</filename> file
142 demonstrates the required syntax:
143 <literallayout class='monospaced'>
144 # We have a conf and classes directory, add to BBPATH
145 BBPATH .= ":${LAYERDIR}"
146
147 # We have recipes-* directories, add to BBFILES
148 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
149 ${LAYERDIR}/recipes-*/*/*.bbappend"
150
151 BBFILE_COLLECTIONS += "yoctobsp"
152 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
153 BBFILE_PRIORITY_yoctobsp = "5"
154 LAYERVERSION_yoctobsp = "2"
155 </literallayout></para>
156 <para>Here is an explanation of the example:
157 <itemizedlist>
158 <listitem><para>The configuration and
159 classes directory is appended to
160 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>.
161 <note>
162 All non-distro layers, which include all BSP
163 layers, are expected to append the layer
164 directory to the
165 <filename>BBPATH</filename>.
166 On the other hand, distro layers, such as
167 <filename>meta-yocto</filename>, can choose
168 to enforce their own precedence over
169 <filename>BBPATH</filename>.
170 For an example of that syntax, see the
171 <filename>layer.conf</filename> file for
172 the <filename>meta-yocto</filename> layer.
173 </note></para></listitem>
174 <listitem><para>The recipes for the layers are
175 appended to
176 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'>BBFILES</ulink></filename>.
177 </para></listitem>
178 <listitem><para>The
179 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'>BBFILE_COLLECTIONS</ulink></filename>
180 variable is then appended with the layer name.
181 </para></listitem>
182 <listitem><para>The
183 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'>BBFILE_PATTERN</ulink></filename>
184 variable is set to a regular expression and is
185 used to match files from
186 <filename>BBFILES</filename> into a particular
187 layer.
188 In this case,
189 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
190 is used to make <filename>BBFILE_PATTERN</filename> match within the
191 layer's path.</para></listitem>
192 <listitem><para>The
193 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'>BBFILE_PRIORITY</ulink></filename>
194 variable then assigns a priority to the layer.
195 Applying priorities is useful in situations
196 where the same package might appear in multiple
197 layers and allows you to choose what layer
198 should take precedence.</para></listitem>
199 <listitem><para>The
200 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'>LAYERVERSION</ulink></filename>
201 variable optionally specifies the version of a
202 layer as a single number.</para></listitem>
203 </itemizedlist></para>
204 <para>Note the use of the
205 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
206 variable, which expands to the directory of the current
207 layer.</para>
208 <para>Through the use of the <filename>BBPATH</filename>
209 variable, BitBake locates <filename>.bbclass</filename>
210 files, configuration files, and files that are included
211 with <filename>include</filename> and
212 <filename>require</filename> statements.
213 For these cases, BitBake uses the first file that
214 matches the name found in <filename>BBPATH</filename>.
215 This is similar to the way the <filename>PATH</filename>
216 variable is used for binaries.
217 We recommend, therefore, that you use unique
218 <filename>.bbclass</filename> and configuration
219 filenames in your custom layer.</para></listitem>
220 <listitem><para><emphasis>Add Content:</emphasis> Depending
221 on the type of layer, add the content.
222 If the layer adds support for a machine, add the machine
223 configuration in a <filename>conf/machine/</filename>
224 file within the layer.
225 If the layer adds distro policy, add the distro
226 configuration in a <filename>conf/distro/</filename>
227 file with the layer.
228 If the layer introduces new recipes, put the recipes
229 you need in <filename>recipes-*</filename>
230 subdirectories within the layer.
231 <note>In order to be compliant with the Yocto Project,
232 a layer must contain a
233 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout-readme'>README file.</ulink>
234 </note></para></listitem>
235 </orderedlist>
236 </para>
237 </section>
238
239 <section id='best-practices-to-follow-when-creating-layers'>
240 <title>Best Practices to Follow When Creating Layers</title>
241
242 <para>
243 To create layers that are easier to maintain and that will
244 not impact builds for other machines, you should consider the
245 information in the following sections.
246 </para>
247
248 <section id='avoid-overlaying-entire-recipes'>
249 <title>Avoid "Overlaying" Entire Recipes</title>
250
251 <para>
252 Avoid "overlaying" entire recipes from other layers in your
253 configuration.
254 In other words, do not copy an entire recipe into your
255 layer and then modify it.
256 Rather, use <filename>.bbappend</filename> files to override the
257 only those parts of the original recipe you need to modify.
258 </para>
259 </section>
260
261 <section id='avoid-duplicating-include-files'>
262 <title>Avoid Duplicating Include Files</title>
263
264 <para>
265 Avoid duplicating include files.
266 Use <filename>.bbappend</filename> files for each recipe
267 that uses an include file.
268 Or, if you are introducing a new recipe that requires
269 the included file, use the path relative to the original
270 layer directory to refer to the file.
271 For example, use
272 <filename>require recipes-core/somepackage/somefile.inc</filename>
273 instead of <filename>require somefile.inc</filename>.
274 If you're finding you have to overlay the include file,
275 it could indicate a deficiency in the include file in
276 the layer to which it originally belongs.
277 If this is the case, you need to address that deficiency
278 instead of overlaying the include file.
279 </para>
280
281 <para>
282 For example, consider how support plug-ins for the Qt 4
283 database are configured.
284 The Source Directory does not have MySQL or PostgreSQL.
285 However, OpenEmbedded's layer <filename>meta-oe</filename>
286 does.
287 Consequently, <filename>meta-oe</filename> uses
288 <filename>.bbappend</filename> files to modify the
289 <filename>QT_SQL_DRIVER_FLAGS</filename> variable to
290 enable the appropriate plug-ins.
291 This variable was added to the <filename>qt4.inc</filename>
292 include file in the Source Directory specifically to allow
293 the <filename>meta-oe</filename> layer to be able to control
294 which plug-ins are built.
295 </para>
296 </section>
297
298 <section id='structure-your-layers'>
299 <title>Structure Your Layers</title>
300
301 <para>
302 Proper use of overrides within append files and placement
303 of machine-specific files within your layer can ensure that
304 a build is not using the wrong Metadata and negatively
305 impacting a build for a different machine.
306 Following are some examples:
307 <itemizedlist>
308 <listitem><para><emphasis>Modifying Variables to Support
309 a Different Machine:</emphasis>
310 Suppose you have a layer named
311 <filename>meta-one</filename> that adds support
312 for building machine "one".
313 To do so, you use an append file named
314 <filename>base-files.bbappend</filename> and
315 create a dependency on "foo" by altering the
316 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
317 variable:
318 <literallayout class='monospaced'>
319 DEPENDS = "foo"
320 </literallayout>
321 The dependency is created during any build that
322 includes the layer
323 <filename>meta-one</filename>.
324 However, you might not want this dependency
325 for all machines.
326 For example, suppose you are building for
327 machine "two" but your
328 <filename>bblayers.conf</filename> file has the
329 <filename>meta-one</filename> layer included.
330 During the build, the
331 <filename>base-files</filename> for machine
332 "two" will also have the dependency on
333 <filename>foo</filename>.</para>
334 <para>To make sure your changes apply only when
335 building machine "one", use a machine override
336 with the <filename>DEPENDS</filename> statement:
337 <literallayout class='monospaced'>
338 DEPENDS_one = "foo"
339 </literallayout>
340 You should follow the same strategy when using
341 <filename>_append</filename> and
342 <filename>_prepend</filename> operations:
343 <literallayout class='monospaced'>
344 DEPENDS_append_one = " foo"
345 DEPENDS_prepend_one = "foo "
346 </literallayout>
347 <note>
348 Avoiding "+=" and "=+" and using
349 machine-specific
350 <filename>_append</filename>
351 and <filename>_prepend</filename> operations
352 is recommended as well.
353 </note></para></listitem>
354 <listitem><para><emphasis>Place Machine-Specific Files
355 in Machine-Specific Locations:</emphasis>
356 When you have a base recipe, such as
357 <filename>base-files.bb</filename>, that
358 contains a
359 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
360 statement to a file, you can use an append file
361 to cause the build to use your own version of
362 the file.
363 For example, an append file in your layer at
364 <filename>meta-one/recipes-core/base-files/base-files.bbappend</filename>
365 could extend
366 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
367 using
368 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
369 as follows:
370 <literallayout class='monospaced'>
371 FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"
372 </literallayout>
373 The build for machine "one" will pick up your
374 machine-specific file as long as you have the
375 file in
376 <filename>meta-one/recipes-core/base-files/base-files/</filename>.
377 However, if you are building for a different
378 machine and the
379 <filename>bblayers.conf</filename> file includes
380 the <filename>meta-one</filename> layer and
381 the location of your machine-specific file is
382 the first location where that file is found
383 according to <filename>FILESPATH</filename>,
384 builds for all machines will also use that
385 machine-specific file.</para>
386 <para>You can make sure that a machine-specific
387 file is used for a particular machine by putting
388 the file in a subdirectory specific to the
389 machine.
390 For example, rather than placing the file in
391 <filename>meta-one/recipes-core/base-files/base-files/</filename>
392 as shown above, put it in
393 <filename>meta-one/recipes-core/base-files/base-files/one/</filename>.
394 Not only does this make sure the file is used
395 only when building for machine "one" but the
396 build process locates the file more quickly.</para>
397 <para>In summary, you need to place all files
398 referenced from <filename>SRC_URI</filename>
399 in a machine-specific subdirectory within the
400 layer in order to restrict those files to
401 machine-specific builds.</para></listitem>
402 </itemizedlist>
403 </para>
404 </section>
405
406 <section id='other-recommendations'>
407 <title>Other Recommendations</title>
408
409 <para>
410 We also recommend the following:
411 <itemizedlist>
412 <listitem><para>Store custom layers in a Git repository
413 that uses the
414 <filename>meta-&lt;layer_name&gt;</filename> format.
415 </para></listitem>
416 <listitem><para>Clone the repository alongside other
417 <filename>meta</filename> directories in the
418 <link linkend='source-directory'>Source Directory</link>.
419 </para></listitem>
420 </itemizedlist>
421 Following these recommendations keeps your Source Directory and
422 its configuration entirely inside the Yocto Project's core
423 base.
424 </para>
425 </section>
426 </section>
427
428 <section id='enabling-your-layer'>
429 <title>Enabling Your Layer</title>
430
431 <para>
432 Before the OpenEmbedded build system can use your new layer,
433 you need to enable it.
434 To enable your layer, simply add your layer's path to the
435 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename>
436 variable in your <filename>conf/bblayers.conf</filename> file,
437 which is found in the
438 <link linkend='build-directory'>Build Directory</link>.
439 The following example shows how to enable a layer named
440 <filename>meta-mylayer</filename>:
441 <literallayout class='monospaced'>
442 LCONF_VERSION = "6"
443
444 BBPATH = "${TOPDIR}"
445 BBFILES ?= ""
446
447 BBLAYERS ?= " \
448 $HOME/poky/meta \
449 $HOME/poky/meta-yocto \
450 $HOME/poky/meta-yocto-bsp \
451 $HOME/poky/meta-mylayer \
452 "
453
454 BBLAYERS_NON_REMOVABLE ?= " \
455 $HOME/poky/meta \
456 $HOME/poky/meta-yocto \
457 "
458 </literallayout>
459 </para>
460
461 <para>
462 BitBake parses each <filename>conf/layer.conf</filename> file
463 as specified in the <filename>BBLAYERS</filename> variable
464 within the <filename>conf/bblayers.conf</filename> file.
465 During the processing of each
466 <filename>conf/layer.conf</filename> file, BitBake adds the
467 recipes, classes and configurations contained within the
468 particular layer to the source directory.
469 </para>
470 </section>
471
472 <section id='using-bbappend-files'>
473 <title>Using .bbappend Files</title>
474
475 <para>
476 Recipes used to append Metadata to other recipes are called
477 BitBake append files.
478 BitBake append files use the <filename>.bbappend</filename> file
479 type suffix, while the corresponding recipes to which Metadata
480 is being appended use the <filename>.bb</filename> file type
481 suffix.
482 </para>
483
484 <para>
485 A <filename>.bbappend</filename> file allows your layer to make
486 additions or changes to the content of another layer's recipe
487 without having to copy the other recipe into your layer.
488 Your <filename>.bbappend</filename> file resides in your layer,
489 while the main <filename>.bb</filename> recipe file to
490 which you are appending Metadata resides in a different layer.
491 </para>
492
493 <para>
494 Append files must have the same root names as their corresponding
495 recipes.
496 For example, the append file
497 <filename>someapp_&DISTRO;.bbappend</filename> must apply to
498 <filename>someapp_&DISTRO;.bb</filename>.
499 This means the original recipe and append file names are version
500 number-specific.
501 If the corresponding recipe is renamed to update to a newer
502 version, the corresponding <filename>.bbappend</filename> file must
503 be renamed (and possibly updated) as well.
504 During the build process, BitBake displays an error on starting
505 if it detects a <filename>.bbappend</filename> file that does
506 not have a corresponding recipe with a matching name.
507 See the
508 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink>
509 variable for information on how to handle this error.
510 </para>
511
512 <para>
513 Being able to append information to an existing recipe not only
514 avoids duplication, but also automatically applies recipe
515 changes in a different layer to your layer.
516 If you were copying recipes, you would have to manually merge
517 changes as they occur.
518 </para>
519
520 <para>
521 As an example, consider the main formfactor recipe and a
522 corresponding formfactor append file both from the
523 <link linkend='source-directory'>Source Directory</link>.
524 Here is the main formfactor recipe, which is named
525 <filename>formfactor_0.0.bb</filename> and located in the
526 "meta" layer at
527 <filename>meta/recipes-bsp/formfactor</filename>:
528 <literallayout class='monospaced'>
529 DESCRIPTION = "Device formfactor information"
530 SECTION = "base"
531 LICENSE = "MIT"
532 LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
533 file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
534 PR = "r41"
535
536 SRC_URI = "file://config file://machconfig"
537 S = "${WORKDIR}"
538
539 PACKAGE_ARCH = "${MACHINE_ARCH}"
540 INHIBIT_DEFAULT_DEPS = "1"
541
542 do_install() {
543 # Only install file if it has a contents
544 install -d ${D}${sysconfdir}/formfactor/
545 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
546 if [ -s "${S}/machconfig" ]; then
547 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
548 fi
549 }
550 </literallayout>
551 In the main recipe, note the
552 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
553 variable, which tells the OpenEmbedded build system where to
554 find files during the build.
555 </para>
556
557 <para>
558 Following is the append file, which is named
559 <filename>formfactor_0.0.bbappend</filename> and is from the
560 Crown Bay BSP Layer named
561 <filename>meta-intel/meta-crownbay</filename>.
562 The file is in <filename>recipes-bsp/formfactor</filename>:
563 <literallayout class='monospaced'>
564 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
565 </literallayout>
566 </para>
567
568 <para>
569 By default, the build system uses the
570 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
571 variable to locate files.
572 This append file extends the locations by setting the
573 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
574 variable.
575 Setting this variable in the <filename>.bbappend</filename>
576 file is the most reliable and recommended method for adding
577 directories to the search path used by the build system
578 to find files.
579 </para>
580
581 <para>
582 The statement in this example extends the directories to include
583 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>,
584 which resolves to a directory named
585 <filename>formfactor</filename> in the same directory
586 in which the append file resides (i.e.
587 <filename>meta-intel/meta-crownbay/recipes-bsp/formfactor/formfactor</filename>.
588 This implies that you must have the supporting directory
589 structure set up that will contain any files or patches you
590 will be including from the layer.
591 </para>
592
593 <para>
594 Using the immediate expansion assignment operator
595 <filename>:=</filename> is important because of the reference to
596 <filename>THISDIR</filename>.
597 The trailing colon character is important as it ensures that
598 items in the list remain colon-separated.
599 <note><para>BitBake automatically defines the
600 <filename>THISDIR</filename> variable.
601 You should never set this variable yourself.
602 Using <filename>_prepend</filename> ensures your path will
603 be searched prior to other paths in the final list.</para>
604 <para>Also, not all append files add extra files.
605 Many append files simply exist to add build options
606 (e.g. <filename>systemd</filename>).
607 For these cases, it is not necessary to use the
608 "_prepend" part of the statement.</para>
609 </note>
610 </para>
611 </section>
612
613 <section id='prioritizing-your-layer'>
614 <title>Prioritizing Your Layer</title>
615
616 <para>
617 Each layer is assigned a priority value.
618 Priority values control which layer takes precedence if there
619 are recipe files with the same name in multiple layers.
620 For these cases, the recipe file from the layer with a higher
621 priority number takes precedence.
622 Priority values also affect the order in which multiple
623 <filename>.bbappend</filename> files for the same recipe are
624 applied.
625 You can either specify the priority manually, or allow the
626 build system to calculate it based on the layer's dependencies.
627 </para>
628
629 <para>
630 To specify the layer's priority manually, use the
631 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>
632 variable.
633 For example:
634 <literallayout class='monospaced'>
635 BBFILE_PRIORITY_mylayer = "1"
636 </literallayout>
637 </para>
638
639 <note>
640 <para>It is possible for a recipe with a lower version number
641 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
642 in a layer that has a higher priority to take precedence.</para>
643 <para>Also, the layer priority does not currently affect the
644 precedence order of <filename>.conf</filename>
645 or <filename>.bbclass</filename> files.
646 Future versions of BitBake might address this.</para>
647 </note>
648 </section>
649
650 <section id='managing-layers'>
651 <title>Managing Layers</title>
652
653 <para>
654 You can use the BitBake layer management tool to provide a view
655 into the structure of recipes across a multi-layer project.
656 Being able to generate output that reports on configured layers
657 with their paths and priorities and on
658 <filename>.bbappend</filename> files and their applicable
659 recipes can help to reveal potential problems.
660 </para>
661
662 <para>
663 Use the following form when running the layer management tool.
664 <literallayout class='monospaced'>
665 $ bitbake-layers &lt;command&gt; [arguments]
666 </literallayout>
667 The following list describes the available commands:
668 <itemizedlist>
669 <listitem><para><filename><emphasis>help:</emphasis></filename>
670 Displays general help or help on a specified command.
671 </para></listitem>
672 <listitem><para><filename><emphasis>show-layers:</emphasis></filename>
673 Shows the current configured layers.
674 </para></listitem>
675 <listitem><para><filename><emphasis>show-recipes:</emphasis></filename>
676 Lists available recipes and the layers that provide them.
677 </para></listitem>
678 <listitem><para><filename><emphasis>show-overlayed:</emphasis></filename>
679 Lists overlayed recipes.
680 A recipe is overlayed when a recipe with the same name
681 exists in another layer that has a higher layer
682 priority.
683 </para></listitem>
684 <listitem><para><filename><emphasis>show-appends:</emphasis></filename>
685 Lists <filename>.bbappend</filename> files and the
686 recipe files to which they apply.
687 </para></listitem>
688 <listitem><para><filename><emphasis>show-cross-depends:</emphasis></filename>
689 Lists dependency relationships between recipes that
690 cross layer boundaries.
691 </para></listitem>
692 <listitem><para><filename><emphasis>flatten:</emphasis></filename>
693 Flattens the layer configuration into a separate output
694 directory.
695 Flattening your layer configuration builds a "flattened"
696 directory that contains the contents of all layers,
697 with any overlayed recipes removed and any
698 <filename>.bbappend</filename> files appended to the
699 corresponding recipes.
700 You might have to perform some manual cleanup of the
701 flattened layer as follows:
702 <itemizedlist>
703 <listitem><para>Non-recipe files (such as patches)
704 are overwritten.
705 The flatten command shows a warning for these
706 files.
707 </para></listitem>
708 <listitem><para>Anything beyond the normal layer
709 setup has been added to the
710 <filename>layer.conf</filename> file.
711 Only the lowest priority layer's
712 <filename>layer.conf</filename> is used.
713 </para></listitem>
714 <listitem><para>Overridden and appended items from
715 <filename>.bbappend</filename> files need to be
716 cleaned up.
717 The contents of each
718 <filename>.bbappend</filename> end up in the
719 flattened recipe.
720 However, if there are appended or changed
721 variable values, you need to tidy these up
722 yourself.
723 Consider the following example.
724 Here, the <filename>bitbake-layers</filename>
725 command adds the line
726 <filename>#### bbappended ...</filename> so that
727 you know where the following lines originate:
728 <literallayout class='monospaced'>
729 ...
730 DESCRIPTION = "A useful utility"
731 ...
732 EXTRA_OECONF = "--enable-something"
733 ...
734
735 #### bbappended from meta-anotherlayer ####
736
737 DESCRIPTION = "Customized utility"
738 EXTRA_OECONF += "--enable-somethingelse"
739 </literallayout>
740 Ideally, you would tidy up these utilities as
741 follows:
742 <literallayout class='monospaced'>
743 ...
744 DESCRIPTION = "Customized utility"
745 ...
746 EXTRA_OECONF = "--enable-something --enable-somethingelse"
747 ...
748 </literallayout></para></listitem>
749 </itemizedlist></para></listitem>
750 </itemizedlist>
751 </para>
752 </section>
753
754 <section id='creating-a-general-layer-using-the-yocto-layer-script'>
755 <title>Creating a General Layer Using the yocto-layer Script</title>
756
757 <para>
758 The <filename>yocto-layer</filename> script simplifies
759 creating a new general layer.
760 <note>
761 For information on BSP layers, see the
762 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
763 section in the Yocto Project Board Specific (BSP)
764 Developer's Guide.
765 </note>
766 The default mode of the script's operation is to prompt you for
767 information needed to generate the layer:
768 <itemizedlist>
769 <listitem><para>The layer priority
770 </para></listitem>
771 <listitem><para>Whether or not to create a sample recipe.
772 </para></listitem>
773 <listitem><para>Whether or not to create a sample
774 append file.
775 </para></listitem>
776 </itemizedlist>
777 </para>
778
779 <para>
780 Use the <filename>yocto-layer create</filename> sub-command
781 to create a new general layer.
782 In its simplest form, you can create a layer as follows:
783 <literallayout class='monospaced'>
784 $ yocto-layer create mylayer
785 </literallayout>
786 The previous example creates a layer named
787 <filename>meta-mylayer</filename> in the current directory.
788 </para>
789
790 <para>
791 As the <filename>yocto-layer create</filename> command runs,
792 default values for the prompts appear in brackets.
793 Pressing enter without supplying anything for the prompts
794 or pressing enter and providing an invalid response causes the
795 script to accept the default value.
796 Once the script completes, the new layer
797 is created in the current working directory.
798 The script names the layer by prepending
799 <filename>meta-</filename> to the name you provide.
800 </para>
801
802 <para>
803 Minimally, the script creates the following within the layer:
804 <itemizedlist>
805 <listitem><para><emphasis>The <filename>conf</filename>
806 directory:</emphasis>
807 This directory contains the layer's configuration file.
808 The root name for the file is the same as the root name
809 your provided for the layer (e.g.
810 <filename>&lt;layer&gt;.conf</filename>).
811 </para></listitem>
812 <listitem><para><emphasis>The
813 <filename>COPYING.MIT</filename>:</emphasis>
814 The copyright and use notice for the software.
815 </para></listitem>
816 <listitem><para><emphasis>The <filename>README</filename>
817 file:</emphasis>
818 A file describing the contents of your new layer.
819 </para></listitem>
820 </itemizedlist>
821 </para>
822
823 <para>
824 If you choose to generate a sample recipe file, the script
825 prompts you for the name for the recipe and then creates it
826 in <filename>&lt;layer&gt;/recipes-example/example/</filename>.
827 The script creates a <filename>.bb</filename> file and a
828 directory, which contains a sample
829 <filename>helloworld.c</filename> source file and along with
830 a sample patch file.
831 If you do not provide a recipe name, the script uses
832 "example".
833 </para>
834
835 <para>
836 If you choose to generate a sample append file, the script
837 prompts you for the name for the file and then creates it
838 in <filename>&lt;layer&gt;/recipes-example-bbappend/example-bbappend/</filename>.
839 The script creates a <filename>.bbappend</filename> file and a
840 directory, which contains a sample patch file.
841 If you do not provide a recipe name, the script uses
842 "example".
843 The script also prompts you for the version of the append file.
844 The version should match the recipe to which the append file
845 is associated.
846 </para>
847
848 <para>
849 The easiest way to see how the <filename>yocto-layer</filename>
850 script works is to experiment with the script.
851 You can also read the usage information by entering the
852 following:
853 <literallayout class='monospaced'>
854 $ yocto-layer help
855 </literallayout>
856 </para>
857
858 <para>
859 Once you create your general layer, you must add it to your
860 <filename>bblayers.conf</filename> file.
861 Here is an example where a layer named
862 <filename>meta-mylayer</filename> is added:
863 <literallayout class='monospaced'>
864 BBLAYERS = ?" \
865 /usr/local/src/yocto/meta \
866 /usr/local/src/yocto/meta-yocto \
867 /usr/local/src/yocto/meta-yocto-bsp \
868 /usr/local/src/yocto/meta-mylayer \
869 "
870
871 BBLAYERS_NON_REMOVABLE ?= " \
872 /usr/local/src/yocto/meta \
873 /usr/local/src/yocto/meta-yocto \
874 "
875 </literallayout>
876 Adding the layer to this file enables the build system to
877 locate the layer during the build.
878 </para>
879 </section>
880 </section>
881
882 <section id='usingpoky-extend-customimage'>
883 <title>Customizing Images</title>
884
885 <para>
886 You can customize images to satisfy particular requirements.
887 This section describes several methods and provides guidelines for each.
888 </para>
889
890 <section id='usingpoky-extend-customimage-custombb'>
891 <title>Customizing Images Using Custom .bb Files</title>
892
893 <para>
894 One way to get additional software into an image is to create a custom image.
895 The following example shows the form for the two lines you need:
896 <literallayout class='monospaced'>
897 IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
898
899 inherit core-image
900 </literallayout>
901 </para>
902
903 <para>
904 By creating a custom image, a developer has total control
905 over the contents of the image.
906 It is important to use the correct names of packages in the
907 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
908 variable.
909 You must use the OpenEmbedded notation and not the Debian notation for the names
910 (e.g. <filename>eglibc-dev</filename> instead of <filename>libc6-dev</filename>).
911 </para>
912
913 <para>
914 The other method for creating a custom image is to base it on an existing image.
915 For example, if you want to create an image based on <filename>core-image-sato</filename>
916 but add the additional package <filename>strace</filename> to the image,
917 copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a
918 new <filename>.bb</filename> and add the following line to the end of the copy:
919 <literallayout class='monospaced'>
920 IMAGE_INSTALL += "strace"
921 </literallayout>
922 </para>
923 </section>
924
925 <section id='usingpoky-extend-customimage-customtasks'>
926 <title>Customizing Images Using Custom Package Groups</title>
927
928 <para>
929 For complex custom images, the best approach is to create a
930 custom package group recipe that is used to build the image or
931 images.
932 A good example of a package group recipe is
933 <filename>meta/recipes-core/packagegroups/packagegroup-core-boot.bb</filename>.
934 The
935 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename>
936 variable lists the package group packages you wish to produce.
937 <filename>inherit packagegroup</filename> sets appropriate
938 default values and automatically adds <filename>-dev</filename>,
939 <filename>-dbg</filename>, and <filename>-ptest</filename>
940 complementary packages for every package specified in
941 <filename>PACKAGES</filename>.
942 Note that the inherit line should be towards
943 the top of the recipe, certainly before you set
944 <filename>PACKAGES</filename>.
945 For each package you specify in <filename>PACKAGES</filename>,
946 you can use
947 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename>
948 and
949 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename>
950 entries to provide a list of packages the parent task package
951 should contain.
952 Following is an example:
953 <literallayout class='monospaced'>
954 DESCRIPTION = "My Custom Package Groups"
955
956 inherit packagegroup
957
958 PACKAGES = "\
959 packagegroup-custom-apps \
960 packagegroup-custom-tools \
961 "
962
963 RDEPENDS_packagegroup-custom-apps = "\
964 dropbear \
965 portmap \
966 psplash"
967
968 RDEPENDS_packagegroup-custom-tools = "\
969 oprofile \
970 oprofileui-server \
971 lttng-control \
972 lttng-viewer"
973
974 RRECOMMENDS_packagegroup-custom-tools = "\
975 kernel-module-oprofile"
976 </literallayout>
977 </para>
978
979 <para>
980 In the previous example, two package group packages are created with their dependencies and their
981 recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and
982 <filename>packagegroup-custom-tools</filename>.
983 To build an image using these package group packages, you need to add
984 <filename>packagegroup-custom-apps</filename> and/or
985 <filename>packagegroup-custom-tools</filename> to
986 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>.
987 For other forms of image dependencies see the other areas of this section.
988 </para>
989 </section>
990
991 <section id='usingpoky-extend-customimage-imagefeatures'>
992 <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and
993 <filename>EXTRA_IMAGE_FEATURES</filename></title>
994
995 <para>
996 You might want to customize your image by enabling or
997 disabling high-level image features by using the
998 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
999 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
1000 variables.
1001 Although the functions for both variables are nearly equivalent,
1002 best practices dictate using <filename>IMAGE_FEATURES</filename>
1003 from within a recipe and using
1004 <filename>EXTRA_IMAGE_FEATURES</filename> from within
1005 your <filename>local.conf</filename> file, which is found in the
1006 <link linkend='build-directory'>Build Directory</link>.
1007 </para>
1008
1009 <para>
1010 To understand how these features work, the best reference is
1011 <filename>meta/classes/core-image.bbclass</filename>.
1012 In summary, the file looks at the contents of the
1013 <filename>IMAGE_FEATURES</filename> variable and then maps
1014 those contents into a set of package groups.
1015 Based on this information, the build system automatically
1016 adds the appropriate packages to the
1017 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
1018 variable.
1019 Effectively, you are enabling extra features by extending the
1020 class or creating a custom class for use with specialized image
1021 <filename>.bb</filename> files.
1022 </para>
1023
1024 <para>
1025 Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable
1026 from within your local configuration file.
1027 Using a separate area from which to enable features with
1028 this variable helps you avoid overwriting the features in the
1029 image recipe that are enabled with
1030 <filename>IMAGE_FEATURES</filename>.
1031 The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added
1032 to <filename>IMAGE_FEATURES</filename> within
1033 <filename>meta/conf/bitbake.conf</filename>.
1034 </para>
1035
1036 <para>
1037 To illustrate how you can use these variables to modify your
1038 image, consider an example that selects the SSH server.
1039 The Yocto Project ships with two SSH servers you can use
1040 with your images: Dropbear and OpenSSH.
1041 Dropbear is a minimal SSH server appropriate for
1042 resource-constrained environments, while OpenSSH is a
1043 well-known standard SSH server implementation.
1044 By default, the <filename>core-image-sato</filename> image
1045 is configured to use Dropbear.
1046 The <filename>core-image-basic</filename> and
1047 <filename>core-image-lsb</filename> images both
1048 include OpenSSH.
1049 The <filename>core-image-minimal</filename> image does not
1050 contain an SSH server.
1051 </para>
1052
1053 <para>
1054 You can customize your image and change these defaults.
1055 Edit the <filename>IMAGE_FEATURES</filename> variable
1056 in your recipe or use the
1057 <filename>EXTRA_IMAGE_FEATURES</filename> in your
1058 <filename>local.conf</filename> file so that it configures the
1059 image you are working with to include
1060 <filename>ssh-server-dropbear</filename> or
1061 <filename>ssh-server-openssh</filename>.
1062 </para>
1063
1064 <note>
1065 See the
1066 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
1067 section in the Yocto Project Reference Manual for a complete
1068 list of image features that ship with the Yocto Project.
1069 </note>
1070 </section>
1071
1072 <section id='usingpoky-extend-customimage-localconf'>
1073 <title>Customizing Images Using <filename>local.conf</filename></title>
1074
1075 <para>
1076 It is possible to customize image contents by using variables from your
1077 local configuration in your <filename>conf/local.conf</filename> file.
1078 Because it is limited to local use, this method generally only allows you to
1079 add packages and is not as flexible as creating your own customized image.
1080 When you add packages using local variables this way, you need to realize that
1081 these variable changes affect all images at the same time and might not be
1082 what you require.
1083 </para>
1084
1085 <para>
1086 The simplest way to add extra packages to all images is by using the
1087 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1088 variable with the <filename>_append</filename> operator:
1089 <literallayout class='monospaced'>
1090 IMAGE_INSTALL_append = " strace"
1091 </literallayout>
1092 Use of the syntax is important - specifically, the space between
1093 the quote and the package name, which is
1094 <filename>strace</filename> in this example.
1095 This space is required since the <filename>_append</filename>
1096 operator does not add the space.
1097 </para>
1098
1099 <para>
1100 Furthermore, you must use <filename>_append</filename> instead of the <filename>+=</filename>
1101 operator if you want to avoid ordering issues.
1102 The reason for this is because doing so unconditionally appends to the variable and
1103 avoids ordering problems due to the variable being set in image recipes and
1104 <filename>.bbclass</filename> files with operators like <filename>?=</filename>.
1105 Using <filename>_append</filename> ensures the operation takes affect.
1106 </para>
1107
1108 <para>
1109 As shown in its simplest use, <filename>IMAGE_INSTALL_append</filename> affects
1110 all images.
1111 It is possible to extend the syntax so that the variable applies to a specific image only.
1112 Here is an example:
1113 <literallayout class='monospaced'>
1114 IMAGE_INSTALL_append_pn-core-image-minimal = " strace"
1115 </literallayout>
1116 This example adds <filename>strace</filename> to <filename>core-image-minimal</filename>
1117 only.
1118 </para>
1119
1120 <para>
1121 You can add packages using a similar approach through the
1122 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename>
1123 variable.
1124 If you use this variable, only <filename>core-image-*</filename> images are affected.
1125 </para>
1126 </section>
1127 </section>
1128
1129 <section id='usingpoky-extend-addpkg'>
1130 <title>Writing a Recipe to Add a Package to Your Image</title>
1131
1132 <para>
1133 Recipes add packages to your image.
1134 Writing a recipe means creating a <filename>.bb</filename> file that sets some
1135 variables.
1136 For information on variables that are useful for recipes and for information about recipe naming
1137 issues, see the
1138 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>"
1139 section of the Yocto Project Reference Manual.
1140 </para>
1141
1142 <para>
1143 Before writing a recipe from scratch, it is often useful to check
1144 whether someone else has written one already.
1145 OpenEmbedded is a good place to look as it has a wider scope and range of packages.
1146 Because the Yocto Project aims to be compatible with OpenEmbedded, most recipes
1147 you find there should work for you.
1148 </para>
1149
1150 <para>
1151 For new packages, the simplest way to add a recipe is to base it on a similar
1152 pre-existing recipe.
1153 The sections that follow provide some examples that show how to add standard
1154 types of packages.
1155 </para>
1156
1157 <note>
1158 <para>When writing shell functions, you need to be aware of BitBake's
1159 curly brace parsing.
1160 If a recipe uses a closing curly brace within the function and
1161 the character has no leading spaces, BitBake produces a parsing
1162 error.
1163 If you use a pair of curly brace in a shell function, the
1164 closing curly brace must not be located at the start of the line
1165 without leading spaces.</para>
1166 <para>Here is an example that causes BitBake to produce a parsing
1167 error:
1168 <literallayout class='monospaced'>
1169 fakeroot create_shar() {
1170 cat &lt;&lt; "EOF" &gt; ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
1171 usage()
1172 {
1173 echo "test"
1174 ###### The following "}" at the start of the line causes a parsing error ######
1175 }
1176 EOF
1177 }
1178 </literallayout>
1179 Writing the recipe this way avoids the error:
1180 <literallayout class='monospaced'>
1181 fakeroot create_shar() {
1182 cat &lt;&lt; "EOF" &gt; ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
1183 usage()
1184 {
1185 echo "test"
1186 ######The following "}" with a leading space at the start of the line avoids the error ######
1187 }
1188 EOF
1189 }
1190 </literallayout></para>
1191 </note>
1192
1193 <section id='usingpoky-extend-addpkg-singlec'>
1194 <title>Single .c File Package (Hello World!)</title>
1195
1196 <para>
1197 Building an application from a single file that is stored locally (e.g. under
1198 <filename>files/</filename>) requires a recipe that has the file listed in
1199 the
1200 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>
1201 variable.
1202 Additionally, you need to manually write the <filename>do_compile</filename> and
1203 <filename>do_install</filename> tasks.
1204 The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
1205 variable defines the
1206 directory containing the source code, which is set to
1207 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'>
1208 WORKDIR</ulink></filename> in this case - the directory BitBake uses for the build.
1209 <literallayout class='monospaced'>
1210 DESCRIPTION = "Simple helloworld application"
1211 SECTION = "examples"
1212 LICENSE = "MIT"
1213 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
1214 PR = "r0"
1215
1216 SRC_URI = "file://helloworld.c"
1217
1218 S = "${WORKDIR}"
1219
1220 do_compile() {
1221 ${CC} helloworld.c -o helloworld
1222 }
1223
1224 do_install() {
1225 install -d ${D}${bindir}
1226 install -m 0755 helloworld ${D}${bindir}
1227 }
1228 </literallayout>
1229 </para>
1230
1231 <para>
1232 By default, the <filename>helloworld</filename>, <filename>helloworld-dbg</filename>,
1233 and <filename>helloworld-dev</filename> packages are built.
1234 For information on how to customize the packaging process, see the
1235 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application
1236 into Multiple Packages</link>" section.
1237 </para>
1238 </section>
1239
1240 <section id='usingpoky-extend-addpkg-autotools'>
1241 <title>Autotooled Package</title>
1242 <para>
1243 Applications that use Autotools such as <filename>autoconf</filename> and
1244 <filename>automake</filename> require a recipe that has a source archive listed in
1245 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and
1246 also inherits Autotools, which instructs BitBake to use the
1247 <filename>autotools.bbclass</filename> file, which contains the definitions of all the steps
1248 needed to build an Autotool-based application.
1249 The result of the build is automatically packaged.
1250 And, if the application uses NLS for localization, packages with local information are
1251 generated (one package per language).
1252 Following is one example: (<filename>hello_2.3.bb</filename>)
1253 <literallayout class='monospaced'>
1254 DESCRIPTION = "GNU Helloworld application"
1255 SECTION = "examples"
1256 LICENSE = "GPLv2+"
1257 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
1258 PR = "r0"
1259
1260 SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
1261
1262 inherit autotools gettext
1263 </literallayout>
1264 </para>
1265
1266 <para>
1267 The variable
1268 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename>
1269 is used to track source license changes as described in the
1270 "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>" section.
1271 You can quickly create Autotool-based recipes in a manner similar to the previous example.
1272 </para>
1273 </section>
1274
1275 <section id='usingpoky-extend-addpkg-makefile'>
1276 <title>Makefile-Based Package</title>
1277
1278 <para>
1279 Applications that use GNU <filename>make</filename> also require a recipe that has
1280 the source archive listed in
1281 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
1282 You do not need to add a <filename>do_compile</filename> step since by default BitBake
1283 starts the <filename>make</filename> command to compile the application.
1284 If you need additional <filename>make</filename> options, you should store them in the
1285 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'>EXTRA_OEMAKE</ulink></filename>
1286 variable.
1287 BitBake passes these options into the <filename>make</filename> GNU invocation.
1288 Note that a <filename>do_install</filename> task is still required.
1289 Otherwise, BitBake runs an empty <filename>do_install</filename> task by default.
1290 </para>
1291
1292 <para>
1293 Some applications might require extra parameters to be passed to the compiler.
1294 For example, the application might need an additional header path.
1295 You can accomplish this by adding to the
1296 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable.
1297 The following example shows this:
1298 <literallayout class='monospaced'>
1299 CFLAGS_prepend = "-I ${S}/include "
1300 </literallayout>
1301 </para>
1302
1303 <para>
1304 In the following example, <filename>mtd-utils</filename> is a makefile-based package:
1305 <literallayout class='monospaced'>
1306 DESCRIPTION = "Tools for managing memory technology devices."
1307 SECTION = "base"
1308 DEPENDS = "zlib lzo e2fsprogs util-linux"
1309 HOMEPAGE = "http://www.linux-mtd.infradead.org/"
1310 LICENSE = "GPLv2+"
1311 LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
1312 file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
1313
1314 SRC_URI = "git://git.infradead.org/mtd-utils.git;protocol=git;tag=995cfe51b0a3cf32f381c140bf72b21bf91cef1b \
1315 file://add-exclusion-to-mkfs-jffs2-git-2.patch"
1316
1317 S = "${WORKDIR}/git/"
1318
1319 PR = "r1"
1320
1321 EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' \
1322 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'"
1323
1324 do_install () {
1325 oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} \
1326 INCLUDEDIR=${includedir}
1327 install -d ${D}${includedir}/mtd/
1328 for f in ${S}/include/mtd/*.h; do
1329 install -m 0644 $f ${D}${includedir}/mtd/
1330 done
1331 }
1332
1333 PARALLEL_MAKE = ""
1334
1335 BBCLASSEXTEND = "native"
1336 </literallayout>
1337 </para>
1338
1339 <para>
1340 If your sources are available as a tarball instead of a Git repository, you
1341 will need to provide the URL to the tarball as well as an
1342 <filename>md5</filename> or <filename>sha256</filename> sum of
1343 the download.
1344 Here is an example:
1345 <literallayout class='monospaced'>
1346 SRC_URI="ftp://ftp.infradead.org/pub/mtd-utils/mtd-utils-1.4.9.tar.bz2"
1347 SRC_URI[md5sum]="82b8e714b90674896570968f70ca778b"
1348 </literallayout>
1349 You can generate the <filename>md5</filename> or <filename>sha256</filename> sums
1350 by using the <filename>md5sum</filename> or <filename>sha256sum</filename> commands
1351 with the target file as the only argument.
1352 Here is an example:
1353 <literallayout class='monospaced'>
1354 $ md5sum mtd-utils-1.4.9.tar.bz2
1355 82b8e714b90674896570968f70ca778b mtd-utils-1.4.9.tar.bz2
1356 </literallayout>
1357 </para>
1358 </section>
1359
1360 <section id='splitting-an-application-into-multiple-packages'>
1361 <title>Splitting an Application into Multiple Packages</title>
1362
1363 <para>
1364 You can use the variables
1365 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and
1366 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename>
1367 to split an application into multiple packages.
1368 </para>
1369
1370 <para>
1371 Following is an example that uses the <filename>libXpm</filename> recipe.
1372 By default, this recipe generates a single package that contains the library along
1373 with a few binaries.
1374 You can modify the recipe to split the binaries into separate packages:
1375 <literallayout class='monospaced'>
1376 require xorg-lib-common.inc
1377
1378 DESCRIPTION = "X11 Pixmap library"
1379 LICENSE = "X-BSD"
1380 LIC_FILES_CHKSUM = "file://COPYING;md5=3e07763d16963c3af12db271a31abaa5"
1381 DEPENDS += "libxext libsm libxt"
1382 PR = "r3"
1383 PE = "1"
1384
1385 XORG_PN = "libXpm"
1386
1387 PACKAGES =+ "sxpm cxpm"
1388 FILES_cxpm = "${bindir}/cxpm"
1389 FILES_sxpm = "${bindir}/sxpm"
1390 </literallayout>
1391 </para>
1392
1393 <para>
1394 In the previous example, we want to ship the <filename>sxpm</filename>
1395 and <filename>cxpm</filename> binaries in separate packages.
1396 Since <filename>bindir</filename> would be packaged into the main
1397 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
1398 package by default, we prepend the <filename>PACKAGES</filename>
1399 variable so additional package names are added to the start of list.
1400 This results in the extra <filename>FILES_*</filename>
1401 variables then containing information that define which files and
1402 directories go into which packages.
1403 Files included by earlier packages are skipped by latter packages.
1404 Thus, the main <filename>PN</filename> package
1405 does not include the above listed files.
1406 </para>
1407 </section>
1408
1409 <section id='usingpoky-extend-addpkg-postinstalls'>
1410 <title>Post-Installation Scripts</title>
1411
1412 <para>
1413 To add a post-installation script to a package, add a
1414 <filename>pkg_postinst_PACKAGENAME()</filename> function to the
1415 <filename>.bb</filename> file and use
1416 <filename>PACKAGENAME</filename> as the name of the package you want to attach to the
1417 <filename>postinst</filename> script.
1418 Normally,
1419 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
1420 can be used, which automatically expands to <filename>PACKAGENAME</filename>.
1421 A post-installation function has the following structure:
1422 <literallayout class='monospaced'>
1423 pkg_postinst_PACKAGENAME () {
1424 #!/bin/sh -e
1425 # Commands to carry out
1426 }
1427 </literallayout>
1428 </para>
1429
1430 <para>
1431 The script defined in the post-installation function is called when the
1432 root filesystem is created.
1433 If the script succeeds, the package is marked as installed.
1434 If the script fails, the package is marked as unpacked and the script is
1435 executed when the image boots again.
1436 </para>
1437
1438 <para>
1439 Sometimes it is necessary for the execution of a post-installation
1440 script to be delayed until the first boot.
1441 For example, the script might need to be executed on the device itself.
1442 To delay script execution until boot time, use the following structure in the
1443 post-installation script:
1444 <literallayout class='monospaced'>
1445 pkg_postinst_PACKAGENAME () {
1446 #!/bin/sh -e
1447 if [ x"$D" = "x" ]; then
1448 # Actions to carry out on the device go here
1449 else
1450 exit 1
1451 fi
1452 }
1453 </literallayout>
1454 </para>
1455
1456 <para>
1457 The previous example delays execution until the image boots again because the
1458 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'>D</ulink></filename>
1459 variable points
1460 to the directory containing the image when the root filesystem is created at build time but
1461 is unset when executed on the first boot.
1462 </para>
1463 </section>
1464 </section>
1465
1466 <section id="platdev-newmachine">
1467 <title>Adding a New Machine</title>
1468
1469 <para>
1470 Adding a new machine to the Yocto Project is a straightforward process.
1471 This section provides information that gives you an idea of the changes you must make.
1472 The information covers adding machines similar to those the Yocto Project already supports.
1473 Although well within the capabilities of the Yocto Project, adding a totally new architecture
1474 might require
1475 changes to <filename>gcc/eglibc</filename> and to the site information, which is
1476 beyond the scope of this manual.
1477 </para>
1478
1479 <para>
1480 For a complete example that shows how to add a new machine,
1481 see the
1482 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
1483 in the Yocto Project Board Support Package (BSP) Developer's Guide.
1484 </para>
1485
1486 <section id="platdev-newmachine-conffile">
1487 <title>Adding the Machine Configuration File</title>
1488
1489 <para>
1490 To add a machine configuration, you need to add a <filename>.conf</filename> file
1491 with details of the device being added to the <filename>conf/machine/</filename> file.
1492 The name of the file determines the name the OpenEmbedded build system
1493 uses to reference the new machine.
1494 </para>
1495
1496 <para>
1497 The most important variables to set in this file are as follows:
1498 <itemizedlist>
1499 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename>
1500 (e.g. "arm")</para></listitem>
1501 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename>
1502 (see below)</para></listitem>
1503 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename>
1504 (e.g. "apm screen wifi")</para></listitem>
1505 </itemizedlist>
1506 </para>
1507
1508 <para>
1509 You might also need these variables:
1510 <itemizedlist>
1511 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename>
1512 (e.g. "115200 ttyS0")</para></listitem>
1513 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename>
1514 (e.g. "zImage")</para></listitem>
1515 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename>
1516 (e.g. "tar.gz jffs2")</para></listitem>
1517 </itemizedlist>
1518 </para>
1519
1520 <para>
1521 You can find full details on these variables in the reference section.
1522 You can leverage many existing machine <filename>.conf</filename> files from
1523 <filename>meta/conf/machine/</filename>.
1524 </para>
1525 </section>
1526
1527 <section id="platdev-newmachine-kernel">
1528 <title>Adding a Kernel for the Machine</title>
1529
1530 <para>
1531 The OpenEmbedded build system needs to be able to build a kernel for the machine.
1532 You need to either create a new kernel recipe for this machine, or extend an
1533 existing recipe.
1534 You can find several kernel examples in the
1535 Source Directory at <filename>meta/recipes-kernel/linux</filename>
1536 that you can use as references.
1537 </para>
1538
1539 <para>
1540 If you are creating a new recipe, normal recipe-writing rules apply for setting
1541 up a
1542 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
1543 Thus, you need to specify any necessary patches and set
1544 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> to point at the source code.
1545 You need to create a <filename>configure</filename> task that configures the
1546 unpacked kernel with a defconfig.
1547 You can do this by using a <filename>make defconfig</filename> command or,
1548 more commonly, by copying in a suitable <filename>defconfig</filename> file and and then running
1549 <filename>make oldconfig</filename>.
1550 By making use of <filename>inherit kernel</filename> and potentially some of the
1551 <filename>linux-*.inc</filename> files, most other functionality is
1552 centralized and the the defaults of the class normally work well.
1553 </para>
1554
1555 <para>
1556 If you are extending an existing kernel, it is usually a matter of adding a
1557 suitable defconfig file.
1558 The file needs to be added into a location similar to defconfig files
1559 used for other machines in a given kernel.
1560 A possible way to do this is by listing the file in the
1561 <filename>SRC_URI</filename> and adding the machine to the expression in
1562 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>:
1563 <literallayout class='monospaced'>
1564 COMPATIBLE_MACHINE = '(qemux86|qemumips)'
1565 </literallayout>
1566 </para>
1567 </section>
1568
1569 <section id="platdev-newmachine-formfactor">
1570 <title>Adding a Formfactor Configuration File</title>
1571
1572 <para>
1573 A formfactor configuration file provides information about the
1574 target hardware for which the image is being built and information that
1575 the build system cannot obtain from other sources such as the kernel.
1576 Some examples of information contained in a formfactor configuration file include
1577 framebuffer orientation, whether or not the system has a keyboard,
1578 the positioning of the keyboard in relation to the screen, and
1579 the screen resolution.
1580 </para>
1581
1582 <para>
1583 The build system uses reasonable defaults in most cases.
1584 However, if customization is
1585 necessary, you need to create a <filename>machconfig</filename> file
1586 in the <filename>meta/recipes-bsp/formfactor/files</filename>
1587 directory.
1588 This directory contains directories for specific machines such as
1589 <filename>qemuarm</filename> and <filename>qemux86</filename>.
1590 For information about the settings available and the defaults, see the
1591 <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the
1592 same area.
1593 </para>
1594
1595 <para>
1596 Following is an example for qemuarm:
1597 <literallayout class='monospaced'>
1598 HAVE_TOUCHSCREEN=1
1599 HAVE_KEYBOARD=1
1600
1601 DISPLAY_CAN_ROTATE=0
1602 DISPLAY_ORIENTATION=0
1603 #DISPLAY_WIDTH_PIXELS=640
1604 #DISPLAY_HEIGHT_PIXELS=480
1605 #DISPLAY_BPP=16
1606 DISPLAY_DPI=150
1607 DISPLAY_SUBPIXEL_ORDER=vrgb
1608 </literallayout>
1609 </para>
1610 </section>
1611 </section>
1612
1613 <section id="platdev-working-with-libraries">
1614 <title>Working With Libraries</title>
1615
1616 <para>
1617 Libraries are an integral part of your system.
1618 This section describes some common practices you might find
1619 helpful when working with libraries to build your system:
1620 <itemizedlist>
1621 <listitem><para><link linkend='including-static-library-files'>How to include static library files</link>
1622 </para></listitem>
1623 <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link>
1624 </para></listitem>
1625 <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link>
1626 </para></listitem>
1627 </itemizedlist>
1628 </para>
1629
1630 <section id='including-static-library-files'>
1631 <title>Including Static Library Files</title>
1632
1633 <para>
1634 If you are building a library and the library offers static linking, you can control
1635 which static library files (<filename>*.a</filename> files) get included in the
1636 built library.
1637 </para>
1638
1639 <para>
1640 The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
1641 and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink>
1642 variables in the
1643 <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed
1644 by the <filename>do_install</filename> task are packaged.
1645 By default, the <filename>PACKAGES</filename> variable contains
1646 <filename>${PN}-staticdev</filename>, which includes all static library files.
1647 <note>
1648 Some previously released versions of the Yocto Project
1649 defined the static library files through
1650 <filename>${PN}-dev</filename>.
1651 </note>
1652 Following, is part of the BitBake configuration file.
1653 You can see where the static library files are defined:
1654 <literallayout class='monospaced'>
1655 PACKAGES = "${PN}-dbg ${PN} ${PN}-doc ${PN}-dev ${PN}-staticdev ${PN}-locale"
1656 PACKAGES_DYNAMIC = "${PN}-locale-*"
1657 FILES = ""
1658
1659 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
1660 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
1661 ${base_bindir}/* ${base_sbindir}/* \
1662 ${base_libdir}/*${SOLIBS} \
1663 ${datadir}/${BPN} ${libdir}/${BPN}/* \
1664 ${datadir}/pixmaps ${datadir}/applications \
1665 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
1666 ${libdir}/bonobo/servers"
1667
1668 FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
1669 ${datadir}/gnome/help"
1670 SECTION_${PN}-doc = "doc"
1671
1672 FILES_${PN}-dev = "${includedir} ${libdir}/lib*${SOLIBSDEV} ${libdir}/*.la \
1673 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
1674 ${datadir}/aclocal ${base_libdir}/*.o"
1675 SECTION_${PN}-dev = "devel"
1676 ALLOW_EMPTY_${PN}-dev = "1"
1677 RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
1678
1679 FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a"
1680 SECTION_${PN}-staticdev = "devel"
1681 RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
1682 </literallayout>
1683 </para>
1684 </section>
1685
1686 <section id="combining-multiple-versions-library-files-into-one-image">
1687 <title>Combining Multiple Versions of Library Files into One Image</title>
1688
1689 <para>
1690 The build system offers the ability to build libraries with different
1691 target optimizations or architecture formats and combine these together
1692 into one system image.
1693 You can link different binaries in the image
1694 against the different libraries as needed for specific use cases.
1695 This feature is called "Multilib."
1696 </para>
1697
1698 <para>
1699 An example would be where you have most of a system compiled in 32-bit
1700 mode using 32-bit libraries, but you have something large, like a database
1701 engine, that needs to be a 64-bit application and uses 64-bit libraries.
1702 Multilib allows you to get the best of both 32-bit and 64-bit libraries.
1703 </para>
1704
1705 <para>
1706 While the Multilib feature is most commonly used for 32 and 64-bit differences,
1707 the approach the build system uses facilitates different target optimizations.
1708 You could compile some binaries to use one set of libraries and other binaries
1709 to use other different sets of libraries.
1710 The libraries could differ in architecture, compiler options, or other
1711 optimizations.
1712 </para>
1713
1714 <para>
1715 This section overviews the Multilib process only.
1716 For more details on how to implement Multilib, see the
1717 <ulink url='&YOCTO_WIKI_URL;/wiki/Multilib'>Multilib</ulink> wiki
1718 page.
1719 </para>
1720
1721 <para>
1722 Aside from this wiki page, several examples exist in the
1723 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/tree/meta-skeleton'><filename>meta-skeleton</filename></ulink>
1724 layer found in the
1725 <link linkend='source-directory'>Source Directory</link>:
1726 <itemizedlist>
1727 <listitem><para><filename>conf/multilib-example.conf</filename>
1728 configuration file</para></listitem>
1729 <listitem><para><filename>conf/multilib-example2.conf</filename>
1730 configuration file</para></listitem>
1731 <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename>
1732 recipe</para></listitem>
1733 </itemizedlist>
1734 </para>
1735
1736 <section id='preparing-to-use-multilib'>
1737 <title>Preparing to Use Multilib</title>
1738
1739 <para>
1740 User-specific requirements drive the Multilib feature.
1741 Consequently, there is no one "out-of-the-box" configuration that likely
1742 exists to meet your needs.
1743 </para>
1744
1745 <para>
1746 In order to enable Multilib, you first need to ensure your recipe is
1747 extended to support multiple libraries.
1748 Many standard recipes are already extended and support multiple libraries.
1749 You can check in the <filename>meta/conf/multilib.conf</filename>
1750 configuration file in the
1751 <link linkend='source-directory'>Source Directory</link> to see how this is
1752 done using the
1753 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink>
1754 variable.
1755 Eventually, all recipes will be covered and this list will be unneeded.
1756 </para>
1757
1758 <para>
1759 For the most part, the Multilib class extension works automatically to
1760 extend the package name from <filename>${PN}</filename> to
1761 <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename>
1762 is the particular multilib (e.g. "lib32-" or "lib64-").
1763 Standard variables such as
1764 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
1765 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
1766 <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>,
1767 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
1768 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>,
1769 and <filename>PACKAGES_DYNAMIC</filename> are automatically extended by the system.
1770 If you are extending any manual code in the recipe, you can use the
1771 <filename>${MLPREFIX}</filename> variable to ensure those names are extended
1772 correctly.
1773 This automatic extension code resides in <filename>multilib.bbclass</filename>.
1774 </para>
1775 </section>
1776
1777 <section id='using-multilib'>
1778 <title>Using Multilib</title>
1779
1780 <para>
1781 After you have set up the recipes, you need to define the actual
1782 combination of multiple libraries you want to build.
1783 You accomplish this through your <filename>local.conf</filename>
1784 configuration file in the
1785 <link linkend='build-directory'>Build Directory</link>.
1786 An example configuration would be as follows:
1787 <literallayout class='monospaced'>
1788 MACHINE = "qemux86-64"
1789 require conf/multilib.conf
1790 MULTILIBS = "multilib:lib32"
1791 DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
1792 IMAGE_INSTALL = "lib32-connman"
1793 </literallayout>
1794 This example enables an
1795 additional library named <filename>lib32</filename> alongside the
1796 normal target packages.
1797 When combining these "lib32" alternatives, the example uses "x86" for tuning.
1798 For information on this particular tuning, see
1799 <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>.
1800 </para>
1801
1802 <para>
1803 The example then includes <filename>lib32-connman</filename>
1804 in all the images, which illustrates one method of including a
1805 multiple library dependency.
1806 You can use a normal image build to include this dependency,
1807 for example:
1808 <literallayout class='monospaced'>
1809 $ bitbake core-image-sato
1810 </literallayout>
1811 You can also build Multilib packages specifically with a command like this:
1812 <literallayout class='monospaced'>
1813 $ bitbake lib32-connman
1814 </literallayout>
1815 </para>
1816 </section>
1817
1818 <section id='additional-implementation-details'>
1819 <title>Additional Implementation Details</title>
1820
1821 <para>
1822 Different packaging systems have different levels of native Multilib
1823 support.
1824 For the RPM Package Management System, the following implementation details
1825 exist:
1826 <itemizedlist>
1827 <listitem><para>A unique architecture is defined for the Multilib packages,
1828 along with creating a unique deploy folder under
1829 <filename>tmp/deploy/rpm</filename> in the
1830 <link linkend='build-directory'>Build Directory</link>.
1831 For example, consider <filename>lib32</filename> in a
1832 <filename>qemux86-64</filename> image.
1833 The possible architectures in the system are "all", "qemux86_64",
1834 "lib32_qemux86_64", and "lib32_x86".</para></listitem>
1835 <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from
1836 <filename>${PN}</filename> during RPM packaging.
1837 The naming for a normal RPM package and a Multilib RPM package in a
1838 <filename>qemux86-64</filename> system resolves to something similar to
1839 <filename>bash-4.1-r2.x86_64.rpm</filename> and
1840 <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively.
1841 </para></listitem>
1842 <listitem><para>When installing a Multilib image, the RPM backend first
1843 installs the base image and then installs the Multilib libraries.
1844 </para></listitem>
1845 <listitem><para>The build system relies on RPM to resolve the identical files in the
1846 two (or more) Multilib packages.</para></listitem>
1847 </itemizedlist>
1848 </para>
1849
1850 <para>
1851 For the IPK Package Management System, the following implementation details exist:
1852 <itemizedlist>
1853 <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from
1854 <filename>${PN}</filename> during IPK packaging.
1855 The naming for a normal RPM package and a Multilib IPK package in a
1856 <filename>qemux86-64</filename> system resolves to something like
1857 <filename>bash_4.1-r2.x86_64.ipk</filename> and
1858 <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively.
1859 </para></listitem>
1860 <listitem><para>The IPK deploy folder is not modified with
1861 <filename>${MLPREFIX}</filename> because packages with and without
1862 the Multilib feature can exist in the same folder due to the
1863 <filename>${PN}</filename> differences.</para></listitem>
1864 <listitem><para>IPK defines a sanity check for Multilib installation
1865 using certain rules for file comparison, overridden, etc.
1866 </para></listitem>
1867 </itemizedlist>
1868 </para>
1869 </section>
1870 </section>
1871
1872 <section id='installing-multiple-versions-of-the-same-library'>
1873 <title>Installing Multiple Versions of the Same Library</title>
1874
1875 <para>
1876 Situations can exist where you need to install and use
1877 multiple versions of the same library on the same system
1878 at the same time.
1879 These situations almost always exist when a library API
1880 changes and you have multiple pieces of software that
1881 depend on the separate versions of the library.
1882 To accommodate these situations, you can install multiple
1883 versions of the same library in parallel on the same system.
1884 </para>
1885
1886 <para>
1887 The process is straight forward as long as the libraries use
1888 proper versioning.
1889 With properly versioned libraries, all you need to do to
1890 individually specify the libraries is create separate,
1891 appropriately named recipes where the
1892 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the
1893 name includes a portion that differentiates each library version
1894 (e.g.the major part of the version number).
1895 Thus, instead of having a single recipe that loads one version
1896 of a library (e.g. <filename>clutter</filename>), you provide
1897 multiple recipes that result in different versions
1898 of the libraries you want.
1899 As an example, the following two recipes would allow the
1900 two separate versions of the <filename>clutter</filename>
1901 library to co-exist on the same system:
1902 <literallayout class='monospaced'>
1903 clutter-1.6_1.6.20.bb
1904 clutter-1.8_1.8.4.bb
1905 </literallayout>
1906 Additionally, if you have other recipes that depend on a given
1907 library, you need to use the
1908 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
1909 variable to create the dependency.
1910 Continuing with the same example, if you want to have a recipe
1911 depend on the 1.8 version of the <filename>clutter</filename>
1912 library, use the following in your recipe:
1913 <literallayout class='monospaced'>
1914 DEPENDS = "clutter-1.8"
1915 </literallayout>
1916 </para>
1917 </section>
1918 </section>
1919
1920 <section id='creating-partitioned-images'>
1921 <title>Creating Partitioned Images</title>
1922
1923 <para>
1924 Creating an image for a particular hardware target using the
1925 OpenEmbedded build system does not necessarily mean you can boot
1926 that image as is on your device.
1927 Physical devices accept and boot images in various ways depending
1928 on the specifics of the device.
1929 Usually, information about the hardware can tell you what image
1930 format the device requires.
1931 Should your device require multiple partitions on an SD card, flash,
1932 or an HDD, you can use the OpenEmbedded Image Creator
1933 () to create the properly partitioned image.
1934 </para>
1935
1936 <para>
1937 The <filename>wic</filename> command generates partitioned images
1938 from existing OpenEmbedded build artifacts.
1939 Image generation is driven by partitioning commands contained
1940 in an Openembedded kickstart file (<filename>.wks</filename>)
1941 specified either directly on the command-line or as one of a
1942 selection of canned <filename>.wks</filename> files
1943 (see 'wic list images').
1944 When applied to a given set of build artifacts, the result is an
1945 image or set of images that can be directly written onto media and
1946 used on a particular system.
1947 </para>
1948
1949 <para>
1950 This section provides some background information on
1951 <filename>wic</filename>, describes what you need to have in
1952 place to run the tool, provides instruction on how to use
1953 <filename>wic</filename>, and provides several examples.
1954 </para>
1955
1956 <section id='wic-background'>
1957 <title>Background</title>
1958
1959 <para>
1960 This section provides some background on the
1961 <filename>wic</filename> utility.
1962 While none of this information is required to use
1963 <filename>wic</filename>, you might find it interesting.
1964 <itemizedlist>
1965 <listitem><para>
1966 The name "wic" is derived from OpenEmbedded
1967 Image Creator (oeic).
1968 The "oe" diphthong in "oeic" was promoted to the
1969 letter "w", because "oeic" is both difficult to remember and
1970 pronounce.</para></listitem>
1971 <listitem><para>
1972 <filename>wic</filename> is loosely based on the
1973 Meego Image Creator (<filename>mic</filename>)
1974 framework.
1975 The <filename>wic</filename> implementation has been
1976 heavily modified to make direct use of OpenEmbedded
1977 build artifacts instead of package installation and
1978 configuration, which are already incorporated within
1979 the OpenEmbedded artifacts.</para></listitem>
1980 <listitem><para>
1981 <filename>wic</filename> is a completely independent
1982 standalone utility that initially provides
1983 easier-to-use and more flexible replacements for a
1984 couple bits of existing functionality in OE Core's
1985 <filename>directdisk.bbclass</filename> and
1986 <filename>mkefidisk.sh</filename> script.
1987 The replaced scripts are implemented by a
1988 general-purpose partitioning language based on Red Hat
1989 kickstart syntax.
1990 Underlying code for <filename>wic</filename> succeeded
1991 from several projects over time.</para></listitem>
1992 </itemizedlist>
1993 </para>
1994 </section>
1995
1996<!--
1997
1998 <para>
1999 This section covers the mechanics of invoking and providing help for
2000the command and sub-commands; it contains hooks for future commits to
2001connect with the actual functionality, once implemented.
2002 </para>
2003
2004 <para>
2005 Help is integrated into the 'wic' command itself - you can also see
2006that for details and reminders on usage (simply invoke 'wic' without
2007any arguments to get started).
2008 </para>
2009
2010 <note>
2011 Just because 'wic' can generate an image doesn't mean that it
2012will boot on a given machine. 'wic' tries to spot the most obvious
2013usages that are likely to cause problems but, as a relatively
2014low-level tool, it can't in general figure out whether a generated
2015image is appropriate for a given piece of hardware - it's really up to
2016you to provide intelligent inputs to the image creation process. If
2017you suspect that your image isn't working as expected due to some bug
2018or missing feature of the tool, please file a bug report describing
2019the details.
2020 </note>
2021
2022-->
2023
2024 <section id='wic-requirements'>
2025 <title>Requirements</title>
2026
2027 <para>
2028 In order to use the <filename>wic</filename> utility with the
2029 OpenEmbedded Build system, you need to meet the following
2030 requirements:
2031 <itemizedlist>
2032 <listitem><para>The Linux distribution on your
2033 development host must support the Yocto Project.
2034 See the
2035 "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>"
2036 section in the Yocto Project Reference Manual for this
2037 list of distributions.</para></listitem>
2038 <listitem><para>
2039 The standard system utilities, such as
2040 <filename>cp</filename>, must be installed on your
2041 development host system.
2042 </para></listitem>
2043 <listitem><para>
2044 The
2045 <ulink url='http://www.gnu.org/software/parted/'>GNU Parted</ulink>
2046 package must be installed on your development host
2047 system.
2048 </para></listitem>
2049 <listitem><para>
2050 Have the build artifacts already available.
2051 You must already have created an image using the
2052 Openembedded build system (e.g.
2053 <filename>core-image-minimal</filename>.
2054 It might seem redundant to generate an image in order
2055 to create an image using <filename>wic</filename>,
2056 but the artifacts are needed and they are generated
2057 with the build system.</para></listitem>
2058 <listitem><para>
2059 You must have sourced one of the build environment
2060 setup scripts (i.e.
2061 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
2062 or
2063 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
2064 found in the
2065 <link linkend='build-directory'>Build Directory</link>.
2066 </para></listitem>
2067 </itemizedlist>
2068 </para>
2069 </section>
2070
2071 <section id='wic-getting-help'>
2072 <title>Getting Help</title>
2073
2074 <para>
2075 You can get general help for the <filename>wic</filename>
2076 by entering the <filename>wic</filename> command by itself
2077 or by entering the command with a help argument as follows:
2078 <literallayout class='monospaced'>
2079 $ wic -h
2080 $ wic --help
2081 </literallayout>
2082 </para>
2083
2084 <para>
2085 Currently, <filename>wic</filename> supports two commands:
2086 <filename>create</filename> and <filename>list</filename>.
2087 You can get help for these commands as follows:
2088 <literallayout class='monospaced'>
2089 $ wic help &lt;command&gt;
2090 </literallayout>
2091 </para>
2092
2093 <para>
2094 You can find more out about the images
2095 <filename>wic</filename> creates using the provided
2096 kickstart files with the following form of the command:
2097 <literallayout class='monospaced'>
2098 $ wic list &lt;image&gt; help
2099 </literallayout>
2100 Where <filename>&lt;image&gt;</filename> is either
2101 <filename>directdisk</filename> or
2102 <filename>mkefidisk</filename>.
2103 </para>
2104 </section>
2105
2106 <section id='operational-modes'>
2107 <title>Operational Modes</title>
2108
2109 <para>
2110 You can run <filename>wic</filename> in two modes: Raw and
2111 Cooked:
2112 <itemizedlist>
2113 <listitem><para><emphasis>Raw Mode:</emphasis>
2114 You explicitly specify build artifacts through
2115 command-line arguments.</para></listitem>
2116 <listitem><para><emphasis>Cooked Mode:</emphasis>
2117 The current
2118 <ulink url='&YOCTO_DOCS_REF_URL;var-MACHINE'><filename>MACHINE</filename></ulink>
2119 setting and image name are used to automatically locate
2120 and provide the build artifacts.</para></listitem>
2121 </itemizedlist>
2122 </para>
2123
2124 <section id='raw-mode'>
2125 <title>Raw Mode</title>
2126
2127 <para>
2128 The general form of the 'wic' command in raw mode is:
2129 <literallayout class='monospaced'>
2130 $ wic create &lt;image_name&gt;.wks -r &lt;rootfs_dir&gt; -b &lt;bootimg_dir&gt; /
2131 -k &lt;kernel_dir&gt; -n &lt;native_sysroot&gt;
2132 </literallayout>
2133 <note>
2134 You do not need root privileges to run
2135 <filename>wic</filename>.
2136 In fact, you should not run as root when using the
2137 utility.
2138 </note>
2139 </para>
2140
2141 <para>
2142 Following is a description of the <filename>wic</filename>
2143 parameters and options:
2144 <itemizedlist>
2145 <listitem><para><emphasis><filename>&lt;image_name&gt;.wks</filename>:</emphasis>
2146 An OpenEmbedded kickstart file.
2147 You can provide your own custom file or use a
2148 file from a set of provided files as described
2149 following this list.</para></listitem>
2150 <listitem><para><emphasis><filename>-r &lt;rootfs_dir&gt;</filename>:</emphasis>
2151 Specifies the path to the root filesystem directory
2152 to be used and the <filename>.wks</filename>
2153 root filesystem source.</para></listitem>
2154 <listitem><para><emphasis><filename>-b &lt;bootimg_dir&gt;</filename>:</emphasis>
2155 Specifies the path to the directory that contains
2156 the boot artifacts (e.g. the
2157 <filename>EFI</filename> or
2158 <filename>syslinux</filename> directories) to use
2159 as the <filename>.wks</filename> boot image source.
2160 </para></listitem>
2161 <listitem><para><emphasis><filename>-k &lt;kernel_dir&gt;</filename>:</emphasis>
2162 Specifies the path to the dir containing the kernel
2163 to use in the <filename>.wks</filename> boot
2164 image.</para></listitem>
2165 <listitem><para><emphasis><filename>-n &lt;native_sysroot&gt;</filename>:</emphasis>
2166 Specifies the path to the native sysroot
2167 that contains the tools used to build the image.
2168 </para></listitem>
2169 </itemizedlist>
2170 </para>
2171 </section>
2172
2173 <section id='cooked-mode'>
2174 <title>Cooked Mode</title>
2175
2176 <para>
2177 The general form of the <filename>wic</filename> command
2178 using Cooked Mode is:
2179 <literallayout class='monospaced'>
2180 $ wic create &lt;kickstart_file&gt; -e &lt;image_name&gt;
2181 </literallayout>
2182 This form is the simplest and most user-friendly, as it
2183 does not requre specifying all individual parameters.
2184 All you need to provide is your own
2185 <filename>.wks</filename> file or one provided with the
2186 release.
2187 </para>
2188
2189 <para>
2190 Following is a description of the <filename>wic</filename>
2191 parameters and options:
2192 <itemizedlist>
2193 <listitem><para><emphasis><filename>&lt;kickstart&gt;</filename>:</emphasis>
2194 An OpenEmbedded kickstart file.
2195 You can provide your own custom file or supplied
2196 file.</para></listitem>
2197 <listitem><para><emphasis><filename>-e &lt;image_name&gt;</filename>:</emphasis>
2198 Specifies the image built using the OpenEmbedded
2199 build system.</para></listitem>
2200 </itemizedlist>
2201 </para>
2202 </section>
2203 </section>
2204
2205 <section id='using-a-provided-kickstart_file'>
2206 <title>Using a Provided Kickstart File</title>
2207
2208 <para>
2209 If you do not want to create your own
2210 <filename>.wks</filename> file, you can use a provided
2211 file.
2212 Use the following command to list the available files:
2213 <literallayout class='monospaced'>
2214 $ wic list images
2215 mkefidisk Create an EFI disk image
2216 directdisk Create a 'pcbios' direct disk image
2217 </literallayout>
2218 When you use a provided file, you do not have to use the
2219 <filename>.wks</filename> extension.
2220 Here is an example in Raw Mode that uses the
2221 <filename>directdisk</filename> file:
2222 <literallayout class='monospaced'>
2223 $ wic create directdisk -r &lt;rootfs_dir&gt; -b &lt;bootimg_dir&gt; \
2224 -k &lt;kernel_dir&gt; -n &lt;native_sysroot&gt;
2225 </literallayout>
2226 </para>
2227 </section>
2228
2229 <section id='wic-usage-examples'>
2230 <title>Examples</title>
2231
2232 <para>
2233 This section provides several examples that show how to use
2234 the <filename>wic</filename> utility.
2235 All the examples assume the list of requirements in the
2236 "<link linkend='wic-requirements'>Requirements</link>" section
2237 have been met.
2238 The examples assume the previously generated image is
2239 <filename>core-image-minimal</filename>.
2240 </para>
2241
2242 <section id='generate-an-image-using-a-provided-kickstart-file'>
2243 <title>Generate an Image using a Provided Kickstart File</title>
2244
2245 <para>
2246 This example runs in Cooked Mode and uses the
2247 <filename>mkefidisk</filename> kickstart file:
2248 <literallayout class='monospaced'>
2249 $ wic create mkefidisk -e core-image-minimal
2250 Checking basic build environment...
2251 Done.
2252
2253 Creating image(s)...
2254
2255 Info: The new image(s) can be found here:
2256 /var/tmp/wic/build/mkefidisk-201310230946-sda.direct
2257
2258 The following build artifacts were used to create the image(s):
2259 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs
2260 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg
2261 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
2262 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
2263
2264
2265 The image(s) were created using OE kickstart file:
2266 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks
2267 </literallayout>
2268 This example shows the easiest way to create an image
2269 by running in Cooked Mode and using the
2270 <filename>&lt;-e&gt;</filename> option with a
2271 provided kickstart file.
2272 All that is necessary is to specify the image
2273 used to generate the artifacts.
2274 Your <filename>local.conf</filename> needs to have the
2275 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
2276 variable set to the machine you are using, which is
2277 "minnow" in this example.
2278 </para>
2279
2280 <para>
2281 The output specifies exactly which image were
2282 created and where the image was created.
2283 The output also names the artifacts used and the exact
2284 <filename>.wks</filename> script that was used to generate
2285 the image.
2286 <note>
2287 You should always verify the details provided in the
2288 output to make sure that the imagewas indeed created
2289 exactly as expected.
2290 </note>
2291 </para>
2292
2293 <para>
2294 Continuing with the example, you can now directly
2295 <filename>dd</filename> the image to a USB stick, or
2296 whatever media for which you built your image,
2297 and boot the resulting media:
2298 <literallayout class='monospaced'>
2299 $ sudo dd if=/var/tmp/wic/build/mkefidisk-201310230946-sda.direct of=/dev/sdb
2300 [sudo] password for trz:
2301 182274+0 records in
2302 182274+0 records out
2303 93324288 bytes (93 MB) copied, 14.4777 s, 6.4 MB/s
2304 [trz@empanada ~]$ sudo eject /dev/sdb
2305 </literallayout>
2306 </para>
2307 </section>
2308
2309 <section id='using-a-modified-kickstart-file'>
2310 <title>Using a Modified Kickstart File</title>
2311
2312 <para>
2313 Because <filename>wic</filename> image creation is driven
2314 by the kickstart file, it is easy to drive image creation
2315 by changing the parameters in the file.
2316 This next example demonstrates that through modification
2317 of the <filename>directdisk</filename> kickstart file.
2318 </para>
2319
2320 <para>
2321 As mentioned earlier, you can use the command
2322 <filename>wic list images</filename> to show the list
2323 of provided kickstart files.
2324 The directory in which these files reside is in the
2325 <link linkend='source-directory'>Source Directory</link>
2326 in <filename>scripts/lib/image/canned-wks/</filename>.
2327 Because the available files reside in this directory, you
2328 can create and add your own custom files to the directory.
2329 Subsequent use of the <filename>wic list images</filename>
2330 command would then include your kickstart files.
2331 </para>
2332
2333 <para>
2334 In this example, the existing
2335 <filename>directdisk</filename> file already does most
2336 of what is needed.
2337 However, for the hardware in this example, the image will
2338 need to boot from <filename>sdb</filename> instead of
2339 <filename>sda</filename>, which is what the
2340 <filename>directdisk</filename> kickstart file uses.
2341 </para>
2342
2343 <para>
2344 The example begins by making a copy of the
2345 <filename>directdisk.wks</filename> file in the
2346 <filename>scripts/lib/image/canned-wks</filename>
2347 directory and then changing the lines that specify the
2348 target disk from which to boot.
2349 <literallayout class='monospaced'>
2350 $ cp /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks
2351 </literallayout>
2352 Next, the example modifies the
2353 <filename>directdisksdb.wks</filename> file and changes all
2354 instances of "<filename>--ondisk sda</filename>"
2355 to "<filename>--ondisk sdb</filename>".
2356 The example changes the following two lines and leaves the
2357 remaining lines untouched:
2358 <literallayout class='monospaced'>
2359 part /boot --source bootimg --ondisk sdb --fstype=msdos --label boot --active --align 1024
2360 part / --source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024
2361 </literallayout>
2362 (I AM HERE)
2363 Once the lines are changed, Once we've made that change, we generate a directdisksdb image,
2364pointing the process at the core-image-minimal artifacts for the nuc
2365(Next Unit of Computing), selected as our current MACHINE in
2366local.conf.
2367 </para>
2368
2369 <para>
2370 Once we've set the build up, we run a core-image-minimal nuc build:
2371 <literallayout class='monospaced'>
2372 $ bitbake core-image-minimal
2373 </literallayout>
2374 Once the build is finished, we can then use nuc to create our
2375directdisk image for the nuc to boot. In this case, we'll use the
2376'-e' option to have wic discover the appropriate build artifacts and
2377generate the image:
2378 <literallayout class='monospaced'>
2379 $ wic create directdisksdb -e core-image-minimal
2380 Checking basic build environment...
2381 Done.
2382
2383 Creating image(s)...
2384
2385 Info: The new image(s) can be found here:
2386 /var/tmp/wic/build/directdisksdb-201310231131-sdb.direct
2387
2388 The following build artifacts were used to create the image(s):
2389 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/nuc-poky-linux/core-image-minimal/1.0-r0/rootfs
2390 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/share
2391 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/src/kernel
2392 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
2393
2394
2395 The image(s) were created using OE kickstart file:
2396 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks
2397 </literallayout>
2398 Using the path specified in the output for the image name and
2399location, you can now directly dd the image to a USB stick or whatever
2400media you built the image for, and boot the resulting media:
2401 <literallayout class='monospaced'>
2402 $ sudo dd if=/var/tmp/wic/build/directdisksdb-201310231131-sdb.direct of=/dev/sdb
2403 86018+0 records in
2404 86018+0 records out
2405 44041216 bytes (44 MB) copied, 13.0734 s, 3.4 MB/s
2406 [trz@empanada tmp]$ sudo eject /dev/sdb
2407 </literallayout>
2408 Of course, you can just use the directdisk image directly if you don't
2409have any special needs.
2410 </para>
2411
2412 <para>
2413 Here'we're creating a wic image based on core-image-minimal and
2414crownbay-noemgd, which works right out of the box.
2415 <literallayout class='monospaced'>
2416 $ wic create directdisk -e core-image-minimal
2417
2418 Checking basic build environment...
2419 Done.
2420
2421 Creating image(s)...
2422
2423 Info: The new image(s) can be found here:
2424 /var/tmp/wic/build/directdisk-201309252350-sda.direct
2425
2426 The following build artifacts were used to create the image(s):
2427
2428 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
2429 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
2430 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
2431 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
2432
2433 The image(s) were created using OE kickstart file:
2434 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks
2435 </literallayout>
2436 Finally, here's an example that doesn't take the easy way out and
2437manually specifies each build artifact, along with a non-canned .wks
2438file, and also uses the -o option to have wic create the output
2439somewhere other than the default /var/tmp/wic:
2440 <literallayout class='monospaced'>
2441 $ wic create ~/test.wks -o /home/trz/testwic --rootfs-dir /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs --bootimg-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share --kernel-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel --native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
2442
2443 Creating image(s)...
2444
2445 Info: The new image(s) can be found here:
2446 /home/trz/testwic/build/test-201309260032-sda.direct
2447
2448 The following build artifacts were used to create the image(s):
2449
2450 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
2451 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
2452 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
2453 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
2454
2455 The image(s) were created using OE kickstart file:
2456 /home/trz/test.wks
2457 </literallayout>
2458 In this case, we didn't need to have the proper machine selected in
2459local.conf - we manually specified each artifact and therefore wic
2460doesn't need further information from the build system.
2461 </para>
2462
2463 <para>
2464 Finally, here's an example of the actual partition language commands
2465used to generate the mkefidisk image i.e. these are the contents of the
2466mkefidisk.wks OE kickstart file:
2467 <literallayout class='monospaced'>
2468 # short-description: Create an EFI disk image
2469 # long-description: Creates a partitioned EFI disk image that the user
2470 # can directly dd to boot media.
2471
2472 part /boot --source bootimg --ondisk sda --fstype=efi --label msdos --active --align 1024
2473
2474 part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024
2475
2476 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
2477
2478 bootloader --timeout=10 --append="rootwait rootfstype=ext3 console=ttyPCH0,115200 console=tty0 vmalloc=256MB snd-hda- intel.enable_msi=0"
2479 </literallayout>
2480 </para>
2481 </section>
2482 </section>
2483
2484 <section id='openembedded-kickstart-wks-reference'>
2485 <title>OpenEmbedded Kickstart (.wks) Reference</title>
2486
2487 <para>
2488 The current 'wic' implementation supports only the basic kickstart
2489partitioning commands: 'partition' (or 'part' for short) and
2490'bootloader'.
2491 </para>
2492
2493 <para>
2494 They are listed below and mostly follow the syntax and meaning of the
2495standard kickstart options for those commands. The documentation below
2496is based on the Fedora kickstart documentation of the same commands,
2497but modified to reflect wic capabilities. For reference:
2498 <literallayout class='monospaced'>
2499 http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
2500 http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader
2501 </literallayout>
2502 </para>
2503
2504 <section id='command-part-or-partition'>
2505 <title>Command: part or partition</title>
2506
2507 <para>
2508 Creates a partition on the system.
2509 Use the following syntax:
2510 <literallayout class='monospaced'>
2511 part &lt;mntpoint&gt;
2512 </literallayout>
2513 The &lt;mntpoint&gt; is where the partition will be mounted and must be of
2514one of the following forms:
2515 <itemizedlist>
2516 <listitem><para><filename>/&lt;path&gt;</filename>:
2517 For example, <filename>/</filename>,
2518 <filename>/usr</filename>, and
2519 <filename>/home</filename></para></listitem>
2520 <listitem><para><filename>swap</filename>:
2521 The partition will be used as swap space.
2522 </para></listitem>
2523 </itemizedlist>
2524 </para>
2525
2526 <para>
2527 Following are the supported options:
2528 <literallayout class='monospaced'>
2529 --size
2530 The minimum partition size in megabytes. Specify an integer value
2531 here such as 500. Do not append the number with MB. Not needed if
2532 --source is used.
2533
2534 --source
2535 bootimg
2536 rootfs
2537
2538 The --source option is a wic-specific option that can currently
2539 have one of two values, 'bootimg' or 'rootfs'.
2540
2541 If '--source rootfs' is used, it tells the wic command to create a
2542 partition as large as needed to fill with the contents of /rootfs
2543 (specified by the -r 'wic' option) and to fill it with the
2544 contents of /rootfs.
2545
2546 If '--source bootimg' is used, it tells the wic command to create
2547 a partition as large as needed to fill with the contents of the
2548 boot partition (specified by the -b 'wic' option). Exactly what
2549 those contents are depend on the value of the --fstype option for
2550 that partition. If '--fstype=efi' is specified, the boot
2551 artifacts contained in HDDDIR are used, and if '--fstype=msdos' is
2552 specified, the boot artifacts found in STAGING_DATADIR are used.
2553
2554 --ondisk or --ondrive
2555 Forces the partition to be created on a particular disk.
2556
2557 --fstype
2558 Sets the file system type for the partition. Valid values are:
2559 msdos
2560 efi
2561 ext4
2562 ext3
2563 ext2
2564 btrfs
2565 swap
2566
2567 --label label
2568 Specify the label to give to the filesystem to be made on the
2569 partition. If the given label is already in use by another
2570 filesystem, a new label will be created for this partition.
2571
2572 --active
2573 Mark the partition as active.
2574
2575 --align (in kB)
2576 The '--align' option is a mic-specific option that says to start a
2577 partition on an x kB boundary.
2578 </literallayout>
2579 </para>
2580 </section>
2581
2582 <section id='command-bootloader'>
2583 <title>Command: bootloader</title>
2584
2585 <para>
2586 This command specifies how the boot loader should be installed.
2587 </para>
2588
2589 <para>
2590 Following are the supported options:
2591 <literallayout class='monospaced'>
2592 --timeout
2593 Specify the number of seconds before the bootloader times out and
2594 boots the default option.
2595
2596 --append
2597 Specifies kernel parameters. These will be added to the syslinux
2598 APPEND or grub kernel command line.
2599
2600 The boot type is determined by the fstype of the /boot mountpoint. If
2601 the fstype is 'msdos' the boot type is 'pcbios', otherwise it's the
2602 fstype, which currently be: 'efi' (more to be added later).
2603
2604 If the boot type is 'efi', the image will use grub and has one
2605 menuentry: 'boot'.
2606
2607 If the boot type is 'pcbios', the image will use syslinux and has one
2608 menu label: 'boot'.
2609
2610 Future updates will implement more options - using anything not
2611 explicitly supported can result in unpredictable results.
2612 </literallayout>
2613 </para>
2614 </section>
2615 </section>
2616 </section>
2617
2618 <section id='configuring-the-kernel'>
2619 <title>Configuring the Kernel</title>
2620
2621 <para>
2622 Configuring the Yocto Project kernel consists of making sure the <filename>.config</filename>
2623 file has all the right information in it for the image you are building.
2624 You can use the <filename>menuconfig</filename> tool and configuration fragments to
2625 make sure your <filename>.config</filename> file is just how you need it.
2626 This section describes how to use <filename>menuconfig</filename>, create and use
2627 configuration fragments, and how to interactively tweak your <filename>.config</filename>
2628 file to create the leanest kernel configuration file possible.
2629 </para>
2630
2631 <para>
2632 For more information on kernel configuration, see the
2633 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
2634 section in the Yocto Project Linux Kernel Development Manual.
2635 </para>
2636
2637 <section id='using-menuconfig'>
2638 <title>Using&nbsp;&nbsp;<filename>menuconfig</filename></title>
2639
2640 <para>
2641 The easiest way to define kernel configurations is to set them through the
2642 <filename>menuconfig</filename> tool.
2643 This tool provides an interactive method with which
2644 to set kernel configurations.
2645 For general information on <filename>menuconfig</filename>, see
2646 <ulink url='http://en.wikipedia.org/wiki/Menuconfig'></ulink>.
2647 </para>
2648
2649 <para>
2650 To use the <filename>menuconfig</filename> tool in the Yocto Project development
2651 environment, you must build the tool using BitBake.
2652 Thus, the environment must be set up using the
2653 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
2654 or
2655 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>
2656 script found in the
2657 <link linkend='build-directory'>Build Directory</link>.
2658 The following commands build and invoke <filename>menuconfig</filename> assuming the
2659 <link linkend='source-directory'>Source Directory</link>
2660 top-level folder is <filename>~/poky</filename>:
2661 <literallayout class='monospaced'>
2662 $ cd poky
2663 $ source oe-init-build-env
2664 $ bitbake linux-yocto -c menuconfig
2665 </literallayout>
2666 Once <filename>menuconfig</filename> comes up, its standard interface allows you to
2667 interactively examine and configure all the kernel configuration parameters.
2668 After making your changes, simply exit the tool and save your changes to
2669 create an updated version of the <filename>.config</filename> configuration file.
2670 </para>
2671
2672 <para>
2673 Consider an example that configures the <filename>linux-yocto-3.4</filename>
2674 kernel.
2675 The OpenEmbedded build system recognizes this kernel as
2676 <filename>linux-yocto</filename>.
2677 Thus, the following commands from the shell in which you previously sourced the
2678 environment initialization script cleans the shared state cache and the
2679 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
2680 directory and then builds and launches <filename>menuconfig</filename>:
2681 <literallayout class='monospaced'>
2682 $ bitbake linux-yocto -c menuconfig
2683 </literallayout>
2684 </para>
2685
2686 <para>
2687 Once <filename>menuconfig</filename> launches, use the interface
2688 to navigate through the selections to find the configuration settings in
2689 which you are interested.
2690 For example, consider the <filename>CONFIG_SMP</filename> configuration setting.
2691 You can find it at <filename>Processor Type and Features</filename> under
2692 the configuration selection <filename>Symmetric Multi-processing Support</filename>.
2693 After highlighting the selection, use the arrow keys to select or deselect
2694 the setting.
2695 When you are finished with all your selections, exit out and save them.
2696 </para>
2697
2698 <para>
2699 Saving the selections updates the <filename>.config</filename> configuration file.
2700 This is the file that the OpenEmbedded build system uses to configure the
2701 kernel during the build.
2702 You can find and examine this file in the Build Directory in
2703 <filename>tmp/work/</filename>.
2704 The actual <filename>.config</filename> is located in the area where the
2705 specific kernel is built.
2706 For example, if you were building a Linux Yocto kernel based on the
2707 Linux 3.4 kernel and you were building a QEMU image targeted for
2708 <filename>x86</filename> architecture, the
2709 <filename>.config</filename> file would be located here:
2710 <literallayout class='monospaced'>
2711 poky/build/tmp/work/qemux86-poky-linux/linux-yocto-3.4.11+git1+84f...
2712 ...656ed30-r1/linux-qemux86-standard-build
2713 </literallayout>
2714 <note>
2715 The previous example directory is artificially split and many of the characters
2716 in the actual filename are omitted in order to make it more readable.
2717 Also, depending on the kernel you are using, the exact pathname
2718 for <filename>linux-yocto-3.4...</filename> might differ.
2719 </note>
2720 </para>
2721
2722 <para>
2723 Within the <filename>.config</filename> file, you can see the kernel settings.
2724 For example, the following entry shows that symmetric multi-processor support
2725 is not set:
2726 <literallayout class='monospaced'>
2727 # CONFIG_SMP is not set
2728 </literallayout>
2729 </para>
2730
2731 <para>
2732 A good method to isolate changed configurations is to use a combination of the
2733 <filename>menuconfig</filename> tool and simple shell commands.
2734 Before changing configurations with <filename>menuconfig</filename>, copy the
2735 existing <filename>.config</filename> and rename it to something else,
2736 use <filename>menuconfig</filename> to make
2737 as many changes an you want and save them, then compare the renamed configuration
2738 file against the newly created file.
2739 You can use the resulting differences as your base to create configuration fragments
2740 to permanently save in your kernel layer.
2741 <note>
2742 Be sure to make a copy of the <filename>.config</filename> and don't just
2743 rename it.
2744 The build system needs an existing <filename>.config</filename>
2745 from which to work.
2746 </note>
2747 </para>
2748 </section>
2749
2750 <section id='creating-config-fragments'>
2751 <title>Creating Configuration Fragments</title>
2752
2753 <para>
2754 Configuration fragments are simply kernel options that appear in a file
2755 placed where the OpenEmbedded build system can find and apply them.
2756 Syntactically, the configuration statement is identical to what would appear
2757 in the <filename>.config</filename> file, which is in the
2758 <link linkend='build-directory'>Build Directory</link> in
2759 <filename>tmp/work/&lt;arch&gt;-poky-linux/linux-yocto-&lt;release-specific-string&gt;/linux-&lt;arch&gt;-&lt;build-type&gt;</filename>.
2760 </para>
2761
2762 <para>
2763 It is simple to create a configuration fragment.
2764 For example, issuing the following from the shell creates a configuration fragment
2765 file named <filename>my_smp.cfg</filename> that enables multi-processor support
2766 within the kernel:
2767 <literallayout class='monospaced'>
2768 $ echo "CONFIG_SMP=y" >> my_smp.cfg
2769 </literallayout>
2770 <note>
2771 All configuration files must use the <filename>.cfg</filename> extension in order
2772 for the OpenEmbedded build system to recognize them as a configuration fragment.
2773 </note>
2774 </para>
2775
2776 <para>
2777 Where do you put your configuration files?
2778 You can place these configuration files in the same area pointed to by
2779 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
2780 The OpenEmbedded build system will pick up the configuration and add it to the
2781 kernel's configuration.
2782 For example, suppose you had a set of configuration options in a file called
2783 <filename>myconfig.cfg</filename>.
2784 If you put that file inside a directory named <filename>linux-yocto</filename>
2785 that resides in the same directory as the kernel's append file and then add
2786 a <filename>SRC_URI</filename> statement such as the following to the kernel's append file,
2787 those configuration options will be picked up and applied when the kernel is built.
2788 <literallayout class='monospaced'>
2789 SRC_URI += "file://myconfig.cfg"
2790 </literallayout>
2791 </para>
2792
2793 <para>
2794 As mentioned earlier, you can group related configurations into multiple files and
2795 name them all in the <filename>SRC_URI</filename> statement as well.
2796 For example, you could group separate configurations specifically for Ethernet and graphics
2797 into their own files and add those by using a <filename>SRC_URI</filename> statement like the
2798 following in your append file:
2799 <literallayout class='monospaced'>
2800 SRC_URI += "file://myconfig.cfg \
2801 file://eth.cfg \
2802 file://gfx.cfg"
2803 </literallayout>
2804 </para>
2805 </section>
2806
2807 <section id='fine-tuning-the-kernel-configuration-file'>
2808 <title>Fine-Tuning the Kernel Configuration File</title>
2809
2810 <para>
2811 You can make sure the <filename>.config</filename> file is as lean or efficient as
2812 possible by reading the output of the kernel configuration fragment audit,
2813 noting any issues, making changes to correct the issues, and then repeating.
2814 </para>
2815
2816 <para>
2817 As part of the kernel build process, the
2818 <filename>kernel_configcheck</filename> task runs.
2819 This task validates the kernel configuration by checking the final
2820 <filename>.config</filename> file against the input files.
2821 During the check, the task produces warning messages for the following
2822 issues:
2823 <itemizedlist>
2824 <listitem><para>Requested options that did not make the final
2825 <filename>.config</filename> file.</para></listitem>
2826 <listitem><para>Configuration items that appear twice in the same
2827 configuration fragment.</para></listitem>
2828 <listitem><para>Configuration items tagged as "required" were overridden.
2829 </para></listitem>
2830 <listitem><para>A board overrides a non-board specific option.</para></listitem>
2831 <listitem><para>Listed options not valid for the kernel being processed.
2832 In other words, the option does not appear anywhere.</para></listitem>
2833 </itemizedlist>
2834 <note>
2835 The <filename>kernel_configcheck</filename> task can also optionally report
2836 if an option is overridden during processing.
2837 </note>
2838 </para>
2839
2840 <para>
2841 For each output warning, a message points to the file
2842 that contains a list of the options and a pointer to the config
2843 fragment that defines them.
2844 Collectively, the files are the key to streamlining the configuration.
2845 </para>
2846
2847 <para>
2848 To streamline the configuration, do the following:
2849 <orderedlist>
2850 <listitem><para>Start with a full configuration that you know
2851 works - it builds and boots successfully.
2852 This configuration file will be your baseline.</para></listitem>
2853 <listitem><para>Separately run the <filename>configme</filename> and
2854 <filename>kernel_configcheck</filename> tasks.</para></listitem>
2855 <listitem><para>Take the resulting list of files from the
2856 <filename>kernel_configcheck</filename> task warnings and do the following:
2857 <itemizedlist>
2858 <listitem><para>Drop values that are redefined in the fragment but do not
2859 change the final <filename>.config</filename> file.</para></listitem>
2860 <listitem><para>Analyze and potentially drop values from the
2861 <filename>.config</filename> file that override required
2862 configurations.</para></listitem>
2863 <listitem><para>Analyze and potentially remove non-board specific options.
2864 </para></listitem>
2865 <listitem><para>Remove repeated and invalid options.</para></listitem>
2866 </itemizedlist></para></listitem>
2867 <listitem><para>After you have worked through the output of the kernel configuration
2868 audit, you can re-run the <filename>configme</filename>
2869 and <filename>kernel_configcheck</filename> tasks to see the results of your
2870 changes.
2871 If you have more issues, you can deal with them as described in the
2872 previous step.</para></listitem>
2873 </orderedlist>
2874 </para>
2875
2876 <para>
2877 Iteratively working through steps two through four eventually yields
2878 a minimal, streamlined configuration file.
2879 Once you have the best <filename>.config</filename>, you can build the Linux
2880 Yocto kernel.
2881 </para>
2882 </section>
2883 </section>
2884
2885 <section id="patching-the-kernel">
2886 <title>Patching the Kernel</title>
2887
2888 <para>
2889 Patching the kernel involves changing or adding configurations to an existing kernel,
2890 changing or adding recipes to the kernel that are needed to support specific hardware features,
2891 or even altering the source code itself.
2892 <note>
2893 You can use the <filename>yocto-kernel</filename> script
2894 found in the <link linkend='source-directory'>Source Directory</link>
2895 under <filename>scripts</filename> to manage kernel patches and configuration.
2896 See the "<ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'>Managing kernel Patches and Config Items with yocto-kernel</ulink>"
2897 section in the Yocto Project Board Support Packages (BSP) Developer's Guide for
2898 more information.</note>
2899 </para>
2900
2901 <para>
2902 This example creates a simple patch by adding some QEMU emulator console
2903 output at boot time through <filename>printk</filename> statements in the kernel's
2904 <filename>calibrate.c</filename> source code file.
2905 Applying the patch and booting the modified image causes the added
2906 messages to appear on the emulator's console.
2907 </para>
2908
2909 <para>
2910 The example assumes a clean build exists for the <filename>qemux86</filename>
2911 machine in a Source Directory named <filename>poky</filename>.
2912 Furthermore, the <link linkend='build-directory'>Build Directory</link> is
2913 <filename>build</filename> and is located in <filename>poky</filename> and
2914 the kernel is based on the Linux 3.4 kernel.
2915 For general information on how to configure the most efficient build, see the
2916 "<ulink url='&YOCTO_DOCS_QS_URL;#building-image'>Building an Image</ulink>" section
2917 in the Yocto Project Quick Start.
2918 </para>
2919
2920 <para>
2921 Also, for more information on patching the kernel, see the
2922 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#applying-patches'>Applying Patches</ulink>"
2923 section in the Yocto Project Linux Kernel Development Manual.
2924 </para>
2925
2926 <section id='create-a-layer-for-your-changes'>
2927 <title>Create a Layer for your Changes</title>
2928
2929 <para>
2930 The first step is to create a layer so you can isolate your changes:
2931 <literallayout class='monospaced'>
2932 $cd ~/poky
2933 $mkdir meta-mylayer
2934 </literallayout>
2935 Creating a directory that follows the Yocto Project layer naming
2936 conventions sets up the layer for your changes.
2937 The layer is where you place your configuration files, append
2938 files, and patch files.
2939 To learn more about creating a layer and filling it with the
2940 files you need, see the "<link linkend='understanding-and-creating-layers'>Understanding
2941 and Creating Layers</link>" section.
2942 </para>
2943 </section>
2944
2945 <section id='finding-the-kernel-source-code'>
2946 <title>Finding the Kernel Source Code</title>
2947
2948 <para>
2949 Each time you build a kernel image, the kernel source code is fetched
2950 and unpacked into the following directory:
2951 <literallayout class='monospaced'>
2952 ${S}/linux
2953 </literallayout>
2954 See the "<link linkend='finding-the-temporary-source-code'>Finding the Temporary Source Code</link>"
2955 section and the
2956 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> variable
2957 for more information about where source is kept during a build.
2958 </para>
2959
2960 <para>
2961 For this example, we are going to patch the
2962 <filename>init/calibrate.c</filename> file
2963 by adding some simple console <filename>printk</filename> statements that we can
2964 see when we boot the image using QEMU.
2965 </para>
2966 </section>
2967
2968 <section id='creating-the-patch'>
2969 <title>Creating the Patch</title>
2970
2971 <para>
2972 Two methods exist by which you can create the patch:
2973 <link linkend='using-a-git-workflow'>Git workflow</link> and
2974 <link linkend='using-a-quilt-workflow'>Quilt workflow</link>.
2975 For kernel patches, the Git workflow is more appropriate.
2976 This section assumes the Git workflow and shows the steps specific to
2977 this example.
2978 <orderedlist>
2979 <listitem><para><emphasis>Change the working directory</emphasis>:
2980 Change to where the kernel source code is before making
2981 your edits to the <filename>calibrate.c</filename> file:
2982 <literallayout class='monospaced'>
2983 $ cd ~/poky/build/tmp/work/qemux86-poky-linux/linux-yocto-${PV}-${PR}/linux
2984 </literallayout>
2985 Because you are working in an established Git repository,
2986 you must be in this directory in order to commit your changes
2987 and create the patch file.
2988 <note>The <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> and
2989 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> variables
2990 represent the version and revision for the
2991 <filename>linux-yocto</filename> recipe.
2992 The <filename>PV</filename> variable includes the Git meta and machine
2993 hashes, which make the directory name longer than you might
2994 expect.
2995 </note></para></listitem>
2996 <listitem><para><emphasis>Edit the source file</emphasis>:
2997 Edit the <filename>init/calibrate.c</filename> file to have the
2998 following changes:
2999 <literallayout class='monospaced'>
3000 void __cpuinit calibrate_delay(void)
3001 {
3002 unsigned long lpj;
3003 static bool printed;
3004 int this_cpu = smp_processor_id();
3005
3006 printk("*************************************\n");
3007 printk("* *\n");
3008 printk("* HELLO YOCTO KERNEL *\n");
3009 printk("* *\n");
3010 printk("*************************************\n");
3011
3012 if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
3013 .
3014 .
3015 .
3016 </literallayout></para></listitem>
3017 <listitem><para><emphasis>Stage and commit your changes</emphasis>:
3018 These Git commands display the modified file, stage it, and then
3019 commit the file:
3020 <literallayout class='monospaced'>
3021 $ git status
3022 $ git add init/calibrate.c
3023 $ git commit -m "calibrate: Add printk example"
3024 </literallayout></para></listitem>
3025 <listitem><para><emphasis>Generate the patch file</emphasis>:
3026 This Git command creates the a patch file named
3027 <filename>0001-calibrate-Add-printk-example.patch</filename>
3028 in the current directory.
3029 <literallayout class='monospaced'>
3030 $ git format-patch -1
3031 </literallayout>
3032 </para></listitem>
3033 </orderedlist>
3034 </para>
3035 </section>
3036
3037 <section id='set-up-your-layer-for-the-build'>
3038 <title>Set Up Your Layer for the Build</title>
3039
3040 <para>These steps get your layer set up for the build:
3041 <orderedlist>
3042 <listitem><para><emphasis>Create additional structure</emphasis>:
3043 Create the additional layer structure:
3044 <literallayout class='monospaced'>
3045 $ cd ~/poky/meta-mylayer
3046 $ mkdir conf
3047 $ mkdir recipes-kernel
3048 $ mkdir recipes-kernel/linux
3049 $ mkdir recipes-kernel/linux/linux-yocto
3050 </literallayout>
3051 The <filename>conf</filename> directory holds your configuration files, while the
3052 <filename>recipes-kernel</filename> directory holds your append file and
3053 your patch file.</para></listitem>
3054 <listitem><para><emphasis>Create the layer configuration file</emphasis>:
3055 Move to the <filename>meta-mylayer/conf</filename> directory and create
3056 the <filename>layer.conf</filename> file as follows:
3057 <literallayout class='monospaced'>
3058 # We have a conf and classes directory, add to BBPATH
3059 BBPATH .= ":${LAYERDIR}"
3060
3061 # We have recipes-* directories, add to BBFILES
3062 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
3063 ${LAYERDIR}/recipes-*/*/*.bbappend"
3064
3065 BBFILE_COLLECTIONS += "mylayer"
3066 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/"
3067 BBFILE_PRIORITY_mylayer = "5"
3068 </literallayout>
3069 Notice <filename>mylayer</filename> as part of the last three
3070 statements.</para></listitem>
3071 <listitem><para><emphasis>Create the kernel recipe append file</emphasis>:
3072 Move to the <filename>meta-mylayer/recipes-kernel/linux</filename> directory and create
3073 the <filename>linux-yocto_3.4.bbappend</filename> file as follows:
3074 <literallayout class='monospaced'>
3075 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
3076
3077 SRC_URI += "file://0001-calibrate-Add-printk-example.patch"
3078
3079 PRINC := "${@int(PRINC) + 1}"
3080 </literallayout>
3081 The <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
3082 and <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
3083 statements enable the OpenEmbedded build system to find the patch file.
3084 For more information on using append files, see the
3085 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>"
3086 section.
3087 </para></listitem>
3088 <listitem><para><emphasis>Put the patch file in your layer</emphasis>:
3089 Move the <filename>0001-calibrate-Add-printk-example.patch</filename> file to
3090 the <filename>meta-mylayer/recipes-kernel/linux/linux-yocto</filename>
3091 directory.</para></listitem>
3092 </orderedlist>
3093 </para>
3094 </section>
3095
3096 <section id='set-up-for-the-build'>
3097 <title>Set Up for the Build</title>
3098
3099 <para>
3100 Do the following to make sure the build parameters are set up for the example.
3101 Once you set up these build parameters, they do not have to change unless you
3102 change the target architecture of the machine you are building:
3103 <itemizedlist>
3104 <listitem><para><emphasis>Build for the correct target architecture:</emphasis> Your
3105 selected <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
3106 definition within the <filename>local.conf</filename> file in the
3107 <link linkend='build-directory'>Build Directory</link>
3108 specifies the target architecture used when building the Linux kernel.
3109 By default, <filename>MACHINE</filename> is set to
3110 <filename>qemux86</filename>, which specifies a 32-bit
3111 <trademark class='registered'>Intel</trademark> Architecture
3112 target machine suitable for the QEMU emulator.</para></listitem>
3113 <listitem><para><emphasis>Identify your <filename>meta-mylayer</filename>
3114 layer:</emphasis> The
3115 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink>
3116 variable in the
3117 <filename>bblayers.conf</filename> file found in the
3118 <filename>poky/build/conf</filename> directory needs to have the path to your local
3119 <filename>meta-mylayer</filename> layer.
3120 By default, the <filename>BBLAYERS</filename> variable contains paths to
3121 <filename>meta</filename>, <filename>meta-yocto</filename>, and
3122 <filename>meta-yocto-bsp</filename> in the
3123 <filename>poky</filename> Git repository.
3124 Add the path to your <filename>meta-mylayer</filename> location:
3125 <literallayout class='monospaced'>
3126 BBLAYERS ?= " \
3127 $HOME/poky/meta \
3128 $HOME/poky/meta-yocto \
3129 $HOME/poky/meta-yocto-bsp \
3130 $HOME/poky/meta-mylayer \
3131 "
3132
3133 BBLAYERS_NON_REMOVABLE ?= " \
3134 $HOME/poky/meta \
3135 $HOME/poky/meta-yocto \
3136 "
3137 </literallayout></para></listitem>
3138 </itemizedlist>
3139 </para>
3140 </section>
3141
3142 <section id='build-the-modified-qemu-kernel-image'>
3143 <title>Build the Modified QEMU Kernel Image</title>
3144
3145 <para>
3146 The following steps build your modified kernel image:
3147 <orderedlist>
3148 <listitem><para><emphasis>Be sure your build environment is initialized</emphasis>:
3149 Your environment should be set up since you previously sourced
3150 the
3151 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
3152 script.
3153 If it is not, source the script again from <filename>poky</filename>.
3154 <literallayout class='monospaced'>
3155 $ cd ~/poky
3156 $ source &OE_INIT_FILE;
3157 </literallayout>
3158 </para></listitem>
3159 <listitem><para><emphasis>Clean up</emphasis>:
3160 Be sure to clean the shared state out by running the
3161 <filename>cleansstate</filename> BitBake task as follows from your Build Directory:
3162 <literallayout class='monospaced'>
3163 $ bitbake -c cleansstate linux-yocto
3164 </literallayout></para>
3165 <para><note>Never remove any files by hand from the <filename>tmp/deploy</filename>
3166 directory inside the
3167 <link linkend='build-directory'>Build Directory</link>.
3168 Always use the various BitBake clean tasks to clear out previous
3169 build artifacts.
3170 </note></para></listitem>
3171 <listitem><para><emphasis>Build the image</emphasis>:
3172 Next, build the kernel image using this command:
3173 <literallayout class='monospaced'>
3174 $ bitbake -k linux-yocto
3175 </literallayout></para></listitem>
3176 </orderedlist>
3177 </para>
3178 </section>
3179
3180 <section id='boot-the-image-and-verify-your-changes'>
3181 <title>Boot the Image and Verify Your Changes</title>
3182
3183 <para>
3184 These steps boot the image and allow you to see the changes
3185 <orderedlist>
3186 <listitem><para><emphasis>Boot the image</emphasis>:
3187 Boot the modified image in the QEMU emulator
3188 using this command:
3189 <literallayout class='monospaced'>
3190 $ runqemu qemux86
3191 </literallayout></para></listitem>
3192 <listitem><para><emphasis>Verify the changes</emphasis>:
3193 Log into the machine using <filename>root</filename> with no password and then
3194 use the following shell command to scroll through the console's boot output.
3195 <literallayout class='monospaced'>
3196 # dmesg | less
3197 </literallayout>
3198 You should see the results of your <filename>printk</filename> statements
3199 as part of the output.</para></listitem>
3200 </orderedlist>
3201 </para>
3202 </section>
3203 </section>
3204
3205 <section id='creating-your-own-distribution'>
3206 <title>Creating Your Own Distribution</title>
3207
3208 <para>
3209 When you build an image using the Yocto Project and
3210 do not alter any distribution
3211 <link linkend='metadata'>Metadata</link>, you are creating a
3212 Poky distribution.
3213 If you wish to gain more control over package alternative
3214 selections, compile-time options, and other low-level
3215 configurations, you can create your own distribution.
3216 </para>
3217
3218 <para>
3219 To create your own distribution, the basic steps consist of
3220 creating your own distribution layer, creating your own
3221 distribution configuration file, and then adding any needed
3222 code and Metadata to the layer.
3223 The following steps provide some more detail:
3224 <itemizedlist>
3225 <listitem><para><emphasis>Create a layer for your new distro:</emphasis>
3226 Create your distribution layer so that you can keep your
3227 Metadata and code for the distribution separate.
3228 It is strongly recommended that you create and use your own
3229 layer for configuration and code.
3230 Using your own layer as compared to just placing
3231 configurations in a <filename>local.conf</filename>
3232 configuration file makes it easier to reproduce the same
3233 build configuration when using multiple build machines.
3234 See the
3235 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
3236 section for information on how to quickly set up a layer.
3237 </para></listitem>
3238 <listitem><para><emphasis>Create the distribution configuration file:</emphasis>
3239 The distribution configuration file needs to be created in
3240 the <filename>conf/distro</filename> directory of your
3241 layer.
3242 You need to name it using your distribution name
3243 (e.g. <filename>mydistro.conf</filename>).</para>
3244 <para>You can split out parts of your configuration file
3245 into include files and then "require" them from within
3246 your distribution configuration file.
3247 Be sure to place the include files in the
3248 <filename>conf/distro/include</filename> directory of
3249 your layer.
3250 A common example usage of include files would be to
3251 separate out the selection of desired version and revisions
3252 for individual recipes.
3253</para>
3254 <para>Your configuration file needs to set the following
3255 required variables:
3256 <literallayout class='monospaced'>
3257 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink> [required]
3258 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink> [required]
3259 </literallayout>
3260 These following variables are optional and you typically
3261 set them from the distribution configuration file:
3262 <literallayout class='monospaced'>
3263 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> [optional]
3264 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink> [optional]
3265 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink> [optional]
3266 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink> [optional]
3267 </literallayout>
3268 <tip>
3269 If you want to base your distribution configuration file
3270 on the very basic configuration from OE-Core, you
3271 can use
3272 <filename>conf/distro/defaultsetup.conf</filename> as
3273 a reference and just include variables that differ
3274 as compared to <filename>defaultsetup.conf</filename>.
3275 Alternatively, you can create a distribution
3276 configuration file from scratch using the
3277 <filename>defaultsetup.conf</filename> file
3278 or configuration files from other distributions
3279 such as Poky or Angstrom as references.
3280 </tip></para></listitem>
3281 <listitem><para><emphasis>Provide miscellaneous variables:</emphasis>
3282 Be sure to define any other variables for which you want to
3283 create a default or enforce as part of the distribution
3284 configuration.
3285 You can include nearly any variable from the
3286 <filename>local.conf</filename> file.
3287 The variables you use are not limited to the list in the
3288 previous bulleted item.</para></listitem>
3289 <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis>
3290 In your <filename>local.conf</filename> file in the
3291 <link linkend='build-directory'>Build Directory</link>,
3292 set your
3293 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
3294 variable to point to your distribution's configuration file.
3295 For example, if your distribution's configuration file is
3296 named <filename>mydistro.conf</filename>, then you point
3297 to it as follows:
3298 <literallayout class='monospaced'>
3299 DISTRO = "mydistro"
3300 </literallayout></para></listitem>
3301 <listitem><para><emphasis>Add more to the layer if necessary:</emphasis>
3302 Use your layer to hold other information needed for the
3303 distribution:
3304 <itemizedlist>
3305 <listitem><para>Add recipes for installing
3306 distro-specific configuration files that are not
3307 already installed by another recipe.
3308 If you have distro-specific configuration files
3309 that are included by an existing recipe, you should
3310 add a <filename>.bbappend</filename> for those.
3311 For general information and recommendations
3312 on how to add recipes to your layer, see the
3313 "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>"
3314 and
3315 "<link linkend='best-practices-to-follow-when-creating-layers'>Best Practices to Follow When Creating Layers</link>"
3316 sections.</para></listitem>
3317 <listitem><para>Add any image recipes that are specific
3318 to your distribution.</para></listitem>
3319 <listitem><para>Add a <filename>psplash</filename>
3320 append file for a branded splash screen.
3321 For information on append files, see the
3322 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>"
3323 section.</para></listitem>
3324 <listitem><para>Add any other append files to make
3325 custom changes that are specific to individual
3326 recipes.</para></listitem>
3327 </itemizedlist></para></listitem>
3328 </itemizedlist>
3329 </para>
3330 </section>
3331
3332 <section id='building-a-tiny-system'>
3333 <title>Building a Tiny System</title>
3334
3335 <para>
3336 Very small distributions have some significant advantages such
3337 as requiring less on-die or in-package memory (cheaper), better
3338 performance through efficient cache usage, lower power requirements
3339 due to less memory, faster boot times, and reduced development
3340 overhead.
3341 Some real-world examples where a very small distribution gives
3342 you distinct advantages are digital cameras, medical devices,
3343 and small headless systems.
3344 </para>
3345
3346 <para>
3347 This section presents information that shows you how you can
3348 trim your distribution to even smaller sizes than the
3349 <filename>poky-tiny</filename> distribution, which is around
3350 5 Mbytes, that can be built out-of-the-box using the Yocto Project.
3351 </para>
3352
3353 <section id='tiny-system-overview'>
3354 <title>Overview</title>
3355
3356 <para>
3357 The following list presents the overall steps you need to
3358 consider and perform to create distributions with smaller
3359 root filesystems, faster boot times, maintain your critical
3360 functionality, and avoid initial RAM disks:
3361 <itemizedlist>
3362 <listitem><para>Determine your goals and guiding
3363 principles.</para></listitem>
3364 <listitem><para>Understand what gives your image size.
3365 </para></listitem>
3366 <listitem><para>Reduce the size of the root filesystem.
3367 </para></listitem>
3368 <listitem><para>Reduce the size of the kernel.
3369 </para></listitem>
3370 <listitem><para>Eliminate packaging requirements.
3371 </para></listitem>
3372 <listitem><para>Look for other ways to minimize size.
3373 </para></listitem>
3374 <listitem><para>Iterate on the process.</para></listitem>
3375 </itemizedlist>
3376 </para>
3377 </section>
3378
3379 <section id='goals-and-guiding-principles'>
3380 <title>Goals and Guiding Principles</title>
3381
3382 <para>
3383 Before you can reach your destination, you need to know
3384 where you are going.
3385 Here is an example list that you can use as a guide when
3386 creating very small distributions:
3387 <itemizedlist>
3388 <listitem><para>Determine how much space you need
3389 (e.g. a kernel that is 1 Mbyte or less and
3390 a root filesystem that is 3 Mbytes or less).
3391 </para></listitem>
3392 <listitem><para>Find the areas that are currently
3393 taking 90% of the space and concentrate on reducing
3394 those areas.
3395 </para></listitem>
3396 <listitem><para>Do not create any difficult "hacks"
3397 to achieve your goals.</para></listitem>
3398 <listitem><para>Leverage the device-specific
3399 options.</para></listitem>
3400 <listitem><para>Work in a separate layer so that you
3401 keep changes isolated.
3402 For information on how to create layers, see
3403 the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section.
3404 </para></listitem>
3405 </itemizedlist>
3406 </para>
3407 </section>
3408
3409 <section id='understand-what-gives-your-image-size'>
3410 <title>Understand What Gives Your Image Size</title>
3411
3412 <para>
3413 It is easiest to have something to start with when creating
3414 your own distribution.
3415 You can use the Yocto Project out-of-the-box to create the
3416 <filename>poky-tiny</filename> distribution.
3417 Ultimately, you will want to make changes in your own
3418 distribution that are likely modeled after
3419 <filename>poky-tiny</filename>.
3420 <note>
3421 To use <filename>poky-tiny</filename> in your build,
3422 set the
3423 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
3424 variable in your
3425 <filename>local.conf</filename> file to "poky-tiny"
3426 as described in the
3427 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
3428 section.
3429 </note>
3430 </para>
3431
3432 <para>
3433 Understanding some memory concepts will help you reduce the
3434 system size.
3435 Memory consists of static, dynamic, and temporary memory.
3436 Static memory is the TEXT (code), DATA (initialized data
3437 in the code), and BSS (uninitialized data) sections.
3438 Dynamic memory contains memory that is allocated at runtime,
3439 stacks, hash tables, and so forth.
3440 Temporary memory is recovered after the boot process.
3441 This memory consists of memory used for decompressing
3442 the kernel and for the <filename>__init__</filename>
3443 functions.
3444 </para>
3445
3446 <para>
3447 To help you see where you currently are with kernel and root
3448 filesystem sizes, you can use two tools found in the
3449 <link linkend='source-directory'>Source Directory</link> in
3450 the <filename>scripts</filename> directory:
3451 <itemizedlist>
3452 <listitem><para><filename>ksize.py</filename>: Reports
3453 component sizes for the kernel build objects.
3454 </para></listitem>
3455 <listitem><para><filename>dirsize.py</filename>: Reports
3456 component sizes for the root filesystem.</para></listitem>
3457 </itemizedlist>
3458 This next tool and command helps you organize configuration
3459 fragments and view file dependencies in a human-readable form:
3460 <itemizedlist>
3461 <listitem><para><filename>merge_config.sh</filename>:
3462 Helps you manage configuration files and fragments
3463 within the kernel.
3464 With this tool, you can merge individual configuration
3465 fragments together.
3466 The tool allows you to make overrides and warns you
3467 of any missing configuration options.
3468 The tool is ideal for allowing you to iterate on
3469 configurations, create minimal configurations, and
3470 create configuration files for different machines
3471 without having to duplicate your process.</para>
3472 <para>The <filename>merge_config.sh</filename> script is
3473 part of the Linux Yocto kernel Git repository in the
3474 <filename>scripts/kconfig</filename> directory.</para>
3475 <para>For more information on configuration fragments,
3476 see the
3477 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>"
3478 section of the Yocto Project Linux Kernel Development
3479 Manual and the "<link linkend='creating-config-fragments'>Creating Configuration Fragments</link>"
3480 section, which is in this manual.</para></listitem>
3481 <listitem><para><filename>bitbake -u depexp -g &lt;bitbake_target&gt;</filename>:
3482 Using the BitBake command with these options brings up
3483 a Dependency Explorer from which you can view file
3484 dependencies.
3485 Understanding these dependencies allows you to make
3486 informed decisions when cutting out various pieces of the
3487 kernel and root filesystem.</para></listitem>
3488 </itemizedlist>
3489 </para>
3490 </section>
3491
3492 <section id='trim-the-root-filesystem'>
3493 <title>Trim the Root Filesystem</title>
3494
3495 <para>
3496 The root filesystem is made up of packages for booting,
3497 libraries, and applications.
3498 To change things, you can configure how the packaging happens,
3499 which changes the way you build them.
3500 You can also tweak the filesystem itself or select a different
3501 filesystem.
3502 </para>
3503
3504 <para>
3505 First, find out what is hogging your root filesystem by running the
3506 <filename>dirsize.py</filename> script from your root directory:
3507 <literallayout class='monospaced'>
3508 $ cd &lt;root-directory-of-image&gt;
3509 $ dirsize.py 100000 > dirsize-100k.log
3510 $ cat dirsize-100k.log
3511 </literallayout>
3512 You can apply a filter to the script to ignore files under
3513 a certain size.
3514 This example filters out anything below 100 Kbytes.
3515 The sizes reported by the tool are uncompressed and thus,
3516 will be smaller by a relatively constant factor in a
3517 compressed root filesystem.
3518 When you examine your log file, you can focus on areas of the
3519 root filesystem that take up large amounts of memory.
3520 </para>
3521
3522 <para>
3523 You need to be sure that what you eliminate does not cripple
3524 the functionality you need.
3525 One way to see how packages relate to each other is by using
3526 the Dependency Explorer UI with the BitBake command:
3527 <literallayout class='monospaced'>
3528 $ cd &lt;image-directory&gt;
3529 $ bitbake -u depexp -g &lt;image&gt;
3530 </literallayout>
3531 Use the interface to select potential packages you wish to
3532 eliminate and see their dependency relationships.
3533 </para>
3534
3535 <para>
3536 When deciding how to reduce the size, get rid of packages that
3537 result in minimal impact on the feature set.
3538 For example, you might not need a VGA display.
3539 Or, you might be able to get by with <filename>devtmpfs</filename>
3540 and <filename>mdev</filename> instead of
3541 <filename>udev</filename>.
3542 </para>
3543
3544 <para>
3545 Use the <filename>local.conf</filename> file to make changes.
3546 For example, to eliminate <filename>udev</filename> and
3547 <filename>glib</filename>, set the following in the
3548 local configuration file:
3549 <literallayout class='monospaced'>
3550 VIRTUAL-RUNTIME_dev_manager = ""
3551 </literallayout>
3552 </para>
3553
3554 <para>
3555 Finally, you should consider exactly the type of root
3556 filesystem you need to meet your needs while also reducing
3557 its size.
3558 For example, consider <filename>cramfs</filename>,
3559 <filename>squashfs</filename>, <filename>ubifs</filename>,
3560 <filename>ext2</filename>, or an <filename>initramfs</filename>
3561 using <filename>initramfs</filename>.
3562 Be aware that <filename>ext3</filename> requires a 1 Mbyte
3563 journal.
3564 If you are okay with running read-only you do not need this
3565 journal.
3566 </para>
3567
3568 <note>
3569 After each round of elimination, you need to rebuild your
3570 system and then use the tools to see the effects of your
3571 reductions.
3572 </note>
3573
3574
3575 </section>
3576
3577 <section id='trim-the-kernel'>
3578 <title>Trim the Kernel</title>
3579
3580 <para>
3581 The kernel is built by including policies for hardware-independent
3582 aspects.
3583 What subsystems do you enable?
3584 For what architecture are you building?
3585 Which drivers do you build by default.
3586 <note>You can modify the kernel source if you want to help
3587 with boot time.
3588 </note>
3589 </para>
3590
3591 <para>
3592 Run the <filename>ksize.py</filename> script from the top-level
3593 Linux build directory to get an idea of what is making up
3594 the kernel:
3595 <literallayout class='monospaced'>
3596 $ cd &lt;top-level-linux-build-directory&gt;
3597 $ ksize.py > ksize.log
3598 $ cat ksize.log
3599 </literallayout>
3600 When you examine the log, you will see how much space is
3601 taken up with the built-in <filename>.o</filename> files for
3602 drivers, networking, core kernel files, filesystem, sound,
3603 and so forth.
3604 The sizes reported by the tool are uncompressed and thus,
3605 will be smaller by a relatively constant factor in a compressed
3606 kernel image.
3607 Look to reduce the areas that are large and taking up around
3608 the "90% rule."
3609 </para>
3610
3611 <para>
3612 To examine, or drill down, into any particular area, use the
3613 <filename>-d</filename> option with the script:
3614 <literallayout class='monospaced'>
3615 $ ksize.py -d > ksize.log
3616 </literallayout>
3617 Using this option breaks out the individual file information
3618 for each area of the kernel (e.g. drivers, networking, and
3619 so forth).
3620 </para>
3621
3622 <para>
3623 Use your log file to see what you can eliminate from the kernel
3624 based on features you can let go.
3625 For example, if you are not going to need sound, you do not
3626 need any drivers that support sound.
3627 </para>
3628
3629 <para>
3630 After figuring out what to eliminate, you need to reconfigure
3631 the kernel to reflect those changes during the next build.
3632 You could run <filename>menuconfig</filename> and make all your
3633 changes at once.
3634 However, that makes it difficult to see the effects of your
3635 individual eliminations and also makes it difficult to replicate
3636 the changes for perhaps another target device.
3637 A better method is to start with no configurations using
3638 <filename>allnoconfig</filename>, create configuration
3639 fragments for individual changes, and then manage the
3640 fragments into a single configuration file using
3641 <filename>merge_config.sh</filename>.
3642 The tool makes it easy for you to iterate using the
3643 configuration change and build cycle.
3644 </para>
3645
3646 <para>
3647 Each time you make configuration changes, you need to rebuild
3648 the kernel and check to see what impact your changes had on
3649 the overall size.
3650 </para>
3651 </section>
3652
3653 <section id='remove-package-management-requirements'>
3654 <title>Remove Package Management Requirements</title>
3655
3656 <para>
3657 Packaging requirements add size to the image.
3658 One way to reduce the size of the image is to remove all the
3659 packaging requirements from the image.
3660 This reduction includes both removing the package manager
3661 and its unique dependencies as well as removing the package
3662 management data itself.
3663 </para>
3664
3665 <para>
3666 To eliminate all the packaging requirements for an image,
3667 follow these steps:
3668 <orderedlist>
3669 <listitem><para>Put the following line in your main
3670 recipe for the image to remove package management
3671 data files:
3672 <literallayout class='monospaced'>
3673 ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ;
3674 </literallayout>
3675 For example, the recipe for the
3676 <filename>core-image-minimal</filename> image contains
3677 this line.
3678 You can also add the line to the
3679 <filename>local.conf</filename> configuration file.
3680 </para></listitem>
3681 <listitem><para>Be sure that "package-management" is not
3682 part of your
3683 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
3684 statement for the image.
3685 When you remove this feature, you are removing the
3686 package manager as well as its dependencies
3687 from the root filesystem.
3688 </para></listitem>
3689 </orderedlist>
3690 </para>
3691 </section>
3692
3693 <section id='look-for-other-ways-to-minimize-size'>
3694 <title>Look for Other Ways to Minimize Size</title>
3695
3696 <para>
3697 Depending on your particular circumstances, other areas that you
3698 can trim likely exist.
3699 The key to finding these areas is through tools and methods
3700 described here combined with experimentation and iteration.
3701 Here are a couple of areas to experiment with:
3702 <itemizedlist>
3703 <listitem><para><filename>eglibc</filename>:
3704 In general, follow this process:
3705 <orderedlist>
3706 <listitem><para>Remove <filename>eglibc</filename>
3707 features from
3708 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
3709 that you think you do not need.</para></listitem>
3710 <listitem><para>Build your distribution.
3711 </para></listitem>
3712 <listitem><para>If the build fails due to missing
3713 symbols in a package, determine if you can
3714 reconfigure the package to not need those
3715 features.
3716 For example, change the configuration to not
3717 support wide character support as is done for
3718 <filename>ncurses</filename>.
3719 Or, if support for those characters is needed,
3720 determine what <filename>eglibc</filename>
3721 features provide the support and restore the
3722 configuration.
3723 </para></listitem>
3724 <listitem><para>Rebuild and repeat the process.
3725 </para></listitem>
3726 </orderedlist></para></listitem>
3727 <listitem><para><filename>busybox</filename>:
3728 For BusyBox, use a process similar as described for
3729 <filename>eglibc</filename>.
3730 A difference is you will need to boot the resulting
3731 system to see if you are able to do everything you
3732 expect from the running system.
3733 You need to be sure to integrate configuration fragments
3734 into Busybox because BusyBox handles its own core
3735 features and then allows you to add configuration
3736 fragments on top.
3737 </para></listitem>
3738 </itemizedlist>
3739 </para>
3740 </section>
3741
3742 <section id='iterate-on-the-process'>
3743 <title>Iterate on the Process</title>
3744
3745 <para>
3746 If you have not reached your goals on system size, you need
3747 to iterate on the process.
3748 The process is the same.
3749 Use the tools and see just what is taking up 90% of the root
3750 filesystem and the kernel.
3751 Decide what you can eliminate without limiting your device
3752 beyond what you need.
3753 </para>
3754
3755 <para>
3756 Depending on your system, a good place to look might be
3757 Busybox, which provides a stripped down
3758 version of Unix tools in a single, executable file.
3759 You might be able to drop virtual terminal services or perhaps
3760 ipv6.
3761 </para>
3762 </section>
3763 </section>
3764
3765 <section id='working-with-packages'>
3766 <title>Working with Packages</title>
3767
3768 <para>
3769 This section describes a few tasks that involve packages:
3770 <itemizedlist>
3771 <listitem><para>Excluding packages from an image
3772 </para></listitem>
3773 <listitem><para>Incrementing a package revision number
3774 </para></listitem>
3775 <listitem><para>Handling a package name alias
3776 </para></listitem>
3777 <listitem><para>Handling optional module packaging
3778 </para></listitem>
3779 <listitem><para>Using Runtime Package Management
3780 </para></listitem>
3781 <listitem><para>Setting up and running package test
3782 (ptest)
3783 </para></listitem>
3784 </itemizedlist>
3785 </para>
3786
3787 <section id='excluding-packages-from-an-image'>
3788 <title>Excluding Packages from an Image</title>
3789
3790 <para>
3791 You might find it necessary to prevent specific packages
3792 from being installed into an image.
3793 If so, you can use several variables to direct the build
3794 system to essentially ignore installing recommended packages
3795 or to not install a package at all.
3796 </para>
3797
3798 <para>
3799 The following list introduces variables you can use to
3800 prevent packages from being installed into your image.
3801 Each of these variables only works with IPK and RPM
3802 package types.
3803 Support for Debian packages does not exist.
3804 Also, you can use these variables from your
3805 <filename>local.conf</filename> file or attach them to a
3806 specific image recipe by using a recipe name override.
3807 For more detail on the variables, see the descriptions in the
3808 Yocto Project Reference Manual's glossary chapter.
3809 <itemizedlist>
3810 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>:
3811 Use this variable to specify "recommended-only"
3812 packages that you do not want installed.
3813 </para></listitem>
3814 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>:
3815 Use this variable to prevent all "recommended-only"
3816 packages from being installed.
3817 </para></listitem>
3818 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>:
3819 Use this variable to prevent specific packages from
3820 being installed regardless of whether they are
3821 "recommended-only" or not.
3822 You need to realize that the build process could
3823 fail with an error when you
3824 prevent the installation of a package whose presence
3825 is required by an installed package.
3826 </para></listitem>
3827 </itemizedlist>
3828 </para>
3829 </section>
3830
3831 <section id='incrementing-a-package-revision-number'>
3832 <title>Incrementing a Package Revision Number</title>
3833
3834 <para>
3835 If a committed change results in changing the package output,
3836 then the value of the
3837 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
3838 variable needs to be increased (or "bumped").
3839 Increasing <filename>PR</filename> occurs one of two ways:
3840 <itemizedlist>
3841 <listitem><para>Automatically using a Package Revision
3842 Service (PR Service).</para></listitem>
3843 <listitem><para>Manually incrementing the
3844 <filename>PR</filename> variable.</para></listitem>
3845 </itemizedlist>
3846 </para>
3847
3848 <para>
3849 Given that one of the challenges any build system and its
3850 users face is how to maintain a package feed that is compatible
3851 with existing package manager applications such as
3852 RPM, APT, and OPKG, using an automated system is much
3853 preferred over a manual system.
3854 In either system, the main requirement is that version
3855 numbering increases in a linear fashion and that a number of
3856 version components exist that support that linear progression.
3857 </para>
3858
3859 <para>
3860 The following two sections provide information on the PR Service
3861 and on manual <filename>PR</filename> bumping.
3862 </para>
3863
3864 <section id='working-with-a-pr-service'>
3865 <title>Working With a PR Service</title>
3866
3867 <para>
3868 As mentioned, attempting to maintain revision numbers in the
3869 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
3870 is error prone, inaccurate and causes problems for people
3871 submitting recipes.
3872 Conversely, the PR Service automatically generates
3873 increasing numbers, particularly the revision field,
3874 which removes the human element.
3875 <note>
3876 For additional information on using a PR Service, you
3877 can see the
3878 <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink>
3879 wiki page.
3880 </note>
3881 </para>
3882
3883 <para>
3884 The Yocto Project uses variables in order of
3885 decreasing priority to facilitate revision numbering (i.e.
3886 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>,
3887 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
3888 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
3889 for epoch, version and revision, respectively).
3890 The values are highly dependent on the policies and
3891 procedures of a given distribution and package feed.
3892 </para>
3893
3894 <para>
3895 Because the OpenEmbedded build system uses
3896 "<ulink url='&YOCTO_DOCS_REF_URL;#checksums'>signatures</ulink>",
3897 which are unique to a given build, the build system
3898 knows when to rebuild packages.
3899 All the inputs into a given task are represented by a
3900 signature, which can trigger a rebuild when different.
3901 Thus, the build system itself does not rely on the
3902 <filename>PR</filename> numbers to trigger a rebuild.
3903 The signatures, however, can be used to generate
3904 <filename>PR</filename> values.
3905 </para>
3906
3907 <para>
3908 The PR Service works with both
3909 <filename>OEBasic</filename> and
3910 <filename>OEBasicHash</filename> generators.
3911 The value of <filename>PR</filename> bumps when the
3912 checksum changes and the different generator mechanisms
3913 change signatures under different circumstances.
3914 </para>
3915
3916 <para>
3917 As implemented, the build system includes values from
3918 the PR Service into the <filename>PR</filename> field as
3919 an addition using the form "<filename>.x</filename>" so
3920 <filename>r0</filename> becomes <filename>r0.1</filename>,
3921 <filename>r0.2</filename> and so forth.
3922 This scheme allows existing <filename>PR</filename> values
3923 to be used for whatever reasons, which include manual
3924 <filename>PR</filename> bumps should it be necessary.
3925 </para>
3926
3927 <para>
3928 By default, the PR Service is not enabled or running.
3929 Thus, the packages generated are just "self consistent".
3930 The build system adds and removes packages and
3931 there are no guarantees about upgrade paths but images
3932 will be consistent and correct with the latest changes.
3933 </para>
3934
3935 <para>
3936 The simplest form for a PR Service is for it to exist
3937 for a single host development system that builds the
3938 package feed (building system).
3939 For this scenario, you can enable the PR Service by adding
3940 the following to your <filename>local.conf</filename>
3941 file in the
3942 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
3943 <literallayout class='monospaced'>
3944 PRSERV_HOST = "localhost:0"
3945 </literallayout>
3946 Once the service is started, packages will automatically
3947 get increasing <filename>PR</filename> values and
3948 BitBake will take care of starting and stopping the server.
3949 </para>
3950
3951 <para>
3952 If you have a more complex setup where multiple host
3953 development systems work against a common, shared package
3954 feed, you have a single PR Service running and it is
3955 connected to each building system.
3956 For this scenario, you need to start the PR Service using
3957 the <filename>bitbake-prserv</filename> command:
3958 <literallayout class='monospaced'>
3959 bitbake-prserv &dash;&dash;host &lt;ip&gt; &dash;&dash;port &lt;port&gt; &dash;&dash;start
3960 </literallayout>
3961 In addition to hand-starting the service, you need to
3962 update the <filename>local.conf</filename> file of each
3963 building system as described earlier so each system
3964 points to the server and port.
3965 </para>
3966
3967 <para>
3968 It is also recommended you use Build History, which adds
3969 some sanity checks to package versions, in conjunction with
3970 the server that is running the PR Service.
3971 To enable build history, add the following to each building
3972 system's <filename>local.conf</filename> file:
3973 <literallayout class='monospaced'>
3974 # It is recommended to activate "buildhistory" for testing the PR service
3975 INHERIT += "buildhistory"
3976 BUILDHISTORY_COMMIT = "1"
3977 </literallayout>
3978 For information on Build History, see the
3979 "<ulink url='&YOCTO_DOCS_REF_URL;#maintaining-build-output-quality'>Maintaining Build Output Quality</ulink>"
3980 section in the Yocto Project Reference Manual.
3981 </para>
3982
3983 <note>
3984 <para>The OpenEmbedded build system does not maintain
3985 <filename>PR</filename> information as part of the
3986 shared state (sstate) packages.
3987 If you maintain an sstate feed, its expected that either
3988 all your building systems that contribute to the sstate
3989 feed use a shared PR Service, or you do not run a PR
3990 Service on any of your building systems.
3991 Having some systems use a PR Service while others do
3992 not leads to obvious problems.</para>
3993 <para>For more information on shared state, see the
3994 "<ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>"
3995 section in the Yocto Project Reference Manual.</para>
3996 </note>
3997 </section>
3998
3999 <section id='manually-bumping-pr'>
4000 <title>Manually Bumping PR</title>
4001
4002 <para>
4003 The alternative to setting up a PR Service is to manually
4004 bump the
4005 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
4006 variable.
4007 </para>
4008
4009 <para>
4010 If a committed change results in changing the package output,
4011 then the value of the PR variable needs to be increased
4012 (or "bumped") as part of that commit.
4013 For new recipes you should add the <filename>PR</filename>
4014 variable and set its initial value equal to "r0", which is the default.
4015 Even though the default value is "r0", the practice of adding it to a new recipe makes
4016 it harder to forget to bump the variable when you make changes
4017 to the recipe in future.
4018 </para>
4019
4020 <para>
4021 If you are sharing a common <filename>.inc</filename> file with multiple recipes,
4022 you can also use the
4023 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename>
4024 variable to ensure that
4025 the recipes sharing the <filename>.inc</filename> file are rebuilt when the
4026 <filename>.inc</filename> file itself is changed.
4027 The <filename>.inc</filename> file must set <filename>INC_PR</filename>
4028 (initially to "r0"), and all recipes referring to it should set <filename>PR</filename>
4029 to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed.
4030 If the <filename>.inc</filename> file is changed then its
4031 <filename>INC_PR</filename> should be incremented.
4032 </para>
4033
4034 <para>
4035 When upgrading the version of a package, assuming the
4036 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename>
4037 changes, the <filename>PR</filename> variable should be
4038 reset to "r0" (or "$(INC_PR).0" if you are using
4039 <filename>INC_PR</filename>).
4040 </para>
4041
4042 <para>
4043 Usually, version increases occur only to packages.
4044 However, if for some reason <filename>PV</filename> changes but does not
4045 increase, you can increase the
4046 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename>
4047 variable (Package Epoch).
4048 The <filename>PE</filename> variable defaults to "0".
4049 </para>
4050
4051 <para>
4052 Version numbering strives to follow the
4053 <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'>
4054 Debian Version Field Policy Guidelines</ulink>.
4055 These guidelines define how versions are compared and what "increasing" a version means.
4056 </para>
4057 </section>
4058 </section>
4059
4060 <section id="usingpoky-configuring-DISTRO_PN_ALIAS">
4061 <title>Handling a Package Name Alias</title>
4062 <para>
4063 Sometimes a package name you are using might exist under an alias or as a similarly named
4064 package in a different distribution.
4065 The OpenEmbedded build system implements a <filename>distro_check</filename>
4066 task that automatically connects to major distributions
4067 and checks for these situations.
4068 If the package exists under a different name in a different distribution, you get a
4069 <filename>distro_check</filename> mismatch.
4070 You can resolve this problem by defining a per-distro recipe name alias using the
4071 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_PN_ALIAS'>DISTRO_PN_ALIAS</ulink></filename>
4072 variable.
4073 </para>
4074
4075 <para>
4076 Following is an example that shows how you specify the <filename>DISTRO_PN_ALIAS</filename>
4077 variable:
4078 <literallayout class='monospaced'>
4079 DISTRO_PN_ALIAS_pn-PACKAGENAME = "distro1=package_name_alias1 \
4080 distro2=package_name_alias2 \
4081 distro3=package_name_alias3 \
4082 ..."
4083 </literallayout>
4084 </para>
4085
4086 <para>
4087 If you have more than one distribution alias, separate them with a space.
4088 Note that the build system currently automatically checks the
4089 Fedora, OpenSUSE, Debian, Ubuntu,
4090 and Mandriva distributions for source package recipes without having to specify them
4091 using the <filename>DISTRO_PN_ALIAS</filename> variable.
4092 For example, the following command generates a report that lists the Linux distributions
4093 that include the sources for each of the recipes.
4094 <literallayout class='monospaced'>
4095 $ bitbake world -f -c distro_check
4096 </literallayout>
4097 The results are stored in the <filename>build/tmp/log/distro_check-${DATETIME}.results</filename>
4098 file found in the
4099 <link linkend='source-directory'>Source Directory</link>.
4100 </para>
4101 </section>
4102
4103 <section id='handling-optional-module-packaging'>
4104 <title>Handling Optional Module Packaging</title>
4105
4106 <para>
4107 Many pieces of software split functionality into optional
4108 modules (or plug-ins) and the plug-ins that are built
4109 might depend on configuration options.
4110 To avoid having to duplicate the logic that determines what
4111 modules are available in your recipe or to avoid having
4112 to package each module by hand, the OpenEmbedded build system
4113 provides functionality to handle module packaging dynamically.
4114 </para>
4115
4116 <para>
4117 To handle optional module packaging, you need to do two things:
4118 <itemizedlist>
4119 <listitem><para>Ensure the module packaging is actually
4120 done</para></listitem>
4121 <listitem><para>Ensure that any dependencies on optional
4122 modules from other recipes are satisfied by your recipe
4123 </para></listitem>
4124 </itemizedlist>
4125 </para>
4126
4127 <section id='making-sure-the-packaging-is-done'>
4128 <title>Making Sure the Packaging is Done</title>
4129
4130 <para>
4131 To ensure the module packaging actually gets done, you use
4132 the <filename>do_split_packages</filename> function within
4133 the <filename>populate_packages</filename> Python function
4134 in your recipe.
4135 The <filename>do_split_packages</filename> function
4136 searches for a pattern of files or directories under a
4137 specified path and creates a package for each one it finds
4138 by appending to the
4139 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
4140 variable and setting the appropriate values for
4141 <filename>FILES_packagename</filename>,
4142 <filename>RDEPENDS_packagename</filename>,
4143 <filename>DESCRIPTION_packagename</filename>, and so forth.
4144 Here is an example from the <filename>lighttpd</filename>
4145 recipe:
4146 <literallayout class='monospaced'>
4147 python populate_packages_prepend () {
4148 lighttpd_libdir = d.expand('${libdir}')
4149 do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
4150 'lighttpd-module-%s', 'Lighttpd module for %s',
4151 extra_depends='')
4152 }
4153 </literallayout>
4154 The previous example specifies a number of things in the
4155 call to <filename>do_split_packages</filename>.
4156 <itemizedlist>
4157 <listitem><para>A directory within the files installed
4158 by your recipe through <filename>do_install</filename>
4159 in which to search.</para></listitem>
4160 <listitem><para>A regular expression to match module
4161 files in that directory.
4162 In the example, note the parentheses () that mark
4163 the part of the expression from which the module
4164 name should be derived.</para></listitem>
4165 <listitem><para>A pattern to use for the package names.
4166 </para></listitem>
4167 <listitem><para>A description for each package.
4168 </para></listitem>
4169 <listitem><para>An empty string for
4170 <filename>extra_depends</filename>, which disables
4171 the default dependency on the main
4172 <filename>lighttpd</filename> package.
4173 Thus, if a file in <filename>${libdir}</filename>
4174 called <filename>mod_alias.so</filename> is found,
4175 a package called <filename>lighttpd-module-alias</filename>
4176 is created for it and the
4177 <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink>
4178 is set to "Lighttpd module for alias".</para></listitem>
4179 </itemizedlist>
4180 </para>
4181
4182 <para>
4183 Often, packaging modules is as simple as the previous
4184 example.
4185 However, more advanced options exist that you can use
4186 within <filename>do_split_packages</filename> to modify its
4187 behavior.
4188 And, if you need to, you can add more logic by specifying
4189 a hook function that is called for each package.
4190 It is also perfectly acceptable to call
4191 <filename>do_split_packages</filename> multiple times if
4192 you have more than one set of modules to package.
4193 </para>
4194
4195 <para>
4196 For more examples that show how to use
4197 <filename>do_split_packages</filename>, see the
4198 <filename>connman.inc</filename> file in the
4199 <filename>meta/recipes-connectivity/connman/</filename>
4200 directory of the <filename>poky</filename>
4201 <link linkend='yocto-project-repositories'>source repository</link>.
4202 You can also find examples in
4203 <filename>meta/classes/kernel.bbclass</filename>.
4204 </para>
4205
4206 <para>
4207 Following is a reference that shows
4208 <filename>do_split_packages</filename> mandatory and
4209 optional arguments:
4210 <literallayout class='monospaced'>
4211 Mandatory arguments
4212
4213 root
4214 The path in which to search
4215 file_regex
4216 Regular expression to match searched files.
4217 Use parentheses () to mark the part of this
4218 expression that should be used to derive the
4219 module name (to be substituted where %s is
4220 used in other function arguments as noted below)
4221 output_pattern
4222 Pattern to use for the package names. Must
4223 include %s.
4224 description
4225 Description to set for each package. Must
4226 include %s.
4227
4228 Optional arguments
4229
4230 postinst
4231 Postinstall script to use for all packages
4232 (as a string)
4233 recursive
4234 True to perform a recursive search - default
4235 False
4236 hook
4237 A hook function to be called for every match.
4238 The function will be called with the following
4239 arguments (in the order listed):
4240
4241 f
4242 Full path to the file/directory match
4243 pkg
4244 The package name
4245 file_regex
4246 As above
4247 output_pattern
4248 As above
4249 modulename
4250 The module name derived using file_regex
4251
4252 extra_depends
4253 Extra runtime dependencies (RDEPENDS) to be
4254 set for all packages. The default value of None
4255 causes a dependency on the main package
4256 (${PN}) - if you do not want this, pass empty
4257 string '' for this parameter.
4258 aux_files_pattern
4259 Extra item(s) to be added to FILES for each
4260 package. Can be a single string item or a list
4261 of strings for multiple items. Must include %s.
4262 postrm
4263 postrm script to use for all packages (as a
4264 string)
4265 allow_dirs
4266 True to allow directories to be matched -
4267 default False
4268 prepend
4269 If True, prepend created packages to PACKAGES
4270 instead of the default False which appends them
4271 match_path
4272 match file_regex on the whole relative path to
4273 the root rather than just the file name
4274 aux_files_pattern_verbatim
4275 Extra item(s) to be added to FILES for each
4276 package, using the actual derived module name
4277 rather than converting it to something legal
4278 for a package name. Can be a single string item
4279 or a list of strings for multiple items. Must
4280 include %s.
4281 allow_links
4282 True to allow symlinks to be matched - default
4283 False
4284 </literallayout>
4285 </para>
4286 </section>
4287
4288 <section id='satisfying-dependencies'>
4289 <title>Satisfying Dependencies</title>
4290
4291 <para>
4292 The second part for handling optional module packaging
4293 is to ensure that any dependencies on optional modules
4294 from other recipes are satisfied by your recipe.
4295 You can be sure these dependencies are satisfied by
4296 using the
4297 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable.
4298 Here is an example that continues with the
4299 <filename>lighttpd</filename> recipe shown earlier:
4300 <literallayout class='monospaced'>
4301 PACKAGES_DYNAMIC = "lighttpd-module-.*"
4302 </literallayout>
4303 The name specified in the regular expression can of
4304 course be anything.
4305 In this example, it is <filename>lighttpd-module-</filename>
4306 and is specified as the prefix to ensure that any
4307 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
4308 and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>
4309 on a package name starting with the prefix are satisfied
4310 during build time.
4311 If you are using <filename>do_split_packages</filename>
4312 as described in the previous section, the value you put in
4313 <filename>PACKAGES_DYNAMIC</filename> should correspond to
4314 the name pattern specified in the call to
4315 <filename>do_split_packages</filename>.
4316 </para>
4317 </section>
4318 </section>
4319
4320 <section id='using-runtime-package-management'>
4321 <title>Using Runtime Package Management</title>
4322
4323 <para>
4324 During a build, BitBake always transforms a recipe into one or
4325 more packages.
4326 For example, BitBake takes the <filename>bash</filename> recipe
4327 and currently produces the <filename>bash-dbg</filename>,
4328 <filename>bash-staticdev</filename>,
4329 <filename>bash-dev</filename>, <filename>bash-doc</filename>,
4330 <filename>bash-locale</filename>, and
4331 <filename>bash</filename> packages.
4332 Not all generated packages are included in an image.
4333 </para>
4334
4335 <para>
4336 In several situations, you might need to update, add, remove,
4337 or query the packages on a target device at runtime
4338 (i.e. without having to generate a new image).
4339 Examples of such situations include:
4340 <itemizedlist>
4341 <listitem><para>
4342 You want to provide in-the-field updates to deployed
4343 devices (e.g. security updates).
4344 </para></listitem>
4345 <listitem><para>
4346 You want to have a fast turn-around development cycle
4347 for one or more applications that run on your device.
4348 </para></listitem>
4349 <listitem><para>
4350 You want to temporarily install the "debug" packages
4351 of various applications on your device so that
4352 debugging can be greatly improved by allowing
4353 access to symbols and source debugging.
4354 </para></listitem>
4355 <listitem><para>
4356 You want to deploy a more minimal package selection of
4357 your device but allow in-the-field updates to add a
4358 larger selection for customization.
4359 </para></listitem>
4360 </itemizedlist>
4361 </para>
4362
4363 <para>
4364 In all these situations, you have something similar to a more
4365 traditional Linux distribution in that in-field devices
4366 are able to receive pre-compiled packages from a server for
4367 installation or update.
4368 Being able to install these packages on a running,
4369 in-field device is what is termed "runtime package
4370 management".
4371 </para>
4372
4373 <para>
4374 In order to use runtime package management, you
4375 need a host/server machine that serves up the pre-compiled
4376 packages plus the required metadata.
4377 You also need package manipulation tools on the target.
4378 The build machine is a likely candidate to act as the server.
4379 However, that machine does not necessarily have to be the
4380 package server.
4381 The build machine could push its artifacts to another machine
4382 that acts as the server (e.g. Internet-facing).
4383 </para>
4384
4385 <para>
4386 A simple build that targets just one device produces
4387 more than one package database.
4388 In other words, the packages produced by a build are separated
4389 out into a couple of different package groupings based on
4390 criteria such as the target's CPU architecture, the target
4391 board, or the C library used on the target.
4392 For example, a build targeting the <filename>qemuarm</filename>
4393 device produces the following three package databases:
4394 <filename>all</filename>, <filename>armv5te</filename>, and
4395 <filename>qemuarm</filename>.
4396 If you wanted your <filename>qemuarm</filename> device to be
4397 aware of all the packages that were available to it,
4398 you would need to point it to each of these databases
4399 individually.
4400 In a similar way, a traditional Linux distribution usually is
4401 configured to be aware of a number of software repositories
4402 from which it retrieves packages.
4403 </para>
4404
4405 <para>
4406 Using runtime package management is completely optional and
4407 not required for a successful build or deployment in any
4408 way.
4409 But if you want to make use of runtime package management,
4410 you need to do a couple things above and beyond the basics.
4411 The remainder of this section describes what you need to do.
4412 </para>
4413
4414 <section id='runtime-package-management-build'>
4415 <title>Build Considerations</title>
4416
4417 <para>
4418 This section describes build considerations that you need
4419 to be aware of in order to provide support for runtime
4420 package management.
4421 </para>
4422
4423 <para>
4424 When BitBake generates packages it needs to know
4425 what format(s) to use.
4426 In your configuration, you use the
4427 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
4428 variable to specify the format.
4429 <note>
4430 You can choose to have more than one format but you must
4431 provide at least one.
4432 </note>
4433 </para>
4434
4435 <para>
4436 If you would like your image to start off with a basic
4437 package database of the packages in your current build
4438 as well as have the relevant tools available on the
4439 target for runtime package management, you can include
4440 "package-management" in the
4441 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
4442 variable.
4443 Including "package-management" in this
4444 configuration variable ensures that when the image
4445 is assembled for your target, the image includes
4446 the currently-known package databases as well as
4447 the target-specific tools required for runtime
4448 package management to be performed on the target.
4449 However, this is not strictly necessary.
4450 You could start your image off without any databases
4451 but only include the required on-target package
4452 tool(s).
4453 As an example, you could include "opkg" in your
4454 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
4455 variable if you are using the IPK package format.
4456 You can then initialize your target's package database(s)
4457 later once your image is up and running.
4458 </para>
4459
4460 <para>
4461 Whenever you perform any sort of build step that can
4462 potentially generate a package or modify an existing
4463 package, it is always a good idea to re-generate the
4464 package index with:
4465 <literallayout class='monospaced'>
4466 $ bitbake package-index
4467 </literallayout>
4468 Realize that it is not sufficient to simply do the
4469 following:
4470 <literallayout class='monospaced'>
4471 $ bitbake &lt;some-package&gt; package-index
4472 </literallayout>
4473 This is because BitBake does not properly schedule the
4474 <filename>package-index</filename> target fully after any
4475 other target has completed.
4476 Thus, be sure to run the package update step separately.
4477 </para>
4478
4479 <para>
4480 As described below in the
4481 "<link linkend='runtime-package-management-target-ipk'>Using IPK</link>"
4482 section, if you are using IPK as your package format, you
4483 can make use of the
4484 <filename>distro-feed-configs</filename> recipe provided
4485 by <filename>meta-oe</filename> in order to configure your
4486 target to use your IPK databases.
4487 </para>
4488
4489 <para>
4490 When your build is complete, your packages reside in the
4491 <filename>${TMPDIR}/deploy/&lt;package-format&gt;</filename>
4492 directory.
4493 For example, if <filename>${TMPDIR}</filename>
4494 is <filename>tmp</filename> and your selected package type
4495 is IPK, then your IPK packages are available in
4496 <filename>tmp/deploy/ipk</filename>.
4497 </para>
4498 </section>
4499
4500 <section id='runtime-package-management-server'>
4501 <title>Host or Server Machine Setup</title>
4502
4503 <para>
4504 Typically, packages are served from a server using
4505 HTTP.
4506 However, other protocols are possible.
4507 If you want to use HTTP, then setup and configure a
4508 web server, such as Apache 2 or lighttpd, on the machine
4509 serving the packages.
4510 </para>
4511
4512 <para>
4513 As previously mentioned, the build machine can act as the
4514 package server.
4515 In the following sections that describe server machine
4516 setups, the build machine is assumed to also be the server.
4517 </para>
4518
4519 <section id='package-server-apache'>
4520 <title>Serving Packages via Apache 2</title>
4521
4522 <para>
4523 This example assumes you are using the Apache 2
4524 server:
4525 <orderedlist>
4526 <listitem><para>
4527 Add the directory to your Apache
4528 configuration, which you can find at
4529 <filename>/etc/httpd/conf/httpd.conf</filename>.
4530 Use commands similar to these on the
4531 development system.
4532 These example commands assume a top-level
4533 <link linkend='source-directory'>Source Directory</link>
4534 named <filename>poky</filename> in your home
4535 directory.
4536 The example also assumes an RPM package type.
4537 If you are using a different package type, such
4538 as IPK, use "ipk" in the pathnames:
4539 <literallayout class='monospaced'>
4540 &lt;VirtualHost *:80&gt;
4541 ....
4542 Alias /rpm ~/poky/build/tmp/deploy/rpm
4543 &lt;Directory "~/poky/build/tmp/deploy/rpm"&gt;
4544 Options +Indexes
4545 &lt;/Directory&gt;
4546 &lt;/VirtualHost&gt;
4547 </literallayout></para></listitem>
4548 <listitem><para>
4549 Reload the Apache configuration as described
4550 in this step.
4551 For all commands, be sure you have root
4552 privileges.
4553 </para>
4554
4555 <para>
4556 If your development system is using Fedora or
4557 CentOS, use the following:
4558 <literallayout class='monospaced'>
4559 # service httpd reload
4560 </literallayout>
4561 For Ubuntu and Debian, use the following:
4562 <literallayout class='monospaced'>
4563 # /etc/init.d/apache2 reload
4564 </literallayout>
4565 For OpenSUSE, use the following:
4566 <literallayout class='monospaced'>
4567 # /etc/init.d/apache2 reload
4568 </literallayout></para></listitem>
4569 <listitem><para>
4570 If you are using Security-Enhanced Linux
4571 (SELinux), you need to label the files as
4572 being accessible through Apache.
4573 Use the following command from the development
4574 host.
4575 This example assumes RPM package types:
4576 <literallayout class='monospaced'>
4577 # chcon -R -h -t httpd_sys_content_t tmp/deploy/rpm
4578 </literallayout></para></listitem>
4579 </orderedlist>
4580 </para>
4581 </section>
4582
4583 <section id='package-server-lighttpd'>
4584 <title>Serving Packages via lighttpd</title>
4585
4586 <para>
4587 If you are using lighttpd, all you need
4588 to do is to provide a link from your
4589 <filename>${TMPDIR}/deploy/&lt;package-format&gt;</filename>
4590 directory to lighttpd's document-root.
4591 You can determine the specifics of your lighttpd
4592 installation by looking through its configuration file,
4593 which is usually found at:
4594 <filename>/etc/lighttpd/lighttpd.conf</filename>.
4595 </para>
4596
4597 <para>
4598 For example, if you are using IPK, lighttpd's
4599 document-root is set to
4600 <filename>/var/www/lighttpd</filename>, and you had
4601 packages for a target named "BOARD",
4602 then you might create a link from your build location
4603 to lighttpd's document-root as follows:
4604 <literallayout class='monospaced'>
4605 # ln -s $(PWD)/tmp/deploy/ipk /var/www/lighttpd/BOARD-dir
4606 </literallayout>
4607 </para>
4608
4609 <para>
4610 At this point, you need to start the lighttpd server.
4611 The method used to start the server varies by
4612 distribution.
4613 However, one basic method that starts it by hand is:
4614 <literallayout class='monospaced'>
4615 # lighttpd -f /etc/lighttpd/lighttpd.conf
4616 </literallayout>
4617 </para>
4618 </section>
4619 </section>
4620
4621 <section id='runtime-package-management-target'>
4622 <title>Target Setup</title>
4623
4624 <para>
4625 Setting up the target differs depending on the
4626 package management system.
4627 This section provides information for RPM and IPK.
4628 </para>
4629
4630 <section id='runtime-package-management-target-rpm'>
4631 <title>Using RPM</title>
4632
4633 <para>
4634 The application for performing runtime package
4635 management of RPM packages on the target is called
4636 <filename>smart</filename>.
4637 </para>
4638
4639 <para>
4640 On the target machine, you need to inform
4641 <filename>smart</filename> of every package database
4642 you want to use.
4643 As an example, suppose your target device can use the
4644 following three package databases from a server named
4645 <filename>server.name</filename>:
4646 <filename>all</filename>, <filename>i586</filename>,
4647 and <filename>qemux86</filename>.
4648 Given this example, issue the following commands on the
4649 target:
4650 <literallayout class='monospaced'>
4651 # smart channel --add all type=rpm-md baseurl=http://server.name/rpm/all
4652 # smart channel --add i585 type=rpm-md baseurl=http://server.name/rpm/i586
4653 # smart channel --add qemux86 type=rpm-md baseurl=http://server.name/rpm/qemux86
4654 </literallayout>
4655 Also from the target machine, fetch the repository
4656 information using this command:
4657 <literallayout class='monospaced'>
4658 # smart update
4659 </literallayout>
4660 You can now use the <filename>smart query</filename>
4661 and <filename>smart install</filename> commands to
4662 find and install packages from the repositories.
4663 </para>
4664 </section>
4665
4666 <section id='runtime-package-management-target-ipk'>
4667 <title>Using IPK</title>
4668
4669 <para>
4670 The application for performing runtime package
4671 management of IPK packages on the target is called
4672 <filename>opkg</filename>.
4673 </para>
4674
4675 <para>
4676 In order to inform <filename>opkg</filename> of the
4677 package databases you want to use, simply create one
4678 or more <filename>*.conf</filename> files in the
4679 <filename>/etc/opkg</filename> directory on the target.
4680 The <filename>opkg</filename> application uses them
4681 to find its available package databases.
4682 As an example, suppose you configured your HTTP server
4683 on your machine named
4684 <filename>www.mysite.com</filename> to serve files
4685 from a <filename>BOARD-dir</filename> directory under
4686 its document-root.
4687 In this case, you might create a configuration
4688 file on the target called
4689 <filename>/etc/opkg/base-feeds.conf</filename> that
4690 contains:
4691 <literallayout class='monospaced'>
4692 src/gz all http://www.mysite.com/BOARD-dir/all
4693 src/gz armv7a http://www.mysite.com/BOARD-dir/armv7a
4694 src/gz beagleboard http://www.mysite.com/BOARD-dir/beagleboard
4695 </literallayout>
4696 </para>
4697
4698 <para>
4699 As a way of making it easier to generate and make
4700 these IPK configuration files available on your
4701 target, simply define
4702 <ulink url='&YOCTO_DOCS_REF_URL;#var-FEED_DEPLOYDIR_BASE_URI'><filename>FEED_DEPLOYDIR_BASE_URI</filename></ulink>
4703 to point to your server and the location within the
4704 document-root which contains the databases.
4705 For example: if you are serving your packages over
4706 HTTP, your server's IP address is 192.168.7.1, and
4707 your databases are located in a directory called
4708 <filename>BOARD-dir</filename> underneath your HTTP
4709 server's document-root, you need to set
4710 <filename>FEED_DEPLOYDIR_BASE_URI</filename> to
4711 <filename>http://192.168.7.1/BOARD-dir</filename> and
4712 a set of configuration files will be generated for you
4713 in your target to work with this feed.
4714 </para>
4715
4716 <para>
4717 On the target machine, fetch (or refresh) the
4718 repository information using this command:
4719 <literallayout class='monospaced'>
4720 # opkg update
4721 </literallayout>
4722 You can now use the <filename>opkg list</filename> and
4723 <filename>opkg install</filename> commands to find and
4724 install packages from the repositories.
4725 </para>
4726 </section>
4727 </section>
4728 </section>
4729
4730 <section id='testing-packages-with-ptest'>
4731 <title>Testing Packages With ptest</title>
4732
4733 <para>
4734 A Package Test (ptest) runs tests against packages built
4735 by the OpenEmbedded build system on the target machine.
4736 A ptest contains at least two items: the actual test, and
4737 a shell script (<filename>run-ptest</filename>) that starts
4738 the test.
4739 The shell script that starts the test must not contain
4740 the actual test, the script only starts it.
4741 On the other hand, the test can be anything from a simple
4742 shell script that runs a binary and checks the output to
4743 an elaborate system of test binaries and data files.
4744 </para>
4745
4746 <para>
4747 The test generates output in the format used by
4748 Automake:
4749 <literallayout class='monospaced'>
4750 &lt;result&gt;: &lt;testname&gt;
4751 </literallayout>
4752 where the result can be <filename>PASS</filename>,
4753 <filename>FAIL</filename>, or <filename>SKIP</filename>,
4754 and the testname can be any identifying string.
4755 </para>
4756
4757 <note>
4758 With this release of the Yocto Project, three recipes exist
4759 that are "ptest-enabled": <filename>bash</filename>,
4760 <filename>glib-2.0</filename>, and
4761 <filename>dbus</filename>.
4762 These three recipes are Autotool-enabled.
4763 </note>
4764
4765 <section id='adding-ptest-to-your-build'>
4766 <title>Adding ptest to Your Build</title>
4767
4768 <para>
4769 To add package testing to your build, add the
4770 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
4771 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
4772 variables to your <filename>local.conf</filename> file,
4773 which is found in the
4774 <link linkend='build-directory'>Build Directory</link>:
4775 <literallayout class='monospaced'>
4776 DISTRO_FEATURES_append = " ptest"
4777 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
4778 </literallayout>
4779 Once your build is complete, the ptest files are installed
4780 into the <filename>/usr/lib/&lt;package&gt;/ptest</filename>
4781 directory within the image, where
4782 <filename>&lt;package&gt;</filename> is the name of the
4783 package.
4784 </para>
4785 </section>
4786
4787 <section id='running-ptest'>
4788 <title>Running ptest</title>
4789
4790 <para>
4791 The <filename>ptest-runner</filename> package installs a
4792 shell script that loops through all installed ptest test
4793 suites and runs them in sequence.
4794 Consequently, you might want to add this package to
4795 your image.
4796 </para>
4797 </section>
4798
4799 <section id='getting-your-package-ready'>
4800 <title>Getting Your Package Ready</title>
4801
4802 <para>
4803 In order to enable a recipe to run installed ptests
4804 on target hardware,
4805 you need to prepare the recipes that build the packages
4806 you want to test.
4807 Here is what you have to do for each recipe:
4808 <itemizedlist>
4809 <listitem><para><emphasis>Be sure the recipe
4810 inherits ptest:</emphasis>
4811 Include the following line in each recipe:
4812 <literallayout class='monospaced'>
4813 inherit ptest
4814 </literallayout>
4815 </para></listitem>
4816 <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis>
4817 This script starts your test.
4818 Locate the script where you will refer to it
4819 using
4820 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
4821 Here is an example that starts a test for
4822 <filename>dbus</filename>:
4823 <literallayout class='monospaced'>
4824 #!/bin/sh
4825 cd test
4826 make -k runtest-TESTS
4827 </literallayout>
4828 </para></listitem>
4829 <listitem><para><emphasis>Ensure dependencies are
4830 met:</emphasis>
4831 If the test adds build or runtime dependencies
4832 that normally do not exist for the package
4833 (such as requiring "make" to run the test suite),
4834 use the
4835 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
4836 and
4837 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
4838 variables in your recipe in order for the package
4839 to meet the dependencies.
4840 Here is an example where the package has a runtime
4841 dependency on "make":
4842 <literallayout class='monospaced'>
4843 RDEPENDS_${PN}-ptest += "make"
4844 </literallayout>
4845 </para></listitem>
4846 <listitem><para><emphasis>Add a function to build the
4847 test suite:</emphasis>
4848 Not many packages support cross-compilation of
4849 their test suites.
4850 Consequently, you usually need to add a
4851 cross-compilation function to the package.
4852 </para>
4853 <para>Many packages based on Automake compile and
4854 run the test suite by using a single command
4855 such as <filename>make check</filename>.
4856 However, the native <filename>make check</filename>
4857 builds and runs on the same computer, while
4858 cross-compiling requires that the package is built
4859 on the host but executed on the target.
4860 The built version of Automake that ships with the
4861 Yocto Project includes a patch that separates
4862 building and execution.
4863 Consequently, packages that use the unaltered,
4864 patched version of <filename>make check</filename>
4865 automatically cross-compiles.</para>
4866 <para>However, you still must add a
4867 <filename>do_compile_ptest</filename> function to
4868 build the test suite.
4869 Add a function similar to the following to your
4870 recipe:
4871 <literallayout class='monospaced'>
4872 do_compile_ptest() {
4873 oe_runmake buildtest-TESTS
4874 }
4875 </literallayout>
4876 </para></listitem>
4877 <listitem><para><emphasis>Ensure special configurations
4878 are set:</emphasis>
4879 If the package requires special configurations
4880 prior to compiling the test code, you must
4881 insert a <filename>do_configure_ptest</filename>
4882 function into the recipe.
4883 </para></listitem>
4884 <listitem><para><emphasis>Install the test
4885 suite:</emphasis>
4886 The <filename>ptest.bbclass</filename> class
4887 automatically copies the file
4888 <filename>run-ptest</filename> to the target and
4889 then runs make <filename>install-ptest</filename>
4890 to run the tests.
4891 If this is not enough, you need to create a
4892 <filename>do_install_ptest</filename> function and
4893 make sure it gets called after the
4894 "make install-ptest" completes.
4895 </para></listitem>
4896 </itemizedlist>
4897 </para>
4898 </section>
4899 </section>
4900 </section>
4901
4902 <section id="building-software-from-an-external-source">
4903 <title>Building Software from an External Source</title>
4904
4905 <para>
4906 By default, the OpenEmbedded build system uses the
4907 <link linkend='build-directory'>Build Directory</link> to
4908 build source code.
4909 The build process involves fetching the source files, unpacking
4910 them, and then patching them if necessary before the build takes
4911 place.
4912 </para>
4913
4914 <para>
4915 Situations exist where you might want to build software from source
4916 files that are external to and thus outside of the
4917 OpenEmbedded build system.
4918 For example, suppose you have a project that includes a new BSP with
4919 a heavily customized kernel.
4920 And, you want to minimize exposing the build system to the
4921 development team so that they can focus on their project and
4922 maintain everyone's workflow as much as possible.
4923 In this case, you want a kernel source directory on the development
4924 machine where the development occurs.
4925 You want the recipe's
4926 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
4927 variable to point to the external directory and use it as is, not
4928 copy it.
4929 </para>
4930
4931 <para>
4932 To build from software that comes from an external source, all you
4933 need to do is inherit
4934 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></ulink>
4935 and then set the
4936 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>
4937 variable to point to your external source code.
4938 Here are the statements to put in your
4939 <filename>local.conf</filename> file:
4940 <literallayout class='monospaced'>
4941 INHERIT += "externalsrc"
4942 EXTERNALSRC_pn-myrecipe = "/some/path/to/your/source/tree"
4943 </literallayout>
4944 </para>
4945
4946 <para>
4947 By default, <filename>externalsrc.bbclass</filename> builds
4948 the source code in a directory separate from the external source
4949 directory as specified by
4950 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>.
4951 If you need to have the source built in the same directory in
4952 which it resides, or some other nominated directory, you can set
4953 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink>
4954 to point to that directory:
4955 <literallayout class='monospaced'>
4956 EXTERNALSRC_BUILD_pn-myrecipe = "/path/to/my/source/tree"
4957 </literallayout>
4958 </para>
4959 </section>
4960
4961 <section id="selecting-an-initialization-manager">
4962 <title>Selecting an Initialization Manager</title>
4963
4964 <para>
4965 By default, the Yocto Project uses
4966 <filename>SysVinit</filename> as the initialization manager.
4967 However, support also exists for <filename>systemd</filename>,
4968 which is a full replacement for <filename>init</filename> with
4969 parallel starting of services, reduced shell overhead and other
4970 features that are used by many distributions.
4971 </para>
4972
4973 <para>
4974 If you want to use <filename>sysvinit</filename>, you do
4975 not have to do anything.
4976 But, if you want to use <filename>systemd</filename>, you must
4977 take some steps as described in the following sections.
4978 </para>
4979
4980<!--
4981 <note>
4982 It is recommended that you create your own distribution configuration
4983 file to hold these settings instead of using your
4984 <filename>local.conf</filename> file.
4985 For information on creating your own distribution, see the
4986 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
4987 section.
4988 </note>
4989-->
4990
4991 <section id='using-systemd-exclusively'>
4992 <title>Using systemd Exclusively</title>
4993
4994 <para>
4995 Set the following variables in your distribution configuration
4996 file as follows:
4997 <literallayout class='monospaced'>
4998 DISTRO_FEATURES_append = " systemd"
4999 VIRTUAL-RUNTIME_init_manager = "systemd"
5000 </literallayout>
5001 You can also prevent the <filename>sysvinit</filename>
5002 distribution feature from
5003 being automatically enabled as follows:
5004 <literallayout class='monospaced'>
5005 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
5006 </literallayout>
5007 Doing so removes any redundant <filename>sysvinit</filename>
5008 scripts.
5009 </para>
5010
5011 <para>
5012 For information on the backfill variable, see
5013 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
5014 in the Yocto Project Reference Manual.
5015 </para>
5016 </section>
5017
5018 <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'>
5019 <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title>
5020
5021 <para>
5022 Set the following variables in your distribution configuration
5023 file as follows:
5024 <literallayout class='monospaced'>
5025 DISTRO_FEATURES_append = " systemd"
5026 VIRTUAL-RUNTIME_init_manager = "systemd"
5027 </literallayout>
5028 Doing so causes your main image to use the
5029 <filename>packagegroup-core-boot.bb</filename> recipe and
5030 <filename>systemd</filename>.
5031 The rescue/minimal image cannot use this package group.
5032 However, it can install <filename>sysvinit</filename>
5033 and the appropriate packages will have support for both
5034 <filename>systemd</filename> and <filename>sysvinit</filename>.
5035 </para>
5036 </section>
5037 </section>
5038
5039 <section id='excluding-recipes-from-the-build'>
5040 <title>Excluding Recipes From the Build</title>
5041
5042 <para>
5043 You might find that there are groups of recipes or append files
5044 that you want to filter out of the build process.
5045 Usually, this is not necessary.
5046 However, on rare occasions where you might want to use a
5047 layer but exclude parts that are causing problems, such
5048 as introducing a different version of a recipe, you can
5049 use
5050 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBMASK'><filename>BBMASK</filename></ulink>
5051 to exclude the recipe.
5052 </para>
5053
5054 <para>
5055 It is possible to filter or mask out <filename>.bb</filename> and
5056 <filename>.bbappend</filename> files.
5057 You can do this by providing an expression with the
5058 <filename>BBMASK</filename> variable.
5059 Here is an example:
5060 <literallayout class='monospaced'>
5061 BBMASK = "/meta-mymachine/recipes-maybe/"
5062 </literallayout>
5063 Here, all <filename>.bb</filename> and
5064 <filename>.bbappend</filename> files in the directory that match
5065 the expression are ignored during the build process.
5066 </para>
5067
5068 <note>
5069 The value you provide is passed to Python's regular expression
5070 compiler.
5071 The expression is compared against the full paths to the files.
5072 For complete syntax information, see Python's documentation at
5073 <ulink url='http://docs.python.org/release/2.3/lib/re-syntax.html'></ulink>.
5074 </note>
5075 </section>
5076
5077 <section id="platdev-appdev-srcrev">
5078 <title>Using an External SCM</title>
5079
5080 <para>
5081 If you're working on a recipe that pulls from an external Source Code Manager (SCM), it
5082 is possible to have the OpenEmbedded build system notice new recipe changes added to the
5083 SCM and then build the resulting package that depends on the new recipes by using the latest
5084 versions.
5085 This only works for SCMs from which it is possible to get a sensible revision number for changes.
5086 Currently, you can do this with Apache Subversion (SVN), Git, and Bazaar (BZR) repositories.
5087 </para>
5088
5089 <para>
5090 To enable this behavior, simply add the following to the <filename>local.conf</filename>
5091 configuration file found in the
5092 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
5093 <literallayout class='monospaced'>
5094 SRCREV_pn-&lt;PN&gt; = "${AUTOREV}"
5095 </literallayout>
5096 where <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>
5097 is the name of the recipe for which you want to enable automatic source
5098 revision updating.
5099 </para>
5100 </section>
5101
5102 <section id='creating-a-read-only-root-filesystem'>
5103 <title>Creating a Read-Only Root Filesystem</title>
5104
5105 <para>
5106 Suppose, for security reasons, you need to disable
5107 your target device's root filesystem's write permissions
5108 (i.e. you need a read-only root filesystem).
5109 Or, perhaps you are running the device's operating system
5110 from a read-only storage device.
5111 For either case, you can customize your image for
5112 that behavior.
5113 </para>
5114
5115 <note>
5116 Supporting a read-only root filesystem requires that the system and
5117 applications do not try to write to the root filesystem.
5118 You must configure all parts of the target system to write
5119 elsewhere, or to gracefully fail in the event of failing to
5120 write to the root filesystem.
5121 </note>
5122
5123 <section id='creating-the-root-filesystem'>
5124 <title>Creating the Root Filesystem</title>
5125
5126 <para>
5127 To create the read-only root filesystem, simply add the
5128 <filename>read-only-rootfs</filename> feature to your image.
5129 Using either of the following statements in your
5130 image recipe or from within the
5131 <filename>local.conf</filename> file found in the
5132 <link linkend='build-directory'>Build Directory</link>
5133 causes the build system to create a read-only root filesystem:
5134 <literallayout class='monospaced'>
5135 IMAGE_FEATURES = "read-only-rootfs"
5136 </literallayout>
5137 or
5138 <literallayout class='monospaced'>
5139 EXTRA_IMAGE_FEATURES = "read-only-rootfs"
5140 </literallayout>
5141 </para>
5142
5143 <para>
5144 For more information on how to use these variables, see the
5145 "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>"
5146 section.
5147 For information on the variables, see
5148 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
5149 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>.
5150 </para>
5151 </section>
5152
5153 <section id='post-installation-scripts'>
5154 <title>Post-Installation Scripts</title>
5155
5156 <para>
5157 It is very important that you make sure all
5158 post-Installation (<filename>pkg_postinst</filename>) scripts
5159 for packages that are installed into the image can be run
5160 at the time when the root filesystem is created during the
5161 build on the host system.
5162 These scripts cannot attempt to run during first-boot on the
5163 target device.
5164 With the <filename>read-only-rootfs</filename> feature enabled,
5165 the build system checks during root filesystem creation to make
5166 sure all post-installation scripts succeed.
5167 If any of these scripts still need to be run after the root
5168 filesystem is created, the build immediately fails.
5169 These checks during build time ensure that the build fails
5170 rather than the target device fails later during its
5171 initial boot operation.
5172 </para>
5173
5174 <para>
5175 Most of the common post-installation scripts generated by the
5176 build system for the out-of-the-box Yocto Project are engineered
5177 so that they can run during root filesystem creation
5178 (e.g. post-installation scripts for caching fonts).
5179 However, if you create and add custom scripts, you need
5180 to be sure they can be run during file system creation.
5181 </para>
5182
5183 <para>
5184 Here are some common problems that prevent
5185 post-installation scripts from running during root filesystem
5186 creation:
5187 <itemizedlist>
5188 <listitem><para><emphasis>Not using $D in front of absolute paths:</emphasis>
5189 The build system defines
5190 <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
5191 at root filesystem creation time, and
5192 it is blank when run on the target device.
5193 This implies two purposes for <filename>$D</filename>:
5194 ensuring paths are valid in both the host and target
5195 environments, and checking to determine which
5196 environment is being used as a method for taking
5197 appropriate actions.
5198 </para></listitem>
5199 <listitem><para><emphasis>Attempting to run processes that are
5200 specific to or dependent on the target
5201 architecture:</emphasis>
5202 You can work around these attempts by using native
5203 tools to accomplish the same tasks, or
5204 by alternatively running the processes under QEMU,
5205 which has the <filename>qemu_run_binary</filename>
5206 function.
5207 For more information, see the
5208 <filename>meta/classes/qemu.bbclass</filename>
5209 class in the
5210 <link linkend='source-directory'>Source Directory</link>.
5211 </para></listitem>
5212 </itemizedlist>
5213 </para>
5214 </section>
5215
5216 <section id='areas-with-write-access'>
5217 <title>Areas With Write Access</title>
5218
5219 <para>
5220 With the <filename>read-only-rootfs</filename> feature enabled,
5221 any attempt by the target to write to the root filesystem at
5222 runtime fails.
5223 Consequently, you must make sure that you configure processes
5224 and applications that attempt these types of writes do so
5225 to directories with write access (e.g.
5226 <filename>/tmp</filename> or <filename>/var/run</filename>).
5227 </para>
5228 </section>
5229 </section>
5230
5231 <section id="performing-automated-runtime-testing">
5232 <title>Performing Automated Runtime Testing</title>
5233
5234 <para>
5235 The OpenEmbedded build system makes available a series of automated
5236 tests for images to verify runtime functionality.
5237 <note>
5238 Currently, there is only support for running these tests
5239 under QEMU.
5240 </note>
5241 These tests are written in Python making use of the
5242 <filename>unittest</filename> module, and the majority of them
5243 run commands on the target system over
5244 <filename>ssh</filename>.
5245 This section describes how you set up the environment to use these
5246 tests, run available tests, and write and add your own tests.
5247 </para>
5248
5249 <section id="qemu-image-enabling-tests">
5250 <title>Enabling Tests</title>
5251
5252 <para>
5253 In order to run tests, you need to do the following:
5254 <itemizedlist>
5255 <listitem><para><emphasis>Set up to avoid interaction
5256 with <filename>sudo</filename> for networking:</emphasis>
5257 To accomplish this, you must do one of the
5258 following:
5259 <itemizedlist>
5260 <listitem><para>Add
5261 <filename>NOPASSWD</filename> for your user
5262 in <filename>/etc/sudoers</filename> either for
5263 ALL commands or just for
5264 <filename>runqemu-ifup</filename>.
5265 You must provide the full path as that can
5266 change if you are using multiple clones of the
5267 source repository.
5268 <note>
5269 On some distributions, you also need to
5270 comment out "Defaults requiretty" in
5271 <filename>/etc/sudoers</filename>.
5272 </note></para></listitem>
5273 <listitem><para>Manually configure a tap interface
5274 for your system.</para></listitem>
5275 <listitem><para>Run as root the script in
5276 <filename>scripts/runqemu-gen-tapdevs</filename>,
5277 which should generate a list of tap devices.
5278 This is the option typically chosen for
5279 Autobuilder-type environments.
5280 </para></listitem>
5281 </itemizedlist></para></listitem>
5282 <listitem><para><emphasis>Set the
5283 <filename>DISPLAY</filename> variable:</emphasis>
5284 You need to set this variable so that you have an X
5285 server available (e.g. start
5286 <filename>vncserver</filename> for a headless machine).
5287 </para></listitem>
5288 <listitem><para><emphasis>Be sure your host's firewall
5289 accepts incoming connections from
5290 192.168.7.0/24:</emphasis>
5291 Some of the tests (in particular smart tests) start a
5292 HTTP server on a random high number port, which is
5293 used to serve files to the target.
5294 The smart module serves
5295 <filename>${DEPLOY_DIR}/rpm</filename> so it can run
5296 smart channel commands. That means your host's firewall
5297 must accept incoming connections from 192.168.7.0/24,
5298 which is the default IP range used for tap devices
5299 by <filename>runqemu</filename>.</para></listitem>
5300 </itemizedlist>
5301 </para>
5302
5303 <note>
5304 Regardless of how you initiate the tests, if you built your
5305 image using <filename>rm_work</filename>,
5306 most of the tests will fail with errors because they rely on
5307 <filename>${WORKDIR}/installed_pkgs.txt</filename>.
5308 </note>
5309 </section>
5310
5311 <section id="qemu-image-running-tests">
5312 <title>Running Tests</title>
5313
5314 <para>
5315 You can start the tests automatically or manually:
5316 <itemizedlist>
5317 <listitem><para><emphasis>Automatically Running Tests:</emphasis>
5318 To run the tests automatically after the
5319 OpenEmbedded build system successfully creates an image,
5320 first set the
5321 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
5322 variable to "1" in your <filename>local.conf</filename>
5323 file in the
5324 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
5325 <literallayout class='monospaced'>
5326 TEST_IMAGE = "1"
5327 </literallayout>
5328 Next, simply build your image.
5329 If the image successfully builds, the tests will be
5330 run:
5331 <literallayout class='monospaced'>
5332 bitbake core-image-sato
5333 </literallayout></para></listitem>
5334 <listitem><para><emphasis>Manually Running Tests:</emphasis>
5335 To manually run the tests, first globally inherit
5336 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename></ulink>
5337 by editing your <filename>local.conf</filename> file:
5338 <literallayout class='monospaced'>
5339 INHERIT += "testimage"
5340 </literallayout>
5341 Next, use BitBake to run the tests:
5342 <literallayout class='monospaced'>
5343 bitbake -c testimage &lt;image&gt;
5344 </literallayout></para></listitem>
5345 </itemizedlist>
5346 </para>
5347
5348 <para>
5349 Regardless of how you run the tests, once they start, the
5350 following happens:
5351 <itemizedlist>
5352 <listitem><para>A copy of the root filesystem is written
5353 to <filename>${WORKDIR}/testimage</filename>.
5354 </para></listitem>
5355 <listitem><para>The image is booted under QEMU using the
5356 standard <filename>runqemu</filename> script.
5357 </para></listitem>
5358 <listitem><para>A default timeout of 500 seconds occurs
5359 to allow for the boot process to reach the login prompt.
5360 You can change the timeout period by setting
5361 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink>
5362 in the <filename>local.conf</filename> file.
5363 </para></listitem>
5364 <listitem><para>Once the boot process is reached and the
5365 login prompt appears, the tests run.
5366 The full boot log is written to
5367 <filename>${WORKDIR}/testimage/qemu_boot_log</filename>.
5368 </para></listitem>
5369 <listitem><para>Each test module loads in the order found
5370 in <filename>TEST_SUITES</filename>.
5371 You can find the full output of the commands run over
5372 <filename>ssh</filename> in
5373 <filename>${WORKDIR}/testimgage/ssh_target_log</filename>.
5374 </para></listitem>
5375 <listitem><para>If no failures occur, the task running the
5376 tests ends successfully.
5377 You can find the output from the
5378 <filename>unittest</filename> in the task log at
5379 <filename>${WORKDIR}/temp/log.do_testimage</filename>.
5380 </para></listitem>
5381 </itemizedlist>
5382 </para>
5383
5384 <para>
5385 All test files reside in
5386 <filename>meta/lib/oeqa/runtime</filename> in the
5387 <link linkend='source-directory'>Source Directory</link>.
5388 A test name maps directly to a Python module.
5389 Each test module may contain a number of individual tests.
5390 Tests are usually grouped together by the area
5391 tested (e.g tests for <filename>systemd</filename> reside in
5392 <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
5393 </para>
5394
5395 <para>
5396 You can add tests to any layer provided you place them in the
5397 proper area and you extend
5398 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
5399 in the <filename>local.conf</filename> file as normal.
5400 Be sure that tests reside in
5401 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename>.
5402 <note>
5403 Be sure that module names do not collide with module names
5404 used in the default set of test modules in
5405 <filename>meta/lib/oeqa/runtime</filename>.
5406 </note>
5407 </para>
5408
5409 <para>
5410 You can change the set of tests run by appending or overriding
5411 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>
5412 variable in <filename>local.conf</filename>.
5413 Each name in <filename>TEST_SUITES</filename> represents a
5414 required test for the image.
5415 Test modules named within <filename>TEST_SUITES</filename>
5416 cannot be skipped even if a test is not suitable for an image
5417 (e.g. running the rpm tests on an image without
5418 <filename>rpm</filename>).
5419 Appending "auto" to <filename>TEST_SUITES</filename> causes the
5420 build system to try to run all tests that are suitable for the
5421 image (i.e. each test module may elect to skip itself).
5422 </para>
5423
5424 <para>
5425 The order you list tests in <filename>TEST_SUITES</filename>
5426 is important.
5427 The order influences test dependencies.
5428 Consequently, tests that depend on other tests should be added
5429 after the test on which they depend.
5430 For example, since <filename>ssh</filename> depends on the
5431 <filename>ping</filename> test, <filename>ssh</filename>
5432 needs to come after <filename>ping</filename> in the list.
5433 The test class provides no re-ordering or dependency handling.
5434 <note>
5435 Each module can have multiple classes with multiple test
5436 methods.
5437 And, Python <filename>unittest</filename> rules apply.
5438 </note>
5439 </para>
5440
5441 <para>
5442 Here are some things to keep in mind when running tests:
5443 <itemizedlist>
5444 <listitem><para>The default tests for the image are defined
5445 as:
5446 <literallayout class='monospaced'>
5447 DEFAULT_TEST_SUITES_pn-&lt;image&gt; = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
5448 </literallayout></para></listitem>
5449 <listitem><para>Add your own test to the list of the
5450 by using the following:
5451 <literallayout class='monospaced'>
5452 TEST_SUITES_append = " mytest"
5453 </literallayout></para></listitem>
5454 <listitem><para>Run a specific list of tests as follows:
5455 <literallayout class='monospaced'>
5456 TEST_SUITES = "test1 test2 test3"
5457 </literallayout>
5458 Remember, order is important.
5459 Be sure to place a test that is dependent on another test
5460 later in the order.</para></listitem>
5461 </itemizedlist>
5462 </para>
5463 </section>
5464
5465 <section id="qemu-image-writing-new-tests">
5466 <title>Writing New Tests</title>
5467
5468 <para>
5469 As mentioned previously, all new test files need to be in the
5470 proper place for the build system to find them.
5471 New tests for additional functionality outside of the core
5472 should be added to the layer that adds the functionality, in
5473 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename> (as
5474 long as
5475 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
5476 is extended in the layer's
5477 <filename>layer.conf</filename> file as normal).
5478 Just remember that filenames need to map directly to test
5479 (module) names and that you do not use module names that
5480 collide with existing core tests.
5481 </para>
5482
5483 <para>
5484 To create a new test, start by copying an existing module
5485 (e.g. <filename>syslog.py</filename> or
5486 <filename>gcc.py</filename> are good ones to use).
5487 Test modules can use code from
5488 <filename>meta/lib/oeqa/utils</filename>, which are helper
5489 classes.
5490 </para>
5491
5492 <note>
5493 Structure shell commands such that you rely on them and they
5494 return a single code for success.
5495 Be aware that sometimes you will need to parse the output.
5496 See the <filename>df.py</filename> and
5497 <filename>date.py</filename> modules for examples.
5498 </note>
5499
5500 <para>
5501 You will notice that all test classes inherit
5502 <filename>oeRuntimeTest</filename>, which is found in
5503 <filename>meta/lib/oetest.py</filename>.
5504 This base class offers some helper attributes, which are
5505 described in the following sections:
5506 </para>
5507
5508 <section id='qemu-image-writing-tests-class-methods'>
5509 <title>Class Methods</title>
5510
5511 <para>
5512 Class methods are as follows:
5513 <itemizedlist>
5514 <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis>
5515 Returns "True" if <filename>pkg</filename> is in the
5516 installed package list of the image, which is based
5517 on
5518 <filename>${WORKDIR}/installed_pkgs.txt</filename>
5519 that is generated during the
5520 <filename>do.rootfs</filename> task.
5521 </para></listitem>
5522 <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis>
5523 Returns "True" if the feature is in
5524 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
5525 or
5526 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>.
5527 </para></listitem>
5528 <listitem><para><emphasis><filename>restartTarget(params)</filename>:</emphasis>
5529 Restarts the QEMU image optionally passing
5530 <filename>params</filename> to the
5531 <filename>runqemu</filename> script's
5532 <filename>qemuparams</filename> list (e.g "-m 1024" for
5533 more memory).</para></listitem>
5534 </itemizedlist>
5535 </para>
5536 </section>
5537
5538 <section id='qemu-image-writing-tests-class-attributes'>
5539 <title>Class Attributes</title>
5540
5541 <para>
5542 Class attributes are as follows:
5543 <itemizedlist>
5544 <listitem><para><emphasis><filename>pscmd</filename>:</emphasis>
5545 Equals "ps -ef" if <filename>procps</filename> is
5546 installed in the image.
5547 Otherwise, <filename>pscmd</filename> equals
5548 "ps" (busybox).
5549 </para></listitem>
5550 <listitem><para><emphasis><filename>tc</filename>:</emphasis>
5551 The called text context, which gives access to the
5552 following attributes:
5553 <itemizedlist>
5554 <listitem><para><emphasis><filename>d</filename>:</emphasis>
5555 The BitBake data store, which allows you to
5556 use stuff such as
5557 <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>.
5558 </para></listitem>
5559 <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis>
5560 Used internally.
5561 The tests do not need these.
5562 </para></listitem>
5563 <listitem><para><emphasis><filename>filesdir</filename>:</emphasis>
5564 The absolute path to
5565 <filename>meta/lib/oeqa/runtime/files</filename>,
5566 which contains helper files for tests meant
5567 for copying on the target such as small
5568 files written in C for compilation.
5569 </para></listitem>
5570 <listitem><para><emphasis><filename>qemu</filename>:</emphasis>
5571 Provides access to the
5572 <filename>QemuRunner</filename> object,
5573 which is the class that boots the image.
5574 The <filename>qemu</filename> attribute
5575 provides the following useful attributes:
5576 <itemizedlist>
5577 <listitem><para><emphasis><filename>ip</filename>:</emphasis>
5578 The machine's IP address.
5579 </para></listitem>
5580 <listitem><para><emphasis><filename>host_ip</filename>:</emphasis>
5581 The host IP address, which is only
5582 used by smart tests.
5583 </para></listitem>
5584 </itemizedlist></para></listitem>
5585 <listitem><para><emphasis><filename>target</filename>:</emphasis>
5586 The <filename>SSHControl</filename> object,
5587 which is used for running the following
5588 commands on the image:
5589 <itemizedlist>
5590 <listitem><para><emphasis><filename>host</filename>:</emphasis>
5591 Used internally.
5592 The tests do not use this command.
5593 </para></listitem>
5594 <listitem><para><emphasis><filename>timeout</filename>:</emphasis>
5595 A global timeout for commands run on
5596 the target for the instance of a
5597 test.
5598 The default is 300 seconds.
5599 </para></listitem>
5600 <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis>
5601 The single, most used method.
5602 This command is a wrapper for:
5603 <filename>ssh root@host "cmd"</filename>.
5604 The command returns a tuple:
5605 (status, output), which are what
5606 their names imply - the return code
5607 of 'cmd' and whatever output
5608 it produces.
5609 The optional timeout argument
5610 represents the number of seconds the
5611 test should wait for 'cmd' to
5612 return.
5613 If the argument is "None", the
5614 test uses the default instance's
5615 timeout period, which is 300
5616 seconds.
5617 If the argument is "0", the test
5618 runs until the command returns.
5619 </para></listitem>
5620 <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis>
5621 <filename>scp localpath root@ip:remotepath</filename>.
5622 </para></listitem>
5623 <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis>
5624 <filename>scp root@host:remotepath localpath</filename>.
5625 </para></listitem>
5626 </itemizedlist></para></listitem>
5627 </itemizedlist></para></listitem>
5628 </itemizedlist>
5629 </para>
5630 </section>
5631
5632 <section id='qemu-image-writing-tests-instance-attributes'>
5633 <title>Instance Attributes</title>
5634
5635 <para>
5636 A single instance attribute exists, which is
5637 <filename>target</filename>.
5638 The <filename>target</filename> instance attribute is
5639 identical to the class attribute of the same name, which
5640 is described in the previous section.
5641 This attribute exists as both an instance and class
5642 attribute so tests can use
5643 <filename>self.target.run(cmd)</filename> in instance
5644 methods instead of
5645 <filename>oeRuntimeTest.tc.target.run(cmd)</filename>.
5646 </para>
5647 </section>
5648 </section>
5649 </section>
5650
5651 <section id="platdev-gdb-remotedebug">
5652 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title>
5653
5654 <para>
5655 GDB allows you to examine running programs, which in turn helps you to understand and fix problems.
5656 It also allows you to perform post-mortem style analysis of program crashes.
5657 GDB is available as a package within the Yocto Project and is
5658 installed in SDK images by default.
5659 See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter
5660 in the Yocto Project Reference Manual for a description of these images.
5661 You can find information on GDB at <ulink url="http://sourceware.org/gdb/"/>.
5662 </para>
5663
5664 <tip>
5665 For best results, install <filename>-dbg</filename> packages for
5666 the applications you are going to debug.
5667 Doing so makes extra debug symbols available that give you more
5668 meaningful output.
5669 </tip>
5670
5671 <para>
5672 Sometimes, due to memory or disk space constraints, it is not possible
5673 to use GDB directly on the remote target to debug applications.
5674 These constraints arise because GDB needs to load the debugging information and the
5675 binaries of the process being debugged.
5676 Additionally, GDB needs to perform many computations to locate information such as function
5677 names, variable names and values, stack traces and so forth - even before starting the
5678 debugging process.
5679 These extra computations place more load on the target system and can alter the
5680 characteristics of the program being debugged.
5681 </para>
5682
5683 <para>
5684 To help get past the previously mentioned constraints, you can use Gdbserver.
5685 Gdbserver runs on the remote target and does not load any debugging information
5686 from the debugged process.
5687 Instead, a GDB instance processes the debugging information that is run on a
5688 remote computer - the host GDB.
5689 The host GDB then sends control commands to Gdbserver to make it stop or start the debugged
5690 program, as well as read or write memory regions of that debugged program.
5691 All the debugging information loaded and processed as well
5692 as all the heavy debugging is done by the host GDB.
5693 Offloading these processes gives the Gdbserver running on the target a chance to remain
5694 small and fast.
5695 </para>
5696
5697 <para>
5698 Because the host GDB is responsible for loading the debugging information and
5699 for doing the necessary processing to make actual debugging happen, the
5700 user has to make sure the host can access the unstripped binaries complete
5701 with their debugging information and also be sure the target is compiled with no optimizations.
5702 The host GDB must also have local access to all the libraries used by the
5703 debugged program.
5704 Because Gdbserver does not need any local debugging information, the binaries on
5705 the remote target can remain stripped.
5706 However, the binaries must also be compiled without optimization
5707 so they match the host's binaries.
5708 </para>
5709
5710 <para>
5711 To remain consistent with GDB documentation and terminology, the binary being debugged
5712 on the remote target machine is referred to as the "inferior" binary.
5713 For documentation on GDB see the
5714 <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>.
5715 </para>
5716
5717 <para>
5718 The remainder of this section describes the steps you need to take
5719 to debug using the GNU project debugger.
5720 </para>
5721
5722 <section id='platdev-gdb-remotedebug-setup'>
5723 <title>Set Up the Cross-Development Debugging Environment</title>
5724
5725 <para>
5726 Before you can initiate a remote debugging session, you need
5727 to be sure you have set up the cross-development environment,
5728 toolchain, and sysroot.
5729 The "<ulink url='&YOCTO_DOCS_ADT_URL;#adt-prepare'>Preparing for Application Development</ulink>"
5730 chapter of the Yocto Project Application Developer's Guide
5731 describes this process.
5732 Be sure you have read that chapter and have set up
5733 your environment.
5734 </para>
5735 </section>
5736
5737 <section id="platdev-gdb-remotedebug-launch-gdbserver">
5738 <title>Launch Gdbserver on the Target</title>
5739
5740 <para>
5741 Make sure Gdbserver is installed on the target.
5742 If it is not, install the package
5743 <filename>gdbserver</filename>, which needs the
5744 <filename>libthread-db1</filename> package.
5745 </para>
5746
5747 <para>
5748 Here is an example that when entered from the host
5749 connects to the target and launches Gdbserver in order to
5750 "debug" a binary named <filename>helloworld</filename>:
5751 <literallayout class='monospaced'>
5752 $ gdbserver localhost:2345 /usr/bin/helloworld
5753 </literallayout>
5754 Gdbserver should now be listening on port 2345 for debugging
5755 commands coming from a remote GDB process that is running on
5756 the host computer.
5757 Communication between Gdbserver and the host GDB are done
5758 using TCP.
5759 To use other communication protocols, please refer to the
5760 <ulink url='http://www.gnu.org/software/gdb/'>Gdbserver documentation</ulink>.
5761 </para>
5762 </section>
5763
5764 <section id="platdev-gdb-remotedebug-launch-gdb">
5765 <title>Launch GDB on the Host Computer</title>
5766
5767 <para>
5768 Running GDB on the host computer takes a number of stages, which
5769 this section describes.
5770 </para>
5771
5772 <section id="platdev-gdb-remotedebug-launch-gdb-buildcross">
5773 <title>Build the Cross-GDB Package</title>
5774 <para>
5775 A suitable GDB cross-binary is required that runs on your
5776 host computer but also knows about the the ABI of the
5777 remote target.
5778 You can get this binary from the
5779 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
5780 Here is an example where the toolchain has been installed
5781 in the default directory
5782 <filename>/opt/poky/&DISTRO;</filename>:
5783 <literallayout class='monospaced'>
5784 /opt/poky/1.4/sysroots/i686-pokysdk-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi/arm-poky-linux-gnueabi-gdb
5785 </literallayout>
5786 where <filename>arm</filename> is the target architecture
5787 and <filename>linux-gnueabi</filename> is the target ABI.
5788 </para>
5789
5790 <para>
5791 Alternatively, you can use BitBake to build the
5792 <filename>gdb-cross</filename> binary.
5793 Here is an example:
5794 <literallayout class='monospaced'>
5795 $ bitbake gdb-cross
5796 </literallayout>
5797 Once the binary is built, you can find it here:
5798 <literallayout class='monospaced'>
5799 tmp/sysroots/&lt;host-arch&gt;/usr/bin/&lt;target-platform&gt;/&lt;target-abi&gt;-gdb
5800 </literallayout>
5801 </para>
5802 </section>
5803
5804 <section id='create-the-gdb-initialization-file'>
5805 <title>Create the GDB Initialization File and Point to Your Root Filesystem</title>
5806
5807 <para>
5808 Aside from the GDB cross-binary, you also need a GDB
5809 initialization file in the same top directory in which
5810 your binary resides.
5811 When you start GDB on your host development system, GDB
5812 finds this initialization file and executes all the
5813 commands within.
5814 For information on the <filename>.gdbinit</filename>, see
5815 "<ulink url='http://sourceware.org/gdb/onlinedocs/gdb/'>Debugging with GDB</ulink>",
5816 which is maintained by
5817 <ulink url='http://www.sourceware.org'>sourceware.org</ulink>.
5818 </para>
5819
5820 <para>
5821 You need to add a statement in the
5822 <filename>.gdbinit</filename> file that points to your
5823 root filesystem.
5824 Here is an example that points to the root filesystem for
5825 an ARM-based target device:
5826 <literallayout class='monospaced'>
5827 set sysroot /home/jzhang/sysroot_arm
5828 </literallayout>
5829 </para>
5830 </section>
5831
5832 <section id="platdev-gdb-remotedebug-launch-gdb-launchhost">
5833 <title>Launch the Host GDB</title>
5834
5835 <para>
5836 Before launching the host GDB, you need to be sure
5837 you have sourced the cross-debugging environment script,
5838 which if you installed the root filesystem in the default
5839 location is at <filename>/opt/poky/&DISTRO;</filename>
5840 and begins with the string "environment-setup".
5841 For more information, see the
5842 "<ulink url='&YOCTO_DOCS_ADT_URL;#setting-up-the-cross-development-environment'>Setting Up the Cross-Development Environment</ulink>"
5843 section in the Yocto Project Application Developer's
5844 Guide.
5845 </para>
5846
5847 <para>
5848 Finally, switch to the directory where the binary resides
5849 and run the <filename>cross-gdb</filename> binary.
5850 Provide the binary file you are going to debug.
5851 For example, the following command continues with the
5852 example used in the previous section by loading
5853 the <filename>helloworld</filename> binary as well as the
5854 debugging information:
5855 <literallayout class='monospaced'>
5856 $ arm-poky-linux-gnuabi-gdb helloworld
5857 </literallayout>
5858 The commands in your <filename>.gdbinit</filename> execute
5859 and the GDB prompt appears.
5860 </para>
5861 </section>
5862 </section>
5863
5864 <section id='platdev-gdb-connect-to-the-remote-gdb-server'>
5865 <title>Connect to the Remote GDB Server</title>
5866
5867 <para>
5868 From the target, you need to connect to the remote GDB
5869 server that is running on the host.
5870 You need to specify the remote host and port.
5871 Here is the command continuing with the example:
5872 <literallayout class='monospaced'>
5873 target remote 192.168.7.2:2345
5874 </literallayout>
5875 </para>
5876 </section>
5877
5878 <section id="platdev-gdb-remotedebug-launch-gdb-using">
5879 <title>Use the Debugger</title>
5880
5881 <para>
5882 You can now proceed with debugging as normal - as if you were debugging
5883 on the local machine.
5884 For example, to instruct GDB to break in the "main" function and then
5885 continue with execution of the inferior binary use the following commands
5886 from within GDB:
5887 <literallayout class='monospaced'>
5888 (gdb) break main
5889 (gdb) continue
5890 </literallayout>
5891 </para>
5892
5893 <para>
5894 For more information about using GDB, see the project's online documentation at
5895 <ulink url="http://sourceware.org/gdb/download/onlinedocs/"/>.
5896 </para>
5897 </section>
5898 </section>
5899
5900 <section id="examining-builds-using-toaster">
5901 <title>Examining Builds Using the Toaster API</title>
5902
5903 <para>
5904 Toaster is an Application Programming Interface (API) and
5905 web-based interface to the OpenEmbedded build system, which uses
5906 BitBake.
5907 Both interfaces are based on a Representational State Transfer
5908 (REST) API that queries for and returns build information using
5909 <filename>GET</filename> and <filename>JSON</filename>.
5910 These types of search operations retrieve sets of objects from
5911 a data store used to collect build information.
5912 The results contain all the data for the objects being returned.
5913 You can order the results of the search by key and the search
5914 parameters are consistent for all object types.
5915 </para>
5916
5917 <para>
5918 Using the interfaces you can do the following:
5919 <itemizedlist>
5920 <listitem><para>See information about the tasks executed
5921 and reused during the build.</para></listitem>
5922 <listitem><para>See what is built (recipes and
5923 packages) and what packages were installed into the final
5924 image.</para></listitem>
5925 <listitem><para>See performance-related information such
5926 as build time, CPU usage, and disk I/O.</para></listitem>
5927 <listitem><para>Examine error, warning and trace messages
5928 to aid in debugging.</para></listitem>
5929 </itemizedlist>
5930 </para>
5931
5932 <note>
5933 <para>This release of Toaster provides you with information
5934 about a BitBake run.
5935 The tool does not allow you to configure and launch a build.
5936 However, future development includes plans to integrate the
5937 configuration and build launching capabilities of
5938 <ulink url='&YOCTO_HOME_URL;/tools-resources/projects/hob'>Hob</ulink>.
5939 </para>
5940 <para>For more information on using Hob to build an image,
5941 see the
5942 "<link linkend='image-development-using-hob'>Image Development Using Hob</link>"
5943 section.</para>
5944 </note>
5945
5946 <para>
5947 The remainder of this section describes what you need to have in
5948 place to use Toaster, how to start it, use it, and stop it.
5949 For additional information on installing and running Toaster, see the
5950 "<ulink url='https://wiki.yoctoproject.org/wiki/Toaster#Installation_and_Running'>Installation and Running</ulink>"
5951 section of the "Toaster" wiki page.
5952 For complete information on the API and its search operation
5953 URI, parameters, and responses, see the
5954 <ulink url='https://wiki.yoctoproject.org/wiki/REST_API_Contracts'>REST API Contracts</ulink>
5955 Wiki page.
5956 </para>
5957
5958 <section id='starting-toaster'>
5959 <title>Starting Toaster</title>
5960
5961 <para>
5962 Getting set up to use and start Toaster is simple.
5963 First, be sure you have met the following requirements:
5964 <itemizedlist>
5965 <listitem><para>You have set up your
5966 <link linkend='source-directory'>Source Directory</link>
5967 by cloning the upstream <filename>poky</filename>
5968 repository.
5969 See the
5970 <link linkend='local-yp-release'>Yocto Project Release</link>
5971 item for information on how to set up the Source
5972 Directory.</para></listitem>
5973 <listitem><para>You have checked out the
5974 <filename>dora-toaster</filename> branch:
5975 <literallayout class='monospaced'>
5976 $ cd poky
5977 $ git checkout -b dora-toaster origin/dora-toaster
5978 </literallayout></para></listitem>
5979 <listitem><para>Be sure your build machine has
5980 <ulink url='http://en.wikipedia.org/wiki/Django_%28web_framework%29'>Django</ulink>
5981 version 1.4.5 installed.</para></listitem>
5982 <listitem><para>Make sure that port 8000 and 8200 are
5983 free (i.e. they have no servers on them).
5984 </para></listitem>
5985 </itemizedlist>
5986 </para>
5987
5988 <para>
5989 Once you have met the requirements, follow these steps to
5990 start Toaster running in the background of your shell:
5991 <orderedlist>
5992 <listitem><para><emphasis>Set up your build environment:</emphasis>
5993 Source a build environment script (i.e.
5994 <filename>oe-init-build-env</filename> or
5995 <filename>oe-init-build-env-memres</filename>).
5996 </para></listitem>
5997 <listitem><para><emphasis>Prepare your local configuration file:</emphasis>
5998 Toaster needs the Toaster class enabled
5999 in Bitbake in order to record target image package
6000 information.
6001 You can enable it by adding the following line to your
6002 <filename>conf/local.conf</filename> file:
6003 <literallayout class='monospaced'>
6004 INHERIT += "toaster"
6005 </literallayout>
6006 Toaster also needs Build History enabled in Bitbake in
6007 order to record target image package information.
6008 You can enable this by adding the following two lines
6009 to your <filename>conf/local.conf</filename> file:
6010 <literallayout class='monospaced'>
6011 INHERIT += "buildhistory"
6012 BUILDHISTORY_COMMIT = "1"
6013 </literallayout></para></listitem>
6014 <listitem><para><emphasis>Start Toaster:</emphasis>
6015 Start the Toaster service using this
6016 command from within your build directory:
6017 <literallayout class='monospaced'>
6018 $ source toaster start
6019 </literallayout></para></listitem>
6020 <note>
6021 The Toaster must be started and running in order
6022 for it to collect data.
6023 </note>
6024 </orderedlist>
6025 </para>
6026
6027 <para>
6028 When Toaster starts, it creates some additional files in your
6029 Build Directory.
6030 Deleting these files will cause you to lose data or interrupt
6031 Toaster:
6032 <itemizedlist>
6033 <listitem><para><emphasis><filename>toaster.sqlite</filename>:</emphasis>
6034 Toaster's database file.</para></listitem>
6035 <listitem><para><emphasis><filename>toaster_web.log</filename>:</emphasis>
6036 The log file of the web server.</para></listitem>
6037 <listitem><para><emphasis><filename>toaster_ui.log</filename>:</emphasis>
6038 The log file of the user interface component.
6039 </para></listitem>
6040 <listitem><para><emphasis><filename>toastermain.pid</filename>:</emphasis>
6041 The PID of the web server.</para></listitem>
6042 <listitem><para><emphasis><filename>toasterui.pid</filename>:</emphasis>
6043 The PID of the DSI data bridge.</para></listitem>
6044 <listitem><para><emphasis><filename>bitbake-cookerdaemon.log</filename>:</emphasis>
6045 The BitBake server's log file.</para></listitem>
6046 </itemizedlist>
6047 </para>
6048 </section>
6049
6050 <section id='using-toaster'>
6051 <title>Using Toaster</title>
6052
6053 <para>
6054 Once Toaster is running, it logs information for any BitBake
6055 run from your Build Directory.
6056 This logging is automatic.
6057 All you need to do is access and use the information.
6058 </para>
6059
6060 <para>
6061 You access the information one of two ways:
6062 <itemizedlist>
6063 <listitem><para>Open a Browser and type enter in the
6064 <filename>http://localhost:8000</filename> URL.
6065 </para></listitem>
6066 <listitem><para>Use the <filename>xdg-open</filename>
6067 tool from the shell and pass it the same URL.
6068 </para></listitem>
6069 </itemizedlist>
6070 Either method opens the home page for the Toaster interface,
6071 which is temporary for this release.
6072 </para>
6073 </section>
6074
6075 <section id='examining-toaster-data'>
6076 <title>Examining Toaster Data</title>
6077
6078 <para>
6079 The Toaster database is persistent regardless of whether you
6080 start or stop the service.
6081 </para>
6082
6083 <para>
6084 Toaster's interface shows you a list of builds
6085 (successful and unsuccessful) for which it has data.
6086 You can click on any build to see related information.
6087 This information includes configuration details, information
6088 about tasks, all recipes and packages built and their
6089 dependencies, packages installed in your final image,
6090 execution time, CPU usage and disk I/O per task.
6091 </para>
6092 </section>
6093
6094 <section id='stopping-toaster'>
6095 <title>Stopping Toaster</title>
6096
6097 <para>
6098 Stop the Toaster service with the following command:
6099 <literallayout class='monospaced'>
6100 $ source toaster stop
6101 </literallayout>
6102 The service stops but the Toaster database remains persistent.
6103 </para>
6104 </section>
6105 </section>
6106
6107 <section id="platdev-oprofile">
6108 <title>Profiling with OProfile</title>
6109
6110 <para>
6111 <ulink url="http://oprofile.sourceforge.net/">OProfile</ulink> is a
6112 statistical profiler well suited for finding performance
6113 bottlenecks in both user-space software and in the kernel.
6114 This profiler provides answers to questions like "Which functions does my application spend
6115 the most time in when doing X?"
6116 Because the OpenEmbedded build system is well integrated with OProfile, it makes profiling
6117 applications on target hardware straightforward.
6118 <note>
6119 For more information on how to set up and run OProfile, see the
6120 "<ulink url='&YOCTO_DOCS_PROF_URL;#profile-manual-oprofile'>OProfile</ulink>"
6121 section in the Yocto Project Profiling and Tracing Manual.
6122 </note>
6123 </para>
6124
6125 <para>
6126 To use OProfile, you need an image that has OProfile installed.
6127 The easiest way to do this is with <filename>tools-profile</filename> in the
6128 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'>IMAGE_FEATURES</ulink></filename> variable.
6129 You also need debugging symbols to be available on the system where the analysis
6130 takes place.
6131 You can gain access to the symbols by using <filename>dbg-pkgs</filename> in the
6132 <filename>IMAGE_FEATURES</filename> variable or by
6133 installing the appropriate <filename>-dbg</filename> packages.
6134 </para>
6135
6136 <para>
6137 For successful call graph analysis, the binaries must preserve the frame
6138 pointer register and should also be compiled with the
6139 <filename>-fno-omit-framepointer</filename> flag.
6140 You can achieve this by setting the
6141 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SELECTED_OPTIMIZATION'>SELECTED_OPTIMIZATION</ulink></filename>
6142 variable with the following options:
6143 <literallayout class='monospaced'>
6144 -fexpensive-optimizations
6145 -fno-omit-framepointer
6146 -frename-registers
6147 -O2
6148 </literallayout>
6149 You can also achieve it by setting the
6150 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_BUILD'>DEBUG_BUILD</ulink></filename>
6151 variable to "1" in the <filename>local.conf</filename> configuration file.
6152 If you use the <filename>DEBUG_BUILD</filename> variable,
6153 you also add extra debugging information that can make the debug
6154 packages large.
6155 </para>
6156
6157 <section id="platdev-oprofile-target">
6158 <title>Profiling on the Target</title>
6159
6160 <para>
6161 Using OProfile you can perform all the profiling work on the target device.
6162 A simple OProfile session might look like the following:
6163 </para>
6164
6165 <para>
6166 <literallayout class='monospaced'>
6167 # opcontrol --reset
6168 # opcontrol --start --separate=lib --no-vmlinux -c 5
6169 .
6170 .
6171 [do whatever is being profiled]
6172 .
6173 .
6174 # opcontrol --stop
6175 $ opreport -cl
6176 </literallayout>
6177 </para>
6178
6179 <para>
6180 In this example, the <filename>reset</filename> command clears any previously profiled data.
6181 The next command starts OProfile.
6182 The options used when starting the profiler separate dynamic library data
6183 within applications, disable kernel profiling, and enable callgraphing up to
6184 five levels deep.
6185 <note>
6186 To profile the kernel, you would specify the
6187 <filename>--vmlinux=/path/to/vmlinux</filename> option.
6188 The <filename>vmlinux</filename> file is usually in the source directory in the
6189 <filename>/boot/</filename> directory and must match the running kernel.
6190 </note>
6191 </para>
6192
6193 <para>
6194 After you perform your profiling tasks, the next command stops the profiler.
6195 After that, you can view results with the <filename>opreport</filename> command with options
6196 to see the separate library symbols and callgraph information.
6197 </para>
6198
6199 <para>
6200 Callgraphing logs information about time spent in functions and about a function's
6201 calling function (parent) and called functions (children).
6202 The higher the callgraphing depth, the more accurate the results.
6203 However, higher depths also increase the logging overhead.
6204 Consequently, you should take care when setting the callgraphing depth.
6205 <note>
6206 On ARM, binaries need to have the frame pointer enabled for callgraphing to work.
6207 To accomplish this use the <filename>-fno-omit-framepointer</filename> option
6208 with <filename>gcc</filename>.
6209 </note>
6210 </para>
6211
6212 <para>
6213 For more information on using OProfile, see the OProfile
6214 online documentation at
6215 <ulink url="http://oprofile.sourceforge.net/docs/"/>.
6216 </para>
6217 </section>
6218
6219 <section id="platdev-oprofile-oprofileui">
6220 <title>Using OProfileUI</title>
6221
6222 <para>
6223 A graphical user interface for OProfile is also available.
6224 You can download and build this interface from the Yocto Project at
6225 <ulink url="&YOCTO_GIT_URL;/cgit.cgi/oprofileui/"></ulink>.
6226 If the "tools-profile" image feature is selected, all necessary binaries
6227 are installed onto the target device for OProfileUI interaction.
6228 For a list of image features that ship with the Yocto Project,
6229 see the
6230 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-features-image'>Images</ulink>"
6231 section in the Yocto Project Reference Manual.
6232 </para>
6233
6234 <para>
6235 Even though the source directory usually includes all needed patches on the target device, you
6236 might find you need other OProfile patches for recent OProfileUI features.
6237 If so, see the <ulink url='&YOCTO_GIT_URL;/cgit.cgi/oprofileui/tree/README'>
6238 OProfileUI README</ulink> for the most recent information.
6239 </para>
6240
6241 <section id="platdev-oprofile-oprofileui-online">
6242 <title>Online Mode</title>
6243
6244 <para>
6245 Using OProfile in online mode assumes a working network connection with the target
6246 hardware.
6247 With this connection, you just need to run "oprofile-server" on the device.
6248 By default, OProfile listens on port 4224.
6249 <note>
6250 You can change the port using the <filename>--port</filename> command-line
6251 option.
6252 </note>
6253 </para>
6254
6255 <para>
6256 The client program is called <filename>oprofile-viewer</filename> and its UI is relatively
6257 straightforward.
6258 You access key functionality through the buttons on the toolbar, which
6259 are duplicated in the menus.
6260 Here are the buttons:
6261 <itemizedlist>
6262 <listitem><para><emphasis>Connect:</emphasis> Connects to the remote host.
6263 You can also supply the IP address or hostname.</para></listitem>
6264 <listitem><para><emphasis>Disconnect:</emphasis> Disconnects from the target.
6265 </para></listitem>
6266 <listitem><para><emphasis>Start:</emphasis> Starts profiling on the device.
6267 </para></listitem>
6268 <listitem><para><emphasis>Stop:</emphasis> Stops profiling on the device and
6269 downloads the data to the local host.
6270 Stopping the profiler generates the profile and displays it in the viewer.
6271 </para></listitem>
6272 <listitem><para><emphasis>Download:</emphasis> Downloads the data from the
6273 target and generates the profile, which appears in the viewer.</para></listitem>
6274 <listitem><para><emphasis>Reset:</emphasis> Resets the sample data on the device.
6275 Resetting the data removes sample information collected from previous
6276 sampling runs.
6277 Be sure you reset the data if you do not want to include old sample information.
6278 </para></listitem>
6279 <listitem><para><emphasis>Save:</emphasis> Saves the data downloaded from the
6280 target to another directory for later examination.</para></listitem>
6281 <listitem><para><emphasis>Open:</emphasis> Loads previously saved data.
6282 </para></listitem>
6283 </itemizedlist>
6284 </para>
6285
6286 <para>
6287 The client downloads the complete 'profile archive' from
6288 the target to the host for processing.
6289 This archive is a directory that contains the sample data, the object files,
6290 and the debug information for the object files.
6291 The archive is then converted using the <filename>oparchconv</filename> script, which is
6292 included in this distribution.
6293 The script uses <filename>opimport</filename> to convert the archive from
6294 the target to something that can be processed on the host.
6295 </para>
6296
6297 <para>
6298 Downloaded archives reside in the
6299 <link linkend='build-directory'>Build Directory</link> in
6300 <filename>tmp</filename> and are cleared up when they are no longer in use.
6301 </para>
6302
6303 <para>
6304 If you wish to perform kernel profiling, you need to be sure
6305 a <filename>vmlinux</filename> file that matches the running kernel is available.
6306 In the source directory, that file is usually located in
6307 <filename>/boot/vmlinux-KERNELVERSION</filename>, where
6308 <filename>KERNEL-version</filename> is the version of the kernel.
6309 The OpenEmbedded build system generates separate <filename>vmlinux</filename>
6310 packages for each kernel it builds.
6311 Thus, it should just be a question of making sure a matching package is
6312 installed (e.g. <filename>opkg install kernel-vmlinux</filename>).
6313 The files are automatically installed into development and profiling images
6314 alongside OProfile.
6315 A configuration option exists within the OProfileUI settings page that you can use to
6316 enter the location of the <filename>vmlinux</filename> file.
6317 </para>
6318
6319 <para>
6320 Waiting for debug symbols to transfer from the device can be slow, and it
6321 is not always necessary to actually have them on the device for OProfile use.
6322 All that is needed is a copy of the filesystem with the debug symbols present
6323 on the viewer system.
6324 The "<link linkend='platdev-gdb-remotedebug-launch-gdb'>Launch GDB on the Host Computer</link>"
6325 section covers how to create such a directory with
6326 the <link linkend='source-directory'>Source Directory</link>
6327 and how to use the OProfileUI Settings Dialog to specify the location.
6328 If you specify the directory, it will be used when the file checksums
6329 match those on the system you are profiling.
6330 </para>
6331 </section>
6332
6333 <section id="platdev-oprofile-oprofileui-offline">
6334 <title>Offline Mode</title>
6335
6336 <para>
6337 If network access to the target is unavailable, you can generate
6338 an archive for processing in <filename>oprofile-viewer</filename> as follows:
6339 <literallayout class='monospaced'>
6340 # opcontrol --reset
6341 # opcontrol --start --separate=lib --no-vmlinux -c 5
6342 .
6343 .
6344 [do whatever is being profiled]
6345 .
6346 .
6347 # opcontrol --stop
6348 # oparchive -o my_archive
6349 </literallayout>
6350 </para>
6351
6352 <para>
6353 In the above example, <filename>my_archive</filename> is the name of the
6354 archive directory where you would like the profile archive to be kept.
6355 After the directory is created, you can copy it to another host and load it
6356 using <filename>oprofile-viewer</filename> open functionality.
6357 If necessary, the archive is converted.
6358 </para>
6359 </section>
6360 </section>
6361 </section>
6362
6363 <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'>
6364 <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title>
6365
6366 <para>
6367 One of the concerns for a development organization using open source
6368 software is how to maintain compliance with various open source
6369 licensing during the lifecycle of the product.
6370 While this section does not provide legal advice or
6371 comprehensively cover all scenarios, it does
6372 present methods that you can use to
6373 assist you in meeting the compliance requirements during a software
6374 release.
6375 </para>
6376
6377 <para>
6378 With hundreds of different open source licenses that the Yocto
6379 Project tracks, it is difficult to know the requirements of each
6380 and every license.
6381 However, we can begin to cover the requirements of the major FLOSS licenses, by
6382 assuming that there are three main areas of concern:
6383 <itemizedlist>
6384 <listitem><para>Source code must be provided.</para></listitem>
6385 <listitem><para>License text for the software must be
6386 provided.</para></listitem>
6387 <listitem><para>Compilation scripts and modifications to the
6388 source code must be provided.
6389 </para></listitem>
6390 </itemizedlist>
6391 There are other requirements beyond the scope of these
6392 three and the methods described in this section
6393 (e.g. the mechanism through which source code is distributed).
6394 </para>
6395
6396 <para>
6397 As different organizations have different methods of complying with
6398 open source licensing, this section is not meant to imply that
6399 there is only one single way to meet your compliance obligations,
6400 but rather to describe one method of achieving compliance.
6401 The remainder of this section describes methods supported to meet the
6402 previously mentioned three requirements.
6403 Once you take steps to meet these requirements,
6404 and prior to releasing images, sources, and the build system,
6405 you should audit all artifacts to ensure completeness.
6406 <note>
6407 The Yocto Project generates a license manifest during
6408 image creation that is located
6409 in <filename>${DEPLOY_DIR}/licenses/&lt;image_name-datestamp&gt;</filename>
6410 to assist with any audits.
6411 </note>
6412 </para>
6413
6414 <section id='providing-the-source-code'>
6415 <title>Providing the Source Code</title>
6416
6417 <para>
6418 Compliance activities should begin before you generate the
6419 final image.
6420 The first thing you should look at is the requirement that
6421 tops the list for most compliance groups - providing
6422 the source.
6423 The Yocto Project has a few ways of meeting this
6424 requirement.
6425 </para>
6426
6427 <para>
6428 One of the easiest ways to meet this requirement is
6429 to provide the entire
6430 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
6431 used by the build.
6432 This method, however, has a few issues.
6433 The most obvious is the size of the directory since it includes
6434 all sources used in the build and not just the source used in
6435 the released image.
6436 It will include toolchain source, and other artifacts, which
6437 you would not generally release.
6438 However, the more serious issue for most companies is accidental
6439 release of proprietary software.
6440 The Yocto Project provides an archiver class to help avoid
6441 some of these concerns.
6442 See the
6443 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'>Archiving Sources - <filename>archive*.bbclass</filename></ulink>"
6444 section in the Yocto Project Reference Manual for information
6445 on this class.
6446 </para>
6447
6448 <para>
6449 Before you employ <filename>DL_DIR</filename> or the
6450 archiver class, you need to decide how you choose to
6451 provide source.
6452 The source archiver class can generate tarballs and SRPMs
6453 and can create them with various levels of compliance in mind.
6454 One way of doing this (but certainly not the only way) is to
6455 release just the original source as a tarball.
6456 You can do this by adding the following to the
6457 <filename>local.conf</filename> file found in the
6458 <link linkend='build-directory'>Build Directory</link>:
6459 <literallayout class='monospaced'>
6460 ARCHIVER_MODE ?= "original"
6461 ARCHIVER_CLASS = "${@'archive-${ARCHIVER_MODE}-source' if
6462 ARCHIVER_MODE != 'none' else ''}"
6463 INHERIT += "${ARCHIVER_CLASS}"
6464 SOURCE_ARCHIVE_PACKAGE_TYPE = "tar"
6465 </literallayout>
6466 During the creation of your image, the source from all
6467 recipes that deploy packages to the image is placed within
6468 subdirectories of
6469 <filename>DEPLOY_DIR/sources</filename> based on the
6470 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
6471 for each recipe.
6472 Releasing the entire directory enables you to comply with
6473 requirements concerning providing the unmodified source.
6474 It is important to note that the size of the directory can
6475 get large.
6476 </para>
6477
6478 <para>
6479 A way to help mitigate the size issue is to only release
6480 tarballs for licenses that require the release of
6481 source.
6482 Let's assume you are only concerned with GPL code as
6483 identified with the following:
6484 <literallayout class='monospaced'>
6485 $ cd poky/build/tmp/deploy/sources
6486 $ mkdir ~/gpl_source_release
6487 $ for dir in */*GPL*; do cp -r $dir ~/gpl_source_release; done
6488 </literallayout>
6489 At this point, you could create a tarball from the
6490 <filename>gpl_source_release</filename> directory and
6491 provide that to the end user.
6492 This method would be a step toward achieving compliance
6493 with section 3a of GPLv2 and with section 6 of GPLv3.
6494 </para>
6495 </section>
6496
6497 <section id='providing-license-text'>
6498 <title>Providing License Text</title>
6499
6500 <para>
6501 One requirement that is often overlooked is inclusion
6502 of license text.
6503 This requirement also needs to be dealt with prior to
6504 generating the final image.
6505 Some licenses require the license text to accompany
6506 the binary.
6507 You can achieve this by adding the following to your
6508 <filename>local.conf</filename> file:
6509 <literallayout class='monospaced'>
6510 COPY_LIC_MANIFEST = "1"
6511 COPY_LIC_DIRS = "1"
6512 </literallayout>
6513 Adding these statements to the configuration file ensures
6514 that the licenses collected during package generation
6515 are included on your image.
6516 As the source archiver has already archived the original
6517 unmodified source that contains the license files,
6518 you would have already met the requirements for inclusion
6519 of the license information with source as defined by the GPL
6520 and other open source licenses.
6521 </para>
6522 </section>
6523
6524 <section id='providing-compilation-scripts-and-source-code-modifications'>
6525 <title>Providing Compilation Scripts and Source Code Modifications</title>
6526
6527 <para>
6528 At this point, we have addressed all we need to address
6529 prior to generating the image.
6530 The next two requirements are addressed during the final
6531 packaging of the release.
6532 </para>
6533
6534 <para>
6535 By releasing the version of the OpenEmbedded build system
6536 and the layers used during the build, you will be providing both
6537 compilation scripts and the source code modifications in one
6538 step.
6539 </para>
6540
6541 <para>
6542 If the deployment team has a
6543 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink>
6544 and a distro layer, and those those layers are used to patch,
6545 compile, package, or modify (in any way) any open source
6546 software included in your released images, you
6547 may be required to to release those layers under section 3 of
6548 GPLv2 or section 1 of GPLv3.
6549 One way of doing that is with a clean
6550 checkout of the version of the Yocto Project and layers used
6551 during your build.
6552 Here is an example:
6553 <literallayout class='monospaced'>
6554 # We built using the &DISTRO_NAME; branch of the poky repo
6555 $ git clone -b &DISTRO_NAME; git://git.yoctoproject.org/poky
6556 $ cd poky
6557 # We built using the release_branch for our layers
6558 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
6559 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
6560 # clean up the .git repos
6561 $ find . -name ".git" -type d -exec rm -rf {} \;
6562 </literallayout>
6563 One thing a development organization might want to consider
6564 for end-user convenience is to modify
6565 <filename>meta-yocto/conf/bblayers.conf.sample</filename> to
6566 ensure that when the end user utilizes the released build
6567 system to build an image, the development organization's
6568 layers are included in the <filename>bblayers.conf</filename>
6569 file automatically:
6570 <literallayout class='monospaced'>
6571 # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
6572 # changes incompatibly
6573 LCONF_VERSION = "6"
6574
6575 BBPATH = "${TOPDIR}"
6576 BBFILES ?= ""
6577
6578 BBLAYERS ?= " \
6579 ##OEROOT##/meta \
6580 ##OEROOT##/meta-yocto \
6581 ##OEROOT##/meta-yocto-bsp \
6582 ##OEROOT##/meta-mylayer \
6583 "
6584
6585 BBLAYERS_NON_REMOVABLE ?= " \
6586 ##OEROOT##/meta \
6587 ##OEROOT##/meta-yocto \
6588 "
6589 </literallayout>
6590 Creating and providing an archive of the
6591 <link linkend='metadata'>Metadata</link> layers
6592 (recipes, configuration files, and so forth)
6593 enables you to meet your
6594 requirements to include the scripts to control compilation
6595 as well as any modifications to the original source.
6596 </para>
6597 </section>
6598 </section>
6599</chapter>
6600
6601<!--
6602vim: expandtab tw=80 ts=4
6603-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-10-31 20:37:27 +0000