summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual/dev-manual-common-tasks.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.xml')
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml5913
1 files changed, 5913 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
new file mode 100644
index 0000000000..e2e89a032c
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -0,0 +1,5913 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='extendpoky'>
6
7<title>Common Tasks</title>
8 <para>
9 This chapter describes fundamental procedures such as creating layers,
10 adding new software packages, extending or customizing images,
11 porting work to new hardware (adding a new machine), and so forth.
12 You will find the procedures documented here occur often in the
13 develop cycle using the Yocto Project.
14 </para>
15
16 <section id="understanding-and-creating-layers">
17 <title>Understanding and Creating Layers</title>
18
19 <para>
20 The OpenEmbedded build system supports organizing
21 <link linkend='metadata'>Metadata</link> into multiple layers.
22 Layers allow you to isolate different types of customizations from
23 each other.
24 You might find it tempting to keep everything in one layer when
25 working on a single project.
26 However, the more modular you organize your Metadata, the easier
27 it is to cope with future changes.
28 </para>
29
30 <para>
31 To illustrate how layers are used to keep things modular, consider
32 machine customizations.
33 These types of customizations typically reside in a special layer,
34 rather than a general layer, called a Board Specific Package (BSP)
35 Layer.
36 Furthermore, the machine customizations should be isolated from
37 recipes and Metadata that support a new GUI environment,
38 for example.
39 This situation gives you a couple of layers: one for the machine
40 configurations, and one for the GUI environment.
41 It is important to understand, however, that the BSP layer can
42 still make machine-specific additions to recipes within the GUI
43 environment layer without polluting the GUI layer itself
44 with those machine-specific changes.
45 You can accomplish this through a recipe that is a BitBake append
46 (<filename>.bbappend</filename>) file, which is described later
47 in this section.
48 </para>
49
50 <para>
51 </para>
52
53 <section id='yocto-project-layers'>
54 <title>Layers</title>
55
56 <para>
57 The <link linkend='source-directory'>Source Directory</link>
58 contains both general layers and BSP
59 layers right out of the box.
60 You can easily identify layers that ship with a
61 Yocto Project release in the Source Directory by their
62 folder names.
63 Folders that are layers begin with the string
64 <filename>meta</filename>.
65 <note>
66 It is not a requirement that a layer begins with the
67 string <filename>meta</filename>.
68 </note>
69 For example, when you set up the Source Directory structure,
70 you will see several layers:
71 <filename>meta</filename>, <filename>meta-hob</filename>,
72 <filename>meta-skeleton</filename>,
73 <filename>meta-yocto</filename>, and
74 <filename>meta-yocto-bsp</filename>.
75 Each of these folders is a layer.
76 </para>
77
78 <para>
79 Furthermore, if you set up a local copy of the
80 <filename>meta-intel</filename> Git repository
81 and then explore the folder of that general layer,
82 you will discover many BSP layers inside.
83 For more information on BSP layers, see the
84 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
85 section in the Yocto Project Board Support Package (BSP)
86 Developer's Guide.
87 </para>
88 </section>
89
90 <section id='creating-your-own-layer'>
91 <title>Creating Your Own Layer</title>
92
93 <para>
94 It is very easy to create your own layers to use with the
95 OpenEmbedded build system.
96 The Yocto Project ships with scripts that speed up creating
97 general layers and BSP layers.
98 This section describes the steps you perform by hand to create
99 a layer so that you can better understand them.
100 For information about the layer-creation scripts, see the
101 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
102 section in the Yocto Project Board Support Package (BSP)
103 Developer's Guide and the
104 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
105 section further down in this manual.
106 </para>
107
108 <para>
109 Follow these general steps to create your layer:
110 <orderedlist>
111 <listitem><para><emphasis>Check Existing Layers:</emphasis>
112 Before creating a new layer, you should be sure someone
113 has not already created a layer containing the Metadata
114 you need.
115 You can see the
116 <ulink url='http://layers.openembedded.org/layerindex/layers/'><filename>OpenEmbedded Metadata Index</filename></ulink>
117 for a list of layers from the OpenEmbedded community
118 that can be used in the Yocto Project.
119 </para></listitem>
120 <listitem><para><emphasis>Create a Directory:</emphasis>
121 Create the directory for your layer.
122 While not strictly required, prepend the name of the
123 folder with the string <filename>meta-</filename>.
124 For example:
125 <literallayout class='monospaced'>
126 meta-mylayer
127 meta-GUI_xyz
128 meta-mymachine
129 </literallayout>
130 </para></listitem>
131 <listitem><para><emphasis>Create a Layer Configuration
132 File:</emphasis>
133 Inside your new layer folder, you need to create a
134 <filename>conf/layer.conf</filename> file.
135 It is easiest to take an existing layer configuration
136 file and copy that to your layer's
137 <filename>conf</filename> directory and then modify the
138 file as needed.</para>
139 <para>The
140 <filename>meta-yocto-bsp/conf/layer.conf</filename> file
141 demonstrates the required syntax:
142 <literallayout class='monospaced'>
143 # We have a conf and classes directory, add to BBPATH
144 BBPATH .= ":${LAYERDIR}"
145
146 # We have recipes-* directories, add to BBFILES
147 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
148 ${LAYERDIR}/recipes-*/*/*.bbappend"
149
150 BBFILE_COLLECTIONS += "yoctobsp"
151 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
152 BBFILE_PRIORITY_yoctobsp = "5"
153 LAYERVERSION_yoctobsp = "2"
154 </literallayout></para>
155 <para>Here is an explanation of the example:
156 <itemizedlist>
157 <listitem><para>The configuration and
158 classes directory is appended to
159 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>.
160 <note>
161 All non-distro layers, which include all BSP
162 layers, are expected to append the layer
163 directory to the
164 <filename>BBPATH</filename>.
165 On the other hand, distro layers, such as
166 <filename>meta-yocto</filename>, can choose
167 to enforce their own precedence over
168 <filename>BBPATH</filename>.
169 For an example of that syntax, see the
170 <filename>layer.conf</filename> file for
171 the <filename>meta-yocto</filename> layer.
172 </note></para></listitem>
173 <listitem><para>The recipes for the layers are
174 appended to
175 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'>BBFILES</ulink></filename>.
176 </para></listitem>
177 <listitem><para>The
178 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'>BBFILE_COLLECTIONS</ulink></filename>
179 variable is then appended with the layer name.
180 </para></listitem>
181 <listitem><para>The
182 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'>BBFILE_PATTERN</ulink></filename>
183 variable is set to a regular expression and is
184 used to match files from
185 <filename>BBFILES</filename> into a particular
186 layer.
187 In this case,
188 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
189 is used to make <filename>BBFILE_PATTERN</filename> match within the
190 layer's path.</para></listitem>
191 <listitem><para>The
192 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'>BBFILE_PRIORITY</ulink></filename>
193 variable then assigns a priority to the layer.
194 Applying priorities is useful in situations
195 where the same package might appear in multiple
196 layers and allows you to choose what layer
197 should take precedence.</para></listitem>
198 <listitem><para>The
199 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'>LAYERVERSION</ulink></filename>
200 variable optionally specifies the version of a
201 layer as a single number.</para></listitem>
202 </itemizedlist></para>
203 <para>Note the use of the
204 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
205 variable, which expands to the directory of the current
206 layer.</para>
207 <para>Through the use of the <filename>BBPATH</filename>
208 variable, BitBake locates <filename>.bbclass</filename>
209 files, configuration files, and files that are included
210 with <filename>include</filename> and
211 <filename>require</filename> statements.
212 For these cases, BitBake uses the first file that
213 matches the name found in <filename>BBPATH</filename>.
214 This is similar to the way the <filename>PATH</filename>
215 variable is used for binaries.
216 We recommend, therefore, that you use unique
217 <filename>.bbclass</filename> and configuration
218 filenames in your custom layer.</para></listitem>
219 <listitem><para><emphasis>Add Content:</emphasis> Depending
220 on the type of layer, add the content.
221 If the layer adds support for a machine, add the machine
222 configuration in a <filename>conf/machine/</filename>
223 file within the layer.
224 If the layer adds distro policy, add the distro
225 configuration in a <filename>conf/distro/</filename>
226 file with the layer.
227 If the layer introduces new recipes, put the recipes
228 you need in <filename>recipes-*</filename>
229 subdirectories within the layer.
230 <note>In order to be compliant with the Yocto Project,
231 a layer must contain a
232 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout-readme'>README file.</ulink>
233 </note></para></listitem>
234 </orderedlist>
235 </para>
236 </section>
237
238 <section id='best-practices-to-follow-when-creating-layers'>
239 <title>Best Practices to Follow When Creating Layers</title>
240
241 <para>
242 To create layers that are easier to maintain and that will
243 not impact builds for other machines, you should consider the
244 information in the following sections.
245 </para>
246
247 <section id='avoid-overlaying-entire-recipes'>
248 <title>Avoid "Overlaying" Entire Recipes</title>
249
250 <para>
251 Avoid "overlaying" entire recipes from other layers in your
252 configuration.
253 In other words, do not copy an entire recipe into your
254 layer and then modify it.
255 Use <filename>.bbappend</filename> files to override the
256 parts of the recipe you need to modify.
257 </para>
258 </section>
259
260 <section id='avoid-duplicating-include-files'>
261 <title>Avoid Duplicating Include Files</title>
262
263 <para>
264 Avoid duplicating include files.
265 Use <filename>.bbappend</filename> files for each recipe
266 that uses an include file.
267 Or, if you are introducing a new recipe that requires
268 the included file, use the path relative to the original
269 layer directory to refer to the file.
270 For example, use
271 <filename>require recipes-core/somepackage/somefile.inc</filename>
272 instead of <filename>require somefile.inc</filename>.
273 If you're finding you have to overlay the include file,
274 it could indicate a deficiency in the include file in
275 the layer to which it originally belongs.
276 If this is the case, you need to address that deficiency
277 instead of overlaying the include file.
278 For example, consider how support plug-ins for the Qt 4
279 database are configured.
280 The Source Directory does not have MySQL or PostgreSQL.
281 However, OpenEmbedded's layer <filename>meta-oe</filename>
282 does.
283 Consequently, <filename>meta-oe</filename> uses
284 <filename>.bbappend</filename> files to modify the
285 <filename>QT_SQL_DRIVER_FLAGS</filename> variable to
286 enable the appropriate plug-ins.
287 This variable was added to the <filename>qt4.inc</filename>
288 include file in the Source Directory specifically to allow
289 the <filename>meta-oe</filename> layer to be able to control
290 which plug-ins are built.
291 </para>
292 </section>
293
294 <section id='structure-your-layers'>
295 <title>Structure Your Layers</title>
296
297 <para>
298 Proper use of overrides within append files and placement
299 of machine-specific files within your layer can ensure that
300 a build is not using the wrong Metadata and negatively
301 impacting a build for a different machine.
302 Following are some examples:
303 <itemizedlist>
304 <listitem><para><emphasis>Modifying Variables to Support
305 a Different Machine:</emphasis>
306 Suppose you have a layer named
307 <filename>meta-one</filename> that adds support
308 for building machine "one".
309 To do so, you use an append file named
310 <filename>base-files.bbappend</filename> and
311 create a dependency on "foo" by altering the
312 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
313 variable:
314 <literallayout class='monospaced'>
315 DEPENDS = "foo"
316 </literallayout>
317 The dependency is created during any build that
318 includes the layer
319 <filename>meta-one</filename>.
320 However, you might not want this dependency
321 for all machines.
322 For example, suppose you are building for
323 machine "two" but your
324 <filename>bblayers.conf</filename> file has the
325 <filename>meta-one</filename> layer included.
326 During the build, the
327 <filename>base-files</filename> for machine
328 "two" will also have the dependency on
329 <filename>foo</filename>.</para>
330 <para>To make sure your changes apply only when
331 building machine "one", use a machine override
332 with the <filename>DEPENDS</filename> statement:
333 <literallayout class='monospaced'>
334 DEPENDS_one = "foo"
335 </literallayout>
336 You should follow the same strategy when using
337 <filename>_append</filename> and
338 <filename>_prepend</filename> operations:
339 <literallayout class='monospaced'>
340 DEPENDS_append_one = " foo"
341 DEPENDS_prepend_one = "foo "
342 </literallayout>
343 <note>
344 Avoiding "+=" and "=+" and using
345 machine-specific
346 <filename>_append</filename>
347 and <filename>_prepend</filename> operations
348 is recommended as well.
349 </note></para></listitem>
350 <listitem><para><emphasis>Place Machine-Specific Files
351 in Machine-Specific Locations:</emphasis>
352 When you have a base recipe, such as
353 <filename>base-files.bb</filename>, that
354 contains a
355 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
356 statement to a file, you can use an append file
357 to cause the build to use your own version of
358 the file.
359 For example, an append file in your layer at
360 <filename>/meta-one/recipes-core/base-files/base-files.bbappend</filename>
361 could extend
362 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
363 using
364 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
365 as follows:
366 <literallayout class='monospaced'>
367 FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"
368 </literallayout>
369 The build for machine "one" will pick up your
370 machine-specific file as long as you have the
371 file in
372 <filename>/meta-one/recipes-core/base-files/base-files/</filename>.
373 However, if you are building for a different
374 machine and the
375 <filename>bblayers.conf</filename> file includes
376 the <filename>meta-one</filename> layer and
377 the location of your machine-specific file is
378 the first location where that file is found
379 according to <filename>FILESPATH</filename>,
380 builds for all machines will also use that
381 machine-specific file.</para>
382 <para>You can make sure that a machine-specific
383 file is used for a particular machine by putting
384 the file in a subdirectory specific to the
385 machine.
386 For example, rather than placing the file in
387 <filename>/meta-one/recipes-core/base-files/base-files/</filename>
388 as shown above, put it in
389 <filename>/meta-one/recipes-core/base-files/base-files/one/</filename>.
390 Not only does this make sure the file is used
391 only when building for machine "one" but the
392 build process locates the file more quickly.</para>
393 <para>In summary, you need to place all files
394 referenced from <filename>SRC_URI</filename>
395 in a machine-specific subdirectory within the
396 layer in order to restrict those files to
397 machine-specific builds.</para></listitem>
398 </itemizedlist>
399 </para>
400 </section>
401
402 <section id='other-recommendations'>
403 <title>Other Recommendations</title>
404
405 <para>
406 We also recommend the following:
407 <itemizedlist>
408 <listitem><para>Store custom layers in a Git repository
409 that uses the
410 <filename>meta-&lt;layer_name&gt;</filename> format.
411 </para></listitem>
412 <listitem><para>Clone the repository alongside other
413 <filename>meta</filename> directories in the
414 <link linkend='source-directory'>Source Directory</link>.
415 </para></listitem>
416 </itemizedlist>
417 Following these recommendations keeps your Source Directory and
418 its configuration entirely inside the Yocto Project's core
419 base.
420 </para>
421 </section>
422 </section>
423
424 <section id='enabling-your-layer'>
425 <title>Enabling Your Layer</title>
426
427 <para>
428 Before the OpenEmbedded build system can use your new layer,
429 you need to enable it.
430 To enable your layer, simply add your layer's path to the
431 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename>
432 variable in your <filename>conf/bblayers.conf</filename> file,
433 which is found in the
434 <link linkend='build-directory'>Build Directory</link>.
435 The following example shows how to enable a layer named
436 <filename>meta-mylayer</filename>:
437 <literallayout class='monospaced'>
438 LCONF_VERSION = "6"
439
440 BBPATH = "${TOPDIR}"
441 BBFILES ?= ""
442
443 BBLAYERS ?= " \
444 $HOME/poky/meta \
445 $HOME/poky/meta-yocto \
446 $HOME/poky/meta-yocto-bsp \
447 $HOME/poky/meta-mylayer \
448 "
449
450 BBLAYERS_NON_REMOVABLE ?= " \
451 $HOME/poky/meta \
452 $HOME/poky/meta-yocto \
453 "
454 </literallayout>
455 </para>
456
457 <para>
458 BitBake parses each <filename>conf/layer.conf</filename> file
459 as specified in the <filename>BBLAYERS</filename> variable
460 within the <filename>conf/bblayers.conf</filename> file.
461 During the processing of each
462 <filename>conf/layer.conf</filename> file, BitBake adds the
463 recipes, classes and configurations contained within the
464 particular layer to the source directory.
465 </para>
466 </section>
467
468 <section id='using-bbappend-files'>
469 <title>Using .bbappend Files</title>
470
471 <para>
472 Recipes used to append Metadata to other recipes are called
473 BitBake append files.
474 BitBake append files use the <filename>.bbappend</filename> file
475 type suffix, while the corresponding recipes to which Metadata
476 is being appended use the <filename>.bb</filename> file type
477 suffix.
478 </para>
479
480 <para>
481 A <filename>.bbappend</filename> file allows your layer to make
482 additions or changes to the content of another layer's recipe
483 without having to copy the other recipe into your layer.
484 Your <filename>.bbappend</filename> file resides in your layer,
485 while the main <filename>.bb</filename> recipe file to
486 which you are appending Metadata resides in a different layer.
487 </para>
488
489 <para>
490 Append files must have the same root names as their corresponding
491 recipes.
492 For example, the append file
493 <filename>someapp_&DISTRO;.bbappend</filename> must apply to
494 <filename>someapp_&DISTRO;.bb</filename>.
495 This means the original recipe and append file names are version
496 number-specific.
497 If the corresponding recipe is renamed to update to a newer
498 version, the corresponding <filename>.bbappend</filename> file must
499 be renamed as well.
500 During the build process, BitBake displays an error on starting
501 if it detects a <filename>.bbappend</filename> file that does
502 not have a corresponding recipe with a matching name.
503 See the
504 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink>
505 variable for information on how to handle this error.
506 </para>
507
508 <para>
509 Being able to append information to an existing recipe not only
510 avoids duplication, but also automatically applies recipe
511 changes in a different layer to your layer.
512 If you were copying recipes, you would have to manually merge
513 changes as they occur.
514 </para>
515
516 <para>
517 As an example, consider the main formfactor recipe and a
518 corresponding formfactor append file both from the
519 <link linkend='source-directory'>Source Directory</link>.
520 Here is the main formfactor recipe, which is named
521 <filename>formfactor_0.0.bb</filename> and located in the
522 "meta" layer at
523 <filename>meta/recipes-bsp/formfactor</filename>:
524 <literallayout class='monospaced'>
525 DESCRIPTION = "Device formfactor information"
526 SECTION = "base"
527 LICENSE = "MIT"
528 LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
529 file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
530 PR = "r41"
531
532 SRC_URI = "file://config file://machconfig"
533 S = "${WORKDIR}"
534
535 PACKAGE_ARCH = "${MACHINE_ARCH}"
536 INHIBIT_DEFAULT_DEPS = "1"
537
538 do_install() {
539 # Only install file if it has a contents
540 install -d ${D}${sysconfdir}/formfactor/
541 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
542 if [ -s "${S}/machconfig" ]; then
543 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
544 fi
545 }
546 </literallayout>
547 In the main recipe, note the
548 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
549 variable, which tells the OpenEmbedded build system where to
550 find files during the build.
551 </para>
552
553 <para>
554 Following is the append file, which is named
555 <filename>formfactor_0.0.bbappend</filename> and is from the
556 Crown Bay BSP Layer named
557 <filename>meta-intel/meta-crownbay</filename>.
558 The file is in <filename>recipes-bsp/formfactor</filename>:
559 <literallayout class='monospaced'>
560 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
561 </literallayout>
562 </para>
563
564 <para>
565 By default, the build system uses the
566 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
567 variable to locate files.
568 This append file extends the locations by setting the
569 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
570 variable.
571 Setting this variable in the <filename>.bbappend</filename>
572 file is the most reliable and recommended method for adding
573 directories to the search path used by the build system
574 to find files.
575 </para>
576
577 <para>
578 The statement in this example extends the directories to include
579 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>,
580 which resolves to a directory named
581 <filename>formfactor</filename> in the same directory
582 in which the append file resides (i.e.
583 <filename>meta-intel/meta-crownbay/recipes-bsp/formfactor/formfactor</filename>.
584 This implies that you must have the supporting directory
585 structure set up that will contain any files or patches you
586 will be including from the layer.
587 </para>
588
589 <para>
590 Using the immediate expansion assignment operator
591 <filename>:=</filename> is important because of the reference to
592 <filename>THISDIR</filename>.
593 The trailing colon character is important as it ensures that
594 items in the list remain colon-separated.
595 <note><para>BitBake automatically defines the
596 <filename>THISDIR</filename> variable.
597 You should never set this variable yourself.
598 Using <filename>_prepend</filename> ensures your path will
599 be searched prior to other paths in the final list.</para>
600 <para>Also, not all append files add extra files.
601 Many append files simply exist to add build options
602 (e.g. <filename>systemd</filename>).
603 For these cases, it is not necessary to use the
604 "_prepend" part of the statement.</para>
605 </note>
606 </para>
607 </section>
608
609 <section id='prioritizing-your-layer'>
610 <title>Prioritizing Your Layer</title>
611
612 <para>
613 Each layer is assigned a priority value.
614 Priority values control which layer takes precedence if there
615 are recipe files with the same name in multiple layers.
616 For these cases, the recipe file from the layer with a higher
617 priority number takes precedence.
618 Priority values also affect the order in which multiple
619 <filename>.bbappend</filename> files for the same recipe are
620 applied.
621 You can either specify the priority manually, or allow the
622 build system to calculate it based on the layer's dependencies.
623 </para>
624
625 <para>
626 To specify the layer's priority manually, use the
627 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>
628 variable.
629 For example:
630 <literallayout class='monospaced'>
631 BBFILE_PRIORITY_mylayer = "1"
632 </literallayout>
633 </para>
634
635 <note>
636 <para>It is possible for a recipe with a lower version number
637 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
638 in a layer that has a higher priority to take precedence.</para>
639 <para>Also, the layer priority does not currently affect the
640 precedence order of <filename>.conf</filename>
641 or <filename>.bbclass</filename> files.
642 Future versions of BitBake might address this.</para>
643 </note>
644 </section>
645
646 <section id='managing-layers'>
647 <title>Managing Layers</title>
648
649 <para>
650 You can use the BitBake layer management tool to provide a view
651 into the structure of recipes across a multi-layer project.
652 Being able to generate output that reports on configured layers
653 with their paths and priorities and on
654 <filename>.bbappend</filename> files and their applicable
655 recipes can help to reveal potential problems.
656 </para>
657
658 <para>
659 Use the following form when running the layer management tool.
660 <literallayout class='monospaced'>
661 $ bitbake-layers &lt;command&gt; [arguments]
662 </literallayout>
663 The following list describes the available commands:
664 <itemizedlist>
665 <listitem><para><filename><emphasis>help:</emphasis></filename>
666 Displays general help or help on a specified command.
667 </para></listitem>
668 <listitem><para><filename><emphasis>show-layers:</emphasis></filename>
669 Shows the current configured layers.
670 </para></listitem>
671 <listitem><para><filename><emphasis>show-recipes:</emphasis></filename>
672 Lists available recipes and the layers that provide them.
673 </para></listitem>
674 <listitem><para><filename><emphasis>show-overlayed:</emphasis></filename>
675 Lists overlayed recipes.
676 A recipe is overlayed when a recipe with the same name
677 exists in another layer that has a higher layer
678 priority.
679 </para></listitem>
680 <listitem><para><filename><emphasis>show-appends:</emphasis></filename>
681 Lists <filename>.bbappend</filename> files and the
682 recipe files to which they apply.
683 </para></listitem>
684 <listitem><para><filename><emphasis>show-cross-depends:</emphasis></filename>
685 Lists dependency relationships between recipes that
686 cross layer boundaries.
687 </para></listitem>
688 <listitem><para><filename><emphasis>flatten:</emphasis></filename>
689 Flattens the layer configuration into a separate output
690 directory.
691 Flattening your layer configuration builds a "flattened"
692 directory that contains the contents of all layers,
693 with any overlayed recipes removed and any
694 <filename>.bbappend</filename> files appended to the
695 corresponding recipes.
696 You might have to perform some manual cleanup of the
697 flattened layer as follows:
698 <itemizedlist>
699 <listitem><para>Non-recipe files (such as patches)
700 are overwritten.
701 The flatten command shows a warning for these
702 files.
703 </para></listitem>
704 <listitem><para>Anything beyond the normal layer
705 setup has been added to the
706 <filename>layer.conf</filename> file.
707 Only the lowest priority layer's
708 <filename>layer.conf</filename> is used.
709 </para></listitem>
710 <listitem><para>Overridden and appended items from
711 <filename>.bbappend</filename> files need to be
712 cleaned up.
713 The contents of each
714 <filename>.bbappend</filename> end up in the
715 flattened recipe.
716 However, if there are appended or changed
717 variable values, you need to tidy these up
718 yourself.
719 Consider the following example.
720 Here, the <filename>bitbake-layers</filename>
721 command adds the line
722 <filename>#### bbappended ...</filename> so that
723 you know where the following lines originate:
724 <literallayout class='monospaced'>
725 ...
726 DESCRIPTION = "A useful utility"
727 ...
728 EXTRA_OECONF = "--enable-something"
729 ...
730
731 #### bbappended from meta-anotherlayer ####
732
733 DESCRIPTION = "Customized utility"
734 EXTRA_OECONF += "--enable-somethingelse"
735 </literallayout>
736 Ideally, you would tidy up these utilities as
737 follows:
738 <literallayout class='monospaced'>
739 ...
740 DESCRIPTION = "Customized utility"
741 ...
742 EXTRA_OECONF = "--enable-something --enable-somethingelse"
743 ...
744 </literallayout></para></listitem>
745 </itemizedlist></para></listitem>
746 </itemizedlist>
747 </para>
748 </section>
749
750 <section id='creating-a-general-layer-using-the-yocto-layer-script'>
751 <title>Creating a General Layer Using the yocto-layer Script</title>
752
753 <para>
754 The <filename>yocto-layer</filename> script simplifies
755 creating a new general layer.
756 <note>
757 For information on BSP layers, see the
758 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
759 section in the Yocto Project Board Specific (BSP)
760 Developer's Guide.
761 </note>
762 The default mode of the script's operation is to prompt you for
763 information needed to generate the layer:
764 <itemizedlist>
765 <listitem><para>The layer priority
766 </para></listitem>
767 <listitem><para>Whether or not to create a sample recipe.
768 </para></listitem>
769 <listitem><para>Whether or not to create a sample
770 append file.
771 </para></listitem>
772 </itemizedlist>
773 </para>
774
775 <para>
776 Use the <filename>yocto-layer create</filename> sub-command
777 to create a new general layer.
778 In its simplest form, you can create a layer as follows:
779 <literallayout class='monospaced'>
780 $ yocto-layer create mylayer
781 </literallayout>
782 The previous example creates a layer named
783 <filename>meta-mylayer</filename> in the current directory.
784 </para>
785
786 <para>
787 As the <filename>yocto-layer create</filename> command runs,
788 default values for the prompts appear in brackets.
789 Pressing enter without supplying anything for the prompts
790 or pressing enter and providing an invalid response causes the
791 script to accept the default value.
792 Once the script completes, the new layer
793 is created in the current working directory.
794 The script names the layer by prepending
795 <filename>meta-</filename> to the name you provide.
796 </para>
797
798 <para>
799 Minimally, the script creates the following within the layer:
800 <itemizedlist>
801 <listitem><para><emphasis>The <filename>conf</filename>
802 directory:</emphasis>
803 This directory contains the layer's configuration file.
804 The root name for the file is the same as the root name
805 your provided for the layer (e.g.
806 <filename>&lt;layer&gt;.conf</filename>).
807 </para></listitem>
808 <listitem><para><emphasis>The
809 <filename>COPYING.MIT</filename>:</emphasis>
810 The copyright and use notice for the software.
811 </para></listitem>
812 <listitem><para><emphasis>The <filename>README</filename>
813 file:</emphasis>
814 A file describing the contents of your new layer.
815 </para></listitem>
816 </itemizedlist>
817 </para>
818
819 <para>
820 If you choose to generate a sample recipe file, the script
821 prompts you for the name for the recipe and then creates it
822 in <filename>&lt;layer&gt;/recipes-example/example/</filename>.
823 The script creates a <filename>.bb</filename> file and a
824 directory, which contains a sample
825 <filename>helloworld.c</filename> source file and along with
826 a sample patch file.
827 If you do not provide a recipe name, the script uses
828 "example".
829 </para>
830
831 <para>
832 If you choose to generate a sample append file, the script
833 prompts you for the name for the file and then creates it
834 in <filename>&lt;layer&gt;/recipes-example-bbappend/example-bbappend/</filename>.
835 The script creates a <filename>.bbappend</filename> file and a
836 directory, which contains a sample patch file.
837 If you do not provide a recipe name, the script uses
838 "example".
839 The script also prompts you for the version of the append file.
840 The version should match the recipe to which the append file
841 is associated.
842 </para>
843
844 <para>
845 The easiest way to see how the <filename>yocto-layer</filename>
846 script works is to experiment with the script.
847 You can also read the usage information by entering the
848 following:
849 <literallayout class='monospaced'>
850 $ yocto-layer help
851 </literallayout>
852 </para>
853
854 <para>
855 Once you create your general layer, you must add it to your
856 <filename>bblayers.conf</filename> file.
857 Here is an example where a layer named
858 <filename>meta-mylayer</filename> is added:
859 <literallayout class='monospaced'>
860 BBLAYERS = ?" \
861 /usr/local/src/yocto/meta \
862 /usr/local/src/yocto/meta-yocto \
863 /usr/local/src/yocto/meta-yocto-bsp \
864 /usr/local/src/yocto/meta-mylayer \
865 "
866
867 BBLAYERS_NON_REMOVABLE ?= " \
868 /usr/local/src/yocto/meta \
869 /usr/local/src/yocto/meta-yocto \
870 "
871 </literallayout>
872 Adding the layer to this file enables the build system to
873 locate the layer during the build.
874 </para>
875 </section>
876 </section>
877
878 <section id='usingpoky-extend-customimage'>
879 <title>Customizing Images</title>
880
881 <para>
882 You can customize images to satisfy particular requirements.
883 This section describes several methods and provides guidelines for each.
884 </para>
885
886 <section id='usingpoky-extend-customimage-custombb'>
887 <title>Customizing Images Using Custom .bb Files</title>
888
889 <para>
890 One way to get additional software into an image is to create a custom image.
891 The following example shows the form for the two lines you need:
892 <literallayout class='monospaced'>
893 IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
894
895 inherit core-image
896 </literallayout>
897 </para>
898
899 <para>
900 By creating a custom image, a developer has total control
901 over the contents of the image.
902 It is important to use the correct names of packages in the
903 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
904 variable.
905 You must use the OpenEmbedded notation and not the Debian notation for the names
906 (e.g. <filename>eglibc-dev</filename> instead of <filename>libc6-dev</filename>).
907 </para>
908
909 <para>
910 The other method for creating a custom image is to base it on an existing image.
911 For example, if you want to create an image based on <filename>core-image-sato</filename>
912 but add the additional package <filename>strace</filename> to the image,
913 copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a
914 new <filename>.bb</filename> and add the following line to the end of the copy:
915 <literallayout class='monospaced'>
916 IMAGE_INSTALL += "strace"
917 </literallayout>
918 </para>
919 </section>
920
921 <section id='usingpoky-extend-customimage-customtasks'>
922 <title>Customizing Images Using Custom Package Groups</title>
923
924 <para>
925 For complex custom images, the best approach is to create a
926 custom package group recipe that is used to build the image or
927 images.
928 A good example of a package group recipe is
929 <filename>meta/recipes-core/packagegroups/packagegroup-core-boot.bb</filename>.
930 The
931 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename>
932 variable lists the package group packages you wish to produce.
933 <filename>inherit packagegroup</filename> sets appropriate
934 default values and automatically adds <filename>-dev</filename>,
935 <filename>-dbg</filename>, and <filename>-ptest</filename>
936 complementary packages for every package specified in
937 <filename>PACKAGES</filename>.
938 Note that the inherit line should be towards
939 the top of the recipe, certainly before you set
940 <filename>PACKAGES</filename>.
941 For each package you specify in <filename>PACKAGES</filename>,
942 you can use
943 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename>
944 and
945 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename>
946 entries to provide a list of packages the parent task package
947 should contain.
948 Following is an example:
949 <literallayout class='monospaced'>
950 DESCRIPTION = "My Custom Package Groups"
951
952 inherit packagegroup
953
954 PACKAGES = "\
955 packagegroup-custom-apps \
956 packagegroup-custom-tools \
957 "
958
959 RDEPENDS_packagegroup-custom-apps = "\
960 dropbear \
961 portmap \
962 psplash"
963
964 RDEPENDS_packagegroup-custom-tools = "\
965 oprofile \
966 oprofileui-server \
967 lttng-control \
968 lttng-viewer"
969
970 RRECOMMENDS_packagegroup-custom-tools = "\
971 kernel-module-oprofile"
972 </literallayout>
973 </para>
974
975 <para>
976 In the previous example, two package group packages are created with their dependencies and their
977 recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and
978 <filename>packagegroup-custom-tools</filename>.
979 To build an image using these package group packages, you need to add
980 <filename>packagegroup-custom-apps</filename> and/or
981 <filename>packagegroup-custom-tools</filename> to
982 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>.
983 For other forms of image dependencies see the other areas of this section.
984 </para>
985 </section>
986
987 <section id='usingpoky-extend-customimage-imagefeatures'>
988 <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and
989 <filename>EXTRA_IMAGE_FEATURES</filename></title>
990
991 <para>
992 You might want to customize your image by enabling or
993 disabling high-level image features by using the
994 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
995 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
996 variables.
997 Although the functions for both variables are nearly equivalent,
998 best practices dictate using <filename>IMAGE_FEATURES</filename>
999 from within a recipe and using
1000 <filename>EXTRA_IMAGE_FEATURES</filename> from within
1001 your <filename>local.conf</filename> file, which is found in the
1002 <link linkend='build-directory'>Build Directory</link>.
1003 </para>
1004
1005 <para>
1006 To understand how these features work, the best reference is
1007 <filename>meta/classes/core-image.bbclass</filename>.
1008 In summary, the file looks at the contents of the
1009 <filename>IMAGE_FEATURES</filename> variable and then maps
1010 those contents into a set of package groups.
1011 Based on this information, the build system automatically
1012 adds the appropriate packages to the
1013 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
1014 variable.
1015 Effectively, you are enabling extra features by extending the
1016 class or creating a custom class for use with specialized image
1017 <filename>.bb</filename> files.
1018 </para>
1019
1020 <para>
1021 Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable
1022 from within your local configuration file.
1023 Using a separate area from which to enable features with
1024 this variable helps you avoid overwriting the features in the
1025 image recipe that are enabled with
1026 <filename>IMAGE_FEATURES</filename>.
1027 The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added
1028 to <filename>IMAGE_FEATURES</filename> within
1029 <filename>meta/conf/bitbake.conf</filename>.
1030 </para>
1031
1032 <para>
1033 To illustrate how you can use these variables to modify your
1034 image, consider an example that selects the SSH server.
1035 The Yocto Project ships with two SSH servers you can use
1036 with your images: Dropbear and OpenSSH.
1037 Dropbear is a minimal SSH server appropriate for
1038 resource-constrained environments, while OpenSSH is a
1039 well-known standard SSH server implementation.
1040 By default, the <filename>core-image-sato</filename> image
1041 is configured to use Dropbear.
1042 The <filename>core-image-basic</filename> and
1043 <filename>core-image-lsb</filename> images both
1044 include OpenSSH.
1045 The <filename>core-image-minimal</filename> image does not
1046 contain an SSH server.
1047 </para>
1048
1049 <para>
1050 You can customize your image and change these defaults.
1051 Edit the <filename>IMAGE_FEATURES</filename> variable
1052 in your recipe or use the
1053 <filename>EXTRA_IMAGE_FEATURES</filename> in your
1054 <filename>local.conf</filename> file so that it configures the
1055 image you are working with to include
1056 <filename>ssh-server-dropbear</filename> or
1057 <filename>ssh-server-openssh</filename>.
1058 </para>
1059
1060 <note>
1061 See the
1062 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
1063 section in the Yocto Project Reference Manual for a complete
1064 list of image features that ship with the Yocto Project.
1065 </note>
1066 </section>
1067
1068 <section id='usingpoky-extend-customimage-localconf'>
1069 <title>Customizing Images Using <filename>local.conf</filename></title>
1070
1071 <para>
1072 It is possible to customize image contents by using variables from your
1073 local configuration in your <filename>conf/local.conf</filename> file.
1074 Because it is limited to local use, this method generally only allows you to
1075 add packages and is not as flexible as creating your own customized image.
1076 When you add packages using local variables this way, you need to realize that
1077 these variable changes affect all images at the same time and might not be
1078 what you require.
1079 </para>
1080
1081 <para>
1082 The simplest way to add extra packages to all images is by using the
1083 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1084 variable with the <filename>_append</filename> operator:
1085 <literallayout class='monospaced'>
1086 IMAGE_INSTALL_append = " strace"
1087 </literallayout>
1088 Use of the syntax is important - specifically, the space between
1089 the quote and the package name, which is
1090 <filename>strace</filename> in this example.
1091 This space is required since the <filename>_append</filename>
1092 operator does not add the space.
1093 </para>
1094
1095 <para>
1096 Furthermore, you must use <filename>_append</filename> instead of the <filename>+=</filename>
1097 operator if you want to avoid ordering issues.
1098 The reason for this is because doing so unconditionally appends to the variable and
1099 avoids ordering problems due to the variable being set in image recipes and
1100 <filename>.bbclass</filename> files with operators like <filename>?=</filename>.
1101 Using <filename>_append</filename> ensures the operation takes affect.
1102 </para>
1103
1104 <para>
1105 As shown in its simplest use, <filename>IMAGE_INSTALL_append</filename> affects
1106 all images.
1107 It is possible to extend the syntax so that the variable applies to a specific image only.
1108 Here is an example:
1109 <literallayout class='monospaced'>
1110 IMAGE_INSTALL_append_pn-core-image-minimal = " strace"
1111 </literallayout>
1112 This example adds <filename>strace</filename> to <filename>core-image-minimal</filename>
1113 only.
1114 </para>
1115
1116 <para>
1117 You can add packages using a similar approach through the
1118 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename>
1119 variable.
1120 If you use this variable, only <filename>core-image-*</filename> images are affected.
1121 </para>
1122 </section>
1123 </section>
1124
1125 <section id='usingpoky-extend-addpkg'>
1126 <title>Writing a Recipe to Add a Package to Your Image</title>
1127
1128 <para>
1129 Recipes add packages to your image.
1130 Writing a recipe means creating a <filename>.bb</filename> file that sets some
1131 variables.
1132 For information on variables that are useful for recipes and for information about recipe naming
1133 issues, see the
1134 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>"
1135 section of the Yocto Project Reference Manual.
1136 </para>
1137
1138 <para>
1139 Before writing a recipe from scratch, it is often useful to check
1140 whether someone else has written one already.
1141 OpenEmbedded is a good place to look as it has a wider scope and range of packages.
1142 Because the Yocto Project aims to be compatible with OpenEmbedded, most recipes
1143 you find there should work for you.
1144 </para>
1145
1146 <para>
1147 For new packages, the simplest way to add a recipe is to base it on a similar
1148 pre-existing recipe.
1149 The sections that follow provide some examples that show how to add standard
1150 types of packages.
1151 </para>
1152
1153 <note>
1154 <para>When writing shell functions, you need to be aware of BitBake's
1155 curly brace parsing.
1156 If a recipe uses a closing curly brace within the function and
1157 the character has no leading spaces, BitBake produces a parsing
1158 error.
1159 If you use a pair of curly brace in a shell function, the
1160 closing curly brace must not be located at the start of the line
1161 without leading spaces.</para>
1162 <para>Here is an example that causes BitBake to produce a parsing
1163 error:
1164 <literallayout class='monospaced'>
1165 fakeroot create_shar() {
1166 cat &lt;&lt; "EOF" &gt; ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
1167 usage()
1168 {
1169 echo "test"
1170 ###### The following "}" at the start of the line causes a parsing error ######
1171 }
1172 EOF
1173 }
1174 </literallayout>
1175 Writing the recipe this way avoids the error:
1176 <literallayout class='monospaced'>
1177 fakeroot create_shar() {
1178 cat &lt;&lt; "EOF" &gt; ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
1179 usage()
1180 {
1181 echo "test"
1182 ######The following "}" with a leading space at the start of the line avoids the error ######
1183 }
1184 EOF
1185 }
1186 </literallayout></para>
1187 </note>
1188
1189 <section id='usingpoky-extend-addpkg-singlec'>
1190 <title>Single .c File Package (Hello World!)</title>
1191
1192 <para>
1193 Building an application from a single file that is stored locally (e.g. under
1194 <filename>files/</filename>) requires a recipe that has the file listed in
1195 the
1196 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>
1197 variable.
1198 Additionally, you need to manually write the <filename>do_compile</filename> and
1199 <filename>do_install</filename> tasks.
1200 The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
1201 variable defines the
1202 directory containing the source code, which is set to
1203 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'>
1204 WORKDIR</ulink></filename> in this case - the directory BitBake uses for the build.
1205 <literallayout class='monospaced'>
1206 DESCRIPTION = "Simple helloworld application"
1207 SECTION = "examples"
1208 LICENSE = "MIT"
1209 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
1210 PR = "r0"
1211
1212 SRC_URI = "file://helloworld.c"
1213
1214 S = "${WORKDIR}"
1215
1216 do_compile() {
1217 ${CC} helloworld.c -o helloworld
1218 }
1219
1220 do_install() {
1221 install -d ${D}${bindir}
1222 install -m 0755 helloworld ${D}${bindir}
1223 }
1224 </literallayout>
1225 </para>
1226
1227 <para>
1228 By default, the <filename>helloworld</filename>, <filename>helloworld-dbg</filename>,
1229 and <filename>helloworld-dev</filename> packages are built.
1230 For information on how to customize the packaging process, see the
1231 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application
1232 into Multiple Packages</link>" section.
1233 </para>
1234 </section>
1235
1236 <section id='usingpoky-extend-addpkg-autotools'>
1237 <title>Autotooled Package</title>
1238 <para>
1239 Applications that use Autotools such as <filename>autoconf</filename> and
1240 <filename>automake</filename> require a recipe that has a source archive listed in
1241 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and
1242 also inherits Autotools, which instructs BitBake to use the
1243 <filename>autotools.bbclass</filename> file, which contains the definitions of all the steps
1244 needed to build an Autotool-based application.
1245 The result of the build is automatically packaged.
1246 And, if the application uses NLS for localization, packages with local information are
1247 generated (one package per language).
1248 Following is one example: (<filename>hello_2.3.bb</filename>)
1249 <literallayout class='monospaced'>
1250 DESCRIPTION = "GNU Helloworld application"
1251 SECTION = "examples"
1252 LICENSE = "GPLv2+"
1253 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
1254 PR = "r0"
1255
1256 SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
1257
1258 inherit autotools gettext
1259 </literallayout>
1260 </para>
1261
1262 <para>
1263 The variable
1264 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename>
1265 is used to track source license changes as described in the
1266 "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>" section.
1267 You can quickly create Autotool-based recipes in a manner similar to the previous example.
1268 </para>
1269 </section>
1270
1271 <section id='usingpoky-extend-addpkg-makefile'>
1272 <title>Makefile-Based Package</title>
1273
1274 <para>
1275 Applications that use GNU <filename>make</filename> also require a recipe that has
1276 the source archive listed in
1277 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
1278 You do not need to add a <filename>do_compile</filename> step since by default BitBake
1279 starts the <filename>make</filename> command to compile the application.
1280 If you need additional <filename>make</filename> options, you should store them in the
1281 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'>EXTRA_OEMAKE</ulink></filename>
1282 variable.
1283 BitBake passes these options into the <filename>make</filename> GNU invocation.
1284 Note that a <filename>do_install</filename> task is still required.
1285 Otherwise, BitBake runs an empty <filename>do_install</filename> task by default.
1286 </para>
1287
1288 <para>
1289 Some applications might require extra parameters to be passed to the compiler.
1290 For example, the application might need an additional header path.
1291 You can accomplish this by adding to the
1292 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable.
1293 The following example shows this:
1294 <literallayout class='monospaced'>
1295 CFLAGS_prepend = "-I ${S}/include "
1296 </literallayout>
1297 </para>
1298
1299 <para>
1300 In the following example, <filename>mtd-utils</filename> is a makefile-based package:
1301 <literallayout class='monospaced'>
1302 DESCRIPTION = "Tools for managing memory technology devices."
1303 SECTION = "base"
1304 DEPENDS = "zlib lzo e2fsprogs util-linux"
1305 HOMEPAGE = "http://www.linux-mtd.infradead.org/"
1306 LICENSE = "GPLv2+"
1307 LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
1308 file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
1309
1310 SRC_URI = "git://git.infradead.org/mtd-utils.git;protocol=git;tag=995cfe51b0a3cf32f381c140bf72b21bf91cef1b \
1311 file://add-exclusion-to-mkfs-jffs2-git-2.patch"
1312
1313 S = "${WORKDIR}/git/"
1314
1315 PR = "r1"
1316
1317 EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' \
1318 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'"
1319
1320 do_install () {
1321 oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} \
1322 INCLUDEDIR=${includedir}
1323 install -d ${D}${includedir}/mtd/
1324 for f in ${S}/include/mtd/*.h; do
1325 install -m 0644 $f ${D}${includedir}/mtd/
1326 done
1327 }
1328
1329 PARALLEL_MAKE = ""
1330
1331 BBCLASSEXTEND = "native"
1332 </literallayout>
1333 </para>
1334
1335 <para>
1336 If your sources are available as a tarball instead of a Git repository, you
1337 will need to provide the URL to the tarball as well as an
1338 <filename>md5</filename> or <filename>sha256</filename> sum of
1339 the download.
1340 Here is an example:
1341 <literallayout class='monospaced'>
1342 SRC_URI="ftp://ftp.infradead.org/pub/mtd-utils/mtd-utils-1.4.9.tar.bz2"
1343 SRC_URI[md5sum]="82b8e714b90674896570968f70ca778b"
1344 </literallayout>
1345 You can generate the <filename>md5</filename> or <filename>sha256</filename> sums
1346 by using the <filename>md5sum</filename> or <filename>sha256sum</filename> commands
1347 with the target file as the only argument.
1348 Here is an example:
1349 <literallayout class='monospaced'>
1350 $ md5sum mtd-utils-1.4.9.tar.bz2
1351 82b8e714b90674896570968f70ca778b mtd-utils-1.4.9.tar.bz2
1352 </literallayout>
1353 </para>
1354 </section>
1355
1356 <section id='splitting-an-application-into-multiple-packages'>
1357 <title>Splitting an Application into Multiple Packages</title>
1358
1359 <para>
1360 You can use the variables
1361 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and
1362 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename>
1363 to split an application into multiple packages.
1364 </para>
1365
1366 <para>
1367 Following is an example that uses the <filename>libXpm</filename> recipe.
1368 By default, this recipe generates a single package that contains the library along
1369 with a few binaries.
1370 You can modify the recipe to split the binaries into separate packages:
1371 <literallayout class='monospaced'>
1372 require xorg-lib-common.inc
1373
1374 DESCRIPTION = "X11 Pixmap library"
1375 LICENSE = "X-BSD"
1376 LIC_FILES_CHKSUM = "file://COPYING;md5=3e07763d16963c3af12db271a31abaa5"
1377 DEPENDS += "libxext libsm libxt"
1378 PR = "r3"
1379 PE = "1"
1380
1381 XORG_PN = "libXpm"
1382
1383 PACKAGES =+ "sxpm cxpm"
1384 FILES_cxpm = "${bindir}/cxpm"
1385 FILES_sxpm = "${bindir}/sxpm"
1386 </literallayout>
1387 </para>
1388
1389 <para>
1390 In the previous example, we want to ship the <filename>sxpm</filename>
1391 and <filename>cxpm</filename> binaries in separate packages.
1392 Since <filename>bindir</filename> would be packaged into the main
1393 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
1394 package by default, we prepend the <filename>PACKAGES</filename>
1395 variable so additional package names are added to the start of list.
1396 This results in the extra <filename>FILES_*</filename>
1397 variables then containing information that define which files and
1398 directories go into which packages.
1399 Files included by earlier packages are skipped by latter packages.
1400 Thus, the main <filename>PN</filename> package
1401 does not include the above listed files.
1402 </para>
1403 </section>
1404
1405 <section id='usingpoky-extend-addpkg-postinstalls'>
1406 <title>Post-Installation Scripts</title>
1407
1408 <para>
1409 To add a post-installation script to a package, add a
1410 <filename>pkg_postinst_PACKAGENAME()</filename> function to the
1411 <filename>.bb</filename> file and use
1412 <filename>PACKAGENAME</filename> as the name of the package you want to attach to the
1413 <filename>postinst</filename> script.
1414 Normally,
1415 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
1416 can be used, which automatically expands to <filename>PACKAGENAME</filename>.
1417 A post-installation function has the following structure:
1418 <literallayout class='monospaced'>
1419 pkg_postinst_PACKAGENAME () {
1420 #!/bin/sh -e
1421 # Commands to carry out
1422 }
1423 </literallayout>
1424 </para>
1425
1426 <para>
1427 The script defined in the post-installation function is called when the
1428 root filesystem is created.
1429 If the script succeeds, the package is marked as installed.
1430 If the script fails, the package is marked as unpacked and the script is
1431 executed when the image boots again.
1432 </para>
1433
1434 <para>
1435 Sometimes it is necessary for the execution of a post-installation
1436 script to be delayed until the first boot.
1437 For example, the script might need to be executed on the device itself.
1438 To delay script execution until boot time, use the following structure in the
1439 post-installation script:
1440 <literallayout class='monospaced'>
1441 pkg_postinst_PACKAGENAME () {
1442 #!/bin/sh -e
1443 if [ x"$D" = "x" ]; then
1444 # Actions to carry out on the device go here
1445 else
1446 exit 1
1447 fi
1448 }
1449 </literallayout>
1450 </para>
1451
1452 <para>
1453 The previous example delays execution until the image boots again because the
1454 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'>D</ulink></filename>
1455 variable points
1456 to the directory containing the image when the root filesystem is created at build time but
1457 is unset when executed on the first boot.
1458 </para>
1459 </section>
1460 </section>
1461
1462 <section id="platdev-newmachine">
1463 <title>Adding a New Machine</title>
1464
1465 <para>
1466 Adding a new machine to the Yocto Project is a straightforward process.
1467 This section provides information that gives you an idea of the changes you must make.
1468 The information covers adding machines similar to those the Yocto Project already supports.
1469 Although well within the capabilities of the Yocto Project, adding a totally new architecture
1470 might require
1471 changes to <filename>gcc/eglibc</filename> and to the site information, which is
1472 beyond the scope of this manual.
1473 </para>
1474
1475 <para>
1476 For a complete example that shows how to add a new machine,
1477 see the
1478 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
1479 in the Yocto Project Board Support Package (BSP) Developer's Guide.
1480 </para>
1481
1482 <section id="platdev-newmachine-conffile">
1483 <title>Adding the Machine Configuration File</title>
1484
1485 <para>
1486 To add a machine configuration, you need to add a <filename>.conf</filename> file
1487 with details of the device being added to the <filename>conf/machine/</filename> file.
1488 The name of the file determines the name the OpenEmbedded build system
1489 uses to reference the new machine.
1490 </para>
1491
1492 <para>
1493 The most important variables to set in this file are as follows:
1494 <itemizedlist>
1495 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename>
1496 (e.g. "arm")</para></listitem>
1497 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename>
1498 (see below)</para></listitem>
1499 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename>
1500 (e.g. "apm screen wifi")</para></listitem>
1501 </itemizedlist>
1502 </para>
1503
1504 <para>
1505 You might also need these variables:
1506 <itemizedlist>
1507 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename>
1508 (e.g. "115200 ttyS0")</para></listitem>
1509 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename>
1510 (e.g. "zImage")</para></listitem>
1511 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename>
1512 (e.g. "tar.gz jffs2")</para></listitem>
1513 </itemizedlist>
1514 </para>
1515
1516 <para>
1517 You can find full details on these variables in the reference section.
1518 You can leverage many existing machine <filename>.conf</filename> files from
1519 <filename>meta/conf/machine/</filename>.
1520 </para>
1521 </section>
1522
1523 <section id="platdev-newmachine-kernel">
1524 <title>Adding a Kernel for the Machine</title>
1525
1526 <para>
1527 The OpenEmbedded build system needs to be able to build a kernel for the machine.
1528 You need to either create a new kernel recipe for this machine, or extend an
1529 existing recipe.
1530 You can find several kernel examples in the
1531 Source Directory at <filename>meta/recipes-kernel/linux</filename>
1532 that you can use as references.
1533 </para>
1534
1535 <para>
1536 If you are creating a new recipe, normal recipe-writing rules apply for setting
1537 up a
1538 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
1539 Thus, you need to specify any necessary patches and set
1540 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> to point at the source code.
1541 You need to create a <filename>configure</filename> task that configures the
1542 unpacked kernel with a defconfig.
1543 You can do this by using a <filename>make defconfig</filename> command or,
1544 more commonly, by copying in a suitable <filename>defconfig</filename> file and and then running
1545 <filename>make oldconfig</filename>.
1546 By making use of <filename>inherit kernel</filename> and potentially some of the
1547 <filename>linux-*.inc</filename> files, most other functionality is
1548 centralized and the the defaults of the class normally work well.
1549 </para>
1550
1551 <para>
1552 If you are extending an existing kernel, it is usually a matter of adding a
1553 suitable defconfig file.
1554 The file needs to be added into a location similar to defconfig files
1555 used for other machines in a given kernel.
1556 A possible way to do this is by listing the file in the
1557 <filename>SRC_URI</filename> and adding the machine to the expression in
1558 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>:
1559 <literallayout class='monospaced'>
1560 COMPATIBLE_MACHINE = '(qemux86|qemumips)'
1561 </literallayout>
1562 </para>
1563 </section>
1564
1565 <section id="platdev-newmachine-formfactor">
1566 <title>Adding a Formfactor Configuration File</title>
1567
1568 <para>
1569 A formfactor configuration file provides information about the
1570 target hardware for which the image is being built and information that
1571 the build system cannot obtain from other sources such as the kernel.
1572 Some examples of information contained in a formfactor configuration file include
1573 framebuffer orientation, whether or not the system has a keyboard,
1574 the positioning of the keyboard in relation to the screen, and
1575 the screen resolution.
1576 </para>
1577
1578 <para>
1579 The build system uses reasonable defaults in most cases.
1580 However, if customization is
1581 necessary, you need to create a <filename>machconfig</filename> file
1582 in the <filename>meta/recipes-bsp/formfactor/files</filename>
1583 directory.
1584 This directory contains directories for specific machines such as
1585 <filename>qemuarm</filename> and <filename>qemux86</filename>.
1586 For information about the settings available and the defaults, see the
1587 <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the
1588 same area.
1589 </para>
1590
1591 <para>
1592 Following is an example for qemuarm:
1593 <literallayout class='monospaced'>
1594 HAVE_TOUCHSCREEN=1
1595 HAVE_KEYBOARD=1
1596
1597 DISPLAY_CAN_ROTATE=0
1598 DISPLAY_ORIENTATION=0
1599 #DISPLAY_WIDTH_PIXELS=640
1600 #DISPLAY_HEIGHT_PIXELS=480
1601 #DISPLAY_BPP=16
1602 DISPLAY_DPI=150
1603 DISPLAY_SUBPIXEL_ORDER=vrgb
1604 </literallayout>
1605 </para>
1606 </section>
1607 </section>
1608
1609 <section id="platdev-working-with-libraries">
1610 <title>Working With Libraries</title>
1611
1612 <para>
1613 Libraries are an integral part of your system.
1614 This section describes some common practices you might find
1615 helpful when working with libraries to build your system:
1616 <itemizedlist>
1617 <listitem><para><link linkend='including-static-library-files'>How to include static library files</link>
1618 </para></listitem>
1619 <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link>
1620 </para></listitem>
1621 <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link>
1622 </para></listitem>
1623 </itemizedlist>
1624 </para>
1625
1626 <section id='including-static-library-files'>
1627 <title>Including Static Library Files</title>
1628
1629 <para>
1630 If you are building a library and the library offers static linking, you can control
1631 which static library files (<filename>*.a</filename> files) get included in the
1632 built library.
1633 </para>
1634
1635 <para>
1636 The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
1637 and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink>
1638 variables in the
1639 <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed
1640 by the <filename>do_install</filename> task are packaged.
1641 By default, the <filename>PACKAGES</filename> variable contains
1642 <filename>${PN}-staticdev</filename>, which includes all static library files.
1643 <note>
1644 Some previously released versions of the Yocto Project
1645 defined the static library files through
1646 <filename>${PN}-dev</filename>.
1647 </note>
1648 Following, is part of the BitBake configuration file.
1649 You can see where the static library files are defined:
1650 <literallayout class='monospaced'>
1651 PACKAGES = "${PN}-dbg ${PN} ${PN}-doc ${PN}-dev ${PN}-staticdev ${PN}-locale"
1652 PACKAGES_DYNAMIC = "${PN}-locale-*"
1653 FILES = ""
1654
1655 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
1656 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
1657 ${base_bindir}/* ${base_sbindir}/* \
1658 ${base_libdir}/*${SOLIBS} \
1659 ${datadir}/${BPN} ${libdir}/${BPN}/* \
1660 ${datadir}/pixmaps ${datadir}/applications \
1661 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
1662 ${libdir}/bonobo/servers"
1663
1664 FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
1665 ${datadir}/gnome/help"
1666 SECTION_${PN}-doc = "doc"
1667
1668 FILES_${PN}-dev = "${includedir} ${libdir}/lib*${SOLIBSDEV} ${libdir}/*.la \
1669 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
1670 ${datadir}/aclocal ${base_libdir}/*.o"
1671 SECTION_${PN}-dev = "devel"
1672 ALLOW_EMPTY_${PN}-dev = "1"
1673 RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
1674
1675 FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a"
1676 SECTION_${PN}-staticdev = "devel"
1677 RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
1678 </literallayout>
1679 </para>
1680 </section>
1681
1682 <section id="combining-multiple-versions-library-files-into-one-image">
1683 <title>Combining Multiple Versions of Library Files into One Image</title>
1684
1685 <para>
1686 The build system offers the ability to build libraries with different
1687 target optimizations or architecture formats and combine these together
1688 into one system image.
1689 You can link different binaries in the image
1690 against the different libraries as needed for specific use cases.
1691 This feature is called "Multilib."
1692 </para>
1693
1694 <para>
1695 An example would be where you have most of a system compiled in 32-bit
1696 mode using 32-bit libraries, but you have something large, like a database
1697 engine, that needs to be a 64-bit application and uses 64-bit libraries.
1698 Multilib allows you to get the best of both 32-bit and 64-bit libraries.
1699 </para>
1700
1701 <para>
1702 While the Multilib feature is most commonly used for 32 and 64-bit differences,
1703 the approach the build system uses facilitates different target optimizations.
1704 You could compile some binaries to use one set of libraries and other binaries
1705 to use other different sets of libraries.
1706 The libraries could differ in architecture, compiler options, or other
1707 optimizations.
1708 </para>
1709
1710 <para>
1711 This section overviews the Multilib process only.
1712 For more details on how to implement Multilib, see the
1713 <ulink url='&YOCTO_WIKI_URL;/wiki/Multilib'>Multilib</ulink> wiki
1714 page.
1715 </para>
1716
1717 <para>
1718 Aside from this wiki page, several examples exist in the
1719 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/tree/meta-skeleton'><filename>meta-skeleton</filename></ulink>
1720 layer found in the
1721 <link linkend='source-directory'>Source Directory</link>:
1722 <itemizedlist>
1723 <listitem><para><filename>conf/multilib-example.conf</filename>
1724 configuration file</para></listitem>
1725 <listitem><para><filename>conf/multilib-example2.conf</filename>
1726 configuration file</para></listitem>
1727 <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename>
1728 recipe</para></listitem>
1729 </itemizedlist>
1730 </para>
1731
1732 <section id='preparing-to-use-multilib'>
1733 <title>Preparing to Use Multilib</title>
1734
1735 <para>
1736 User-specific requirements drive the Multilib feature.
1737 Consequently, there is no one "out-of-the-box" configuration that likely
1738 exists to meet your needs.
1739 </para>
1740
1741 <para>
1742 In order to enable Multilib, you first need to ensure your recipe is
1743 extended to support multiple libraries.
1744 Many standard recipes are already extended and support multiple libraries.
1745 You can check in the <filename>meta/conf/multilib.conf</filename>
1746 configuration file in the
1747 <link linkend='source-directory'>Source Directory</link> to see how this is
1748 done using the
1749 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink>
1750 variable.
1751 Eventually, all recipes will be covered and this list will be unneeded.
1752 </para>
1753
1754 <para>
1755 For the most part, the Multilib class extension works automatically to
1756 extend the package name from <filename>${PN}</filename> to
1757 <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename>
1758 is the particular multilib (e.g. "lib32-" or "lib64-").
1759 Standard variables such as
1760 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
1761 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
1762 <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>,
1763 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
1764 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>,
1765 and <filename>PACKAGES_DYNAMIC</filename> are automatically extended by the system.
1766 If you are extending any manual code in the recipe, you can use the
1767 <filename>${MLPREFIX}</filename> variable to ensure those names are extended
1768 correctly.
1769 This automatic extension code resides in <filename>multilib.bbclass</filename>.
1770 </para>
1771 </section>
1772
1773 <section id='using-multilib'>
1774 <title>Using Multilib</title>
1775
1776 <para>
1777 After you have set up the recipes, you need to define the actual
1778 combination of multiple libraries you want to build.
1779 You accomplish this through your <filename>local.conf</filename>
1780 configuration file in the
1781 <link linkend='build-directory'>Build Directory</link>.
1782 An example configuration would be as follows:
1783 <literallayout class='monospaced'>
1784 MACHINE = "qemux86-64"
1785 require conf/multilib.conf
1786 MULTILIBS = "multilib:lib32"
1787 DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
1788 IMAGE_INSTALL = "lib32-connman"
1789 </literallayout>
1790 This example enables an
1791 additional library named <filename>lib32</filename> alongside the
1792 normal target packages.
1793 When combining these "lib32" alternatives, the example uses "x86" for tuning.
1794 For information on this particular tuning, see
1795 <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>.
1796 </para>
1797
1798 <para>
1799 The example then includes <filename>lib32-connman</filename>
1800 in all the images, which illustrates one method of including a
1801 multiple library dependency.
1802 You can use a normal image build to include this dependency,
1803 for example:
1804 <literallayout class='monospaced'>
1805 $ bitbake core-image-sato
1806 </literallayout>
1807 You can also build Multilib packages specifically with a command like this:
1808 <literallayout class='monospaced'>
1809 $ bitbake lib32-connman
1810 </literallayout>
1811 </para>
1812 </section>
1813
1814 <section id='additional-implementation-details'>
1815 <title>Additional Implementation Details</title>
1816
1817 <para>
1818 Different packaging systems have different levels of native Multilib
1819 support.
1820 For the RPM Package Management System, the following implementation details
1821 exist:
1822 <itemizedlist>
1823 <listitem><para>A unique architecture is defined for the Multilib packages,
1824 along with creating a unique deploy folder under
1825 <filename>tmp/deploy/rpm</filename> in the
1826 <link linkend='build-directory'>Build Directory</link>.
1827 For example, consider <filename>lib32</filename> in a
1828 <filename>qemux86-64</filename> image.
1829 The possible architectures in the system are "all", "qemux86_64",
1830 "lib32_qemux86_64", and "lib32_x86".</para></listitem>
1831 <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from
1832 <filename>${PN}</filename> during RPM packaging.
1833 The naming for a normal RPM package and a Multilib RPM package in a
1834 <filename>qemux86-64</filename> system resolves to something similar to
1835 <filename>bash-4.1-r2.x86_64.rpm</filename> and
1836 <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively.
1837 </para></listitem>
1838 <listitem><para>When installing a Multilib image, the RPM backend first
1839 installs the base image and then installs the Multilib libraries.
1840 </para></listitem>
1841 <listitem><para>The build system relies on RPM to resolve the identical files in the
1842 two (or more) Multilib packages.</para></listitem>
1843 </itemizedlist>
1844 </para>
1845
1846 <para>
1847 For the IPK Package Management System, the following implementation details exist:
1848 <itemizedlist>
1849 <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from
1850 <filename>${PN}</filename> during IPK packaging.
1851 The naming for a normal RPM package and a Multilib IPK package in a
1852 <filename>qemux86-64</filename> system resolves to something like
1853 <filename>bash_4.1-r2.x86_64.ipk</filename> and
1854 <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively.
1855 </para></listitem>
1856 <listitem><para>The IPK deploy folder is not modified with
1857 <filename>${MLPREFIX}</filename> because packages with and without
1858 the Multilib feature can exist in the same folder due to the
1859 <filename>${PN}</filename> differences.</para></listitem>
1860 <listitem><para>IPK defines a sanity check for Multilib installation
1861 using certain rules for file comparison, overridden, etc.
1862 </para></listitem>
1863 </itemizedlist>
1864 </para>
1865 </section>
1866 </section>
1867
1868 <section id='installing-multiple-versions-of-the-same-library'>
1869 <title>Installing Multiple Versions of the Same Library</title>
1870
1871 <para>
1872 Situations can exist where you need to install and use
1873 multiple versions of the same library on the same system
1874 at the same time.
1875 These situations almost always exist when a library API
1876 changes and you have multiple pieces of software that
1877 depend on the separate versions of the library.
1878 To accommodate these situations, you can install multiple
1879 versions of the same library in parallel on the same system.
1880 </para>
1881
1882 <para>
1883 The process is straight forward as long as the libraries use
1884 proper versioning.
1885 With properly versioned libraries, all you need to do to
1886 individually specify the libraries is create separate,
1887 appropriately named recipes where the
1888 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the
1889 name includes a portion that differentiates each library version
1890 (e.g.the major part of the version number).
1891 Thus, instead of having a single recipe that loads one version
1892 of a library (e.g. <filename>clutter</filename>), you provide
1893 multiple recipes that result in different versions
1894 of the libraries you want.
1895 As an example, the following two recipes would allow the
1896 two separate versions of the <filename>clutter</filename>
1897 library to co-exist on the same system:
1898 <literallayout class='monospaced'>
1899 clutter-1.6_1.6.20.bb
1900 clutter-1.8_1.8.4.bb
1901 </literallayout>
1902 Additionally, if you have other recipes that depend on a given
1903 library, you need to use the
1904 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
1905 variable to create the dependency.
1906 Continuing with the same example, if you want to have a recipe
1907 depend on the 1.8 version of the <filename>clutter</filename>
1908 library, use the following in your recipe:
1909 <literallayout class='monospaced'>
1910 DEPENDS = "clutter-1.8"
1911 </literallayout>
1912 </para>
1913 </section>
1914 </section>
1915
1916 <section id='configuring-the-kernel'>
1917 <title>Configuring the Kernel</title>
1918
1919 <para>
1920 Configuring the Yocto Project kernel consists of making sure the <filename>.config</filename>
1921 file has all the right information in it for the image you are building.
1922 You can use the <filename>menuconfig</filename> tool and configuration fragments to
1923 make sure your <filename>.config</filename> file is just how you need it.
1924 This section describes how to use <filename>menuconfig</filename>, create and use
1925 configuration fragments, and how to interactively tweak your <filename>.config</filename>
1926 file to create the leanest kernel configuration file possible.
1927 </para>
1928
1929 <para>
1930 For more information on kernel configuration, see the
1931 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
1932 section in the Yocto Project Linux Kernel Development Manual.
1933 </para>
1934
1935 <section id='using-menuconfig'>
1936 <title>Using&nbsp;&nbsp;<filename>menuconfig</filename></title>
1937
1938 <para>
1939 The easiest way to define kernel configurations is to set them through the
1940 <filename>menuconfig</filename> tool.
1941 This tool provides an interactive method with which
1942 to set kernel configurations.
1943 For general information on <filename>menuconfig</filename>, see
1944 <ulink url='http://en.wikipedia.org/wiki/Menuconfig'></ulink>.
1945 </para>
1946
1947 <para>
1948 To use the <filename>menuconfig</filename> tool in the Yocto Project development
1949 environment, you must build the tool using BitBake.
1950 Thus, the environment must be set up using the
1951 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
1952 or
1953 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>
1954 script found in the
1955 <link linkend='build-directory'>Build Directory</link>.
1956 The following commands build and invoke <filename>menuconfig</filename> assuming the
1957 <link linkend='source-directory'>Source Directory</link>
1958 top-level folder is <filename>~/poky</filename>:
1959 <literallayout class='monospaced'>
1960 $ cd ~/poky
1961 $ source oe-init-build-env
1962 $ bitbake linux-yocto -c menuconfig
1963 </literallayout>
1964 Once <filename>menuconfig</filename> comes up, its standard interface allows you to
1965 interactively examine and configure all the kernel configuration parameters.
1966 After making your changes, simply exit the tool and save your changes to
1967 create an updated version of the <filename>.config</filename> configuration file.
1968 </para>
1969
1970 <para>
1971 Consider an example that configures the <filename>linux-yocto-3.4</filename>
1972 kernel.
1973 The OpenEmbedded build system recognizes this kernel as
1974 <filename>linux-yocto</filename>.
1975 Thus, the following commands from the shell in which you previously sourced the
1976 environment initialization script cleans the shared state cache and the
1977 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
1978 directory and then builds and launches <filename>menuconfig</filename>:
1979 <literallayout class='monospaced'>
1980 $ bitbake linux-yocto -c menuconfig
1981 </literallayout>
1982 </para>
1983
1984 <para>
1985 Once <filename>menuconfig</filename> launches, use the interface
1986 to navigate through the selections to find the configuration settings in
1987 which you are interested.
1988 For example, consider the <filename>CONFIG_SMP</filename> configuration setting.
1989 You can find it at <filename>Processor Type and Features</filename> under
1990 the configuration selection <filename>Symmetric Multi-processing Support</filename>.
1991 After highlighting the selection, use the arrow keys to select or deselect
1992 the setting.
1993 When you are finished with all your selections, exit out and save them.
1994 </para>
1995
1996 <para>
1997 Saving the selections updates the <filename>.config</filename> configuration file.
1998 This is the file that the OpenEmbedded build system uses to configure the
1999 kernel during the build.
2000 You can find and examine this file in the Build Directory in
2001 <filename>tmp/work/</filename>.
2002 The actual <filename>.config</filename> is located in the area where the
2003 specific kernel is built.
2004 For example, if you were building a Linux Yocto kernel based on the
2005 Linux 3.4 kernel and you were building a QEMU image targeted for
2006 <filename>x86</filename> architecture, the
2007 <filename>.config</filename> file would be located here:
2008 <literallayout class='monospaced'>
2009 ~/poky/build/tmp/work/qemux86-poky-linux/linux-yocto-3.4.11+git1+84f...
2010 ...656ed30-r1/linux-qemux86-standard-build
2011 </literallayout>
2012 <note>
2013 The previous example directory is artificially split and many of the characters
2014 in the actual filename are omitted in order to make it more readable.
2015 Also, depending on the kernel you are using, the exact pathname
2016 for <filename>linux-yocto-3.4...</filename> might differ.
2017 </note>
2018 </para>
2019
2020 <para>
2021 Within the <filename>.config</filename> file, you can see the kernel settings.
2022 For example, the following entry shows that symmetric multi-processor support
2023 is not set:
2024 <literallayout class='monospaced'>
2025 # CONFIG_SMP is not set
2026 </literallayout>
2027 </para>
2028
2029 <para>
2030 A good method to isolate changed configurations is to use a combination of the
2031 <filename>menuconfig</filename> tool and simple shell commands.
2032 Before changing configurations with <filename>menuconfig</filename>, copy the
2033 existing <filename>.config</filename> and rename it to something else,
2034 use <filename>menuconfig</filename> to make
2035 as many changes an you want and save them, then compare the renamed configuration
2036 file against the newly created file.
2037 You can use the resulting differences as your base to create configuration fragments
2038 to permanently save in your kernel layer.
2039 <note>
2040 Be sure to make a copy of the <filename>.config</filename> and don't just
2041 rename it.
2042 The build system needs an existing <filename>.config</filename>
2043 from which to work.
2044 </note>
2045 </para>
2046 </section>
2047
2048 <section id='creating-config-fragments'>
2049 <title>Creating Configuration Fragments</title>
2050
2051 <para>
2052 Configuration fragments are simply kernel options that appear in a file
2053 placed where the OpenEmbedded build system can find and apply them.
2054 Syntactically, the configuration statement is identical to what would appear
2055 in the <filename>.config</filename> file, which is in the
2056 <link linkend='build-directory'>Build Directory</link> in
2057 <filename>tmp/work/&lt;arch&gt;-poky-linux/linux-yocto-&lt;release-specific-string&gt;/linux-&lt;arch&gt;-&lt;build-type&gt;</filename>.
2058 </para>
2059
2060 <para>
2061 It is simple to create a configuration fragment.
2062 For example, issuing the following from the shell creates a configuration fragment
2063 file named <filename>my_smp.cfg</filename> that enables multi-processor support
2064 within the kernel:
2065 <literallayout class='monospaced'>
2066 $ echo "CONFIG_SMP=y" >> my_smp.cfg
2067 </literallayout>
2068 <note>
2069 All configuration files must use the <filename>.cfg</filename> extension in order
2070 for the OpenEmbedded build system to recognize them as a configuration fragment.
2071 </note>
2072 </para>
2073
2074 <para>
2075 Where do you put your configuration files?
2076 You can place these configuration files in the same area pointed to by
2077 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
2078 The OpenEmbedded build system will pick up the configuration and add it to the
2079 kernel's configuration.
2080 For example, suppose you had a set of configuration options in a file called
2081 <filename>myconfig.cfg</filename>.
2082 If you put that file inside a directory named <filename>/linux-yocto</filename>
2083 that resides in the same directory as the kernel's append file and then add
2084 a <filename>SRC_URI</filename> statement such as the following to the kernel's append file,
2085 those configuration options will be picked up and applied when the kernel is built.
2086 <literallayout class='monospaced'>
2087 SRC_URI += "file://myconfig.cfg"
2088 </literallayout>
2089 </para>
2090
2091 <para>
2092 As mentioned earlier, you can group related configurations into multiple files and
2093 name them all in the <filename>SRC_URI</filename> statement as well.
2094 For example, you could group separate configurations specifically for Ethernet and graphics
2095 into their own files and add those by using a <filename>SRC_URI</filename> statement like the
2096 following in your append file:
2097 <literallayout class='monospaced'>
2098 SRC_URI += "file://myconfig.cfg \
2099 file://eth.cfg \
2100 file://gfx.cfg"
2101 </literallayout>
2102 </para>
2103 </section>
2104
2105 <section id='fine-tuning-the-kernel-configuration-file'>
2106 <title>Fine-Tuning the Kernel Configuration File</title>
2107
2108 <para>
2109 You can make sure the <filename>.config</filename> file is as lean or efficient as
2110 possible by reading the output of the kernel configuration fragment audit,
2111 noting any issues, making changes to correct the issues, and then repeating.
2112 </para>
2113
2114 <para>
2115 As part of the kernel build process, the
2116 <filename>kernel_configcheck</filename> task runs.
2117 This task validates the kernel configuration by checking the final
2118 <filename>.config</filename> file against the input files.
2119 During the check, the task produces warning messages for the following
2120 issues:
2121 <itemizedlist>
2122 <listitem><para>Requested options that did not make the final
2123 <filename>.config</filename> file.</para></listitem>
2124 <listitem><para>Configuration items that appear twice in the same
2125 configuration fragment.</para></listitem>
2126 <listitem><para>Configuration items tagged as "required" were overridden.
2127 </para></listitem>
2128 <listitem><para>A board overrides a non-board specific option.</para></listitem>
2129 <listitem><para>Listed options not valid for the kernel being processed.
2130 In other words, the option does not appear anywhere.</para></listitem>
2131 </itemizedlist>
2132 <note>
2133 The <filename>kernel_configcheck</filename> task can also optionally report
2134 if an option is overridden during processing.
2135 </note>
2136 </para>
2137
2138 <para>
2139 For each output warning, a message points to the file
2140 that contains a list of the options and a pointer to the config
2141 fragment that defines them.
2142 Collectively, the files are the key to streamlining the configuration.
2143 </para>
2144
2145 <para>
2146 To streamline the configuration, do the following:
2147 <orderedlist>
2148 <listitem><para>Start with a full configuration that you know
2149 works - it builds and boots successfully.
2150 This configuration file will be your baseline.</para></listitem>
2151 <listitem><para>Separately run the <filename>configme</filename> and
2152 <filename>kernel_configcheck</filename> tasks.</para></listitem>
2153 <listitem><para>Take the resulting list of files from the
2154 <filename>kernel_configcheck</filename> task warnings and do the following:
2155 <itemizedlist>
2156 <listitem><para>Drop values that are redefined in the fragment but do not
2157 change the final <filename>.config</filename> file.</para></listitem>
2158 <listitem><para>Analyze and potentially drop values from the
2159 <filename>.config</filename> file that override required
2160 configurations.</para></listitem>
2161 <listitem><para>Analyze and potentially remove non-board specific options.
2162 </para></listitem>
2163 <listitem><para>Remove repeated and invalid options.</para></listitem>
2164 </itemizedlist></para></listitem>
2165 <listitem><para>After you have worked through the output of the kernel configuration
2166 audit, you can re-run the <filename>configme</filename>
2167 and <filename>kernel_configcheck</filename> tasks to see the results of your
2168 changes.
2169 If you have more issues, you can deal with them as described in the
2170 previous step.</para></listitem>
2171 </orderedlist>
2172 </para>
2173
2174 <para>
2175 Iteratively working through steps two through four eventually yields
2176 a minimal, streamlined configuration file.
2177 Once you have the best <filename>.config</filename>, you can build the Linux
2178 Yocto kernel.
2179 </para>
2180 </section>
2181 </section>
2182
2183 <section id="patching-the-kernel">
2184 <title>Patching the Kernel</title>
2185
2186 <para>
2187 Patching the kernel involves changing or adding configurations to an existing kernel,
2188 changing or adding recipes to the kernel that are needed to support specific hardware features,
2189 or even altering the source code itself.
2190 <note>
2191 You can use the <filename>yocto-kernel</filename> script
2192 found in the <link linkend='source-directory'>Source Directory</link>
2193 under <filename>scripts</filename> to manage kernel patches and configuration.
2194 See the "<ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'>Managing kernel Patches and Config Items with yocto-kernel</ulink>"
2195 section in the Yocto Project Board Support Packages (BSP) Developer's Guide for
2196 more information.</note>
2197 </para>
2198
2199 <para>
2200 This example creates a simple patch by adding some QEMU emulator console
2201 output at boot time through <filename>printk</filename> statements in the kernel's
2202 <filename>calibrate.c</filename> source code file.
2203 Applying the patch and booting the modified image causes the added
2204 messages to appear on the emulator's console.
2205 </para>
2206
2207 <para>
2208 The example assumes a clean build exists for the <filename>qemux86</filename>
2209 machine in a Source Directory named <filename>poky</filename>.
2210 Furthermore, the <link linkend='build-directory'>Build Directory</link> is
2211 <filename>build</filename> and is located in <filename>poky</filename> and
2212 the kernel is based on the Linux 3.4 kernel.
2213 For general information on how to configure the most efficient build, see the
2214 "<ulink url='&YOCTO_DOCS_QS_URL;#building-image'>Building an Image</ulink>" section
2215 in the Yocto Project Quick Start.
2216 </para>
2217
2218 <para>
2219 Also, for more information on patching the kernel, see the
2220 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#applying-patches'>Applying Patches</ulink>"
2221 section in the Yocto Project Linux Kernel Development Manual.
2222 </para>
2223
2224 <section id='create-a-layer-for-your-changes'>
2225 <title>Create a Layer for your Changes</title>
2226
2227 <para>
2228 The first step is to create a layer so you can isolate your changes:
2229 <literallayout class='monospaced'>
2230 $cd ~/poky
2231 $mkdir meta-mylayer
2232 </literallayout>
2233 Creating a directory that follows the Yocto Project layer naming
2234 conventions sets up the layer for your changes.
2235 The layer is where you place your configuration files, append
2236 files, and patch files.
2237 To learn more about creating a layer and filling it with the
2238 files you need, see the "<link linkend='understanding-and-creating-layers'>Understanding
2239 and Creating Layers</link>" section.
2240 </para>
2241 </section>
2242
2243 <section id='finding-the-kernel-source-code'>
2244 <title>Finding the Kernel Source Code</title>
2245
2246 <para>
2247 Each time you build a kernel image, the kernel source code is fetched
2248 and unpacked into the following directory:
2249 <literallayout class='monospaced'>
2250 ${S}/linux
2251 </literallayout>
2252 See the "<link linkend='finding-the-temporary-source-code'>Finding the Temporary Source Code</link>"
2253 section and the
2254 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> variable
2255 for more information about where source is kept during a build.
2256 </para>
2257
2258 <para>
2259 For this example, we are going to patch the
2260 <filename>init/calibrate.c</filename> file
2261 by adding some simple console <filename>printk</filename> statements that we can
2262 see when we boot the image using QEMU.
2263 </para>
2264 </section>
2265
2266 <section id='creating-the-patch'>
2267 <title>Creating the Patch</title>
2268
2269 <para>
2270 Two methods exist by which you can create the patch:
2271 <link linkend='using-a-git-workflow'>Git workflow</link> and
2272 <link linkend='using-a-quilt-workflow'>Quilt workflow</link>.
2273 For kernel patches, the Git workflow is more appropriate.
2274 This section assumes the Git workflow and shows the steps specific to
2275 this example.
2276 <orderedlist>
2277 <listitem><para><emphasis>Change the working directory</emphasis>:
2278 Change to where the kernel source code is before making
2279 your edits to the <filename>calibrate.c</filename> file:
2280 <literallayout class='monospaced'>
2281 $ cd ~/poky/build/tmp/work/qemux86-poky-linux/linux-yocto-${PV}-${PR}/linux
2282 </literallayout>
2283 Because you are working in an established Git repository,
2284 you must be in this directory in order to commit your changes
2285 and create the patch file.
2286 <note>The <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> and
2287 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> variables
2288 represent the version and revision for the
2289 <filename>linux-yocto</filename> recipe.
2290 The <filename>PV</filename> variable includes the Git meta and machine
2291 hashes, which make the directory name longer than you might
2292 expect.
2293 </note></para></listitem>
2294 <listitem><para><emphasis>Edit the source file</emphasis>:
2295 Edit the <filename>init/calibrate.c</filename> file to have the
2296 following changes:
2297 <literallayout class='monospaced'>
2298 void __cpuinit calibrate_delay(void)
2299 {
2300 unsigned long lpj;
2301 static bool printed;
2302 int this_cpu = smp_processor_id();
2303
2304 printk("*************************************\n");
2305 printk("* *\n");
2306 printk("* HELLO YOCTO KERNEL *\n");
2307 printk("* *\n");
2308 printk("*************************************\n");
2309
2310 if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
2311 .
2312 .
2313 .
2314 </literallayout></para></listitem>
2315 <listitem><para><emphasis>Stage and commit your changes</emphasis>:
2316 These Git commands display the modified file, stage it, and then
2317 commit the file:
2318 <literallayout class='monospaced'>
2319 $ git status
2320 $ git add init/calibrate.c
2321 $ git commit -m "calibrate: Add printk example"
2322 </literallayout></para></listitem>
2323 <listitem><para><emphasis>Generate the patch file</emphasis>:
2324 This Git command creates the a patch file named
2325 <filename>0001-calibrate-Add-printk-example.patch</filename>
2326 in the current directory.
2327 <literallayout class='monospaced'>
2328 $ git format-patch -1
2329 </literallayout>
2330 </para></listitem>
2331 </orderedlist>
2332 </para>
2333 </section>
2334
2335 <section id='set-up-your-layer-for-the-build'>
2336 <title>Set Up Your Layer for the Build</title>
2337
2338 <para>These steps get your layer set up for the build:
2339 <orderedlist>
2340 <listitem><para><emphasis>Create additional structure</emphasis>:
2341 Create the additional layer structure:
2342 <literallayout class='monospaced'>
2343 $ cd ~/poky/meta-mylayer
2344 $ mkdir conf
2345 $ mkdir recipes-kernel
2346 $ mkdir recipes-kernel/linux
2347 $ mkdir recipes-kernel/linux/linux-yocto
2348 </literallayout>
2349 The <filename>conf</filename> directory holds your configuration files, while the
2350 <filename>recipes-kernel</filename> directory holds your append file and
2351 your patch file.</para></listitem>
2352 <listitem><para><emphasis>Create the layer configuration file</emphasis>:
2353 Move to the <filename>meta-mylayer/conf</filename> directory and create
2354 the <filename>layer.conf</filename> file as follows:
2355 <literallayout class='monospaced'>
2356 # We have a conf and classes directory, add to BBPATH
2357 BBPATH .= ":${LAYERDIR}"
2358
2359 # We have recipes-* directories, add to BBFILES
2360 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
2361 ${LAYERDIR}/recipes-*/*/*.bbappend"
2362
2363 BBFILE_COLLECTIONS += "mylayer"
2364 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/"
2365 BBFILE_PRIORITY_mylayer = "5"
2366 </literallayout>
2367 Notice <filename>mylayer</filename> as part of the last three
2368 statements.</para></listitem>
2369 <listitem><para><emphasis>Create the kernel recipe append file</emphasis>:
2370 Move to the <filename>meta-mylayer/recipes-kernel/linux</filename> directory and create
2371 the <filename>linux-yocto_3.4.bbappend</filename> file as follows:
2372 <literallayout class='monospaced'>
2373 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
2374
2375 SRC_URI += "file://0001-calibrate-Add-printk-example.patch"
2376
2377 PRINC := "${@int(PRINC) + 1}"
2378 </literallayout>
2379 The <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
2380 and <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
2381 statements enable the OpenEmbedded build system to find the patch file.
2382 For more information on using append files, see the
2383 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>"
2384 section.
2385 </para></listitem>
2386 <listitem><para><emphasis>Put the patch file in your layer</emphasis>:
2387 Move the <filename>0001-calibrate-Add-printk-example.patch</filename> file to
2388 the <filename>meta-mylayer/recipes-kernel/linux/linux-yocto</filename>
2389 directory.</para></listitem>
2390 </orderedlist>
2391 </para>
2392 </section>
2393
2394 <section id='set-up-for-the-build'>
2395 <title>Set Up for the Build</title>
2396
2397 <para>
2398 Do the following to make sure the build parameters are set up for the example.
2399 Once you set up these build parameters, they do not have to change unless you
2400 change the target architecture of the machine you are building:
2401 <itemizedlist>
2402 <listitem><para><emphasis>Build for the correct target architecture:</emphasis> Your
2403 selected <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
2404 definition within the <filename>local.conf</filename> file in the
2405 <link linkend='build-directory'>Build Directory</link>
2406 specifies the target architecture used when building the Linux kernel.
2407 By default, <filename>MACHINE</filename> is set to
2408 <filename>qemux86</filename>, which specifies a 32-bit
2409 <trademark class='registered'>Intel</trademark> Architecture
2410 target machine suitable for the QEMU emulator.</para></listitem>
2411 <listitem><para><emphasis>Identify your <filename>meta-mylayer</filename>
2412 layer:</emphasis> The
2413 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink>
2414 variable in the
2415 <filename>bblayers.conf</filename> file found in the
2416 <filename>poky/build/conf</filename> directory needs to have the path to your local
2417 <filename>meta-mylayer</filename> layer.
2418 By default, the <filename>BBLAYERS</filename> variable contains paths to
2419 <filename>meta</filename>, <filename>meta-yocto</filename>, and
2420 <filename>meta-yocto-bsp</filename> in the
2421 <filename>poky</filename> Git repository.
2422 Add the path to your <filename>meta-mylayer</filename> location:
2423 <literallayout class='monospaced'>
2424 BBLAYERS ?= " \
2425 $HOME/poky/meta \
2426 $HOME/poky/meta-yocto \
2427 $HOME/poky/meta-yocto-bsp \
2428 $HOME/poky/meta-mylayer \
2429 "
2430
2431 BBLAYERS_NON_REMOVABLE ?= " \
2432 $HOME/poky/meta \
2433 $HOME/poky/meta-yocto \
2434 "
2435 </literallayout></para></listitem>
2436 </itemizedlist>
2437 </para>
2438 </section>
2439
2440 <section id='build-the-modified-qemu-kernel-image'>
2441 <title>Build the Modified QEMU Kernel Image</title>
2442
2443 <para>
2444 The following steps build your modified kernel image:
2445 <orderedlist>
2446 <listitem><para><emphasis>Be sure your build environment is initialized</emphasis>:
2447 Your environment should be set up since you previously sourced
2448 the
2449 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
2450 script.
2451 If it is not, source the script again from <filename>poky</filename>.
2452 <literallayout class='monospaced'>
2453 $ cd ~/poky
2454 $ source &OE_INIT_FILE;
2455 </literallayout>
2456 </para></listitem>
2457 <listitem><para><emphasis>Clean up</emphasis>:
2458 Be sure to clean the shared state out by running the
2459 <filename>cleansstate</filename> BitBake task as follows from your Build Directory:
2460 <literallayout class='monospaced'>
2461 $ bitbake -c cleansstate linux-yocto
2462 </literallayout></para>
2463 <para><note>Never remove any files by hand from the <filename>tmp/deploy</filename>
2464 directory inside the
2465 <link linkend='build-directory'>Build Directory</link>.
2466 Always use the various BitBake clean tasks to clear out previous
2467 build artifacts.
2468 </note></para></listitem>
2469 <listitem><para><emphasis>Build the image</emphasis>:
2470 Next, build the kernel image using this command:
2471 <literallayout class='monospaced'>
2472 $ bitbake -k linux-yocto
2473 </literallayout></para></listitem>
2474 </orderedlist>
2475 </para>
2476 </section>
2477
2478 <section id='boot-the-image-and-verify-your-changes'>
2479 <title>Boot the Image and Verify Your Changes</title>
2480
2481 <para>
2482 These steps boot the image and allow you to see the changes
2483 <orderedlist>
2484 <listitem><para><emphasis>Boot the image</emphasis>:
2485 Boot the modified image in the QEMU emulator
2486 using this command:
2487 <literallayout class='monospaced'>
2488 $ runqemu qemux86
2489 </literallayout></para></listitem>
2490 <listitem><para><emphasis>Verify the changes</emphasis>:
2491 Log into the machine using <filename>root</filename> with no password and then
2492 use the following shell command to scroll through the console's boot output.
2493 <literallayout class='monospaced'>
2494 # dmesg | less
2495 </literallayout>
2496 You should see the results of your <filename>printk</filename> statements
2497 as part of the output.</para></listitem>
2498 </orderedlist>
2499 </para>
2500 </section>
2501 </section>
2502
2503 <section id='creating-your-own-distribution'>
2504 <title>Creating Your Own Distribution</title>
2505
2506 <para>
2507 When you build an image using the Yocto Project and
2508 do not alter any distribution
2509 <link linkend='metadata'>Metadata</link>, you are creating a
2510 Poky distribution.
2511 If you wish to gain more control over package alternative
2512 selections, compile-time options, and other low-level
2513 configurations, you can create your own distribution.
2514 </para>
2515
2516 <para>
2517 To create your own distribution, the basic steps consist of
2518 creating your own distribution layer, creating your own
2519 distribution configuration file, and then adding any needed
2520 code and Metadata to the layer.
2521 The following steps provide some more detail:
2522 <itemizedlist>
2523 <listitem><para><emphasis>Create a layer for your new distro:</emphasis>
2524 Create your distribution layer so that you can keep your
2525 Metadata and code for the distribution separate.
2526 It is strongly recommended that you create and use your own
2527 layer for configuration and code.
2528 Using your own layer as compared to just placing
2529 configurations in a <filename>local.conf</filename>
2530 configuration file makes it easier to reproduce the same
2531 build configuration when using multiple build machines.
2532 See the
2533 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
2534 section for information on how to quickly set up a layer.
2535 </para></listitem>
2536 <listitem><para><emphasis>Create the distribution configuration file:</emphasis>
2537 The distribution configuration file needs to be created in
2538 the <filename>conf/distro</filename> directory of your
2539 layer.
2540 You need to name it using your distribution name
2541 (e.g. <filename>mydistro.conf</filename>).</para>
2542 <para>You can split out parts of your configuration file
2543 into include files and then "require" them from within
2544 your distribution configuration file.
2545 Be sure to place the include files in the
2546 <filename>conf/distro/include</filename> directory of
2547 your layer.
2548 A common example usage of include files would be to
2549 separate out the selection of desired version and revisions
2550 for individual recipes.
2551</para>
2552 <para>Your configuration file needs to set the following
2553 required variables:
2554 <literallayout class='monospaced'>
2555 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink> [required]
2556 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink> [required]
2557 </literallayout>
2558 These following variables are optional and you typically
2559 set them from the distribution configuration file:
2560 <literallayout class='monospaced'>
2561 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> [optional]
2562 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink> [optional]
2563 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink> [optional]
2564 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink> [optional]
2565 </literallayout>
2566 <tip>
2567 If you want to base your distribution configuration file
2568 on the very basic configuration from OE-Core, you
2569 can use
2570 <filename>conf/distro/defaultsetup.conf</filename> as
2571 a reference and just include variables that differ
2572 as compared to <filename>defaultsetup.conf</filename>.
2573 Alternatively, you can create a distribution
2574 configuration file from scratch using the
2575 <filename>defaultsetup.conf</filename> file
2576 or configuration files from other distributions
2577 such as Poky or Angstrom as references.
2578 </tip></para></listitem>
2579 <listitem><para><emphasis>Provide miscellaneous variables:</emphasis>
2580 Be sure to define any other variables for which you want to
2581 create a default or enforce as part of the distribution
2582 configuration.
2583 You can include nearly any variable from the
2584 <filename>local.conf</filename> file.
2585 The variables you use are not limited to the list in the
2586 previous bulleted item.</para></listitem>
2587 <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis>
2588 In your <filename>local.conf</filename> file in the
2589 <link linkend='build-directory'>Build Directory</link>,
2590 set your
2591 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
2592 variable to point to your distribution's configuration file.
2593 For example, if your distribution's configuration file is
2594 named <filename>mydistro.conf</filename>, then you point
2595 to it as follows:
2596 <literallayout class='monospaced'>
2597 DISTRO = "mydistro"
2598 </literallayout></para></listitem>
2599 <listitem><para><emphasis>Add more to the layer if necessary:</emphasis>
2600 Use your layer to hold other information needed for the
2601 distribution:
2602 <itemizedlist>
2603 <listitem><para>Add recipes for installing
2604 distro-specific configuration files that are not
2605 already installed by another recipe.
2606 If you have distro-specific configuration files
2607 that are included by an existing recipe, you should
2608 add a <filename>.bbappend</filename> for those.
2609 For general information and recommendations
2610 on how to add recipes to your layer, see the
2611 "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>"
2612 and
2613 "<link linkend='best-practices-to-follow-when-creating-layers'>Best Practices to Follow When Creating Layers</link>"
2614 sections.</para></listitem>
2615 <listitem><para>Add any image recipes that are specific
2616 to your distribution.</para></listitem>
2617 <listitem><para>Add a <filename>psplash</filename>
2618 append file for a branded splash screen.
2619 For information on append files, see the
2620 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>"
2621 section.</para></listitem>
2622 <listitem><para>Add any other append files to make
2623 custom changes that are specific to individual
2624 recipes.</para></listitem>
2625 </itemizedlist></para></listitem>
2626 </itemizedlist>
2627 </para>
2628 </section>
2629
2630 <section id='building-a-tiny-system'>
2631 <title>Building a Tiny System</title>
2632
2633 <para>
2634 Very small distributions have some significant advantages such
2635 as requiring less on-die or in-package memory (cheaper), better
2636 performance through efficient cache usage, lower power requirements
2637 due to less memory, faster boot times, and reduced development
2638 overhead.
2639 Some real-world examples where a very small distribution gives
2640 you distinct advantages are digital cameras, medical devices,
2641 and small headless systems.
2642 </para>
2643
2644 <para>
2645 This section presents information that shows you how you can
2646 trim your distribution to even smaller sizes than the
2647 <filename>poky-tiny</filename> distribution, which is around
2648 5 Mbytes, that can be built out-of-the-box using the Yocto Project.
2649 </para>
2650
2651 <section id='tiny-system-overview'>
2652 <title>Overview</title>
2653
2654 <para>
2655 The following list presents the overall steps you need to
2656 consider and perform to create distributions with smaller
2657 root filesystems, faster boot times, maintain your critical
2658 functionality, and avoid initial RAM disks:
2659 <itemizedlist>
2660 <listitem><para>Determine your goals and guiding
2661 principles.</para></listitem>
2662 <listitem><para>Understand what gives your image size.
2663 </para></listitem>
2664 <listitem><para>Reduce the size of the root filesystem.
2665 </para></listitem>
2666 <listitem><para>Reduce the size of the kernel.
2667 </para></listitem>
2668 <listitem><para>Eliminate packaging requirements.
2669 </para></listitem>
2670 <listitem><para>Look for other ways to minimize size.
2671 </para></listitem>
2672 <listitem><para>Iterate on the process.</para></listitem>
2673 </itemizedlist>
2674 </para>
2675 </section>
2676
2677 <section id='goals-and-guiding-principles'>
2678 <title>Goals and Guiding Principles</title>
2679
2680 <para>
2681 Before you can reach your destination, you need to know
2682 where you are going.
2683 Here is an example list that you can use as a guide when
2684 creating very small distributions:
2685 <itemizedlist>
2686 <listitem><para>Determine how much space you need
2687 (e.g. a kernel that is 1 Mbyte or less and
2688 a root filesystem that is 3 Mbytes or less).
2689 </para></listitem>
2690 <listitem><para>Find the areas that are currently
2691 taking 90% of the space and concentrate on reducing
2692 those areas.
2693 </para></listitem>
2694 <listitem><para>Do not create any difficult "hacks"
2695 to achieve your goals.</para></listitem>
2696 <listitem><para>Leverage the device-specific
2697 options.</para></listitem>
2698 <listitem><para>Work in a separate layer so that you
2699 keep changes isolated.
2700 For information on how to create layers, see
2701 the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section.
2702 </para></listitem>
2703 </itemizedlist>
2704 </para>
2705 </section>
2706
2707 <section id='understand-what-gives-your-image-size'>
2708 <title>Understand What Gives Your Image Size</title>
2709
2710 <para>
2711 It is easiest to have something to start with when creating
2712 your own distribution.
2713 You can use the Yocto Project out-of-the-box to create the
2714 <filename>poky-tiny</filename> distribution.
2715 Ultimately, you will want to make changes in your own
2716 distribution that are likely modeled after
2717 <filename>poky-tiny</filename>.
2718 <note>
2719 To use <filename>poky-tiny</filename> in your build,
2720 set the
2721 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
2722 variable in your
2723 <filename>local.conf</filename> file to "poky-tiny"
2724 as described in the
2725 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
2726 section.
2727 </note>
2728 </para>
2729
2730 <para>
2731 Understanding some memory concepts will help you reduce the
2732 system size.
2733 Memory consists of static, dynamic, and temporary memory.
2734 Static memory is the TEXT (code), DATA (initialized data
2735 in the code), and BSS (uninitialized data) sections.
2736 Dynamic memory contains memory that is allocated at runtime,
2737 stacks, hash tables, and so forth.
2738 Temporary memory is recovered after the boot process.
2739 This memory consists of memory used for decompressing
2740 the kernel and for the <filename>__init__</filename>
2741 functions.
2742 </para>
2743
2744 <para>
2745 To help you see where you currently are with kernel and root
2746 filesystem sizes, you can use two tools found in the
2747 <link linkend='source-directory'>Source Directory</link> in
2748 the <filename>scripts</filename> directory:
2749 <itemizedlist>
2750 <listitem><para><filename>ksize.py</filename>: Reports
2751 component sizes for the kernel build objects.
2752 </para></listitem>
2753 <listitem><para><filename>dirsize.py</filename>: Reports
2754 component sizes for the root filesystem.</para></listitem>
2755 </itemizedlist>
2756 This next tool and command helps you organize configuration
2757 fragments and view file dependencies in a human-readable form:
2758 <itemizedlist>
2759 <listitem><para><filename>merge_config.sh</filename>:
2760 Helps you manage configuration files and fragments
2761 within the kernel.
2762 With this tool, you can merge individual configuration
2763 fragments together.
2764 The tool allows you to make overrides and warns you
2765 of any missing configuration options.
2766 The tool is ideal for allowing you to iterate on
2767 configurations, create minimal configurations, and
2768 create configuration files for different machines
2769 without having to duplicate your process.</para>
2770 <para>The <filename>merge_config.sh</filename> script is
2771 part of the Linux Yocto kernel Git repository in the
2772 <filename>scripts/kconfig</filename> directory.</para>
2773 <para>For more information on configuration fragments,
2774 see the
2775 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>"
2776 section of the Yocto Project Linux Kernel Development
2777 Manual and the "<link linkend='creating-config-fragments'>Creating Configuration Fragments</link>"
2778 section, which is in this manual.</para></listitem>
2779 <listitem><para><filename>bitbake -u depexp -g &lt;bitbake_target&gt;</filename>:
2780 Using the BitBake command with these options brings up
2781 a Dependency Explorer from which you can view file
2782 dependencies.
2783 Understanding these dependencies allows you to make
2784 informed decisions when cutting out various pieces of the
2785 kernel and root filesystem.</para></listitem>
2786 </itemizedlist>
2787 </para>
2788 </section>
2789
2790 <section id='trim-the-root-filesystem'>
2791 <title>Trim the Root Filesystem</title>
2792
2793 <para>
2794 The root filesystem is made up of packages for booting,
2795 libraries, and applications.
2796 To change things, you can configure how the packaging happens,
2797 which changes the way you build them.
2798 You can also tweak the filesystem itself or select a different
2799 filesystem.
2800 </para>
2801
2802 <para>
2803 First, find out what is hogging your root filesystem by running the
2804 <filename>dirsize.py</filename> script from your root directory:
2805 <literallayout class='monospaced'>
2806 $ cd &lt;root-directory-of-image&gt;
2807 $ dirsize.py 100000 > dirsize-100k.log
2808 $ cat dirsize-100k.log
2809 </literallayout>
2810 You can apply a filter to the script to ignore files under
2811 a certain size.
2812 This example filters out anything below 100 Kbytes.
2813 The sizes reported by the tool are uncompressed and thus,
2814 will be smaller by a relatively constant factor in a
2815 compressed root filesystem.
2816 When you examine your log file, you can focus on areas of the
2817 root filesystem that take up large amounts of memory.
2818 </para>
2819
2820 <para>
2821 You need to be sure that what you eliminate does not cripple
2822 the functionality you need.
2823 One way to see how packages relate to each other is by using
2824 the Dependency Explorer UI with the BitBake command:
2825 <literallayout class='monospaced'>
2826 $ cd &lt;image-directory&gt;
2827 $ bitbake -u depexp -g &lt;image&gt;
2828 </literallayout>
2829 Use the interface to select potential packages you wish to
2830 eliminate and see their dependency relationships.
2831 </para>
2832
2833 <para>
2834 When deciding how to reduce the size, get rid of packages that
2835 result in minimal impact on the feature set.
2836 For example, you might not need a VGA display.
2837 Or, you might be able to get by with <filename>devtmpfs</filename>
2838 and <filename>mdev</filename> instead of
2839 <filename>udev</filename>.
2840 </para>
2841
2842 <para>
2843 Use the <filename>local.conf</filename> file to make changes.
2844 For example, to eliminate <filename>udev</filename> and
2845 <filename>glib</filename>, set the following in the
2846 local configuration file:
2847 <literallayout class='monospaced'>
2848 VIRTUAL-RUNTIME_dev_manager = ""
2849 </literallayout>
2850 </para>
2851
2852 <para>
2853 Finally, you should consider exactly the type of root
2854 filesystem you need to meet your needs while also reducing
2855 its size.
2856 For example, consider <filename>cramfs</filename>,
2857 <filename>squashfs</filename>, <filename>ubifs</filename>,
2858 <filename>ext2</filename>, or an <filename>initramfs</filename>
2859 using <filename>initramfs</filename>.
2860 Be aware that <filename>ext3</filename> requires a 1 Mbyte
2861 journal.
2862 If you are okay with running read-only you do not need this
2863 journal.
2864 </para>
2865
2866 <note>
2867 After each round of elimination, you need to rebuild your
2868 system and then use the tools to see the effects of your
2869 reductions.
2870 </note>
2871
2872
2873 </section>
2874
2875 <section id='trim-the-kernel'>
2876 <title>Trim the Kernel</title>
2877
2878 <para>
2879 The kernel is built by including policies for hardware-independent
2880 aspects.
2881 What subsystems do you enable?
2882 For what architecture are you building?
2883 Which drivers do you build by default.
2884 <note>You can modify the kernel source if you want to help
2885 with boot time.
2886 </note>
2887 </para>
2888
2889 <para>
2890 Run the <filename>ksize.py</filename> script from the top-level
2891 Linux build directory to get an idea of what is making up
2892 the kernel:
2893 <literallayout class='monospaced'>
2894 $ cd &lt;top-level-linux-build-directory&gt;
2895 $ ksize.py > ksize.log
2896 $ cat ksize.log
2897 </literallayout>
2898 When you examine the log, you will see how much space is
2899 taken up with the built-in <filename>.o</filename> files for
2900 drivers, networking, core kernel files, filesystem, sound,
2901 and so forth.
2902 The sizes reported by the tool are uncompressed and thus,
2903 will be smaller by a relatively constant factor in a compressed
2904 kernel image.
2905 Look to reduce the areas that are large and taking up around
2906 the "90% rule."
2907 </para>
2908
2909 <para>
2910 To examine, or drill down, into any particular area, use the
2911 <filename>-d</filename> option with the script:
2912 <literallayout class='monospaced'>
2913 $ ksize.py -d > ksize.log
2914 </literallayout>
2915 Using this option breaks out the individual file information
2916 for each area of the kernel (e.g. drivers, networking, and
2917 so forth).
2918 </para>
2919
2920 <para>
2921 Use your log file to see what you can eliminate from the kernel
2922 based on features you can let go.
2923 For example, if you are not going to need sound, you do not
2924 need any drivers that support sound.
2925 </para>
2926
2927 <para>
2928 After figuring out what to eliminate, you need to reconfigure
2929 the kernel to reflect those changes during the next build.
2930 You could run <filename>menuconfig</filename> and make all your
2931 changes at once.
2932 However, that makes it difficult to see the effects of your
2933 individual eliminations and also makes it difficult to replicate
2934 the changes for perhaps another target device.
2935 A better method is to start with no configurations using
2936 <filename>allnoconfig</filename>, create configuration
2937 fragments for individual changes, and then manage the
2938 fragments into a single configuration file using
2939 <filename>merge_config.sh</filename>.
2940 The tool makes it easy for you to iterate using the
2941 configuration change and build cycle.
2942 </para>
2943
2944 <para>
2945 Each time you make configuration changes, you need to rebuild
2946 the kernel and check to see what impact your changes had on
2947 the overall size.
2948 </para>
2949 </section>
2950
2951 <section id='remove-package-management-requirements'>
2952 <title>Remove Package Management Requirements</title>
2953
2954 <para>
2955 Packaging requirements add size to the image.
2956 One way to reduce the size of the image is to remove all the
2957 packaging requirements from the image.
2958 This reduction includes both removing the package manager
2959 and its unique dependencies as well as removing the package
2960 management data itself.
2961 </para>
2962
2963 <para>
2964 To eliminate all the packaging requirements for an image,
2965 follow these steps:
2966 <orderedlist>
2967 <listitem><para>Put the following line in your main
2968 recipe for the image to remove package management
2969 data files:
2970 <literallayout class='monospaced'>
2971 ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ;
2972 </literallayout>
2973 For example, the recipe for the
2974 <filename>core-image-minimal</filename> image contains
2975 this line.
2976 You can also add the line to the
2977 <filename>local.conf</filename> configuration file.
2978 </para></listitem>
2979 <listitem><para>Be sure that "package-management" is not
2980 part of your
2981 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
2982 statement for the image.
2983 When you remove this feature, you are removing the
2984 package manager as well as its dependencies
2985 from the root filesystem.
2986 </para></listitem>
2987 </orderedlist>
2988 </para>
2989 </section>
2990
2991 <section id='look-for-other-ways-to-minimize-size'>
2992 <title>Look for Other Ways to Minimize Size</title>
2993
2994 <para>
2995 Depending on your particular circumstances, other areas that you
2996 can trim likely exist.
2997 The key to finding these areas is through tools and methods
2998 described here combined with experimentation and iteration.
2999 Here are a couple of areas to experiment with:
3000 <itemizedlist>
3001 <listitem><para><filename>eglibc</filename>:
3002 In general, follow this process:
3003 <orderedlist>
3004 <listitem><para>Remove <filename>eglibc</filename>
3005 features from
3006 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
3007 that you think you do not need.</para></listitem>
3008 <listitem><para>Build your distribution.
3009 </para></listitem>
3010 <listitem><para>If the build fails due to missing
3011 symbols in a package, determine if you can
3012 reconfigure the package to not need those
3013 features.
3014 For example, change the configuration to not
3015 support wide character support as is done for
3016 <filename>ncurses</filename>.
3017 Or, if support for those characters is needed,
3018 determine what <filename>eglibc</filename>
3019 features provide the support and restore the
3020 configuration.
3021 </para></listitem>
3022 <listitem><para>Rebuild and repeat the process.
3023 </para></listitem>
3024 </orderedlist></para></listitem>
3025 <listitem><para><filename>busybox</filename>:
3026 For BusyBox, use a process similar as described for
3027 <filename>eglibc</filename>.
3028 A difference is you will need to boot the resulting
3029 system to see if you are able to do everything you
3030 expect from the running system.
3031 You need to be sure to integrate configuration fragments
3032 into Busybox because BusyBox handles its own core
3033 features and then allows you to add configuration
3034 fragments on top.
3035 </para></listitem>
3036 </itemizedlist>
3037 </para>
3038 </section>
3039
3040 <section id='iterate-on-the-process'>
3041 <title>Iterate on the Process</title>
3042
3043 <para>
3044 If you have not reached your goals on system size, you need
3045 to iterate on the process.
3046 The process is the same.
3047 Use the tools and see just what is taking up 90% of the root
3048 filesystem and the kernel.
3049 Decide what you can eliminate without limiting your device
3050 beyond what you need.
3051 </para>
3052
3053 <para>
3054 Depending on your system, a good place to look might be
3055 Busybox, which provides a stripped down
3056 version of Unix tools in a single, executable file.
3057 You might be able to drop virtual terminal services or perhaps
3058 ipv6.
3059 </para>
3060 </section>
3061 </section>
3062
3063 <section id='working-with-packages'>
3064 <title>Working with Packages</title>
3065
3066 <para>
3067 This section describes a few tasks that involve packages:
3068 <itemizedlist>
3069 <listitem><para>Excluding packages from an image
3070 </para></listitem>
3071 <listitem><para>Incrementing a package revision number
3072 </para></listitem>
3073 <listitem><para>Handling a package name alias
3074 </para></listitem>
3075 <listitem><para>Handling optional module packaging
3076 </para></listitem>
3077 <listitem><para>Using Runtime Package Management
3078 </para></listitem>
3079 <listitem><para>Setting up and running package test
3080 (ptest)
3081 </para></listitem>
3082 </itemizedlist>
3083 </para>
3084
3085 <section id='excluding-packages-from-an-image'>
3086 <title>Excluding Packages from an Image</title>
3087
3088 <para>
3089 You might find it necessary to prevent specific packages
3090 from being installed into an image.
3091 If so, you can use several variables to direct the build
3092 system to essentially ignore installing recommended packages
3093 or to not install a package at all.
3094 </para>
3095
3096 <para>
3097 The following list introduces variables you can use to
3098 prevent packages from being installed into your image.
3099 Each of these variables only works with IPK and RPM
3100 package types.
3101 Support for Debian packages does not exist.
3102 Also, you can use these variables from your
3103 <filename>local.conf</filename> file or attach them to a
3104 specific image recipe by using a recipe name override.
3105 For more detail on the variables, see the descriptions in the
3106 Yocto Project Reference Manual's glossary chapter.
3107 <itemizedlist>
3108 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>:
3109 Use this variable to specify "recommended-only"
3110 packages that you do not want installed.
3111 </para></listitem>
3112 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>:
3113 Use this variable to prevent all "recommended-only"
3114 packages from being installed.
3115 </para></listitem>
3116 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>:
3117 Use this variable to prevent specific packages from
3118 being installed regardless of whether they are
3119 "recommended-only" or not.
3120 You need to realize that the build process could
3121 fail with an error when you
3122 prevent the installation of a package whose presence
3123 is required by an installed package.
3124 </para></listitem>
3125 </itemizedlist>
3126 </para>
3127 </section>
3128
3129 <section id='incrementing-a-package-revision-number'>
3130 <title>Incrementing a Package Revision Number</title>
3131
3132 <para>
3133 If a committed change results in changing the package output,
3134 then the value of the
3135 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
3136 variable needs to be increased (or "bumped").
3137 Increasing <filename>PR</filename> occurs one of two ways:
3138 <itemizedlist>
3139 <listitem><para>Automatically using a Package Revision
3140 Service (PR Service).</para></listitem>
3141 <listitem><para>Manually incrementing the
3142 <filename>PR</filename> variable.</para></listitem>
3143 </itemizedlist>
3144 </para>
3145
3146 <para>
3147 Given that one of the challenges any build system and its
3148 users face is how to maintain a package feed that is compatible
3149 with existing package manager applications such as
3150 RPM, APT, and OPKG, using an automated system is much
3151 preferred over a manual system.
3152 In either system, the main requirement is that version
3153 numbering increases in a linear fashion and that a number of
3154 version components exist that support that linear progression.
3155 </para>
3156
3157 <para>
3158 The following two sections provide information on the PR Service
3159 and on manual <filename>PR</filename> bumping.
3160 </para>
3161
3162 <section id='working-with-a-pr-service'>
3163 <title>Working With a PR Service</title>
3164
3165 <para>
3166 As mentioned, attempting to maintain revision numbers in the
3167 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
3168 is error prone, inaccurate and causes problems for people
3169 submitting recipes.
3170 Conversely, the PR Service automatically generates
3171 increasing numbers, particularly the revision field,
3172 which removes the human element.
3173 <note>
3174 For additional information on using a PR Service, you
3175 can see the
3176 <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink>
3177 wiki page.
3178 </note>
3179 </para>
3180
3181 <para>
3182 The Yocto Project uses variables in order of
3183 decreasing priority to facilitate revision numbering (i.e.
3184 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>,
3185 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
3186 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
3187 for epoch, version and revision, respectively).
3188 The values are highly dependent on the policies and
3189 procedures of a given distribution and package feed.
3190 </para>
3191
3192 <para>
3193 Because the OpenEmbedded build system uses
3194 "<ulink url='&YOCTO_DOCS_REF_URL;#checksums'>signatures</ulink>",
3195 which are unique to a given build, the build system
3196 knows when to rebuild packages.
3197 All the inputs into a given task are represented by a
3198 signature, which can trigger a rebuild when different.
3199 Thus, the build system itself does not rely on the
3200 <filename>PR</filename> numbers to trigger a rebuild.
3201 The signatures, however, can be used to generate
3202 <filename>PR</filename> values.
3203 </para>
3204
3205 <para>
3206 The PR Service works with both
3207 <filename>OEBasic</filename> and
3208 <filename>OEBasicHash</filename> generators.
3209 The value of <filename>PR</filename> bumps when the
3210 checksum changes and the different generator mechanisms
3211 change signatures under different circumstances.
3212 </para>
3213
3214 <para>
3215 As implemented, the build system includes values from
3216 the PR Service into the <filename>PR</filename> field as
3217 an addition using the form "<filename>.x</filename>" so
3218 <filename>r0</filename> becomes <filename>r0.1</filename>,
3219 <filename>r0.2</filename> and so forth.
3220 This scheme allows existing <filename>PR</filename> values
3221 to be used for whatever reasons, which include manual
3222 <filename>PR</filename> bumps should it be necessary.
3223 </para>
3224
3225 <para>
3226 By default, the PR Service is not enabled or running.
3227 Thus, the packages generated are just "self consistent".
3228 The build system adds and removes packages and
3229 there are no guarantees about upgrade paths but images
3230 will be consistent and correct with the latest changes.
3231 </para>
3232
3233 <para>
3234 The simplest form for a PR Service is for it to exist
3235 for a single host development system that builds the
3236 package feed (building system).
3237 For this scenario, you can enable the PR Service by adding
3238 the following to your <filename>local.conf</filename>
3239 file in the
3240 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
3241 <literallayout class='monospaced'>
3242 PRSERV_HOST = "localhost:0"
3243 </literallayout>
3244 Once the service is started, packages will automatically
3245 get increasing <filename>PR</filename> values and
3246 BitBake will take care of starting and stopping the server.
3247 </para>
3248
3249 <para>
3250 If you have a more complex setup where multiple host
3251 development systems work against a common, shared package
3252 feed, you have a single PR Service running and it is
3253 connected to each building system.
3254 For this scenario, you need to start the PR Service using
3255 the <filename>bitbake-prserv</filename> command:
3256 <literallayout class='monospaced'>
3257 bitbake-prserv &dash;&dash;host &lt;ip&gt; &dash;&dash;port &lt;port&gt; &dash;&dash;start
3258 </literallayout>
3259 In addition to hand-starting the service, you need to
3260 update the <filename>local.conf</filename> file of each
3261 building system as described earlier so each system
3262 points to the server and port.
3263 </para>
3264
3265 <para>
3266 It is also recommended you use Build History, which adds
3267 some sanity checks to package versions, in conjunction with
3268 the server that is running the PR Service.
3269 To enable build history, add the following to each building
3270 system's <filename>local.conf</filename> file:
3271 <literallayout class='monospaced'>
3272 # It is recommended to activate "buildhistory" for testing the PR service
3273 INHERIT += "buildhistory"
3274 BUILDHISTORY_COMMIT = "1"
3275 </literallayout>
3276 For information on Build History, see the
3277 "<ulink url='&YOCTO_DOCS_REF_URL;#maintaining-build-output-quality'>Maintaining Build Output Quality</ulink>"
3278 section in the Yocto Project Reference Manual.
3279 </para>
3280
3281 <note>
3282 <para>The OpenEmbedded build system does not maintain
3283 <filename>PR</filename> information as part of the
3284 shared state (sstate) packages.
3285 If you maintain an sstate feed, its expected that either
3286 all your building systems that contribute to the sstate
3287 feed use a shared PR Service, or you do not run a PR
3288 Service on any of your building systems.
3289 Having some systems use a PR Service while others do
3290 not leads to obvious problems.</para>
3291 <para>For more information on shared state, see the
3292 "<ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>"
3293 section in the Yocto Project Reference Manual.</para>
3294 </note>
3295 </section>
3296
3297 <section id='manually-bumping-pr'>
3298 <title>Manually Bumping PR</title>
3299
3300 <para>
3301 The alternative to setting up a PR Service is to manually
3302 bump the
3303 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
3304 variable.
3305 </para>
3306
3307 <para>
3308 If a committed change results in changing the package output,
3309 then the value of the PR variable needs to be increased
3310 (or "bumped") as part of that commit.
3311 For new recipes you should add the <filename>PR</filename>
3312 variable and set its initial value equal to "r0", which is the default.
3313 Even though the default value is "r0", the practice of adding it to a new recipe makes
3314 it harder to forget to bump the variable when you make changes
3315 to the recipe in future.
3316 </para>
3317
3318 <para>
3319 If you are sharing a common <filename>.inc</filename> file with multiple recipes,
3320 you can also use the
3321 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename>
3322 variable to ensure that
3323 the recipes sharing the <filename>.inc</filename> file are rebuilt when the
3324 <filename>.inc</filename> file itself is changed.
3325 The <filename>.inc</filename> file must set <filename>INC_PR</filename>
3326 (initially to "r0"), and all recipes referring to it should set <filename>PR</filename>
3327 to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed.
3328 If the <filename>.inc</filename> file is changed then its
3329 <filename>INC_PR</filename> should be incremented.
3330 </para>
3331
3332 <para>
3333 When upgrading the version of a package, assuming the
3334 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename>
3335 changes, the <filename>PR</filename> variable should be
3336 reset to "r0" (or "$(INC_PR).0" if you are using
3337 <filename>INC_PR</filename>).
3338 </para>
3339
3340 <para>
3341 Usually, version increases occur only to packages.
3342 However, if for some reason <filename>PV</filename> changes but does not
3343 increase, you can increase the
3344 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename>
3345 variable (Package Epoch).
3346 The <filename>PE</filename> variable defaults to "0".
3347 </para>
3348
3349 <para>
3350 Version numbering strives to follow the
3351 <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'>
3352 Debian Version Field Policy Guidelines</ulink>.
3353 These guidelines define how versions are compared and what "increasing" a version means.
3354 </para>
3355 </section>
3356 </section>
3357
3358 <section id="usingpoky-configuring-DISTRO_PN_ALIAS">
3359 <title>Handling a Package Name Alias</title>
3360 <para>
3361 Sometimes a package name you are using might exist under an alias or as a similarly named
3362 package in a different distribution.
3363 The OpenEmbedded build system implements a <filename>distro_check</filename>
3364 task that automatically connects to major distributions
3365 and checks for these situations.
3366 If the package exists under a different name in a different distribution, you get a
3367 <filename>distro_check</filename> mismatch.
3368 You can resolve this problem by defining a per-distro recipe name alias using the
3369 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_PN_ALIAS'>DISTRO_PN_ALIAS</ulink></filename>
3370 variable.
3371 </para>
3372
3373 <para>
3374 Following is an example that shows how you specify the <filename>DISTRO_PN_ALIAS</filename>
3375 variable:
3376 <literallayout class='monospaced'>
3377 DISTRO_PN_ALIAS_pn-PACKAGENAME = "distro1=package_name_alias1 \
3378 distro2=package_name_alias2 \
3379 distro3=package_name_alias3 \
3380 ..."
3381 </literallayout>
3382 </para>
3383
3384 <para>
3385 If you have more than one distribution alias, separate them with a space.
3386 Note that the build system currently automatically checks the
3387 Fedora, OpenSUSE, Debian, Ubuntu,
3388 and Mandriva distributions for source package recipes without having to specify them
3389 using the <filename>DISTRO_PN_ALIAS</filename> variable.
3390 For example, the following command generates a report that lists the Linux distributions
3391 that include the sources for each of the recipes.
3392 <literallayout class='monospaced'>
3393 $ bitbake world -f -c distro_check
3394 </literallayout>
3395 The results are stored in the <filename>build/tmp/log/distro_check-${DATETIME}.results</filename>
3396 file found in the
3397 <link linkend='source-directory'>Source Directory</link>.
3398 </para>
3399 </section>
3400
3401 <section id='handling-optional-module-packaging'>
3402 <title>Handling Optional Module Packaging</title>
3403
3404 <para>
3405 Many pieces of software split functionality into optional
3406 modules (or plug-ins) and the plug-ins that are built
3407 might depend on configuration options.
3408 To avoid having to duplicate the logic that determines what
3409 modules are available in your recipe or to avoid having
3410 to package each module by hand, the OpenEmbedded build system
3411 provides functionality to handle module packaging dynamically.
3412 </para>
3413
3414 <para>
3415 To handle optional module packaging, you need to do two things:
3416 <itemizedlist>
3417 <listitem><para>Ensure the module packaging is actually
3418 done</para></listitem>
3419 <listitem><para>Ensure that any dependencies on optional
3420 modules from other recipes are satisfied by your recipe
3421 </para></listitem>
3422 </itemizedlist>
3423 </para>
3424
3425 <section id='making-sure-the-packaging-is-done'>
3426 <title>Making Sure the Packaging is Done</title>
3427
3428 <para>
3429 To ensure the module packaging actually gets done, you use
3430 the <filename>do_split_packages</filename> function within
3431 the <filename>populate_packages</filename> Python function
3432 in your recipe.
3433 The <filename>do_split_packages</filename> function
3434 searches for a pattern of files or directories under a
3435 specified path and creates a package for each one it finds
3436 by appending to the
3437 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
3438 variable and setting the appropriate values for
3439 <filename>FILES_packagename</filename>,
3440 <filename>RDEPENDS_packagename</filename>,
3441 <filename>DESCRIPTION_packagename</filename>, and so forth.
3442 Here is an example from the <filename>lighttpd</filename>
3443 recipe:
3444 <literallayout class='monospaced'>
3445 python populate_packages_prepend () {
3446 lighttpd_libdir = d.expand('${libdir}')
3447 do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
3448 'lighttpd-module-%s', 'Lighttpd module for %s',
3449 extra_depends='')
3450 }
3451 </literallayout>
3452 The previous example specifies a number of things in the
3453 call to <filename>do_split_packages</filename>.
3454 <itemizedlist>
3455 <listitem><para>A directory within the files installed
3456 by your recipe through <filename>do_install</filename>
3457 in which to search.</para></listitem>
3458 <listitem><para>A regular expression to match module
3459 files in that directory.
3460 In the example, note the parentheses () that mark
3461 the part of the expression from which the module
3462 name should be derived.</para></listitem>
3463 <listitem><para>A pattern to use for the package names.
3464 </para></listitem>
3465 <listitem><para>A description for each package.
3466 </para></listitem>
3467 <listitem><para>An empty string for
3468 <filename>extra_depends</filename>, which disables
3469 the default dependency on the main
3470 <filename>lighttpd</filename> package.
3471 Thus, if a file in <filename>${libdir}</filename>
3472 called <filename>mod_alias.so</filename> is found,
3473 a package called <filename>lighttpd-module-alias</filename>
3474 is created for it and the
3475 <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink>
3476 is set to "Lighttpd module for alias".</para></listitem>
3477 </itemizedlist>
3478 </para>
3479
3480 <para>
3481 Often, packaging modules is as simple as the previous
3482 example.
3483 However, more advanced options exist that you can use
3484 within <filename>do_split_packages</filename> to modify its
3485 behavior.
3486 And, if you need to, you can add more logic by specifying
3487 a hook function that is called for each package.
3488 It is also perfectly acceptable to call
3489 <filename>do_split_packages</filename> multiple times if
3490 you have more than one set of modules to package.
3491 </para>
3492
3493 <para>
3494 For more examples that show how to use
3495 <filename>do_split_packages</filename>, see the
3496 <filename>connman.inc</filename> file in the
3497 <filename>meta/recipes-connectivity/connman/</filename>
3498 directory of the <filename>poky</filename>
3499 <link linkend='yocto-project-repositories'>source repository</link>.
3500 You can also find examples in
3501 <filename>meta/classes/kernel.bbclass</filename>.
3502 </para>
3503
3504 <para>
3505 Following is a reference that shows
3506 <filename>do_split_packages</filename> mandatory and
3507 optional arguments:
3508 <literallayout class='monospaced'>
3509 Mandatory arguments
3510
3511 root
3512 The path in which to search
3513 file_regex
3514 Regular expression to match searched files.
3515 Use parentheses () to mark the part of this
3516 expression that should be used to derive the
3517 module name (to be substituted where %s is
3518 used in other function arguments as noted below)
3519 output_pattern
3520 Pattern to use for the package names. Must
3521 include %s.
3522 description
3523 Description to set for each package. Must
3524 include %s.
3525
3526 Optional arguments
3527
3528 postinst
3529 Postinstall script to use for all packages
3530 (as a string)
3531 recursive
3532 True to perform a recursive search - default
3533 False
3534 hook
3535 A hook function to be called for every match.
3536 The function will be called with the following
3537 arguments (in the order listed):
3538
3539 f
3540 Full path to the file/directory match
3541 pkg
3542 The package name
3543 file_regex
3544 As above
3545 output_pattern
3546 As above
3547 modulename
3548 The module name derived using file_regex
3549
3550 extra_depends
3551 Extra runtime dependencies (RDEPENDS) to be
3552 set for all packages. The default value of None
3553 causes a dependency on the main package
3554 (${PN}) - if you do not want this, pass empty
3555 string '' for this parameter.
3556 aux_files_pattern
3557 Extra item(s) to be added to FILES for each
3558 package. Can be a single string item or a list
3559 of strings for multiple items. Must include %s.
3560 postrm
3561 postrm script to use for all packages (as a
3562 string)
3563 allow_dirs
3564 True to allow directories to be matched -
3565 default False
3566 prepend
3567 If True, prepend created packages to PACKAGES
3568 instead of the default False which appends them
3569 match_path
3570 match file_regex on the whole relative path to
3571 the root rather than just the file name
3572 aux_files_pattern_verbatim
3573 Extra item(s) to be added to FILES for each
3574 package, using the actual derived module name
3575 rather than converting it to something legal
3576 for a package name. Can be a single string item
3577 or a list of strings for multiple items. Must
3578 include %s.
3579 allow_links
3580 True to allow symlinks to be matched - default
3581 False
3582 </literallayout>
3583 </para>
3584 </section>
3585
3586 <section id='satisfying-dependencies'>
3587 <title>Satisfying Dependencies</title>
3588
3589 <para>
3590 The second part for handling optional module packaging
3591 is to ensure that any dependencies on optional modules
3592 from other recipes are satisfied by your recipe.
3593 You can be sure these dependencies are satisfied by
3594 using the
3595 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable.
3596 Here is an example that continues with the
3597 <filename>lighttpd</filename> recipe shown earlier:
3598 <literallayout class='monospaced'>
3599 PACKAGES_DYNAMIC = "lighttpd-module-.*"
3600 </literallayout>
3601 The name specified in the regular expression can of
3602 course be anything.
3603 In this example, it is <filename>lighttpd-module-</filename>
3604 and is specified as the prefix to ensure that any
3605 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
3606 and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>
3607 on a package name starting with the prefix are satisfied
3608 during build time.
3609 If you are using <filename>do_split_packages</filename>
3610 as described in the previous section, the value you put in
3611 <filename>PACKAGES_DYNAMIC</filename> should correspond to
3612 the name pattern specified in the call to
3613 <filename>do_split_packages</filename>.
3614 </para>
3615 </section>
3616 </section>
3617
3618 <section id='using-runtime-package-management'>
3619 <title>Using Runtime Package Management</title>
3620
3621 <para>
3622 During a build, BitBake always transforms a recipe into one or
3623 more packages.
3624 For example, BitBake takes the <filename>bash</filename> recipe
3625 and currently produces the <filename>bash-dbg</filename>,
3626 <filename>bash-staticdev</filename>,
3627 <filename>bash-dev</filename>, <filename>bash-doc</filename>,
3628 <filename>bash-locale</filename>, and
3629 <filename>bash</filename> packages.
3630 Not all generated packages are included in an image.
3631 </para>
3632
3633 <para>
3634 In several situations, you might need to update, add, remove,
3635 or query the packages on a target device at runtime
3636 (i.e. without having to generate a new image).
3637 Examples of such situations include:
3638 <itemizedlist>
3639 <listitem><para>
3640 You want to provide in-the-field updates to deployed
3641 devices (e.g. security updates).
3642 </para></listitem>
3643 <listitem><para>
3644 You want to have a fast turn-around development cycle
3645 for one or more applications that run on your device.
3646 </para></listitem>
3647 <listitem><para>
3648 You want to temporarily install the "debug" packages
3649 of various applications on your device so that
3650 debugging can be greatly improved by allowing
3651 access to symbols and source debugging.
3652 </para></listitem>
3653 <listitem><para>
3654 You want to deploy a more minimal package selection of
3655 your device but allow in-the-field updates to add a
3656 larger selection for customization.
3657 </para></listitem>
3658 </itemizedlist>
3659 </para>
3660
3661 <para>
3662 In all these situations, you have something similar to a more
3663 traditional Linux distribution in that in-field devices
3664 are able to receive pre-compiled packages from a server for
3665 installation or update.
3666 Being able to install these packages on a running,
3667 in-field device is what is termed "runtime package
3668 management".
3669 </para>
3670
3671 <para>
3672 In order to use runtime package management, you
3673 need a host/server machine that serves up the pre-compiled
3674 packages plus the required metadata.
3675 You also need package manipulation tools on the target.
3676 The build machine is a likely candidate to act as the server.
3677 However, that machine does not necessarily have to be the
3678 package server.
3679 The build machine could push its artifacts to another machine
3680 that acts as the server (e.g. Internet-facing).
3681 </para>
3682
3683 <para>
3684 A simple build that targets just one device produces
3685 more than one package database.
3686 In other words, the packages produced by a build are separated
3687 out into a couple of different package groupings based on
3688 criteria such as the target's CPU architecture, the target
3689 board, or the C library used on the target.
3690 For example, a build targeting the <filename>qemuarm</filename>
3691 device produces the following three package databases:
3692 <filename>all</filename>, <filename>armv5te</filename>, and
3693 <filename>qemuarm</filename>.
3694 If you wanted your <filename>qemuarm</filename> device to be
3695 aware of all the packages that were available to it,
3696 you would need to point it to each of these databases
3697 individually.
3698 In a similar way, a traditional Linux distribution usually is
3699 configured to be aware of a number of software repositories
3700 from which it retrieves packages.
3701 </para>
3702
3703 <para>
3704 Using runtime package management is completely optional and
3705 not required for a successful build or deployment in any
3706 way.
3707 But if you want to make use of runtime package management,
3708 you need to do a couple things above and beyond the basics.
3709 The remainder of this section describes what you need to do.
3710 </para>
3711
3712 <section id='runtime-package-management-build'>
3713 <title>Build Considerations</title>
3714
3715 <para>
3716 This section describes build considerations that you need
3717 to be aware of in order to provide support for runtime
3718 package management.
3719 </para>
3720
3721 <para>
3722 When BitBake generates packages it needs to know
3723 what format(s) to use.
3724 In your configuration, you use the
3725 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
3726 variable to specify the format.
3727 <note>
3728 You can choose to have more than one format but you must
3729 provide at least one.
3730 </note>
3731 </para>
3732
3733 <para>
3734 If you would like your image to start off with a basic
3735 package database of the packages in your current build
3736 as well as have the relevant tools available on the
3737 target for runtime package management, you can include
3738 "package-management" in the
3739 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
3740 variable.
3741 Including "package-management" in this
3742 configuration variable ensures that when the image
3743 is assembled for your target, the image includes
3744 the currently-known package databases as well as
3745 the target-specific tools required for runtime
3746 package management to be performed on the target.
3747 However, this is not strictly necessary.
3748 You could start your image off without any databases
3749 but only include the required on-target package
3750 tool(s).
3751 As an example, you could include "opkg" in your
3752 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
3753 variable if you are using the IPK package format.
3754 You can then initialize your target's package database(s)
3755 later once your image is up and running.
3756 </para>
3757
3758 <para>
3759 Whenever you perform any sort of build step that can
3760 potentially generate a package or modify an existing
3761 package, it is always a good idea to re-generate the
3762 package index with:
3763 <literallayout class='monospaced'>
3764 $ bitbake package-index
3765 </literallayout>
3766 Realize that it is not sufficient to simply do the
3767 following:
3768 <literallayout class='monospaced'>
3769 $ bitbake &lt;some-package&gt; package-index
3770 </literallayout>
3771 This is because BitBake does not properly schedule the
3772 <filename>package-index</filename> target fully after any
3773 other target has completed.
3774 Thus, be sure to run the package update step separately.
3775 </para>
3776
3777 <para>
3778 As described below in the
3779 "<link linkend='runtime-package-management-target-ipk'>Using IPK</link>"
3780 section, if you are using IPK as your package format, you
3781 can make use of the
3782 <filename>distro-feed-configs</filename> recipe provided
3783 by <filename>meta-oe</filename> in order to configure your
3784 target to use your IPK databases.
3785 </para>
3786
3787 <para>
3788 When your build is complete, your packages reside in the
3789 <filename>${TMPDIR}/deploy/&lt;package-format&gt;</filename>
3790 directory.
3791 For example, if <filename>${TMPDIR}</filename>
3792 is <filename>tmp</filename> and your selected package type
3793 is IPK, then your IPK packages are available in
3794 <filename>tmp/deploy/ipk</filename>.
3795 </para>
3796 </section>
3797
3798 <section id='runtime-package-management-server'>
3799 <title>Host or Server Machine Setup</title>
3800
3801 <para>
3802 Typically, packages are served from a server using
3803 HTTP.
3804 However, other protocols are possible.
3805 If you want to use HTTP, then setup and configure a
3806 web server, such as Apache 2 or lighttpd, on the machine
3807 serving the packages.
3808 </para>
3809
3810 <para>
3811 As previously mentioned, the build machine can act as the
3812 package server.
3813 In the following sections that describe server machine
3814 setups, the build machine is assumed to also be the server.
3815 </para>
3816
3817 <section id='package-server-apache'>
3818 <title>Serving Packages via Apache 2</title>
3819
3820 <para>
3821 This example assumes you are using the Apache 2
3822 server:
3823 <orderedlist>
3824 <listitem><para>
3825 Add the directory to your Apache
3826 configuration, which you can find at
3827 <filename>/etc/httpd/conf/httpd.conf</filename>.
3828 Use commands similar to these on the
3829 development system.
3830 These example commands assume a top-level
3831 <link linkend='source-directory'>Source Directory</link>
3832 named <filename>poky</filename> in your home
3833 directory.
3834 The example also assumes an RPM package type.
3835 If you are using a different package type, such
3836 as IPK, use "ipk" in the pathnames:
3837 <literallayout class='monospaced'>
3838 &lt;VirtualHost *:80&gt;
3839 ....
3840 Alias /rpm ~/poky/build/tmp/deploy/rpm
3841 &lt;Directory "~/poky/build/tmp/deploy/rpm"&gt;
3842 Options +Indexes
3843 &lt;/Directory&gt;
3844 &lt;/VirtualHost&gt;
3845 </literallayout></para></listitem>
3846 <listitem><para>
3847 Reload the Apache configuration as described
3848 in this step.
3849 For all commands, be sure you have root
3850 privileges.
3851 </para>
3852
3853 <para>
3854 If your development system is using Fedora or
3855 CentOS, use the following:
3856 <literallayout class='monospaced'>
3857 # service httpd reload
3858 </literallayout>
3859 For Ubuntu and Debian, use the following:
3860 <literallayout class='monospaced'>
3861 # /etc/init.d/apache2 reload
3862 </literallayout>
3863 For OpenSUSE, use the following:
3864 <literallayout class='monospaced'>
3865 # /etc/init.d/apache2 reload
3866 </literallayout></para></listitem>
3867 <listitem><para>
3868 If you are using Security-Enhanced Linux
3869 (SELinux), you need to label the files as
3870 being accessible through Apache.
3871 Use the following command from the development
3872 host.
3873 This example assumes RPM package types:
3874 <literallayout class='monospaced'>
3875 # chcon -R -h -t httpd_sys_content_t tmp/deploy/rpm
3876 </literallayout></para></listitem>
3877 </orderedlist>
3878 </para>
3879 </section>
3880
3881 <section id='package-server-lighttpd'>
3882 <title>Serving Packages via lighttpd</title>
3883
3884 <para>
3885 If you are using lighttpd, all you need
3886 to do is to provide a link from your
3887 <filename>${TMPDIR}/deploy/&lt;package-format&gt;</filename>
3888 directory to lighttpd's document-root.
3889 You can determine the specifics of your lighttpd
3890 installation by looking through its configuration file,
3891 which is usually found at:
3892 <filename>/etc/lighttpd/lighttpd.conf</filename>.
3893 </para>
3894
3895 <para>
3896 For example, if you are using IPK, lighttpd's
3897 document-root is set to
3898 <filename>/var/www/lighttpd</filename>, and you had
3899 packages for a target named "BOARD",
3900 then you might create a link from your build location
3901 to lighttpd's document-root as follows:
3902 <literallayout class='monospaced'>
3903 # ln -s $(PWD)/tmp/deploy/ipk /var/www/lighttpd/BOARD-dir
3904 </literallayout>
3905 </para>
3906
3907 <para>
3908 At this point, you need to start the lighttpd server.
3909 The method used to start the server varies by
3910 distribution.
3911 However, one basic method that starts it by hand is:
3912 <literallayout class='monospaced'>
3913 # lighttpd -f /etc/lighttpd/lighttpd.conf
3914 </literallayout>
3915 </para>
3916 </section>
3917 </section>
3918
3919 <section id='runtime-package-management-target'>
3920 <title>Target Setup</title>
3921
3922 <para>
3923 Setting up the target differs depending on the
3924 package management system.
3925 This section provides information for RPM and IPK.
3926 </para>
3927
3928 <section id='runtime-package-management-target-rpm'>
3929 <title>Using RPM</title>
3930
3931 <para>
3932 The application for performing runtime package
3933 management of RPM packages on the target is called
3934 <filename>smart</filename>.
3935 </para>
3936
3937 <para>
3938 On the target machine, you need to inform
3939 <filename>smart</filename> of every package database
3940 you want to use.
3941 As an example, suppose your target device can use the
3942 following three package databases from a server named
3943 <filename>server.name</filename>:
3944 <filename>all</filename>, <filename>i586</filename>,
3945 and <filename>qemux86</filename>.
3946 Given this example, issue the following commands on the
3947 target:
3948 <literallayout class='monospaced'>
3949 # smart channel --add all type=rpm-md baseurl=http://server.name/rpm/all
3950 # smart channel --add i585 type=rpm-md baseurl=http://server.name/rpm/i586
3951 # smart channel --add qemux86 type=rpm-md baseurl=http://server.name/rpm/qemux86
3952 </literallayout>
3953 Also from the target machine, fetch the repository
3954 information using this command:
3955 <literallayout class='monospaced'>
3956 # smart update
3957 </literallayout>
3958 You can now use the <filename>smart query</filename>
3959 and <filename>smart install</filename> commands to
3960 find and install packages from the repositories.
3961 </para>
3962 </section>
3963
3964 <section id='runtime-package-management-target-ipk'>
3965 <title>Using IPK</title>
3966
3967 <para>
3968 The application for performing runtime package
3969 management of IPK packages on the target is called
3970 <filename>opkg</filename>.
3971 </para>
3972
3973 <para>
3974 In order to inform <filename>opkg</filename> of the
3975 package databases you want to use, simply create one
3976 or more <filename>*.conf</filename> files in the
3977 <filename>/etc/opkg</filename> directory on the target.
3978 The <filename>opkg</filename> application uses them
3979 to find its available package databases.
3980 As an example, suppose you configured your HTTP server
3981 on your machine named
3982 <filename>www.mysite.com</filename> to serve files
3983 from a <filename>BOARD-dir</filename> directory under
3984 its document-root.
3985 In this case, you might create a configuration
3986 file on the target called
3987 <filename>/etc/opkg/base-feeds.conf</filename> that
3988 contains:
3989 <literallayout class='monospaced'>
3990 src/gz all http://www.mysite.com/BOARD-dir/all
3991 src/gz armv7a http://www.mysite.com/BOARD-dir/armv7a
3992 src/gz beagleboard http://www.mysite.com/BOARD-dir/beagleboard
3993 </literallayout>
3994 </para>
3995
3996 <para>
3997 As a way of making it easier to generate and make
3998 these IPK configuration files available on your
3999 target, the <filename>meta-oe</filename> layer
4000 provides a recipe called
4001 <filename>distro-feed-configs</filename>, which
4002 provides a package by the same name.
4003 When you include this package into your image, it will
4004 automatically generate and include a set of
4005 <filename>*.conf</filename> files in the image's
4006 <filename>/etc/opkg</filename> directory that will
4007 provide your target's <filename>opkg</filename>
4008 tool with any and all package databases your build will
4009 generate.
4010 The only catch is that this recipe cannot possibly
4011 imagine your server's DNS name/IP address.
4012 Consequently, somewhere in your configuration you need
4013 to set a variable called
4014 <filename>DISTRO_FEED_URI</filename> to point
4015 to your server and the location within the
4016 document-root that contains the databases.
4017 For example: if you are serving your packages over HTTP,
4018 your server's IP address is 192.168.7.1, and your
4019 databases are located in a directory called
4020 <filename>BOARD-dir</filename> underneath your HTTP
4021 server's document-root, you need to set
4022 <filename>DISTRO_FEED_URI</filename> to
4023 <filename>http://192.168.7.1/BOARD-dir</filename>.
4024 </para>
4025
4026 <para>
4027 On the target machine, fetch (or refresh) the
4028 repository information using this command:
4029 <literallayout class='monospaced'>
4030 # opkg update
4031 </literallayout>
4032 You can now use the <filename>opkg list</filename> and
4033 <filename>opkg install</filename> commands to find and
4034 install packages from the repositories.
4035 </para>
4036 </section>
4037 </section>
4038 </section>
4039
4040 <section id='testing-packages-with-ptest'>
4041 <title>Testing Packages With ptest</title>
4042
4043 <para>
4044 A Package Test (ptest) runs tests against packages built
4045 by the OpenEmbedded build system on the target machine.
4046 A ptest contains at least two items: the actual test, and
4047 a shell script (<filename>run-ptest</filename>) that starts
4048 the test.
4049 The shell script that starts the test must not contain
4050 the actual test, the script only starts it.
4051 On the other hand, the test can be anything from a simple
4052 shell script that runs a binary and checks the output to
4053 an elaborate system of test binaries and data files.
4054 </para>
4055
4056 <para>
4057 The test generates output in the format used by
4058 Automake:
4059 <literallayout class='monospaced'>
4060 &lt;result&gt;: &lt;testname&gt;
4061 </literallayout>
4062 where the result can be <filename>PASS</filename>,
4063 <filename>FAIL</filename>, or <filename>SKIP</filename>,
4064 and the testname can be any identifying string.
4065 </para>
4066
4067 <note>
4068 With this release of the Yocto Project, three recipes exist
4069 that are "ptest-enabled": <filename>bash</filename>,
4070 <filename>glib-2.0</filename>, and
4071 <filename>dbus</filename>.
4072 These three recipes are Autotool-enabled.
4073 </note>
4074
4075 <section id='adding-ptest-to-your-build'>
4076 <title>Adding ptest to Your Build</title>
4077
4078 <para>
4079 To add package testing to your build, add the
4080 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
4081 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
4082 variables to your <filename>local.conf</filename> file,
4083 which is found in the
4084 <link linkend='build-directory'>Build Directory</link>:
4085 <literallayout class='monospaced'>
4086 DISTRO_FEATURES_append = " ptest"
4087 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
4088 </literallayout>
4089 Once your build is complete, the ptest files are installed
4090 into the <filename>/usr/lib/&lt;package&gt;/ptest</filename>
4091 directory within the image, where
4092 <filename>&lt;package&gt;</filename> is the name of the
4093 package.
4094 </para>
4095 </section>
4096
4097 <section id='running-ptest'>
4098 <title>Running ptest</title>
4099
4100 <para>
4101 The <filename>ptest-runner</filename> package installs a
4102 shell script that loops through all installed ptest test
4103 suites and runs them in sequence.
4104 Consequently, you might want to add this package to
4105 your image.
4106 </para>
4107 </section>
4108
4109 <section id='getting-your-package-ready'>
4110 <title>Getting Your Package Ready</title>
4111
4112 <para>
4113 In order to enable a recipe to run installed ptests
4114 on target hardware,
4115 you need to prepare the recipes that build the packages
4116 you want to test.
4117 Here is what you have to do for each recipe:
4118 <itemizedlist>
4119 <listitem><para><emphasis>Be sure the recipe
4120 inherits ptest:</emphasis>
4121 Include the following line in each recipe:
4122 <literallayout class='monospaced'>
4123 inherit ptest
4124 </literallayout>
4125 </para></listitem>
4126 <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis>
4127 This script starts your test.
4128 Locate the script where you will refer to it
4129 using
4130 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
4131 Here is an example that starts a test for
4132 <filename>dbus</filename>:
4133 <literallayout class='monospaced'>
4134 #!/bin/sh
4135 cd test
4136 make -k runtest-TESTS
4137 </literallayout>
4138 </para></listitem>
4139 <listitem><para><emphasis>Ensure dependencies are
4140 met:</emphasis>
4141 If the test adds build or runtime dependencies
4142 that normally do not exist for the package
4143 (such as requiring "make" to run the test suite),
4144 use the
4145 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
4146 and
4147 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
4148 variables in your recipe in order for the package
4149 to meet the dependencies.
4150 Here is an example where the package has a runtime
4151 dependency on "make":
4152 <literallayout class='monospaced'>
4153 RDEPENDS_${PN}-ptest += "make"
4154 </literallayout>
4155 </para></listitem>
4156 <listitem><para><emphasis>Add a function to build the
4157 test suite:</emphasis>
4158 Not many packages support cross-compilation of
4159 their test suites.
4160 Consequently, you usually need to add a
4161 cross-compilation function to the package.
4162 </para>
4163 <para>Many packages based on Automake compile and
4164 run the test suite by using a single command
4165 such as <filename>make check</filename>.
4166 However, the native <filename>make check</filename>
4167 builds and runs on the same computer, while
4168 cross-compiling requires that the package is built
4169 on the host but executed on the target.
4170 The built version of Automake that ships with the
4171 Yocto Project includes a patch that separates
4172 building and execution.
4173 Consequently, packages that use the unaltered,
4174 patched version of <filename>make check</filename>
4175 automatically cross-compiles.</para>
4176 <para>However, you still must add a
4177 <filename>do_compile_ptest</filename> function to
4178 build the test suite.
4179 Add a function similar to the following to your
4180 recipe:
4181 <literallayout class='monospaced'>
4182 do_compile_ptest() {
4183 oe_runmake buildtest-TESTS
4184 }
4185 </literallayout>
4186 </para></listitem>
4187 <listitem><para><emphasis>Ensure special configurations
4188 are set:</emphasis>
4189 If the package requires special configurations
4190 prior to compiling the test code, you must
4191 insert a <filename>do_configure_ptest</filename>
4192 function into the recipe.
4193 </para></listitem>
4194 <listitem><para><emphasis>Install the test
4195 suite:</emphasis>
4196 The <filename>ptest.bbclass</filename> class
4197 automatically copies the file
4198 <filename>run-ptest</filename> to the target and
4199 then runs make <filename>install-ptest</filename>
4200 to run the tests.
4201 If this is not enough, you need to create a
4202 <filename>do_install_ptest</filename> function and
4203 make sure it gets called after the
4204 "make install-ptest" completes.
4205 </para></listitem>
4206 </itemizedlist>
4207 </para>
4208 </section>
4209 </section>
4210 </section>
4211
4212 <section id="building-software-from-an-external-source">
4213 <title>Building Software from an External Source</title>
4214
4215 <para>
4216 By default, the OpenEmbedded build system uses the
4217 <link linkend='build-directory'>Build Directory</link> to
4218 build source code.
4219 The build process involves fetching the source files, unpacking
4220 them, and then patching them if necessary before the build takes
4221 place.
4222 </para>
4223
4224 <para>
4225 Situations exist where you might want to build software from source
4226 files that are external to and thus outside of the
4227 OpenEmbedded build system.
4228 For example, suppose you have a project that includes a new BSP with
4229 a heavily customized kernel.
4230 And, you want to minimize exposing the build system to the
4231 development team so that they can focus on their project and
4232 maintain everyone's workflow as much as possible.
4233 In this case, you want a kernel source directory on the development
4234 machine where the development occurs.
4235 You want the recipe's
4236 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
4237 variable to point to the external directory and use it as is, not
4238 copy it.
4239 </para>
4240
4241 <para>
4242 To build from software that comes from an external source, all you
4243 need to do is inherit
4244 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></ulink>
4245 and then set the
4246 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>
4247 variable to point to your external source code.
4248 Here are the statements to put in your
4249 <filename>local.conf</filename> file:
4250 <literallayout class='monospaced'>
4251 INHERIT += "externalsrc"
4252 EXTERNALSRC_pn-myrecipe = "/some/path/to/your/source/tree"
4253 </literallayout>
4254 </para>
4255
4256 <para>
4257 By default, <filename>externalsrc.bbclass</filename> builds
4258 the source code in a directory separate from the external source
4259 directory as specified by
4260 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>.
4261 If you need to have the source built in the same directory in
4262 which it resides, or some other nominated directory, you can set
4263 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink>
4264 to point to that directory:
4265 <literallayout class='monospaced'>
4266 EXTERNALSRC_BUILD_pn-myrecipe = "/path/to/my/source/tree"
4267 </literallayout>
4268 </para>
4269 </section>
4270
4271 <section id="selecting-an-initialization-manager">
4272 <title>Selecting an Initialization Manager</title>
4273
4274 <para>
4275 By default, the Yocto Project uses
4276 <filename>SysVinit</filename> as the initialization manager.
4277 However, support also exists for <filename>systemd</filename>,
4278 which is a full replacement for <filename>init</filename> with
4279 parallel starting of services, reduced shell overhead and other
4280 features that are used by many distributions.
4281 </para>
4282
4283 <para>
4284 If you want to use <filename>sysvinit</filename>, you do
4285 not have to do anything.
4286 But, if you want to use <filename>systemd</filename>, you must
4287 take some steps as described in the following sections.
4288 </para>
4289
4290<!--
4291 <note>
4292 It is recommended that you create your own distribution configuration
4293 file to hold these settings instead of using your
4294 <filename>local.conf</filename> file.
4295 For information on creating your own distribution, see the
4296 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
4297 section.
4298 </note>
4299-->
4300
4301 <section id='using-systemd-exclusively'>
4302 <title>Using systemd Exclusively</title>
4303
4304 <para>
4305 Set the following variables in your distribution configuration
4306 file as follows:
4307 <literallayout class='monospaced'>
4308 DISTRO_FEATURES_append = " systemd"
4309 VIRTUAL-RUNTIME_init_manager = "systemd"
4310 </literallayout>
4311 You can also prevent the <filename>sysvinit</filename>
4312 distribution feature from
4313 being automatically enabled as follows:
4314 <literallayout class='monospaced'>
4315 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
4316 </literallayout>
4317 Doing so removes any redundant <filename>sysvinit</filename>
4318 scripts.
4319 </para>
4320
4321 <para>
4322 For information on the backfill variable, see
4323 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
4324 in the Yocto Project Reference Manual.
4325 </para>
4326 </section>
4327
4328 <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'>
4329 <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title>
4330
4331 <para>
4332 Set the following variables in your distribution configuration
4333 file as follows:
4334 <literallayout class='monospaced'>
4335 DISTRO_FEATURES_append = " systemd"
4336 VIRTUAL-RUNTIME_init_manager = "systemd"
4337 </literallayout>
4338 Doing so causes your main image to use the
4339 <filename>packagegroup-core-boot.bb</filename> recipe and
4340 <filename>systemd</filename>.
4341 The rescue/minimal image cannot use this package group.
4342 However, it can install <filename>sysvinit</filename>
4343 and the appropriate packages will have support for both
4344 <filename>systemd</filename> and <filename>sysvinit</filename>.
4345 </para>
4346 </section>
4347 </section>
4348
4349 <section id='excluding-recipes-from-the-build'>
4350 <title>Excluding Recipes From the Build</title>
4351
4352 <para>
4353 You might find that there are groups of recipes or append files
4354 that you want to filter out of the build process.
4355 Usually, this is not necessary.
4356 However, on rare occasions where you might want to use a
4357 layer but exclude parts that are causing problems, such
4358 as introducing a different version of a recipe, you can
4359 use
4360 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBMASK'><filename>BBMASK</filename></ulink>
4361 to exclude the recipe.
4362 </para>
4363
4364 <para>
4365 It is possible to filter or mask out <filename>.bb</filename> and
4366 <filename>.bbappend</filename> files.
4367 You can do this by providing an expression with the
4368 <filename>BBMASK</filename> variable.
4369 Here is an example:
4370 <literallayout class='monospaced'>
4371 BBMASK = "/meta-mymachine/recipes-maybe/"
4372 </literallayout>
4373 Here, all <filename>.bb</filename> and
4374 <filename>.bbappend</filename> files in the directory that match
4375 the expression are ignored during the build process.
4376 </para>
4377
4378 <note>
4379 The value you provide is passed to Python's regular expression
4380 compiler.
4381 The expression is compared against the full paths to the files.
4382 For complete syntax information, see Python's documentation at
4383 <ulink url='http://docs.python.org/release/2.3/lib/re-syntax.html'></ulink>.
4384 </note>
4385 </section>
4386
4387 <section id="platdev-appdev-srcrev">
4388 <title>Using an External SCM</title>
4389
4390 <para>
4391 If you're working on a recipe that pulls from an external Source Code Manager (SCM), it
4392 is possible to have the OpenEmbedded build system notice new recipe changes added to the
4393 SCM and then build the resulting package that depends on the new recipes by using the latest
4394 versions.
4395 This only works for SCMs from which it is possible to get a sensible revision number for changes.
4396 Currently, you can do this with Apache Subversion (SVN), Git, and Bazaar (BZR) repositories.
4397 </para>
4398
4399 <para>
4400 To enable this behavior, simply add the following to the <filename>local.conf</filename>
4401 configuration file found in the
4402 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
4403 <literallayout class='monospaced'>
4404 SRCREV_pn-&lt;PN&gt; = "${AUTOREV}"
4405 </literallayout>
4406 where <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>
4407 is the name of the recipe for which you want to enable automatic source
4408 revision updating.
4409 </para>
4410 </section>
4411
4412 <section id='creating-a-read-only-root-filesystem'>
4413 <title>Creating a Read-Only Root Filesystem</title>
4414
4415 <para>
4416 Suppose, for security reasons, you need to disable
4417 your target device's root filesystem's write permissions
4418 (i.e. you need a read-only root filesystem).
4419 Or, perhaps you are running the device's operating system
4420 from a read-only storage device.
4421 For either case, you can customize your image for
4422 that behavior.
4423 </para>
4424
4425 <note>
4426 Supporting a read-only root filesystem requires that the system and
4427 applications do not try to write to the root filesystem.
4428 You must configure all parts of the target system to write
4429 elsewhere, or to gracefully fail in the event of failing to
4430 write to the root filesystem.
4431 </note>
4432
4433 <section id='creating-the-root-filesystem'>
4434 <title>Creating the Root Filesystem</title>
4435
4436 <para>
4437 To create the read-only root filesystem, simply add the
4438 <filename>read-only-rootfs</filename> feature to your image.
4439 Using either of the following statements in your
4440 image recipe or from within the
4441 <filename>local.conf</filename> file found in the
4442 <link linkend='build-directory'>Build Directory</link>
4443 causes the build system to create a read-only root filesystem:
4444 <literallayout class='monospaced'>
4445 IMAGE_FEATURES = "read-only-rootfs"
4446 </literallayout>
4447 or
4448 <literallayout class='monospaced'>
4449 EXTRA_IMAGE_FEATURES = "read-only-rootfs"
4450 </literallayout>
4451 </para>
4452
4453 <para>
4454 For more information on how to use these variables, see the
4455 "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>"
4456 section.
4457 For information on the variables, see
4458 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
4459 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>.
4460 </para>
4461 </section>
4462
4463 <section id='post-installation-scripts'>
4464 <title>Post-Installation Scripts</title>
4465
4466 <para>
4467 It is very important that you make sure all
4468 post-Installation (<filename>pkg_postinst</filename>) scripts
4469 for packages that are installed into the image can be run
4470 at the time when the root filesystem is created during the
4471 build on the host system.
4472 These scripts cannot attempt to run during first-boot on the
4473 target device.
4474 With the <filename>read-only-rootfs</filename> feature enabled,
4475 the build system checks during root filesystem creation to make
4476 sure all post-installation scripts succeed.
4477 If any of these scripts still need to be run after the root
4478 filesystem is created, the build immediately fails.
4479 These checks during build time ensure that the build fails
4480 rather than the target device fails later during its
4481 initial boot operation.
4482 </para>
4483
4484 <para>
4485 Most of the common post-installation scripts generated by the
4486 build system for the out-of-the-box Yocto Project are engineered
4487 so that they can run during root filesystem creation
4488 (e.g. post-installation scripts for caching fonts).
4489 However, if you create and add custom scripts, you need
4490 to be sure they can be run during file system creation.
4491 </para>
4492
4493 <para>
4494 Here are some common problems that prevent
4495 post-installation scripts from running during root filesystem
4496 creation:
4497 <itemizedlist>
4498 <listitem><para><emphasis>Not using $D in front of absolute paths:</emphasis>
4499 The build system defines
4500 <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
4501 at root filesystem creation time, and
4502 it is blank when run on the target device.
4503 This implies two purposes for <filename>$D</filename>:
4504 ensuring paths are valid in both the host and target
4505 environments, and checking to determine which
4506 environment is being used as a method for taking
4507 appropriate actions.
4508 </para></listitem>
4509 <listitem><para><emphasis>Attempting to run processes that are
4510 specific to or dependent on the target
4511 architecture:</emphasis>
4512 You can work around these attempts by using native
4513 tools to accomplish the same tasks, or
4514 by alternatively running the processes under QEMU,
4515 which has the <filename>qemu_run_binary</filename>
4516 function.
4517 For more information, see the
4518 <filename>meta/classes/qemu.bbclass</filename>
4519 class in the
4520 <link linkend='source-directory'>Source Directory</link>.
4521 </para></listitem>
4522 </itemizedlist>
4523 </para>
4524 </section>
4525
4526 <section id='areas-with-write-access'>
4527 <title>Areas With Write Access</title>
4528
4529 <para>
4530 With the <filename>read-only-rootfs</filename> feature enabled,
4531 any attempt by the target to write to the root filesystem at
4532 runtime fails.
4533 Consequently, you must make sure that you configure processes
4534 and applications that attempt these types of writes do so
4535 to directories with write access (e.g.
4536 <filename>/tmp</filename> or <filename>/var/run</filename>).
4537 </para>
4538 </section>
4539 </section>
4540
4541 <section id="performing-automated-runtime-testing">
4542 <title>Performing Automated Runtime Testing</title>
4543
4544 <para>
4545 The OpenEmbedded build system makes available a series of automated
4546 tests for images to verify runtime functionality.
4547 <note>
4548 Currently, there is only support for running these tests
4549 under QEMU.
4550 </note>
4551 These tests are written in Python making use of the
4552 <filename>unittest</filename> module, and the majority of them
4553 run commands on the target system over
4554 <filename>ssh</filename>.
4555 This section describes how you set up the environment to use these
4556 tests, run available tests, and write and add your own tests.
4557 </para>
4558
4559 <section id="qemu-image-enabling-tests">
4560 <title>Enabling Tests</title>
4561
4562 <para>
4563 In order to run tests, you need to do the following:
4564 <itemizedlist>
4565 <listitem><para><emphasis>Set up to avoid interaction
4566 with <filename>sudo</filename> for networking:</emphasis>
4567 To accomplish this, you must do one of the
4568 following:
4569 <itemizedlist>
4570 <listitem><para>Add
4571 <filename>NOPASSWD</filename> for your user
4572 in <filename>/etc/sudoers</filename> either for
4573 ALL commands or just for
4574 <filename>runqemu-ifup</filename>.
4575 You must provide the full path as that can
4576 change if you are using multiple clones of the
4577 source repository.
4578 <note>
4579 On some distributions, you also need to
4580 comment out "Defaults requiretty" in
4581 <filename>/etc/sudoers</filename>.
4582 </note></para></listitem>
4583 <listitem><para>Manually configure a tap interface
4584 for your system.</para></listitem>
4585 <listitem><para>Run as root the script in
4586 <filename>scripts/runqemu-gen-tapdevs</filename>,
4587 which should generate a list of tap devices.
4588 This is the option typically chosen for
4589 Autobuilder-type environments.
4590 </para></listitem>
4591 </itemizedlist></para></listitem>
4592 <listitem><para><emphasis>Set the
4593 <filename>DISPLAY</filename> variable:</emphasis>
4594 You need to set this variable so that you have an X
4595 server available (e.g. start
4596 <filename>vncserver</filename> for a headless machine).
4597 </para></listitem>
4598 <listitem><para><emphasis>Be sure your host's firewall
4599 accepts incoming connections from
4600 192.168.7.0/24:</emphasis>
4601 Some of the tests (in particular smart tests) start a
4602 HTTP server on a random high number port, which is
4603 used to serve files to the target.
4604 The smart module serves
4605 <filename>${DEPLOY_DIR}/rpm</filename> so it can run
4606 smart channel commands. That means your host's firewall
4607 must accept incoming connections from 192.168.7.0/24,
4608 which is the default IP range used for tap devices
4609 by <filename>runqemu</filename>.</para></listitem>
4610 </itemizedlist>
4611 </para>
4612
4613 <note>
4614 Regardless of how you initiate the tests, if you built your
4615 image using <filename>rm_work</filename>,
4616 most of the tests will fail with errors because they rely on
4617 <filename>${WORKDIR}/installed_pkgs.txt</filename>.
4618 </note>
4619 </section>
4620
4621 <section id="qemu-image-running-tests">
4622 <title>Running Tests</title>
4623
4624 <para>
4625 You can start the tests automatically or manually:
4626 <itemizedlist>
4627 <listitem><para><emphasis>Automatically Running Tests:</emphasis>
4628 To run the tests automatically after the
4629 OpenEmbedded build system successfully creates an image,
4630 first set the
4631 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
4632 variable to "1" in your <filename>local.conf</filename>
4633 file in the
4634 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
4635 <literallayout class='monospaced'>
4636 TEST_IMAGE = "1"
4637 </literallayout>
4638 Next, simply build your image.
4639 If the image successfully builds, the tests will be
4640 run:
4641 <literallayout class='monospaced'>
4642 bitbake core-image-sato
4643 </literallayout></para></listitem>
4644 <listitem><para><emphasis>Manually Running Tests:</emphasis>
4645 To manually run the tests, first globally inherit
4646 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename></ulink>
4647 by editing your <filename>local.conf</filename> file:
4648 <literallayout class='monospaced'>
4649 INHERIT += "testimage"
4650 </literallayout>
4651 Next, use BitBake to run the tests:
4652 <literallayout class='monospaced'>
4653 bitbake -c testimage &lt;image&gt;
4654 </literallayout></para></listitem>
4655 </itemizedlist>
4656 </para>
4657
4658 <para>
4659 Regardless of how you run the tests, once they start, the
4660 following happens:
4661 <itemizedlist>
4662 <listitem><para>A copy of the root filesystem is written
4663 to <filename>${WORKDIR}/testimage</filename>.
4664 </para></listitem>
4665 <listitem><para>The image is booted under QEMU using the
4666 standard <filename>runqemu</filename> script.
4667 </para></listitem>
4668 <listitem><para>A default timeout of 500 seconds occurs
4669 to allow for the boot process to reach the login prompt.
4670 You can change the timeout period by setting
4671 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink>
4672 in the <filename>local.conf</filename> file.
4673 </para></listitem>
4674 <listitem><para>Once the boot process is reached and the
4675 login prompt appears, the tests run.
4676 The full boot log is written to
4677 <filename>${WORKDIR}/testimage/qemu_boot_log</filename>.
4678 </para></listitem>
4679 <listitem><para>Each test module loads in the order found
4680 in <filename>TEST_SUITES</filename>.
4681 You can find the full output of the commands run over
4682 <filename>ssh</filename> in
4683 <filename>${WORKDIR}/testimgage/ssh_target_log</filename>.
4684 </para></listitem>
4685 <listitem><para>If no failures occur, the task running the
4686 tests ends successfully.
4687 You can find the output from the
4688 <filename>unittest</filename> in the task log at
4689 <filename>${WORKDIR}/temp/log.do_testimage</filename>.
4690 </para></listitem>
4691 </itemizedlist>
4692 </para>
4693
4694 <para>
4695 All test files reside in
4696 <filename>meta/lib/oeqa/runtime</filename> in the
4697 <link linkend='source-directory'>Source Directory</link>.
4698 A test name maps directly to a Python module.
4699 Each test module may contain a number of individual tests.
4700 Tests are usually grouped together by the area
4701 tested (e.g tests for <filename>systemd</filename> reside in
4702 <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
4703 </para>
4704
4705 <para>
4706 You can add tests to any layer provided you place them in the
4707 proper area and you extend
4708 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
4709 in the <filename>local.conf</filename> file as normal.
4710 Be sure that tests reside in
4711 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename>.
4712 <note>
4713 Be sure that module names do not collide with module names
4714 used in the default set of test modules in
4715 <filename>meta/lib/oeqa/runtime</filename>.
4716 </note>
4717 </para>
4718
4719 <para>
4720 You can change the set of tests run by appending or overriding
4721 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>
4722 variable in <filename>local.conf</filename>.
4723 Each name in <filename>TEST_SUITES</filename> represents a
4724 required test for the image.
4725 Test modules named within <filename>TEST_SUITES</filename>
4726 cannot be skipped even if a test is not suitable for an image
4727 (e.g. running the rpm tests on an image without
4728 <filename>rpm</filename>).
4729 Appending "auto" to <filename>TEST_SUITES</filename> causes the
4730 build system to try to run all tests that are suitable for the
4731 image (i.e. each test module may elect to skip itself).
4732 </para>
4733
4734 <para>
4735 The order you list tests in <filename>TEST_SUITES</filename>
4736 is important.
4737 The order influences test dependencies.
4738 Consequently, tests that depend on other tests should be added
4739 after the test on which they depend.
4740 For example, since <filename>ssh</filename> depends on the
4741 <filename>ping</filename> test, <filename>ssh</filename>
4742 needs to come after <filename>ping</filename> in the list.
4743 The test class provides no re-ordering or dependency handling.
4744 <note>
4745 Each module can have multiple classes with multiple test
4746 methods.
4747 And, Python <filename>unittest</filename> rules apply.
4748 </note>
4749 </para>
4750
4751 <para>
4752 Here are some things to keep in mind when running tests:
4753 <itemizedlist>
4754 <listitem><para>The default tests for the image are defined
4755 as:
4756 <literallayout class='monospaced'>
4757 DEFAULT_TEST_SUITES_pn-&lt;image&gt; = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
4758 </literallayout></para></listitem>
4759 <listitem><para>Add your own test to the list of the
4760 by using the following:
4761 <literallayout class='monospaced'>
4762 TEST_SUITES_append = " mytest"
4763 </literallayout></para></listitem>
4764 <listitem><para>Run a specific list of tests as follows:
4765 <literallayout class='monospaced'>
4766 TEST_SUITES = "test1 test2 test3"
4767 </literallayout>
4768 Remember, order is important.
4769 Be sure to place a test that is dependent on another test
4770 later in the order.</para></listitem>
4771 </itemizedlist>
4772 </para>
4773 </section>
4774
4775 <section id="qemu-image-writing-new-tests">
4776 <title>Writing New Tests</title>
4777
4778 <para>
4779 As mentioned previously, all new test files need to be in the
4780 proper place for the build system to find them.
4781 New tests for additional functionality outside of the core
4782 should be added to the layer that adds the functionality, in
4783 <filename>&lt;layer&gt;/lib/oeqa/runtime</filename> (as
4784 long as
4785 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
4786 is extended in the layer's
4787 <filename>layer.conf</filename> file as normal).
4788 Just remember that filenames need to map directly to test
4789 (module) names and that you do not use module names that
4790 collide with existing core tests.
4791 </para>
4792
4793 <para>
4794 To create a new test, start by copying an existing module
4795 (e.g. <filename>syslog.py</filename> or
4796 <filename>gcc.py</filename> are good ones to use).
4797 Test modules can use code from
4798 <filename>meta/lib/oeqa/utils</filename>, which are helper
4799 classes.
4800 </para>
4801
4802 <note>
4803 Structure shell commands such that you rely on them and they
4804 return a single code for success.
4805 Be aware that sometimes you will need to parse the output.
4806 See the <filename>df.py</filename> and
4807 <filename>date.py</filename> modules for examples.
4808 </note>
4809
4810 <para>
4811 You will notice that all test classes inherit
4812 <filename>oeRuntimeTest</filename>, which is found in
4813 <filename>meta/lib/oetest.py</filename>.
4814 This base class offers some helper attributes, which are
4815 described in the following sections:
4816 </para>
4817
4818 <section id='qemu-image-writing-tests-class-methods'>
4819 <title>Class Methods</title>
4820
4821 <para>
4822 Class methods are as follows:
4823 <itemizedlist>
4824 <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis>
4825 Returns "True" if <filename>pkg</filename> is in the
4826 installed package list of the image, which is based
4827 on
4828 <filename>${WORKDIR}/installed_pkgs.txt</filename>
4829 that is generated during the
4830 <filename>do.rootfs</filename> task.
4831 </para></listitem>
4832 <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis>
4833 Returns "True" if the feature is in
4834 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
4835 or
4836 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>.
4837 </para></listitem>
4838 <listitem><para><emphasis><filename>restartTarget(params)</filename>:</emphasis>
4839 Restarts the QEMU image optionally passing
4840 <filename>params</filename> to the
4841 <filename>runqemu</filename> script's
4842 <filename>qemuparams</filename> list (e.g "-m 1024" for
4843 more memory).</para></listitem>
4844 </itemizedlist>
4845 </para>
4846 </section>
4847
4848 <section id='qemu-image-writing-tests-class-attributes'>
4849 <title>Class Attributes</title>
4850
4851 <para>
4852 Class attributes are as follows:
4853 <itemizedlist>
4854 <listitem><para><emphasis><filename>pscmd</filename>:</emphasis>
4855 Equals "ps -ef" if <filename>procps</filename> is
4856 installed in the image.
4857 Otherwise, <filename>pscmd</filename> equals
4858 "ps" (busybox).
4859 </para></listitem>
4860 <listitem><para><emphasis><filename>tc</filename>:</emphasis>
4861 The called text context, which gives access to the
4862 following attributes:
4863 <itemizedlist>
4864 <listitem><para><emphasis><filename>d</filename>:</emphasis>
4865 The BitBake data store, which allows you to
4866 use stuff such as
4867 <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>.
4868 </para></listitem>
4869 <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis>
4870 Used internally.
4871 The tests do not need these.
4872 </para></listitem>
4873 <listitem><para><emphasis><filename>filesdir</filename>:</emphasis>
4874 The absolute path to
4875 <filename>meta/lib/oeqa/runtime/files</filename>,
4876 which contains helper files for tests meant
4877 for copying on the target such as small
4878 files written in C for compilation.
4879 </para></listitem>
4880 <listitem><para><emphasis><filename>qemu</filename>:</emphasis>
4881 Provides access to the
4882 <filename>QemuRunner</filename> object,
4883 which is the class that boots the image.
4884 The <filename>qemu</filename> attribute
4885 provides the following useful attributes:
4886 <itemizedlist>
4887 <listitem><para><emphasis><filename>ip</filename>:</emphasis>
4888 The machine's IP address.
4889 </para></listitem>
4890 <listitem><para><emphasis><filename>host_ip</filename>:</emphasis>
4891 The host IP address, which is only
4892 used by smart tests.
4893 </para></listitem>
4894 </itemizedlist></para></listitem>
4895 <listitem><para><emphasis><filename>target</filename>:</emphasis>
4896 The <filename>SSHControl</filename> object,
4897 which is used for running the following
4898 commands on the image:
4899 <itemizedlist>
4900 <listitem><para><emphasis><filename>host</filename>:</emphasis>
4901 Used internally.
4902 The tests do not use this command.
4903 </para></listitem>
4904 <listitem><para><emphasis><filename>timeout</filename>:</emphasis>
4905 A global timeout for commands run on
4906 the target for the instance of a
4907 test.
4908 The default is 300 seconds.
4909 </para></listitem>
4910 <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis>
4911 The single, most used method.
4912 This command is a wrapper for:
4913 <filename>ssh root@host "cmd"</filename>.
4914 The command returns a tuple:
4915 (status, output), which are what
4916 their names imply - the return code
4917 of 'cmd' and whatever output
4918 it produces.
4919 The optional timeout argument
4920 represents the number of seconds the
4921 test should wait for 'cmd' to
4922 return.
4923 If the argument is "None", the
4924 test uses the default instance's
4925 timeout period, which is 300
4926 seconds.
4927 If the argument is "0", the test
4928 runs until the command returns.
4929 </para></listitem>
4930 <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis>
4931 <filename>scp localpath root@ip:remotepath</filename>.
4932 </para></listitem>
4933 <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis>
4934 <filename>scp root@host:remotepath localpath</filename>.
4935 </para></listitem>
4936 </itemizedlist></para></listitem>
4937 </itemizedlist></para></listitem>
4938 </itemizedlist>
4939 </para>
4940 </section>
4941
4942 <section id='qemu-image-writing-tests-instance-attributes'>
4943 <title>Instance Attributes</title>
4944
4945 <para>
4946 A single instance attribute exists, which is
4947 <filename>target</filename>.
4948 The <filename>target</filename> instance attribute is
4949 identical to the class attribute of the same name, which
4950 is described in the previous section.
4951 This attribute exists as both an instance and class
4952 attribute so tests can use
4953 <filename>self.target.run(cmd)</filename> in instance
4954 methods instead of
4955 <filename>oeRuntimeTest.tc.target.run(cmd)</filename>.
4956 </para>
4957 </section>
4958 </section>
4959 </section>
4960
4961 <section id="platdev-gdb-remotedebug">
4962 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title>
4963
4964 <para>
4965 GDB allows you to examine running programs, which in turn helps you to understand and fix problems.
4966 It also allows you to perform post-mortem style analysis of program crashes.
4967 GDB is available as a package within the Yocto Project and is
4968 installed in SDK images by default.
4969 See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter
4970 in the Yocto Project Reference Manual for a description of these images.
4971 You can find information on GDB at <ulink url="http://sourceware.org/gdb/"/>.
4972 </para>
4973
4974 <tip>
4975 For best results, install <filename>-dbg</filename> packages for
4976 the applications you are going to debug.
4977 Doing so makes extra debug symbols available that give you more
4978 meaningful output.
4979 </tip>
4980
4981 <para>
4982 Sometimes, due to memory or disk space constraints, it is not possible
4983 to use GDB directly on the remote target to debug applications.
4984 These constraints arise because GDB needs to load the debugging information and the
4985 binaries of the process being debugged.
4986 Additionally, GDB needs to perform many computations to locate information such as function
4987 names, variable names and values, stack traces and so forth - even before starting the
4988 debugging process.
4989 These extra computations place more load on the target system and can alter the
4990 characteristics of the program being debugged.
4991 </para>
4992
4993 <para>
4994 To help get past the previously mentioned constraints, you can use Gdbserver.
4995 Gdbserver runs on the remote target and does not load any debugging information
4996 from the debugged process.
4997 Instead, a GDB instance processes the debugging information that is run on a
4998 remote computer - the host GDB.
4999 The host GDB then sends control commands to Gdbserver to make it stop or start the debugged
5000 program, as well as read or write memory regions of that debugged program.
5001 All the debugging information loaded and processed as well
5002 as all the heavy debugging is done by the host GDB.
5003 Offloading these processes gives the Gdbserver running on the target a chance to remain
5004 small and fast.
5005 </para>
5006
5007 <para>
5008 Because the host GDB is responsible for loading the debugging information and
5009 for doing the necessary processing to make actual debugging happen, the
5010 user has to make sure the host can access the unstripped binaries complete
5011 with their debugging information and also be sure the target is compiled with no optimizations.
5012 The host GDB must also have local access to all the libraries used by the
5013 debugged program.
5014 Because Gdbserver does not need any local debugging information, the binaries on
5015 the remote target can remain stripped.
5016 However, the binaries must also be compiled without optimization
5017 so they match the host's binaries.
5018 </para>
5019
5020 <para>
5021 To remain consistent with GDB documentation and terminology, the binary being debugged
5022 on the remote target machine is referred to as the "inferior" binary.
5023 For documentation on GDB see the
5024 <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>.
5025 </para>
5026
5027 <para>
5028 The remainder of this section describes the steps you need to take
5029 to debug using the GNU project debugger.
5030 </para>
5031
5032 <section id='platdev-gdb-remotedebug-setup'>
5033 <title>Set Up the Cross-Development Debugging Environment</title>
5034
5035 <para>
5036 Before you can initiate a remote debugging session, you need
5037 to be sure you have set up the cross-development environment,
5038 toolchain, and sysroot.
5039 The "<ulink url='&YOCTO_DOCS_ADT_URL;#adt-prepare'>Preparing for Application Development</ulink>"
5040 chapter of the Yocto Project Application Developer's Guide
5041 describes this process.
5042 Be sure you have read that chapter and have set up
5043 your environment.
5044 </para>
5045 </section>
5046
5047 <section id="platdev-gdb-remotedebug-launch-gdbserver">
5048 <title>Launch Gdbserver on the Target</title>
5049
5050 <para>
5051 Make sure Gdbserver is installed on the target.
5052 If it is not, install the package
5053 <filename>gdbserver</filename>, which needs the
5054 <filename>libthread-db1</filename> package.
5055 </para>
5056
5057 <para>
5058 Here is an example that when entered from the host
5059 connects to the target and launches Gdbserver in order to
5060 "debug" a binary named <filename>helloworld</filename>:
5061 <literallayout class='monospaced'>
5062 $ gdbserver localhost:2345 /usr/bin/helloworld
5063 </literallayout>
5064 Gdbserver should now be listening on port 2345 for debugging
5065 commands coming from a remote GDB process that is running on
5066 the host computer.
5067 Communication between Gdbserver and the host GDB are done
5068 using TCP.
5069 To use other communication protocols, please refer to the
5070 <ulink url='http://www.gnu.org/software/gdb/'>Gdbserver documentation</ulink>.
5071 </para>
5072 </section>
5073
5074 <section id="platdev-gdb-remotedebug-launch-gdb">
5075 <title>Launch GDB on the Host Computer</title>
5076
5077 <para>
5078 Running GDB on the host computer takes a number of stages, which
5079 this section describes.
5080 </para>
5081
5082 <section id="platdev-gdb-remotedebug-launch-gdb-buildcross">
5083 <title>Build the Cross-GDB Package</title>
5084 <para>
5085 A suitable GDB cross-binary is required that runs on your
5086 host computer but also knows about the the ABI of the
5087 remote target.
5088 You can get this binary from the
5089 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
5090 Here is an example where the toolchain has been installed
5091 in the default directory
5092 <filename>/opt/poky/&DISTRO;</filename>:
5093 <literallayout class='monospaced'>
5094 /opt/poky/1.4/sysroots/i686-pokysdk-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi/arm-poky-linux-gnueabi-gdb
5095 </literallayout>
5096 where <filename>arm</filename> is the target architecture
5097 and <filename>linux-gnueabi</filename> is the target ABI.
5098 </para>
5099
5100 <para>
5101 Alternatively, you can use BitBake to build the
5102 <filename>gdb-cross</filename> binary.
5103 Here is an example:
5104 <literallayout class='monospaced'>
5105 $ bitbake gdb-cross
5106 </literallayout>
5107 Once the binary is built, you can find it here:
5108 <literallayout class='monospaced'>
5109 tmp/sysroots/&lt;host-arch&gt;/usr/bin/&lt;target-platform&gt;/&lt;target-abi&gt;-gdb
5110 </literallayout>
5111 </para>
5112 </section>
5113
5114 <section id='create-the-gdb-initialization-file'>
5115 <title>Create the GDB Initialization File and Point to Your Root Filesystem</title>
5116
5117 <para>
5118 Aside from the GDB cross-binary, you also need a GDB
5119 initialization file in the same top directory in which
5120 your binary resides.
5121 When you start GDB on your host development system, GDB
5122 finds this initialization file and executes all the
5123 commands within.
5124 For information on the <filename>.gdbinit</filename>, see
5125 "<ulink url='http://sourceware.org/gdb/onlinedocs/gdb/'>Debugging with GDB</ulink>",
5126 which is maintained by
5127 <ulink url='http://www.sourceware.org'>sourceware.org</ulink>.
5128 </para>
5129
5130 <para>
5131 You need to add a statement in the
5132 <filename>.gdbinit</filename> file that points to your
5133 root filesystem.
5134 Here is an example that points to the root filesystem for
5135 an ARM-based target device:
5136 <literallayout class='monospaced'>
5137 set sysroot /home/jzhang/sysroot_arm
5138 </literallayout>
5139 </para>
5140 </section>
5141
5142 <section id="platdev-gdb-remotedebug-launch-gdb-launchhost">
5143 <title>Launch the Host GDB</title>
5144
5145 <para>
5146 Before launching the host GDB, you need to be sure
5147 you have sourced the cross-debugging environment script,
5148 which if you installed the root filesystem in the default
5149 location is at <filename>/opt/poky/&DISTRO;</filename>
5150 and begins with the string "environment-setup".
5151 For more information, see the
5152 "<ulink url='&YOCTO_DOCS_ADT_URL;#setting-up-the-cross-development-environment'>Setting Up the Cross-Development Environment</ulink>"
5153 section in the Yocto Project Application Developer's
5154 Guide.
5155 </para>
5156
5157 <para>
5158 Finally, switch to the directory where the binary resides
5159 and run the <filename>cross-gdb</filename> binary.
5160 Provide the binary file you are going to debug.
5161 For example, the following command continues with the
5162 example used in the previous section by loading
5163 the <filename>helloworld</filename> binary as well as the
5164 debugging information:
5165 <literallayout class='monospaced'>
5166 $ arm-poky-linux-gnuabi-gdb helloworld
5167 </literallayout>
5168 The commands in your <filename>.gdbinit</filename> execute
5169 and the GDB prompt appears.
5170 </para>
5171 </section>
5172 </section>
5173
5174 <section id='platdev-gdb-connect-to-the-remote-gdb-server'>
5175 <title>Connect to the Remote GDB Server</title>
5176
5177 <para>
5178 From the target, you need to connect to the remote GDB
5179 server that is running on the host.
5180 You need to specify the remote host and port.
5181 Here is the command continuing with the example:
5182 <literallayout class='monospaced'>
5183 target remote 192.168.7.2:2345
5184 </literallayout>
5185 </para>
5186 </section>
5187
5188 <section id="platdev-gdb-remotedebug-launch-gdb-using">
5189 <title>Use the Debugger</title>
5190
5191 <para>
5192 You can now proceed with debugging as normal - as if you were debugging
5193 on the local machine.
5194 For example, to instruct GDB to break in the "main" function and then
5195 continue with execution of the inferior binary use the following commands
5196 from within GDB:
5197 <literallayout class='monospaced'>
5198 (gdb) break main
5199 (gdb) continue
5200 </literallayout>
5201 </para>
5202
5203 <para>
5204 For more information about using GDB, see the project's online documentation at
5205 <ulink url="http://sourceware.org/gdb/download/onlinedocs/"/>.
5206 </para>
5207 </section>
5208 </section>
5209
5210 <section id="examining-builds-using-toaster">
5211 <title>Examining Builds Using the Toaster API</title>
5212
5213 <para>
5214 Toaster is an Application Programming Interface (API) and
5215 web-based interface to the OpenEmbedded build system, which uses
5216 BitBake.
5217 Both interfaces are based on a Representational State Transfer
5218 (REST) API that queries for and returns build information using
5219 <filename>GET</filename> and <filename>JSON</filename>.
5220 These types of search operations retrieve sets of objects from
5221 a data store used to collect build information.
5222 The results contain all the data for the objects being returned.
5223 You can order the results of the search by key and the search
5224 parameters are consistent for all object types.
5225 </para>
5226
5227 <para>
5228 Using the interfaces you can do the following:
5229 <itemizedlist>
5230 <listitem><para>See information about the tasks executed
5231 and reused during the build.</para></listitem>
5232 <listitem><para>See what is built (recipes and
5233 packages) and what packages were installed into the final
5234 image.</para></listitem>
5235 <listitem><para>See performance-related information such
5236 as build time, CPU usage, and disk I/O.</para></listitem>
5237 <listitem><para>Examine error, warning and trace messages
5238 to aid in debugging.</para></listitem>
5239 </itemizedlist>
5240 </para>
5241
5242 <note>
5243 <para>This release of Toaster provides you with information
5244 about a BitBake run.
5245 The tool does not allow you to configure and launch a build.
5246 However, future development includes plans to integrate the
5247 configuration and build launching capabilities of
5248 <ulink url='&YOCTO_HOME_URL;/tools-resources/projects/hob'>Hob</ulink>.
5249 </para>
5250 <para>For more information on using Hob to build an image,
5251 see the
5252 "<link linkend='image-development-using-hob'>Image Development Using Hob</link>"
5253 section.</para>
5254 </note>
5255
5256 <para>
5257 The remainder of this section describes what you need to have in
5258 place to use Toaster, how to start it, use it, and stop it.
5259 For additional information on installing and running Toaster, see the
5260 "<ulink url='https://wiki.yoctoproject.org/wiki/Toaster#Installation_and_Running'>Installation and Running</ulink>"
5261 section of the "Toaster" wiki page.
5262 For complete information on the API and its search operation
5263 URI, parameters, and responses, see the
5264 <ulink url='https://wiki.yoctoproject.org/wiki/REST_API_Contracts'>REST API Contracts</ulink>
5265 Wiki page.
5266 </para>
5267
5268 <section id='starting-toaster'>
5269 <title>Starting Toaster</title>
5270
5271 <para>
5272 Getting set up to use and start Toaster is simple.
5273 First, be sure you have met the following requirements:
5274 <itemizedlist>
5275 <listitem><para>You have set up your
5276 <link linkend='source-directory'>Source Directory</link>
5277 by cloning the upstream <filename>poky</filename>
5278 repository.
5279 See the
5280 <link linkend='local-yp-release'>Yocto Project Release</link>
5281 item for information on how to set up the Source
5282 Directory.</para></listitem>
5283 <listitem><para>You have checked out the
5284 <filename>dora-toaster</filename> branch:
5285 <literallayout class='monospaced'>
5286 $ cd ~/poky
5287 $ git checkout -b dora-toaster origin/dora-toaster
5288 </literallayout></para></listitem>
5289 <listitem><para>Be sure your build machine has
5290 <ulink url='http://en.wikipedia.org/wiki/Django_%28web_framework%29'>Django</ulink>
5291 version 1.4.5 installed.</para></listitem>
5292 <listitem><para>Make sure that port 8000 and 8200 are
5293 free (i.e. they have no servers on them).
5294 </para></listitem>
5295 </itemizedlist>
5296 </para>
5297
5298 <para>
5299 Once you have met the requirements, follow these steps to
5300 start Toaster running in the background of your shell:
5301 <orderedlist>
5302 <listitem><para><emphasis>Set up your build environment:</emphasis>
5303 Source a build environment script (i.e.
5304 <filename>oe-init-build-env</filename> or
5305 <filename>oe-init-build-env-memres</filename>).
5306 </para></listitem>
5307 <listitem><para><emphasis>Prepare your local configuration file:</emphasis>
5308 Toaster needs the Toaster class enabled
5309 in Bitbake in order to record target image package
5310 information.
5311 You can enable it by adding the following line to your
5312 <filename>conf/local.conf</filename> file:
5313 <literallayout class='monospaced'>
5314 INHERIT += "toaster"
5315 </literallayout>
5316 Toaster also needs Build History enabled in Bitbake in
5317 order to record target image package information.
5318 You can enable this by adding the following two lines
5319 to your <filename>conf/local.conf</filename> file:
5320 <literallayout class='monospaced'>
5321 INHERIT += "buildhistory"
5322 BUILDHISTORY_COMMIT = "1"
5323 </literallayout></para></listitem>
5324 <listitem><para><emphasis>Start Toaster:</emphasis>
5325 Start the Toaster service using this
5326 command from within your build directory:
5327 <literallayout class='monospaced'>
5328 $ source toaster start
5329 </literallayout></para></listitem>
5330 <note>
5331 The Toaster must be started and running in order
5332 for it to collect data.
5333 </note>
5334 </orderedlist>
5335 </para>
5336
5337 <para>
5338 When Toaster starts, it creates some additional files in your
5339 Build Directory.
5340 Deleting these files will cause you to lose data or interrupt
5341 Toaster:
5342 <itemizedlist>
5343 <listitem><para><emphasis><filename>toaster.sqlite</filename>:</emphasis>
5344 Toaster's database file.</para></listitem>
5345 <listitem><para><emphasis><filename>toaster_web.log</filename>:</emphasis>
5346 The log file of the web server.</para></listitem>
5347 <listitem><para><emphasis><filename>toaster_ui.log</filename>:</emphasis>
5348 The log file of the user interface component.
5349 </para></listitem>
5350 <listitem><para><emphasis><filename>toastermain.pid</filename>:</emphasis>
5351 The PID of the web server.</para></listitem>
5352 <listitem><para><emphasis><filename>toasterui.pid</filename>:</emphasis>
5353 The PID of the DSI data bridge.</para></listitem>
5354 <listitem><para><emphasis><filename>bitbake-cookerdaemon.log</filename>:</emphasis>
5355 The BitBake server's log file.</para></listitem>
5356 </itemizedlist>
5357 </para>
5358 </section>
5359
5360 <section id='using-toaster'>
5361 <title>Using Toaster</title>
5362
5363 <para>
5364 Once Toaster is running, it logs information for any BitBake
5365 run from your Build Directory.
5366 This logging is automatic.
5367 All you need to do is access and use the information.
5368 </para>
5369
5370 <para>
5371 You access the information one of two ways:
5372 <itemizedlist>
5373 <listitem><para>Open a Browser and type enter in the
5374 <filename>http://localhost:8000</filename> URL.
5375 </para></listitem>
5376 <listitem><para>Use the <filename>xdg-open</filename>
5377 tool from the shell and pass it the same URL.
5378 </para></listitem>
5379 </itemizedlist>
5380 Either method opens the home page for the Toaster interface,
5381 which is temporary for this release.
5382 </para>
5383 </section>
5384
5385 <section id='examining-toaster-data'>
5386 <title>Examining Toaster Data</title>
5387
5388 <para>
5389 The Toaster database is persistent regardless of whether you
5390 start or stop the service.
5391 </para>
5392
5393 <para>
5394 Toaster's interface shows you a list of builds
5395 (successful and unsuccessful) for which it has data.
5396 You can click on any build to see related information.
5397 This information includes configuration details, information
5398 about tasks, all recipes and packages built and their
5399 dependencies, packages installed in your final image,
5400 execution time, CPU usage and disk I/O per task.
5401 </para>
5402 </section>
5403
5404 <section id='stopping-toaster'>
5405 <title>Stopping Toaster</title>
5406
5407 <para>
5408 Stop the Toaster service with the following command:
5409 <literallayout class='monospaced'>
5410 $ source toaster stop
5411 </literallayout>
5412 The service stops but the Toaster database remains persistent.
5413 </para>
5414 </section>
5415 </section>
5416
5417 <section id="platdev-oprofile">
5418 <title>Profiling with OProfile</title>
5419
5420 <para>
5421 <ulink url="http://oprofile.sourceforge.net/">OProfile</ulink> is a
5422 statistical profiler well suited for finding performance
5423 bottlenecks in both user-space software and in the kernel.
5424 This profiler provides answers to questions like "Which functions does my application spend
5425 the most time in when doing X?"
5426 Because the OpenEmbedded build system is well integrated with OProfile, it makes profiling
5427 applications on target hardware straightforward.
5428 <note>
5429 For more information on how to set up and run OProfile, see the
5430 "<ulink url='&YOCTO_DOCS_PROF_URL;#profile-manual-oprofile'>OProfile</ulink>"
5431 section in the Yocto Project Profiling and Tracing Manual.
5432 </note>
5433 </para>
5434
5435 <para>
5436 To use OProfile, you need an image that has OProfile installed.
5437 The easiest way to do this is with <filename>tools-profile</filename> in the
5438 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'>IMAGE_FEATURES</ulink></filename> variable.
5439 You also need debugging symbols to be available on the system where the analysis
5440 takes place.
5441 You can gain access to the symbols by using <filename>dbg-pkgs</filename> in the
5442 <filename>IMAGE_FEATURES</filename> variable or by
5443 installing the appropriate <filename>-dbg</filename> packages.
5444 </para>
5445
5446 <para>
5447 For successful call graph analysis, the binaries must preserve the frame
5448 pointer register and should also be compiled with the
5449 <filename>-fno-omit-framepointer</filename> flag.
5450 You can achieve this by setting the
5451 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SELECTED_OPTIMIZATION'>SELECTED_OPTIMIZATION</ulink></filename>
5452 variable with the following options:
5453 <literallayout class='monospaced'>
5454 -fexpensive-optimizations
5455 -fno-omit-framepointer
5456 -frename-registers
5457 -O2
5458 </literallayout>
5459 You can also achieve it by setting the
5460 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_BUILD'>DEBUG_BUILD</ulink></filename>
5461 variable to "1" in the <filename>local.conf</filename> configuration file.
5462 If you use the <filename>DEBUG_BUILD</filename> variable,
5463 you also add extra debugging information that can make the debug
5464 packages large.
5465 </para>
5466
5467 <section id="platdev-oprofile-target">
5468 <title>Profiling on the Target</title>
5469
5470 <para>
5471 Using OProfile you can perform all the profiling work on the target device.
5472 A simple OProfile session might look like the following:
5473 </para>
5474
5475 <para>
5476 <literallayout class='monospaced'>
5477 # opcontrol --reset
5478 # opcontrol --start --separate=lib --no-vmlinux -c 5
5479 .
5480 .
5481 [do whatever is being profiled]
5482 .
5483 .
5484 # opcontrol --stop
5485 $ opreport -cl
5486 </literallayout>
5487 </para>
5488
5489 <para>
5490 In this example, the <filename>reset</filename> command clears any previously profiled data.
5491 The next command starts OProfile.
5492 The options used when starting the profiler separate dynamic library data
5493 within applications, disable kernel profiling, and enable callgraphing up to
5494 five levels deep.
5495 <note>
5496 To profile the kernel, you would specify the
5497 <filename>--vmlinux=/path/to/vmlinux</filename> option.
5498 The <filename>vmlinux</filename> file is usually in the source directory in the
5499 <filename>/boot/</filename> directory and must match the running kernel.
5500 </note>
5501 </para>
5502
5503 <para>
5504 After you perform your profiling tasks, the next command stops the profiler.
5505 After that, you can view results with the <filename>opreport</filename> command with options
5506 to see the separate library symbols and callgraph information.
5507 </para>
5508
5509 <para>
5510 Callgraphing logs information about time spent in functions and about a function's
5511 calling function (parent) and called functions (children).
5512 The higher the callgraphing depth, the more accurate the results.
5513 However, higher depths also increase the logging overhead.
5514 Consequently, you should take care when setting the callgraphing depth.
5515 <note>
5516 On ARM, binaries need to have the frame pointer enabled for callgraphing to work.
5517 To accomplish this use the <filename>-fno-omit-framepointer</filename> option
5518 with <filename>gcc</filename>.
5519 </note>
5520 </para>
5521
5522 <para>
5523 For more information on using OProfile, see the OProfile
5524 online documentation at
5525 <ulink url="http://oprofile.sourceforge.net/docs/"/>.
5526 </para>
5527 </section>
5528
5529 <section id="platdev-oprofile-oprofileui">
5530 <title>Using OProfileUI</title>
5531
5532 <para>
5533 A graphical user interface for OProfile is also available.
5534 You can download and build this interface from the Yocto Project at
5535 <ulink url="&YOCTO_GIT_URL;/cgit.cgi/oprofileui/"></ulink>.
5536 If the "tools-profile" image feature is selected, all necessary binaries
5537 are installed onto the target device for OProfileUI interaction.
5538 For a list of image features that ship with the Yocto Project,
5539 see the
5540 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-features-image'>Images</ulink>"
5541 section in the Yocto Project Reference Manual.
5542 </para>
5543
5544 <para>
5545 Even though the source directory usually includes all needed patches on the target device, you
5546 might find you need other OProfile patches for recent OProfileUI features.
5547 If so, see the <ulink url='&YOCTO_GIT_URL;/cgit.cgi/oprofileui/tree/README'>
5548 OProfileUI README</ulink> for the most recent information.
5549 </para>
5550
5551 <section id="platdev-oprofile-oprofileui-online">
5552 <title>Online Mode</title>
5553
5554 <para>
5555 Using OProfile in online mode assumes a working network connection with the target
5556 hardware.
5557 With this connection, you just need to run "oprofile-server" on the device.
5558 By default, OProfile listens on port 4224.
5559 <note>
5560 You can change the port using the <filename>--port</filename> command-line
5561 option.
5562 </note>
5563 </para>
5564
5565 <para>
5566 The client program is called <filename>oprofile-viewer</filename> and its UI is relatively
5567 straightforward.
5568 You access key functionality through the buttons on the toolbar, which
5569 are duplicated in the menus.
5570 Here are the buttons:
5571 <itemizedlist>
5572 <listitem><para><emphasis>Connect:</emphasis> Connects to the remote host.
5573 You can also supply the IP address or hostname.</para></listitem>
5574 <listitem><para><emphasis>Disconnect:</emphasis> Disconnects from the target.
5575 </para></listitem>
5576 <listitem><para><emphasis>Start:</emphasis> Starts profiling on the device.
5577 </para></listitem>
5578 <listitem><para><emphasis>Stop:</emphasis> Stops profiling on the device and
5579 downloads the data to the local host.
5580 Stopping the profiler generates the profile and displays it in the viewer.
5581 </para></listitem>
5582 <listitem><para><emphasis>Download:</emphasis> Downloads the data from the
5583 target and generates the profile, which appears in the viewer.</para></listitem>
5584 <listitem><para><emphasis>Reset:</emphasis> Resets the sample data on the device.
5585 Resetting the data removes sample information collected from previous
5586 sampling runs.
5587 Be sure you reset the data if you do not want to include old sample information.
5588 </para></listitem>
5589 <listitem><para><emphasis>Save:</emphasis> Saves the data downloaded from the
5590 target to another directory for later examination.</para></listitem>
5591 <listitem><para><emphasis>Open:</emphasis> Loads previously saved data.
5592 </para></listitem>
5593 </itemizedlist>
5594 </para>
5595
5596 <para>
5597 The client downloads the complete 'profile archive' from
5598 the target to the host for processing.
5599 This archive is a directory that contains the sample data, the object files,
5600 and the debug information for the object files.
5601 The archive is then converted using the <filename>oparchconv</filename> script, which is
5602 included in this distribution.
5603 The script uses <filename>opimport</filename> to convert the archive from
5604 the target to something that can be processed on the host.
5605 </para>
5606
5607 <para>
5608 Downloaded archives reside in the
5609 <link linkend='build-directory'>Build Directory</link> in
5610 <filename>/tmp</filename> and are cleared up when they are no longer in use.
5611 </para>
5612
5613 <para>
5614 If you wish to perform kernel profiling, you need to be sure
5615 a <filename>vmlinux</filename> file that matches the running kernel is available.
5616 In the source directory, that file is usually located in
5617 <filename>/boot/vmlinux-KERNELVERSION</filename>, where
5618 <filename>KERNEL-version</filename> is the version of the kernel.
5619 The OpenEmbedded build system generates separate <filename>vmlinux</filename>
5620 packages for each kernel it builds.
5621 Thus, it should just be a question of making sure a matching package is
5622 installed (e.g. <filename>opkg install kernel-vmlinux</filename>).
5623 The files are automatically installed into development and profiling images
5624 alongside OProfile.
5625 A configuration option exists within the OProfileUI settings page that you can use to
5626 enter the location of the <filename>vmlinux</filename> file.
5627 </para>
5628
5629 <para>
5630 Waiting for debug symbols to transfer from the device can be slow, and it
5631 is not always necessary to actually have them on the device for OProfile use.
5632 All that is needed is a copy of the filesystem with the debug symbols present
5633 on the viewer system.
5634 The "<link linkend='platdev-gdb-remotedebug-launch-gdb'>Launch GDB on the Host Computer</link>"
5635 section covers how to create such a directory with
5636 the <link linkend='source-directory'>Source Directory</link>
5637 and how to use the OProfileUI Settings Dialog to specify the location.
5638 If you specify the directory, it will be used when the file checksums
5639 match those on the system you are profiling.
5640 </para>
5641 </section>
5642
5643 <section id="platdev-oprofile-oprofileui-offline">
5644 <title>Offline Mode</title>
5645
5646 <para>
5647 If network access to the target is unavailable, you can generate
5648 an archive for processing in <filename>oprofile-viewer</filename> as follows:
5649 <literallayout class='monospaced'>
5650 # opcontrol --reset
5651 # opcontrol --start --separate=lib --no-vmlinux -c 5
5652 .
5653 .
5654 [do whatever is being profiled]
5655 .
5656 .
5657 # opcontrol --stop
5658 # oparchive -o my_archive
5659 </literallayout>
5660 </para>
5661
5662 <para>
5663 In the above example, <filename>my_archive</filename> is the name of the
5664 archive directory where you would like the profile archive to be kept.
5665 After the directory is created, you can copy it to another host and load it
5666 using <filename>oprofile-viewer</filename> open functionality.
5667 If necessary, the archive is converted.
5668 </para>
5669 </section>
5670 </section>
5671 </section>
5672
5673 <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'>
5674 <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title>
5675
5676 <para>
5677 One of the concerns for a development organization using open source
5678 software is how to maintain compliance with various open source
5679 licensing during the lifecycle of the product.
5680 While this section does not provide legal advice or
5681 comprehensively cover all scenarios, it does
5682 present methods that you can use to
5683 assist you in meeting the compliance requirements during a software
5684 release.
5685 </para>
5686
5687 <para>
5688 With hundreds of different open source licenses that the Yocto
5689 Project tracks, it is difficult to know the requirements of each
5690 and every license.
5691 However, we can begin to cover the requirements of the major FLOSS licenses, by
5692 assuming that there are three main areas of concern:
5693 <itemizedlist>
5694 <listitem><para>Source code must be provided.</para></listitem>
5695 <listitem><para>License text for the software must be
5696 provided.</para></listitem>
5697 <listitem><para>Compilation scripts and modifications to the
5698 source code must be provided.
5699 </para></listitem>
5700 </itemizedlist>
5701 There are other requirements beyond the scope of these
5702 three and the methods described in this section
5703 (e.g. the mechanism through which source code is distributed).
5704 </para>
5705
5706 <para>
5707 As different organizations have different methods of complying with
5708 open source licensing, this section is not meant to imply that
5709 there is only one single way to meet your compliance obligations,
5710 but rather to describe one method of achieving compliance.
5711 The remainder of this section describes methods supported to meet the
5712 previously mentioned three requirements.
5713 Once you take steps to meet these requirements,
5714 and prior to releasing images, sources, and the build system,
5715 you should audit all artifacts to ensure completeness.
5716 <note>
5717 The Yocto Project generates a license manifest during
5718 image creation that is located
5719 in <filename>${DEPLOY_DIR}/licenses/&lt;image_name-datestamp&gt;</filename>
5720 to assist with any audits.
5721 </note>
5722 </para>
5723
5724 <section id='providing-the-source-code'>
5725 <title>Providing the Source Code</title>
5726
5727 <para>
5728 Compliance activities should begin before you generate the
5729 final image.
5730 The first thing you should look at is the requirement that
5731 tops the list for most compliance groups - providing
5732 the source.
5733 The Yocto Project has a few ways of meeting this
5734 requirement.
5735 </para>
5736
5737 <para>
5738 One of the easiest ways to meet this requirement is
5739 to provide the entire
5740 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
5741 used by the build.
5742 This method, however, has a few issues.
5743 The most obvious is the size of the directory since it includes
5744 all sources used in the build and not just the source used in
5745 the released image.
5746 It will include toolchain source, and other artifacts, which
5747 you would not generally release.
5748 However, the more serious issue for most companies is accidental
5749 release of proprietary software.
5750 The Yocto Project provides an archiver class to help avoid
5751 some of these concerns.
5752 See the
5753 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'>Archiving Sources - <filename>archive*.bbclass</filename></ulink>"
5754 section in the Yocto Project Reference Manual for information
5755 on this class.
5756 </para>
5757
5758 <para>
5759 Before you employ <filename>DL_DIR</filename> or the
5760 archiver class, you need to decide how you choose to
5761 provide source.
5762 The source archiver class can generate tarballs and SRPMs
5763 and can create them with various levels of compliance in mind.
5764 One way of doing this (but certainly not the only way) is to
5765 release just the original source as a tarball.
5766 You can do this by adding the following to the
5767 <filename>local.conf</filename> file found in the
5768 <link linkend='build-directory'>Build Directory</link>:
5769 <literallayout class='monospaced'>
5770 ARCHIVER_MODE ?= "original"
5771 ARCHIVER_CLASS = "${@'archive-${ARCHIVER_MODE}-source' if
5772 ARCHIVER_MODE != 'none' else ''}"
5773 INHERIT += "${ARCHIVER_CLASS}"
5774 SOURCE_ARCHIVE_PACKAGE_TYPE = "tar"
5775 </literallayout>
5776 During the creation of your image, the source from all
5777 recipes that deploy packages to the image is placed within
5778 subdirectories of
5779 <filename>DEPLOY_DIR/sources</filename> based on the
5780 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
5781 for each recipe.
5782 Releasing the entire directory enables you to comply with
5783 requirements concerning providing the unmodified source.
5784 It is important to note that the size of the directory can
5785 get large.
5786 </para>
5787
5788 <para>
5789 A way to help mitigate the size issue is to only release
5790 tarballs for licenses that require the release of
5791 source.
5792 Let's assume you are only concerned with GPL code as
5793 identified with the following:
5794 <literallayout class='monospaced'>
5795 $ cd poky/build/tmp/deploy/sources
5796 $ mkdir ~/gpl_source_release
5797 $ for dir in */*GPL*; do cp -r $dir ~/gpl_source_release; done
5798 </literallayout>
5799 At this point, you could create a tarball from the
5800 <filename>gpl_source_release</filename> directory and
5801 provide that to the end user.
5802 This method would be a step toward achieving compliance
5803 with section 3a of GPLv2 and with section 6 of GPLv3.
5804 </para>
5805 </section>
5806
5807 <section id='providing-license-text'>
5808 <title>Providing License Text</title>
5809
5810 <para>
5811 One requirement that is often overlooked is inclusion
5812 of license text.
5813 This requirement also needs to be dealt with prior to
5814 generating the final image.
5815 Some licenses require the license text to accompany
5816 the binary.
5817 You can achieve this by adding the following to your
5818 <filename>local.conf</filename> file:
5819 <literallayout class='monospaced'>
5820 COPY_LIC_MANIFEST = "1"
5821 COPY_LIC_DIRS = "1"
5822 </literallayout>
5823 Adding these statements to the configuration file ensures
5824 that the licenses collected during package generation
5825 are included on your image.
5826 As the source archiver has already archived the original
5827 unmodified source that contains the license files,
5828 you would have already met the requirements for inclusion
5829 of the license information with source as defined by the GPL
5830 and other open source licenses.
5831 </para>
5832 </section>
5833
5834 <section id='providing-compilation-scripts-and-source-code-modifications'>
5835 <title>Providing Compilation Scripts and Source Code Modifications</title>
5836
5837 <para>
5838 At this point, we have addressed all we need to address
5839 prior to generating the image.
5840 The next two requirements are addressed during the final
5841 packaging of the release.
5842 </para>
5843
5844 <para>
5845 By releasing the version of the OpenEmbedded build system
5846 and the layers used during the build, you will be providing both
5847 compilation scripts and the source code modifications in one
5848 step.
5849 </para>
5850
5851 <para>
5852 If the deployment team has a
5853 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink>
5854 and a distro layer, and those those layers are used to patch,
5855 compile, package, or modify (in any way) any open source
5856 software included in your released images, you
5857 may be required to to release those layers under section 3 of
5858 GPLv2 or section 1 of GPLv3.
5859 One way of doing that is with a clean
5860 checkout of the version of the Yocto Project and layers used
5861 during your build.
5862 Here is an example:
5863 <literallayout class='monospaced'>
5864 # We built using the &DISTRO_NAME; branch of the poky repo
5865 $ git clone -b &DISTRO_NAME; git://git.yoctoproject.org/poky
5866 $ cd poky
5867 # We built using the release_branch for our layers
5868 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
5869 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
5870 # clean up the .git repos
5871 $ find . -name ".git" -type d -exec rm -rf {} \;
5872 </literallayout>
5873 One thing a development organization might want to consider
5874 for end-user convenience is to modify
5875 <filename>meta-yocto/conf/bblayers.conf.sample</filename> to
5876 ensure that when the end user utilizes the released build
5877 system to build an image, the development organization's
5878 layers are included in the <filename>bblayers.conf</filename>
5879 file automatically:
5880 <literallayout class='monospaced'>
5881 # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
5882 # changes incompatibly
5883 LCONF_VERSION = "6"
5884
5885 BBPATH = "${TOPDIR}"
5886 BBFILES ?= ""
5887
5888 BBLAYERS ?= " \
5889 ##OEROOT##/meta \
5890 ##OEROOT##/meta-yocto \
5891 ##OEROOT##/meta-yocto-bsp \
5892 ##OEROOT##/meta-mylayer \
5893 "
5894
5895 BBLAYERS_NON_REMOVABLE ?= " \
5896 ##OEROOT##/meta \
5897 ##OEROOT##/meta-yocto \
5898 "
5899 </literallayout>
5900 Creating and providing an archive of the
5901 <link linkend='metadata'>Metadata</link> layers
5902 (recipes, configuration files, and so forth)
5903 enables you to meet your
5904 requirements to include the scripts to control compilation
5905 as well as any modifications to the original source.
5906 </para>
5907 </section>
5908 </section>
5909</chapter>
5910
5911<!--
5912vim: expandtab tw=80 ts=4
5913-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-23 11:52:06 +0000