summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2014-02-11 13:45:54 -0600
committerRichard Purdie <richard.purdie@linuxfoundation.org>2014-03-09 18:59:02 -0700
commitb4b3bf56f60fab51f0c2c66ad34e39208d360d2e (patch)
treef1a10b014b2ad983d8d047d4ca2b743d38fc2d9f
parent34bcd967969a9e31215c230c4a85fff9f2f3b520 (diff)
downloadpoky-b4b3bf56f60fab51f0c2c66ad34e39208d360d2e.tar.gz
bitbake: user-manual: Added new chapter to cover BB execution.
This commit added need files, cleaned up bogus links, and adjusted the build file, which is user-manual.xml. The new chapter file is "user-manual-execution.xml". The file has base text from the bitbake section from the "Closer Look" chapter of the YP ref-manual. This means that I had to initially include some figures from that book for now. I am not including them as part of this commit though as they will likely get renamed. I made sure that all links were removed that were specific to the YP ref-manual. Another change was in the user-manual.xml file to include the new chapter as part of the build. Another block of text for the new chapter was the "Running a Task" section from the user-manual-metadata.xml file. Consequently, that section was deleted from the "Metafile" chapter. Another block of text for the new chapter was the "Parsing and Execution" section from the user-manual-metadata.xml file. Consequently, that section was deleted from the "Metafile chapter. (Bitbake rev: f3b21d1fb711f9625d2ac92d4f4fe0f269242bd7) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat
-rw-r--r--bitbake/doc/user-manual/user-manual-execution.xml690
-rw-r--r--bitbake/doc/user-manual/user-manual-metadata.xml333
-rw-r--r--bitbake/doc/user-manual/user-manual.xml2
3 files changed, 692 insertions, 333 deletions
diff --git a/bitbake/doc/user-manual/user-manual-execution.xml b/bitbake/doc/user-manual/user-manual-execution.xml
new file mode 100644
index 0000000000..365392dbff
--- /dev/null
+++ b/bitbake/doc/user-manual/user-manual-execution.xml
@@ -0,0 +1,690 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id="user-manual-execution">
5 <title>Execution</title>
6
7 <para>
8 Tasks can either be a shell task or a Python task.
9 For shell tasks, BitBake writes a shell script to
10 <filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
11 and then executes the script.
12 The generated shell script contains all the exported variables,
13 and the shell functions with all variables expanded.
14 Output from the shell script goes to the file
15 <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
16 Looking at the expanded shell functions in the run file and
17 the output in the log files is a useful debugging technique.
18 </para>
19
20 <para>
21 For Python tasks, BitBake executes the task internally and logs
22 information to the controlling terminal.
23 Future versions of BitBake will write the functions to files
24 similar to the way shell tasks are handled.
25 Logging will be handled in a way similar to shell tasks as well.
26 </para>
27
28 <para>
29 Once all the tasks have been completed BitBake exits.
30 </para>
31
32 <section id='parsing-and-execution'>
33 <title>Parsing and Execution</title>
34
35 <section id='parsing-overview'>
36 <title>Parsing Overview</title>
37
38 <para>
39 BitBake parses configuration files, classes, recipes, and append
40 files.
41 </para>
42
43 <para>
44 The first thing BitBake does is look for the
45 <filename>bitbake.conf</filename> file.
46 This file resides in the <filename>conf</filename>
47 directory, which must be listed in
48 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
49 </para>
50
51 <para>
52 The <filename>bitbake.conf</filename> file lists other configuration
53 files to include from the <filename>conf</filename> directory below the
54 directories listed in <filename>BBPATH</filename>.
55 In general, the most important of these
56 configuration files from a user's perspective
57 is <filename>local.conf</filename>, which contains the user's
58 customized settings for the build environment.
59 </para>
60
61 <para>
62 Other notable configuration files are the distribution configuration
63 file and the machine configuration file.
64 These configuration files are normally identified by
65 variables unique to the build systems using BitBake.
66 For example, the Yocto Project uses the
67 <filename>DISTRO</filename> and <filename>MACHINE</filename>
68 variables, respectively.
69 </para>
70
71 <para>
72 After parsing of the configuration files, some standard classes are
73 included.
74 The <filename>base.bbclass</filename> file
75 is always included.
76 Other classes that are specified in the configuration using the
77 <link linkend='var-INHERIT'><filename>INHERIT</filename></link>
78 variable are also included.
79 BitBake searches for class files in a "classes" subdirectory under
80 the paths in <filename>BBPATH</filename> in the same way as
81 configuration files.
82 </para>
83
84 <para>
85 After classes are included, the variable
86 <filename>BBFILES</filename> is set, usually in
87 <filename>local.conf</filename>, and defines the list of
88 places to search for recipe and append files.
89 Adding extra content to <filename>BBFILES</filename> is best
90 achieved through the use of BitBake layers.
91 </para>
92
93 <para>
94 BitBake parses each recipe and append file located with
95 <filename>BBFILES</filename> and stores the values of various
96 variables into the datastore.
97 In summary, for each recipe and append file pairing, the configuration
98 plus the base class of variables are set, followed by the data in the
99 recipe file itself, followed by any inherit commands
100 that the recipe file might contain.
101 </para>
102
103 <para>
104 Because parsing recipe and append files is a time consuming
105 process, a cache, referred to as the "setscene"
106 is kept to speed up subsequent parsing.
107 The setscene is invalid if the timestamps of a recipe changes,
108 any of the include files change, configuration files change,
109 or class files on which the recipe file depends change.
110 </para>
111 </section>
112
113 <section id='parsing-configuration-files'>
114 <title>Configuration files</title>
115
116 <para>
117 Prior to parsing configuration files, Bitbake looks
118 at certain variables, including:
119 <itemizedlist>
120 <listitem><para><link linkend='var-BB_ENV_WHITELIST'><filename>BB_ENV_WHITELIST</filename></link></para></listitem>
121 <listitem><para><link linkend='var-BB_PRESERVE_ENV'><filename>BB_PRESERVE_ENV</filename></link></para></listitem>
122 <listitem><para><link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link></para></listitem>
123 <listitem><para><link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link></para></listitem>
124 <listitem><para><link linkend='var-PREFERRED_VERSION'><filename>PREFERRED_VERSION</filename></link></para></listitem>
125 <listitem><para><link linkend='var-PREFERRED_PROVIDERS'><filename>PREFERRED_PROVIDERS</filename></link></para></listitem>
126 </itemizedlist>
127 </para>
128
129 <para>
130 The first kind of metadata in BitBake is configuration metadata.
131 This metadata is global, and therefore affects all packages and
132 tasks that are executed.
133 </para>
134
135 <para>
136 BitBake will first search the current working directory for an
137 optional <filename>conf/bblayers.conf</filename> configuration file.
138 This file is expected to contain a
139 <link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link>
140 variable that is a space delimited list of 'layer' directories.
141 For each directory in this list, a <filename>conf/layer.conf</filename>
142 file will be searched for and parsed with the
143 <link linkend='var-LAYERDIR'><filename>LAYERDIR</filename></link>
144 variable being set to the directory where the layer was found.
145 The idea is these files will setup
146 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
147 and other variables correctly for a given build directory automatically
148 for the user.
149 </para>
150
151 <para>
152 BitBake will then expect to find <filename>conf/bitbake.conf</filename>
153 file somewhere in the user specified <filename>BBPATH</filename>.
154 That configuration file generally has include directives to pull
155 in any other metadata (generally files specific to architecture,
156 machine, local and so on).
157 </para>
158
159 <para>
160 Only variable definitions and include directives are allowed
161 in <filename>.conf</filename> files.
162 The following variables include:
163 <itemizedlist>
164 <listitem><para>
165 <link linkend='var-BITBAKE_UI'><filename>BITBAKE_UI</filename></link>
166 </para></listitem>
167 <listitem><para>
168 <link linkend='var-BBDEBUG'><filename>BBDEBUG</filename></link>
169 </para></listitem>
170 <listitem><para>
171 <link linkend='var-MULTI_PROVIDER_WHITELIST'><filename>MULTI_PROVIDER_WHITELIST</filename></link>
172 </para></listitem>
173 <listitem><para>
174 <link linkend='var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></link>
175 </para></listitem>
176 <listitem><para>
177 <filename>BBPKGS</filename>
178 </para></listitem>
179 <listitem><para>
180 <link linkend='var-BB_DEFAULT_TASK'><filename>BB_DEFAULT_TASK</filename></link>
181 </para></listitem>
182 <listitem><para>
183 <link linkend='var-TOPDIR'><filename>TOPDIR</filename></link>
184 </para></listitem>
185 <listitem><para>
186 <link linkend='var-BB_VERBOSE_LOGS'><filename>BB_VERBOSE_LOGS</filename></link>
187 </para></listitem>
188 <listitem><para>
189 <link linkend='var-BB_NICE_LEVEL'><filename>BB_NICE_LEVEL</filename></link>
190 </para></listitem>
191 <listitem><para>
192 <link linkend='var-BBFILE_COLLECTIONS'><filename>BBFILE_COLLECTIONS</filename></link>
193 </para></listitem>
194 <listitem><para>
195 <link linkend='var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></link>
196 </para></listitem>
197 <listitem><para>
198 <link linkend='var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></link>
199 </para></listitem>
200 <listitem><para>
201 <link linkend='var-BBINCLUDED'><filename>BBINCLUDED</filename></link>
202 </para></listitem>
203 <listitem><para>
204 <link linkend='var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></link>
205 </para></listitem>
206 <listitem><para>
207 <link linkend='var-BUILDNAME'><filename>BUILDNAME</filename></link>
208 </para></listitem>
209 <listitem><para>
210 <link linkend='var-BBMASK'><filename>BBMASK</filename></link>
211 </para></listitem>
212 </itemizedlist>
213 </para>
214
215 <section id='layers'>
216 <title>Layers</title>
217
218 <para>
219 Layers allow you to isolate different types of
220 customizations from each other.
221 While you might find it tempting to keep everything in one layer
222 when working on a single project, the more modular you organize
223 your metadata, the easier it is to cope with future changes.
224 </para>
225
226 <para>
227 To illustrate how you can use layers to keep things modular,
228 consider machine customizations.
229 These types of customizations typically reside in a special layer,
230 rather than a general layer, called a Board Specific Package (BSP) Layer.
231 Furthermore, the machine customizations should be isolated from
232 recipes and metadata that support a new GUI environment, for
233 example.
234 This situation gives you a couple of layers: one for the machine
235 configurations and one for the GUI environment.
236 It is important to understand, however, that the BSP layer can still
237 make machine-specific additions to recipes within
238 the GUI environment layer without polluting the GUI layer itself
239 with those machine-specific changes.
240 You can accomplish this through a recipe that is a BitBake append
241 (<filename>.bbappend</filename>) file.
242 </para>
243
244 <para>
245 There are certain variables specific to layers:
246 <itemizedlist>
247 <listitem><para>
248 <link linkend='var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></link>
249 </para></listitem>
250 <listitem><para>
251 <link linkend='var-LAYERVERSION'><filename>LAYERVERSION</filename></link>
252 </para></listitem>
253 </itemizedlist>
254 </para>
255 </section>
256
257 <section id='schedulers'>
258 <title>Schedulers</title>
259
260 <para>
261 Variables specific to scheduling functionality exist:
262 <itemizedlist>
263 <listitem><para>
264 <link linkend='var-BB_SCHEDULER'><filename>BB_SCHEDULER</filename></link>
265 </para></listitem>
266 <listitem><para>
267 <link linkend='var-BB_SCHEDULERS'><filename>BB_SCHEDULERS</filename></link>
268 </para></listitem>
269 </itemizedlist>
270 </para>
271 </section>
272 </section>
273
274 <section id='metadata-classes'>
275 <title>Classes</title>
276
277 <para>
278 BitBake's rudimentary inheritance mechanism is accomplished using
279 classes.
280 As briefly mentioned in the metadata introduction, BitBake
281 parses a class when an inherit directive is encountered, and it
282 is located in the <filename>classes</filename> directory
283 relative to the directories in
284 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
285 </para>
286 </section>
287
288 <section id='recipe-bb-files'>
289 <title>Recipe (<filename>.bb</filename>) Files</title>
290
291 <para>
292 Recipe files, which are files that have the
293 <filename>.bb</filename> file extension, are logical units of
294 tasks for execution.
295 Normally, that logical unit is a package that needs to be
296 built.
297 </para>
298
299 <para>
300 BitBake obeys all inter-recipe dependencies.
301 </para>
302
303 <para>
304 Recipe files must reside in locations found in the
305 <link linkend='var-BBFILES'><filename>BBFILES</filename></link>
306 variable.
307 </para>
308 </section>
309
310 <section id='append-bbappend-files'>
311 <title>Append (<filename>.bbappend</filename>) Files</title>
312
313 <para>
314 Append files, which are files that have the
315 <filename>.bbappend</filename> file extension, add or
316 extend build information to an existing
317 <link linkend='recipe-bb-files'>recipe file</link>.
318 </para>
319
320 <para>
321 BitBake expects every append file to have a corresponding recipe file.
322 Furthermore, the append file and corresponding recipe file
323 must use the same root filename.
324 The filenames can differ only in the file type suffix used
325 (e.g. <filename>formfactor_0.0.bb</filename> and
326 <filename>formfactor_0.0.bbappend</filename>).
327 </para>
328
329 <para>
330 Information in append files overrides the information in the
331 similarly-named recipe file.
332 </para>
333 </section>
334 </section>
335
336 <section id='bitbake-dev-environment'>
337 <title>BitBake</title>
338
339 <para>
340 The OpenEmbedded build system uses BitBake to produce images.
341 BitBake consists of several functional areas.
342 This section takes a closer look at each of those areas.
343 </para>
344
345 <section id='source-fetching-dev-environment'>
346 <title>Source Fetching</title>
347
348 <para>
349 The first stages of building a recipe are to fetch and unpack
350 the source code:
351 <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" />
352 </para>
353figures/
354 <para>
355 The <filename>do_fetch</filename> and
356 <filename>do_unpack</filename> tasks fetch the source files
357 and unpack them into the work directory.
358 By default, everything is accomplished in the
359 build directory,
360 which has a defined structure.
361 </para>
362
363 <para>
364 Unpacked source files are pointed to by a variable.
365 For example, in the Yocto Project and OpenEmbedded build systems,
366 the <filename>S</filename> variable points to these source files.
367 Each recipe has an area in the Build Directory where the
368 unpacked source code resides.
369 The name of that directory for any given recipe is defined from
370 several different variables.
371 You can see the variables that define these directories
372 by looking at the figure that shows the structure and variables
373 used in the Yocto Project:
374 <itemizedlist>
375 <listitem><para><filename>TMPDIR</filename>
376 </para></listitem>
377 <listitem><para><filename>PACKAGE_ARCH</filename>
378 </para></listitem>
379 <listitem><para><filename>TARGET_OS</filename>
380 </para></listitem>
381 <listitem><para><link linkend='var-PN'><filename>PN</filename></link>
382 </para></listitem>
383 <listitem><para><link linkend='var-PV'><filename>PV</filename></link>
384 </para></listitem>
385 <listitem><para><link linkend='var-PR'><filename>PR</filename></link>
386 </para></listitem>
387 <listitem><para><filename>WORKDIR</filename>
388 </para></listitem>
389 <listitem><para><filename>S</filename>
390 </para></listitem>
391 </itemizedlist>
392 </para>
393
394 <para>
395 Briefly, the <filename>S</filename> directory contains the
396 unpacked source files for a recipe.
397 The <filename>WORKDIR</filename> directory is where all the
398 building goes on for a given recipe.
399 </para>
400 </section>
401
402 <section id='patching-dev-environment'>
403 <title>Patching</title>
404
405 <para>
406 Once source code is fetched and unpacked, BitBake locates
407 patch files and applies them to the source files:
408 <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" />
409 </para>
410
411 <para>
412 The <filename>do_patch</filename> task processes recipes by
413 using the
414 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
415 variable to locate applicable patch files, which by default
416 are <filename>*.patch</filename> or
417 <filename>*.diff</filename> files, or any file if
418 "apply=yes" is specified for the file in
419 <filename>SRC_URI</filename>.
420 </para>
421
422 <para>
423 BitBake finds and applies multiple patches for a single recipe
424 in the order in which it finds the patches.
425 Patches are applied to the recipe's source files located in the
426 <filename>S</filename> directory.
427 </para>
428
429 <para>
430 For more information on how the source directories are
431 created, see the
432 "<link linkend='source-fetching-dev-environment'>Source Fetching</link>"
433 section.
434 </para>
435 </section>
436
437 <section id='configuration-and-compilation-dev-environment'>
438 <title>Configuration and Compilation</title>
439
440 <para>
441 After source code is patched, BitBake executes tasks that
442 configure and compile the source code:
443 <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" />
444 </para>
445
446 <para>
447 This step in the build process consists of three tasks:
448 <itemizedlist>
449 <listitem><para><emphasis><filename>do_configure</filename>:</emphasis>
450 This task configures the source by enabling and
451 disabling any build-time and configuration options for
452 the software being built.
453 Configurations can come from the recipe itself as well
454 as from an inherited class.
455 Additionally, the software itself might configure itself
456 depending on the target for which it is being built.
457 </para>
458
459 <para>The configurations handled by the
460 <filename>do_configure</filename> task are specific
461 to source code configuration for the source code
462 being built by the recipe.</para>
463
464 <para>If you are using the Autotools class
465 (<filename>autotools.bbclass</filename>),
466 you can add additional configuration options by using
467 the <filename>EXTRA_OECONF</filename>
468 variable.
469 For information on how this variable works within
470 that class, see the
471 <filename>meta/classes/autotools.bbclass</filename> file.
472 </para></listitem>
473 <listitem><para><emphasis><filename>do_compile</filename>:</emphasis>
474 Once a configuration task has been satisfied, BitBake
475 compiles the source using the
476 <filename>do_compile</filename> task.
477 Compilation occurs in the directory pointed to by the
478 <link linkend='var-B'><filename>B</filename></link>
479 variable.
480 Realize that the <filename>B</filename> directory is, by
481 default, the same as the
482 <filename>S</filename>
483 directory.</para></listitem>
484 <listitem><para><emphasis><filename>do_install</filename>:</emphasis>
485 Once compilation is done, BitBake executes the
486 <filename>do_install</filename> task.
487 This task copies files from the <filename>B</filename>
488 directory and places them in a holding area pointed to
489 by the <filename>D</filename> variable.</para></listitem>
490 </itemizedlist>
491 </para>
492 </section>
493
494 <section id='package-splitting-dev-environment'>
495 <title>Package Splitting</title>
496
497 <para>
498 After source code is configured and compiled, the
499 OpenEmbedded build system analyzes
500 the results and splits the output into packages:
501 <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" />
502 </para>
503
504 <para>
505 The <filename>do_package</filename> and
506 <filename>do_packagedata</filename> tasks combine to analyze
507 the files found in the <filename>D</filename> directory
508 and split them into subsets based on available packages and
509 files.
510 The analyzing process involves the following as well as other
511 items: splitting out debugging symbols,
512 looking at shared library dependencies between packages,
513 and looking at package relationships.
514 The <filename>do_packagedata</filename> task creates package
515 metadata based on the analysis such that the
516 OpenEmbedded build system can generate the final packages.
517 Working, staged, and intermediate results of the analysis
518 and package splitting process use these areas:
519 <itemizedlist>
520 <listitem><para><filename>PKGD</filename>
521 </para></listitem>
522 <listitem><para><filename>PKGDATA_DIR</filename>
523 </para></listitem>
524 <listitem><para><filename>PKGDESTWORK</filename>
525 </para></listitem>
526 <listitem><para><filename>PKGDEST</filename>
527 </para></listitem>
528 </itemizedlist>
529 The <filename>FILES</filename>
530 variable defines the files that go into each package in
531 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>.
532 If you want details on how this is accomplished in the Yocto Project
533 for example, you can look at the <filename>package.bbclass</filename>
534 file in a Yocto tree.
535 </para>
536
537 <para>
538 Depending on the type of packages being created (RPM, DEB, or
539 IPK), the <filename>do_package_write_*</filename> task
540 creates the actual packages and places them in the
541 Package Feed area, which is
542 <filename>${TMPDIR}/deploy</filename>.
543 <note>
544 Support for creating feeds directly from the
545 <filename>deploy/*</filename> directories does not exist.
546 Creating such feeds usually requires some kind of feed
547 maintenance mechanism that would upload the new packages
548 into an official package feed (e.g. the
549 Ångström distribution).
550 This functionality is highly distribution-specific
551 and thus is not provided out of the box.
552 </note>
553 </para>
554 </section>
555
556 <section id='image-generation-dev-environment'>
557 <title>Image Generation</title>
558
559 <para>
560 Once packages are split and stored in the Package Feeds area,
561 the OpenEmbedded build system uses BitBake to generate the
562 root filesystem image:
563 <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" />
564 </para>
565
566 <para>
567 The image generation process consists of several stages and
568 depends on many variables.
569 The <filename>do_rootfs</filename> task uses these key variables
570 to help create the list of packages to actually install:
571 <itemizedlist>
572 <listitem><para><filename>IMAGE_INSTALL</filename>:
573 Lists out the base set of packages to install from
574 the Package Feeds area.</para></listitem>
575 <listitem><para><filename>PACKAGE_EXCLUDE</filename>:
576 Specifies packages that should not be installed.
577 </para></listitem>
578 <listitem><para><filename>IMAGE_FEATURES</filename>:
579 Specifies features to include in the image.
580 Most of these features map to additional packages for
581 installation.</para></listitem>
582 <listitem><para><filename>PACKAGE_CLASSES</filename>:
583 Specifies the package backend to use and consequently
584 helps determine where to locate packages within the
585 Package Feeds area.</para></listitem>
586 <listitem><para><filename>IMAGE_LINGUAS</filename>:
587 Determines the language(s) for which additional
588 language support packages are installed.
589 </para></listitem>
590 </itemizedlist>
591 </para>
592
593 <para>
594 Package installation is under control of the package manager
595 (e.g. smart/rpm, opkg, or apt/dpkg) regardless of whether or
596 not package management is enabled for the target.
597 At the end of the process, if package management is not
598 enabled for the target, the package manager's data files
599 are deleted from the root filesystem.
600 </para>
601
602 <para>
603 During image generation, the build system attempts to run
604 all post-installation scripts.
605 Any that fail to run on the build host are run on the
606 target when the target system is first booted.
607 If you are using a
608 read-only root filesystem,
609 all the post installation scripts must succeed during the
610 package installation phase since the root filesystem cannot be
611 written into.
612 </para>
613
614 <para>
615 During Optimization, optimizing processes are run across
616 the image.
617 These processes include <filename>mklibs</filename> and
618 <filename>prelink</filename>.
619 The <filename>mklibs</filename> process optimizes the size
620 of the libraries.
621 A <filename>prelink</filename> process optimizes the dynamic
622 linking of shared libraries to reduce start up time of
623 executables.
624 </para>
625
626 <para>
627 Part of the image generation process includes compressing the
628 root filesystem image.
629 Compression is accomplished through several optimization
630 routines designed to reduce the overall size of the image.
631 </para>
632
633 <para>
634 After the root filesystem has been constructed, the image
635 generation process turns everything into an image file or
636 a set of image files.
637 The formats used for the root filesystem depend on the
638 <filename>IMAGE_FSTYPES</filename> variable.
639 </para>
640
641 <note>
642 The entire image generation process is run under Pseudo.
643 Running under Pseudo ensures that the files in the root
644 filesystem have correct ownership.
645 </note>
646 </section>
647
648 <section id='sdk-generation-dev-environment'>
649 <title>SDK Generation</title>
650
651 <para>
652 The OpenEmbedded build system uses BitBake to generate the
653 Software Development Kit (SDK) installer script:
654 <imagedata fileref="figures/sdk-generation.png" align="center" width="6in" depth="7in" />
655 </para>
656
657 <para>
658 Like image generation, the SDK script process consists of
659 several stages and depends on many variables.
660 The <filename>do_populate_sdk</filename> task uses these
661 key variables to help create the list of packages to actually
662 install.
663 </para>
664
665 <para>
666 The <filename>do_populate_sdk</filename> task handles two
667 parts: a target part and a host part.
668 The target part is the part built for the target hardware and
669 includes libraries and headers.
670 The host part is the part of the SDK that runs on the
671 <filename>SDKMACHINE</filename>.
672 </para>
673
674 <para>
675 Once both parts are constructed, the
676 <filename>do_populate_sdk</filename> task performs some cleanup
677 on both parts.
678 After the cleanup, the task creates a cross-development
679 environment setup script and any configuration files that
680 might be needed.
681 </para>
682
683 <para>
684 The final output of the task is the Cross-development
685 toolchain installation script (<filename>.sh</filename> file),
686 which includes the environment setup script.
687 </para>
688 </section>
689 </section>
690</chapter>
diff --git a/bitbake/doc/user-manual/user-manual-metadata.xml b/bitbake/doc/user-manual/user-manual-metadata.xml
index 796d27b151..8d37f6bee1 100644
--- a/bitbake/doc/user-manual/user-manual-metadata.xml
+++ b/bitbake/doc/user-manual/user-manual-metadata.xml
@@ -895,35 +895,6 @@
895 </section> 895 </section>
896 </section> 896 </section>
897 897
898 <section id='running-a-task'>
899 <title>Running a Task</title>
900
901 <para>
902 Tasks can either be a shell task or a Python task.
903 For shell tasks, BitBake writes a shell script to
904 <filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
905 and then executes the script.
906 The generated shell script contains all the exported variables,
907 and the shell functions with all variables expanded.
908 Output from the shell script goes to the file
909 <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
910 Looking at the expanded shell functions in the run file and
911 the output in the log files is a useful debugging technique.
912 </para>
913
914 <para>
915 For Python tasks, BitBake executes the task internally and logs
916 information to the controlling terminal.
917 Future versions of BitBake will write the functions to files
918 similar to the way shell tasks are handled.
919 Logging will be handled in a way similar to shell tasks as well.
920 </para>
921
922 <para>
923 Once all the tasks have been completed BitBake exits.
924 </para>
925 </section>
926
927 <section id='variable-flags'> 898 <section id='variable-flags'>
928 <title>Variable Flags</title> 899 <title>Variable Flags</title>
929 900
@@ -1033,310 +1004,6 @@
1033 </para> 1004 </para>
1034 </section> 1005 </section>
1035 1006
1036 <section id='parsing-and-execution'>
1037 <title>Parsing and Execution</title>
1038
1039 <section id='parsing-overview'>
1040 <title>Parsing Overview</title>
1041
1042 <para>
1043 BitBake parses configuration files, classes, recipes, and append
1044 files.
1045 </para>
1046
1047 <para>
1048 The first thing BitBake does is look for the
1049 <filename>bitbake.conf</filename> file.
1050 This file resides in the <filename>conf</filename>
1051 directory, which must be listed in
1052 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
1053 </para>
1054
1055 <para>
1056 The <filename>bitbake.conf</filename> file lists other configuration
1057 files to include from the <filename>conf</filename> directory below the
1058 directories listed in <filename>BBPATH</filename>.
1059 In general, the most important of these
1060 configuration files from a user's perspective
1061 is <filename>local.conf</filename>, which contains the user's
1062 customized settings for the build environment.
1063 </para>
1064
1065 <para>
1066 Other notable configuration files are the distribution configuration
1067 file and the machine configuration file.
1068 These configuration files are normally identified by
1069 variables unique to the build systems using BitBake.
1070 For example, the Yocto Project uses the
1071 <filename>DISTRO</filename> and <filename>MACHINE</filename>
1072 variables, respectively.
1073 </para>
1074
1075 <para>
1076 After parsing of the configuration files, some standard classes are
1077 included.
1078 The <filename>base.bbclass</filename> file
1079 is always included.
1080 Other classes that are specified in the configuration using the
1081 <link linkend='var-INHERIT'><filename>INHERIT</filename></link>
1082 variable are also included.
1083 BitBake searches for class files in a "classes" subdirectory under
1084 the paths in <filename>BBPATH</filename> in the same way as
1085 configuration files.
1086 </para>
1087
1088 <para>
1089 After classes are included, the variable
1090 <filename>BBFILES</filename> is set, usually in
1091 <filename>local.conf</filename>, and defines the list of
1092 places to search for recipe and append files.
1093 Adding extra content to <filename>BBFILES</filename> is best
1094 achieved through the use of BitBake layers.
1095 </para>
1096
1097 <para>
1098 BitBake parses each recipe and append file located with
1099 <filename>BBFILES</filename> and stores the values of various
1100 variables into the datastore.
1101 In summary, for each recipe and append file pairing, the configuration
1102 plus the base class of variables are set, followed by the data in the
1103 recipe file itself, followed by any inherit commands
1104 that the recipe file might contain.
1105 </para>
1106
1107 <para>
1108 Because parsing recipe and append files is a time consuming
1109 process, a cache, referred to as the "setscene"
1110 is kept to speed up subsequent parsing.
1111 The setscene is invalid if the timestamps of a recipe changes,
1112 any of the include files change, configuration files change,
1113 or class files on which the recipe file depends change.
1114 </para>
1115 </section>
1116
1117 <section id='parsing-configuration-files'>
1118 <title>Configuration files</title>
1119
1120 <para>
1121 Prior to parsing configuration files, Bitbake looks
1122 at certain variables, including:
1123 <itemizedlist>
1124 <listitem><para><link linkend='var-BB_ENV_WHITELIST'><filename>BB_ENV_WHITELIST</filename></link></para></listitem>
1125 <listitem><para><link linkend='var-BB_PRESERVE_ENV'><filename>BB_PRESERVE_ENV</filename></link></para></listitem>
1126 <listitem><para><link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link></para></listitem>
1127 <listitem><para><link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link></para></listitem>
1128 <listitem><para><link linkend='var-PREFERRED_VERSION'><filename>PREFERRED_VERSION</filename></link></para></listitem>
1129 <listitem><para><link linkend='var-PREFERRED_PROVIDERS'><filename>PREFERRED_PROVIDERS</filename></link></para></listitem>
1130 </itemizedlist>
1131 </para>
1132
1133 <para>
1134 The first kind of metadata in BitBake is configuration metadata.
1135 This metadata is global, and therefore affects all packages and
1136 tasks that are executed.
1137 </para>
1138
1139 <para>
1140 BitBake will first search the current working directory for an
1141 optional <filename>conf/bblayers.conf</filename> configuration file.
1142 This file is expected to contain a
1143 <link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link>
1144 variable that is a space delimited list of 'layer' directories.
1145 For each directory in this list, a <filename>conf/layer.conf</filename>
1146 file will be searched for and parsed with the
1147 <link linkend='var-LAYERDIR'><filename>LAYERDIR</filename></link>
1148 variable being set to the directory where the layer was found.
1149 The idea is these files will setup
1150 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
1151 and other variables correctly for a given build directory automatically
1152 for the user.
1153 </para>
1154
1155 <para>
1156 BitBake will then expect to find <filename>conf/bitbake.conf</filename>
1157 file somewhere in the user specified <filename>BBPATH</filename>.
1158 That configuration file generally has include directives to pull
1159 in any other metadata (generally files specific to architecture,
1160 machine, local and so on).
1161 </para>
1162
1163 <para>
1164 Only variable definitions and include directives are allowed
1165 in <filename>.conf</filename> files.
1166 The following variables include:
1167 <itemizedlist>
1168 <listitem><para>
1169 <link linkend='var-BITBAKE_UI'><filename>BITBAKE_UI</filename></link>
1170 </para></listitem>
1171 <listitem><para>
1172 <link linkend='var-BBDEBUG'><filename>BBDEBUG</filename></link>
1173 </para></listitem>
1174 <listitem><para>
1175 <link linkend='var-MULTI_PROVIDER_WHITELIST'><filename>MULTI_PROVIDER_WHITELIST</filename></link>
1176 </para></listitem>
1177 <listitem><para>
1178 <link linkend='var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></link>
1179 </para></listitem>
1180 <listitem><para>
1181 <filename>BBPKGS</filename>
1182 </para></listitem>
1183 <listitem><para>
1184 <link linkend='var-BB_DEFAULT_TASK'><filename>BB_DEFAULT_TASK</filename></link>
1185 </para></listitem>
1186 <listitem><para>
1187 <link linkend='var-TOPDIR'><filename>TOPDIR</filename></link>
1188 </para></listitem>
1189 <listitem><para>
1190 <link linkend='var-BB_VERBOSE_LOGS'><filename>BB_VERBOSE_LOGS</filename></link>
1191 </para></listitem>
1192 <listitem><para>
1193 <link linkend='var-BB_NICE_LEVEL'><filename>BB_NICE_LEVEL</filename></link>
1194 </para></listitem>
1195 <listitem><para>
1196 <link linkend='var-BBFILE_COLLECTIONS'><filename>BBFILE_COLLECTIONS</filename></link>
1197 </para></listitem>
1198 <listitem><para>
1199 <link linkend='var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></link>
1200 </para></listitem>
1201 <listitem><para>
1202 <link linkend='var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></link>
1203 </para></listitem>
1204 <listitem><para>
1205 <link linkend='var-BBINCLUDED'><filename>BBINCLUDED</filename></link>
1206 </para></listitem>
1207 <listitem><para>
1208 <link linkend='var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></link>
1209 </para></listitem>
1210 <listitem><para>
1211 <link linkend='var-BUILDNAME'><filename>BUILDNAME</filename></link>
1212 </para></listitem>
1213 <listitem><para>
1214 <link linkend='var-BBMASK'><filename>BBMASK</filename></link>
1215 </para></listitem>
1216 </itemizedlist>
1217 </para>
1218
1219 <section id='layers'>
1220 <title>Layers</title>
1221
1222 <para>
1223 Layers allow you to isolate different types of
1224 customizations from each other.
1225 While you might find it tempting to keep everything in one layer
1226 when working on a single project, the more modular you organize
1227 your metadata, the easier it is to cope with future changes.
1228 </para>
1229
1230 <para>
1231 To illustrate how you can use layers to keep things modular,
1232 consider machine customizations.
1233 These types of customizations typically reside in a special layer,
1234 rather than a general layer, called a Board Specific Package (BSP) Layer.
1235 Furthermore, the machine customizations should be isolated from
1236 recipes and metadata that support a new GUI environment, for
1237 example.
1238 This situation gives you a couple of layers: one for the machine
1239 configurations and one for the GUI environment.
1240 It is important to understand, however, that the BSP layer can still
1241 make machine-specific additions to recipes within
1242 the GUI environment layer without polluting the GUI layer itself
1243 with those machine-specific changes.
1244 You can accomplish this through a recipe that is a BitBake append
1245 (<filename>.bbappend</filename>) file.
1246 </para>
1247
1248 <para>
1249 There are certain variables specific to layers:
1250 <itemizedlist>
1251 <listitem><para>
1252 <link linkend='var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></link>
1253 </para></listitem>
1254 <listitem><para>
1255 <link linkend='var-LAYERVERSION'><filename>LAYERVERSION</filename></link>
1256 </para></listitem>
1257 </itemizedlist>
1258 </para>
1259 </section>
1260
1261 <section id='schedulers'>
1262 <title>Schedulers</title>
1263
1264 <para>
1265 Variables specific to scheduling functionality exist:
1266 <itemizedlist>
1267 <listitem><para>
1268 <link linkend='var-BB_SCHEDULER'><filename>BB_SCHEDULER</filename></link>
1269 </para></listitem>
1270 <listitem><para>
1271 <link linkend='var-BB_SCHEDULERS'><filename>BB_SCHEDULERS</filename></link>
1272 </para></listitem>
1273 </itemizedlist>
1274 </para>
1275 </section>
1276 </section>
1277
1278 <section id='metadata-classes'>
1279 <title>Classes</title>
1280
1281 <para>
1282 BitBake's rudimentary inheritance mechanism is accomplished using
1283 classes.
1284 As briefly mentioned in the metadata introduction, BitBake
1285 parses a class when an inherit directive is encountered, and it
1286 is located in the <filename>classes</filename> directory
1287 relative to the directories in
1288 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
1289 </para>
1290 </section>
1291
1292 <section id='recipe-bb-files'>
1293 <title>Recipe (<filename>.bb</filename>) Files</title>
1294
1295 <para>
1296 Recipe files, which are files that have the
1297 <filename>.bb</filename> file extension, are logical units of
1298 tasks for execution.
1299 Normally, that logical unit is a package that needs to be
1300 built.
1301 </para>
1302
1303 <para>
1304 BitBake obeys all inter-recipe dependencies.
1305 </para>
1306
1307 <para>
1308 Recipe files must reside in locations found in the
1309 <link linkend='var-BBFILES'><filename>BBFILES</filename></link>
1310 variable.
1311 </para>
1312 </section>
1313
1314 <section id='append-bbappend-files'>
1315 <title>Append (<filename>.bbappend</filename>) Files</title>
1316
1317 <para>
1318 Append files, which are files that have the
1319 <filename>.bbappend</filename> file extension, add or
1320 extend build information to an existing
1321 <link linkend='recipe-bb-files'>recipe file</link>.
1322 </para>
1323
1324 <para>
1325 BitBake expects every append file to have a corresponding recipe file.
1326 Furthermore, the append file and corresponding recipe file
1327 must use the same root filename.
1328 The filenames can differ only in the file type suffix used
1329 (e.g. <filename>formfactor_0.0.bb</filename> and
1330 <filename>formfactor_0.0.bbappend</filename>).
1331 </para>
1332
1333 <para>
1334 Information in append files overrides the information in the
1335 similarly-named recipe file.
1336 </para>
1337 </section>
1338 </section>
1339
1340 <section id='events'> 1007 <section id='events'>
1341 <title>Events</title> 1008 <title>Events</title>
1342 1009
diff --git a/bitbake/doc/user-manual/user-manual.xml b/bitbake/doc/user-manual/user-manual.xml
index 1e8777a4b3..ba690ab243 100644
--- a/bitbake/doc/user-manual/user-manual.xml
+++ b/bitbake/doc/user-manual/user-manual.xml
@@ -75,6 +75,8 @@
75 75
76 <xi:include href="user-manual-intro.xml"/> 76 <xi:include href="user-manual-intro.xml"/>
77 77
78 <xi:include href="user-manual-execution.xml"/>
79
78 <xi:include href="user-manual-metadata.xml"/> 80 <xi:include href="user-manual-metadata.xml"/>
79 81
80 <xi:include href="user-manual-fetching.xml"/> 82 <xi:include href="user-manual-fetching.xml"/>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-22 08:54:45 +0000