summaryrefslogtreecommitdiffstats
path: root/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml
diff options
context:
space:
mode:
Diffstat (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml')
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml644
1 files changed, 644 insertions, 0 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml
new file mode 100644
index 0000000000..ae267b42c8
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml
@@ -0,0 +1,644 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id="bitbake-user-manual-intro">
5 <title>Overview</title>
6
7 <para>
8 Welcome to the BitBake User Manual.
9 This manual provides information on the BitBake tool.
10 The information attempts to be as independent as possible regarding
11 systems that use BitBake, such as the Yocto Project and
12 OpenEmbedded.
13 In some cases, scenarios or examples that within the context of
14 a build system are used in the manual to help with understanding.
15 For these cases, the manual clearly states the context.
16 </para>
17
18 <section id="intro">
19 <title>Introduction</title>
20
21 <para>
22 Fundamentally, BitBake is a generic task execution
23 engine that allows shell and Python tasks to be run
24 efficiently and in parallel while working within
25 complex inter-task dependency constraints.
26 One of BitBake's main users, OpenEmbedded, takes this core
27 and builds embedded Linux software stacks using
28 a task-oriented approach.
29 </para>
30
31 <para>
32 Conceptually, BitBake is similar to GNU Make in
33 some regards but has significant differences:
34 <itemizedlist>
35 <listitem><para>
36 BitBake executes tasks according to provided
37 metadata that builds up the tasks.
38 Metadata is stored in recipe (<filename>.bb</filename>),
39 configuration (<filename>.conf</filename>), and class
40 (<filename>.bbclass</filename>) files and provides
41 BitBake with instructions on what tasks to run and
42 the dependencies between those tasks.
43 </para></listitem>
44 <listitem><para>
45 BitBake includes a fetcher library for obtaining source
46 code from various places such as source control
47 systems or websites.
48 </para></listitem>
49 <listitem><para>
50 The instructions for each unit to be built (e.g. a piece
51 of software) are known as recipe files and
52 contain all the information about the unit
53 (dependencies, source file locations, checksums, description
54 and so on).
55 </para></listitem>
56 <listitem><para>
57 BitBake includes a client/server abstraction and can
58 be used from a command line or used as a service over XMLRPC and
59 has several different user interfaces.
60 </para></listitem>
61 </itemizedlist>
62 </para>
63 </section>
64
65 <section id="history-and-goals">
66 <title>History and Goals</title>
67
68 <para>
69 BitBake was originally a part of the OpenEmbedded project.
70 It was inspired by the Portage package management system
71 used by the Gentoo Linux distribution.
72 On December 7, 2004, OpenEmbedded project team member,
73 Chris Larson split the project into two distinct pieces:
74 <itemizedlist>
75 <listitem><para>BitBake, a generic task executor</para></listitem>
76 <listitem><para>OpenEmbedded, a metadata set utilized by
77 BitBake</para></listitem>
78 </itemizedlist>
79 Today, BitBake is the primary basis of the
80 <ulink url="http://www.openembedded.org/">OpenEmbedded</ulink>
81 project, which is being used to build and maintain Linux
82 distributions such as the Angstrom Distribution and which is used
83 as the build tool for Linux projects such as the Yocto Project.
84 </para>
85
86 <para>
87 Prior to BitBake, no other build tool adequately met the needs of
88 an aspiring embedded Linux distribution.
89 All of the build systems used by traditional desktop Linux
90 distributions lacked important functionality, and none of the
91 ad-hoc Buildroot-based systems, prevalent in the
92 embedded space, were scalable or maintainable.
93 </para>
94
95 <para>
96 Some important original goals for BitBake were:
97 <itemizedlist>
98 <listitem><para>
99 Handle cross-compilation.
100 </para></listitem>
101 <listitem><para>
102 Handle inter-package dependencies (build time on
103 target architecture, build time on native
104 architecture, and runtime).
105 </para></listitem>
106 <listitem><para>
107 Support running any number of tasks within a given
108 package, including, but not limited to, fetching
109 upstream sources, unpacking them, patching them,
110 configuring them, and so forth.
111 </para></listitem>
112 <listitem><para>
113 Be Linux distribution agnostic for both build and
114 target systems.
115 </para></listitem>
116 <listitem><para>
117 Be architecture agnostic.
118 </para></listitem>
119 <listitem><para>
120 Support multiple build and target operating systems
121 (e.g. Cygwin, the BSDs, and so forth).
122 </para></listitem>
123 <listitem><para>
124 Be self contained, rather than tightly
125 integrated into the build machine's root
126 filesystem.
127 </para></listitem>
128 <listitem><para>
129 Handle conditional metadata on the target architecture,
130 operating system, distribution, and machine.
131 </para></listitem>
132 <listitem><para>
133 Be easy to use the tools to supply local metadata and packages
134 against which to operate.
135 </para></listitem>
136 <listitem><para>
137 Be easy to use BitBake to collaborate between multiple
138 projects for their builds.
139 </para></listitem>
140 <listitem><para>
141 Provide an inheritance mechanism that share
142 common metadata between many packages.
143 </para></listitem>
144 </itemizedlist>
145 Over time it became apparent that some further requirements
146 were necessary:
147 <itemizedlist>
148 <listitem><para>
149 Handle variants of a base recipe (e.g. native, sdk,
150 and multilib).
151 </para></listitem>
152 <listitem><para>
153 Split metadata into layers and allow layers
154 to override each other.
155 </para></listitem>
156 <listitem><para>
157 Allow representation of a given set of input variables
158 to a task as a checksum.
159 Based on that checksum, allow acceleration of builds
160 with prebuilt components.
161 </para></listitem>
162 </itemizedlist>
163 BitBake satisfies all the original requirements and many more
164 with extensions being made to the basic functionality to
165 reflect the additional requirements.
166 Flexibility and power have always been the priorities.
167 BitBake is highly extensible and supports embedded Python code and
168 execution of any arbitrary tasks.
169 </para>
170 </section>
171
172 <section id="Concepts">
173 <title>Concepts</title>
174
175 <para>
176 BitBake is a program written in the Python language.
177 At the highest level, BitBake interprets metadata, decides
178 what tasks are required to run, and executes those tasks.
179 Similar to GNU Make, BitBake controls how software is
180 built.
181 GNU Make achieves its control through "makefiles".
182 BitBake uses "recipes".
183 </para>
184
185 <para>
186 BitBake extends the capabilities of a simple
187 tool like GNU Make by allowing for much more complex tasks
188 to be completed, such as assembling entire embedded Linux
189 distributions.
190 </para>
191
192 <para>
193 The remainder of this section introduces several concepts
194 that should be understood in order to better leverage
195 the power of BitBake.
196 </para>
197
198 <section id='recipes'>
199 <title>Recipes</title>
200
201 <para>
202 BitBake Recipes, which are denoted by the file extension
203 <filename>.bb</filename>, are the most basic metadata files.
204 These recipe files provide BitBake with the following:
205 <itemizedlist>
206 <listitem><para>Descriptive information about the package</para></listitem>
207 <listitem><para>The version of the recipe</para></listitem>
208 <listitem><para>Existing Dependencies</para></listitem>
209 <listitem><para>Where the source code resides</para></listitem>
210 <listitem><para>Whether the source code requires any patches</para></listitem>
211 <listitem><para>How to compile the source code</para></listitem>
212 <listitem><para>Where on the target machine to install the
213 package being compiled</para></listitem>
214 </itemizedlist>
215 </para>
216
217 <para>
218 Within the context of BitBake, or any project utilizing BitBake
219 as its build system, files with the <filename>.bb</filename>
220 extension are referred to as recipes.
221 <note>
222 The term "package" is also commonly used to describe recipes.
223 However, since the same word is used to describe packaged
224 output from a project, it is best to maintain a single
225 descriptive term, "recipes".
226 </note>
227 </para>
228 </section>
229
230 <section id='configuration-files'>
231 <title>Configuration Files</title>
232
233 <para>
234 Configuration files, which are denoted by the
235 <filename>.conf</filename> extension, define
236 various configuration variables that govern the project's build
237 process.
238 These files fall into several areas that define
239 machine configuration options, distribution configuration
240 options, compiler tuning options, general common
241 configuration options, and user configuration options.
242 The main configuration file is the sample
243 <filename>bitbake.conf</filename> file, which is
244 located within the BitBake source tree
245 <filename>conf</filename> directory.
246 </para>
247 </section>
248
249 <section id='classes'>
250 <title>Classes</title>
251
252 <para>
253 Class files, which are denoted by the
254 <filename>.bbclass</filename> extension, contain
255 information that is useful to share between metadata files.
256 The BitBake source tree currently comes with one class metadata file
257 called <filename>base.bbclass</filename>.
258 You can find this file in the
259 <filename>classes</filename> directory.
260 The <filename>base.bbclass</filename> is special since it
261 is always included automatically for all recipes
262 and classes.
263 This class contains definitions for standard basic tasks such
264 as fetching, unpacking, configuring (empty by default),
265 compiling (runs any Makefile present), installing (empty by
266 default) and packaging (empty by default).
267 These tasks are often overridden or extended by other classes
268 added during the project development process.
269 </para>
270 </section>
271
272 <section id='layers'>
273 <title>Layers</title>
274
275 <para>
276 Layers allow you to isolate different types of
277 customizations from each other.
278 While you might find it tempting to keep everything in one layer
279 when working on a single project, the more modular you organize
280 your metadata, the easier it is to cope with future changes.
281 </para>
282
283 <para>
284 To illustrate how you can use layers to keep things modular,
285 consider customizations you might make to support a specific target machine.
286 These types of customizations typically reside in a special layer,
287 rather than a general layer, called a Board Specific Package (BSP) Layer.
288 Furthermore, the machine customizations should be isolated from
289 recipes and metadata that support a new GUI environment, for
290 example.
291 This situation gives you a couple of layers: one for the machine
292 configurations and one for the GUI environment.
293 It is important to understand, however, that the BSP layer can still
294 make machine-specific additions to recipes within
295 the GUI environment layer without polluting the GUI layer itself
296 with those machine-specific changes.
297 You can accomplish this through a recipe that is a BitBake append
298 (<filename>.bbappend</filename>) file.
299 </para>
300 </section>
301
302 <section id='append-bbappend-files'>
303 <title>Append Files</title>
304
305 <para>
306 Append files, which are files that have the
307 <filename>.bbappend</filename> file extension, add or
308 extend build information to an existing
309 recipe file.
310 </para>
311
312 <para>
313 BitBake expects every append file to have a corresponding recipe file.
314 Furthermore, the append file and corresponding recipe file
315 must use the same root filename.
316 The filenames can differ only in the file type suffix used
317 (e.g. <filename>formfactor_0.0.bb</filename> and
318 <filename>formfactor_0.0.bbappend</filename>).
319 </para>
320
321 <para>
322 Information in append files overrides the information in the
323 similarly-named recipe file.
324 </para>
325
326 <para>
327 When you name an append file, you can use the
328 wildcard character (%) to allow for matching recipe names.
329 For example, suppose you have an append file named
330 as follows:
331 <literallayout class='monospaced'>
332 busybox_1.21.%.bbappend
333 </literallayout>
334 That append file would match any <filename>busybox_1.21.x.bb</filename>
335 version of the recipe.
336 So, the append file would match the following recipe names:
337 <literallayout class='monospaced'>
338 busybox_1.21.1.bb
339 busybox_1.21.2.bb
340 busybox_1.21.3.bb
341 </literallayout>
342 If the <filename>busybox</filename> recipe was updated to
343 <filename>busybox_1.3.0.bb</filename>, the append name would not
344 match.
345 However, if you named the append file
346 <filename>busybox_1.%.bbappend</filename>, then you would have a match.
347 </para>
348 </section>
349 </section>
350
351 <section id='obtaining-bitbake'>
352 <title>Obtaining BitBake</title>
353
354 <para>
355 You can obtain BitBake several different ways:
356 <itemizedlist>
357 <listitem><para><emphasis>Cloning BitBake:</emphasis>
358 Using Git to clone the BitBake source code repository
359 is the recommended method for obtaining BitBake.
360 Cloning the repository makes it easy to get bug fixes
361 and have access to stable branches and the master
362 branch.
363 Once you have cloned BitBake, you should use
364 the latest stable
365 branch for development since the master branch is for
366 BitBake development and might contain less stable changes.
367 </para>
368 <para>You usually need a version of BitBake
369 that matches the metadata you are using.
370 The metadata is generally backwards compatible but
371 not forward compatible.</para>
372 <para>Here is an example that clones the BitBake repository:
373 <literallayout class='monospaced'>
374 $ git clone git://git.openembedded.org/bitbake
375 </literallayout>
376 This command clones the BitBake Git repository into a
377 directory called <filename>bitbake</filename>.
378 Alternatively, you can
379 designate a directory after the
380 <filename>git clone</filename> command
381 if you want to call the new directory something
382 other than <filename>bitbake</filename>.
383 Here is an example that names the directory
384 <filename>bbdev</filename>:
385 <literallayout class='monospaced'>
386 $ git clone git://git.openembedded.org/bitbake bbdev
387 </literallayout></para></listitem>
388 <listitem><para><emphasis>Installation using your Distribution
389 Package Management System:</emphasis>
390 This method is not
391 recommended because the BitBake version that is
392 provided by your distribution, in most cases,
393 is several
394 releases behind a snapshot of the BitBake repository.
395 </para></listitem>
396 <listitem><para><emphasis>Taking a snapshot of BitBake:</emphasis>
397 Downloading a snapshot of BitBake from the
398 source code repository gives you access to a known
399 branch or release of BitBake.
400 <note>
401 Cloning the Git repository, as described earlier,
402 is the preferred method for getting BitBake.
403 Cloning the repository makes it easier to update as
404 patches are added to the stable branches.
405 </note></para>
406 <para>The following example downloads a snapshot of
407 BitBake version 1.17.0:
408 <literallayout class='monospaced'>
409 $ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz
410 $ tar zxpvf bitbake-1.17.0.tar.gz
411 </literallayout>
412 After extraction of the tarball using the tar utility,
413 you have a directory entitled
414 <filename>bitbake-1.17.0</filename>.
415 </para></listitem>
416 </itemizedlist>
417 </para>
418 </section>
419
420 <section id="bitbake-user-manual-command">
421 <title>The BitBake Command</title>
422
423 <para>
424 The <filename>bitbake</filename> command is the primary interface
425 to the BitBake tool.
426 This section presents the BitBake command syntax and provides
427 several execution examples.
428 </para>
429
430 <section id='usage-and-syntax'>
431 <title>Usage and syntax</title>
432
433 <para>
434 Following is the usage and syntax for BitBake:
435 <literallayout class='monospaced'>
436 $ bitbake -h
437 Usage: bitbake [options] [recipename/target ...]
438
439 Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
440 It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
441 will provide the layer, BBFILES and other configuration information.
442
443 Options:
444 --version show program's version number and exit
445 -h, --help show this help message and exit
446 -b BUILDFILE, --buildfile=BUILDFILE
447 Execute tasks from a specific .bb recipe directly.
448 WARNING: Does not handle any dependencies from other
449 recipes.
450 -k, --continue Continue as much as possible after an error. While the
451 target that failed and anything depending on it cannot
452 be built, as much as possible will be built before
453 stopping.
454 -a, --tryaltconfigs Continue with builds by trying to use alternative
455 providers where possible.
456 -f, --force Force the specified targets/task to run (invalidating
457 any existing stamp file).
458 -c CMD, --cmd=CMD Specify the task to execute. The exact options
459 available depend on the metadata. Some examples might
460 be 'compile' or 'populate_sysroot' or 'listtasks' may
461 give a list of the tasks available.
462 -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
463 Invalidate the stamp for the specified task such as
464 'compile' and then run the default task for the
465 specified target(s).
466 -r PREFILE, --read=PREFILE
467 Read the specified file before bitbake.conf.
468 -R POSTFILE, --postread=POSTFILE
469 Read the specified file after bitbake.conf.
470 -v, --verbose Output more log message data to the terminal.
471 -D, --debug Increase the debug level. You can specify this more
472 than once.
473 -n, --dry-run Don't execute, just go through the motions.
474 -S DUMP_SIGNATURES, --dump-signatures=DUMP_SIGNATURES
475 Dump out the signature construction information, with
476 no task execution. Parameters are passed to the
477 signature handling code, use 'none' if no specific
478 handler is required.
479 -p, --parse-only Quit after parsing the BB recipes.
480 -s, --show-versions Show current and preferred versions of all recipes.
481 -e, --environment Show the global or per-package environment complete
482 with information about where variables were
483 set/changed.
484 -g, --graphviz Save dependency tree information for the specified
485 targets in the dot syntax.
486 -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
487 Assume these dependencies don't exist and are already
488 provided (equivalent to ASSUME_PROVIDED). Useful to
489 make dependency graphs more appealing
490 -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
491 Show debug logging for the specified logging domains
492 -P, --profile Profile the command and save reports.
493 -u UI, --ui=UI The user interface to use (e.g. knotty, hob, depexp).
494 -t SERVERTYPE, --servertype=SERVERTYPE
495 Choose which server to use, process or xmlrpc.
496 --revisions-changed Set the exit code depending on whether upstream
497 floating revisions have changed or not.
498 --server-only Run bitbake without a UI, only starting a server
499 (cooker) process.
500 -B BIND, --bind=BIND The name/address for the bitbake server to bind to.
501 --no-setscene Do not run any setscene tasks. sstate will be ignored
502 and everything needed, built.
503 --remote-server=REMOTE_SERVER
504 Connect to the specified server.
505 -m, --kill-server Terminate the remote server.
506 --observe-only Connect to a server as an observing-only client.
507 --status-only Check the status of the remote bitbake server.
508 </literallayout>
509 </para>
510 </section>
511
512 <section id='bitbake-examples'>
513 <title>Examples</title>
514
515 <para>
516 This section presents some examples showing how to use BitBake.
517 </para>
518
519 <section id='example-executing-a-task-against-a-single-recipe'>
520 <title>Executing a Task Against a Single Recipe</title>
521
522 <para>
523 Executing tasks for a single recipe file is relatively simple.
524 You specify the file in question, and BitBake parses
525 it and executes the specified task.
526 If you do not specify a task, BitBake executes the default
527 task, which is "build”.
528 BitBake obeys inter-task dependencies when doing
529 so.
530 </para>
531
532 <para>
533 The following command runs the build task, which is
534 the default task, on the <filename>foo_1.0.bb</filename>
535 recipe file:
536 <literallayout class='monospaced'>
537 $ bitbake -b foo_1.0.bb
538 </literallayout>
539 The following command runs the clean task on the
540 <filename>foo.bb</filename> recipe file:
541 <literallayout class='monospaced'>
542 $ bitbake -b foo.bb -c clean
543 </literallayout>
544 <note>
545 The "-b" option explicitly does not handle recipe
546 dependencies.
547 Other than for debugging purposes, it is instead
548 recommended that you use the syntax presented in the
549 next section.
550 </note>
551 </para>
552 </section>
553
554 <section id='executing-tasks-against-a-set-of-recipe-files'>
555 <title>Executing Tasks Against a Set of Recipe Files</title>
556
557 <para>
558 There are a number of additional complexities introduced
559 when one wants to manage multiple <filename>.bb</filename>
560 files.
561 Clearly there needs to be a way to tell BitBake what
562 files are available, and of those, which you
563 want to execute.
564 There also needs to be a way for each recipe
565 to express its dependencies, both for build-time and
566 runtime.
567 There must be a way for you to express recipe preferences
568 when multiple recipes provide the same functionality, or when
569 there are multiple versions of a recipe.
570 </para>
571
572 <para>
573 The <filename>bitbake</filename> command, when not using
574 "--buildfile" or "-b" only accepts a "PROVIDES".
575 You cannot provide anything else.
576 By default, a recipe file generally "PROVIDES" its
577 "packagename" as shown in the following example:
578 <literallayout class='monospaced'>
579 $ bitbake foo
580 </literallayout>
581 This next example "PROVIDES" the package name and also uses
582 the "-c" option to tell BitBake to just execute the
583 <filename>do_clean</filename> task:
584 <literallayout class='monospaced'>
585 $ bitbake -c clean foo
586 </literallayout>
587 </para>
588 </section>
589
590 <section id='generating-dependency-graphs'>
591 <title>Generating Dependency Graphs</title>
592
593 <para>
594 BitBake is able to generate dependency graphs using
595 the <filename>dot</filename> syntax.
596 You can convert these graphs into images using the
597 <filename>dot</filename> tool from
598 <ulink url='http://www.graphviz.org'>Graphviz</ulink>.
599 </para>
600
601 <para>
602 When you generate a dependency graph, BitBake writes four files
603 to the current working directory:
604 <itemizedlist>
605 <listitem><para><emphasis><filename>package-depends.dot</filename>:</emphasis>
606 Shows BitBake's knowledge of dependencies between
607 runtime targets.
608 </para></listitem>
609 <listitem><para><emphasis><filename>pn-depends.dot</filename>:</emphasis>
610 Shows dependencies between build-time targets
611 (i.e. recipes).
612 </para></listitem>
613 <listitem><para><emphasis><filename>task-depends.dot</filename>:</emphasis>
614 Shows dependencies between tasks.
615 </para></listitem>
616 <listitem><para><emphasis><filename>pn-buildlist</filename>:</emphasis>
617 Shows a simple list of targets that are to be built.
618 </para></listitem>
619 </itemizedlist>
620 </para>
621
622 <para>
623 To stop depending on common depends, use the "-I" depend
624 option and BitBake omits them from the graph.
625 Leaving this information out can produce more readable graphs.
626 This way, you can remove from the graph
627 <filename>DEPENDS</filename> from inherited classes
628 such as <filename>base.bbclass</filename>.
629 </para>
630
631 <para>
632 Here are two examples that create dependency graphs.
633 The second example omits depends common in OpenEmbedded from
634 the graph:
635 <literallayout class='monospaced'>
636 $ bitbake -g foo
637
638 $ bitbake -g -I virtual/kernel -I eglibc foo
639 </literallayout>
640 </para>
641 </section>
642 </section>
643 </section>
644</chapter>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-23 22:52:39 +0000