From 392de9d5685e17aed868eb5659032103748825f4 Mon Sep 17 00:00:00 2001 From: Richard Purdie Date: Fri, 11 Apr 2014 17:38:18 +0100 Subject: bitbake: doc: Rename user-manual -> bitbake-user-manual This manual gets combined with other manuals and in that context, it helps a lot if its seen as the Bitbake User Manual. Renames are a pain but this is worthwhile so that other docs can correctly be combined with this one. This also clarifies things like google search results which is helpful. (Bitbake rev: 452a62ae0c2793e281d6769fd3e45500a74898d6) Signed-off-by: Richard Purdie --- .../bitbake-user-manual-intro.xml | 644 +++++++++++++++++++++ 1 file changed, 644 insertions(+) create mode 100644 bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml') diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml new file mode 100644 index 0000000000..ae267b42c8 --- /dev/null +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml @@ -0,0 +1,644 @@ + + + + Overview + + + Welcome to the BitBake User Manual. + This manual provides information on the BitBake tool. + The information attempts to be as independent as possible regarding + systems that use BitBake, such as the Yocto Project and + OpenEmbedded. + In some cases, scenarios or examples that within the context of + a build system are used in the manual to help with understanding. + For these cases, the manual clearly states the context. + + +
+ Introduction + + + Fundamentally, BitBake is a generic task execution + engine that allows shell and Python tasks to be run + efficiently and in parallel while working within + complex inter-task dependency constraints. + One of BitBake's main users, OpenEmbedded, takes this core + and builds embedded Linux software stacks using + a task-oriented approach. + + + + Conceptually, BitBake is similar to GNU Make in + some regards but has significant differences: + + + BitBake executes tasks according to provided + metadata that builds up the tasks. + Metadata is stored in recipe (.bb), + configuration (.conf), and class + (.bbclass) files and provides + BitBake with instructions on what tasks to run and + the dependencies between those tasks. + + + BitBake includes a fetcher library for obtaining source + code from various places such as source control + systems or websites. + + + The instructions for each unit to be built (e.g. a piece + of software) are known as recipe files and + contain all the information about the unit + (dependencies, source file locations, checksums, description + and so on). + + + BitBake includes a client/server abstraction and can + be used from a command line or used as a service over XMLRPC and + has several different user interfaces. + + + +
+ +
+ History and Goals + + + BitBake was originally a part of the OpenEmbedded project. + It was inspired by the Portage package management system + used by the Gentoo Linux distribution. + On December 7, 2004, OpenEmbedded project team member, + Chris Larson split the project into two distinct pieces: + + BitBake, a generic task executor + OpenEmbedded, a metadata set utilized by + BitBake + + Today, BitBake is the primary basis of the + OpenEmbedded + project, which is being used to build and maintain Linux + distributions such as the Angstrom Distribution and which is used + as the build tool for Linux projects such as the Yocto Project. + + + + Prior to BitBake, no other build tool adequately met the needs of + an aspiring embedded Linux distribution. + All of the build systems used by traditional desktop Linux + distributions lacked important functionality, and none of the + ad-hoc Buildroot-based systems, prevalent in the + embedded space, were scalable or maintainable. + + + + Some important original goals for BitBake were: + + + Handle cross-compilation. + + + Handle inter-package dependencies (build time on + target architecture, build time on native + architecture, and runtime). + + + Support running any number of tasks within a given + package, including, but not limited to, fetching + upstream sources, unpacking them, patching them, + configuring them, and so forth. + + + Be Linux distribution agnostic for both build and + target systems. + + + Be architecture agnostic. + + + Support multiple build and target operating systems + (e.g. Cygwin, the BSDs, and so forth). + + + Be self contained, rather than tightly + integrated into the build machine's root + filesystem. + + + Handle conditional metadata on the target architecture, + operating system, distribution, and machine. + + + Be easy to use the tools to supply local metadata and packages + against which to operate. + + + Be easy to use BitBake to collaborate between multiple + projects for their builds. + + + Provide an inheritance mechanism that share + common metadata between many packages. + + + Over time it became apparent that some further requirements + were necessary: + + + Handle variants of a base recipe (e.g. native, sdk, + and multilib). + + + Split metadata into layers and allow layers + to override each other. + + + Allow representation of a given set of input variables + to a task as a checksum. + Based on that checksum, allow acceleration of builds + with prebuilt components. + + + BitBake satisfies all the original requirements and many more + with extensions being made to the basic functionality to + reflect the additional requirements. + Flexibility and power have always been the priorities. + BitBake is highly extensible and supports embedded Python code and + execution of any arbitrary tasks. + +
+ +
+ Concepts + + + BitBake is a program written in the Python language. + At the highest level, BitBake interprets metadata, decides + what tasks are required to run, and executes those tasks. + Similar to GNU Make, BitBake controls how software is + built. + GNU Make achieves its control through "makefiles". + BitBake uses "recipes". + + + + BitBake extends the capabilities of a simple + tool like GNU Make by allowing for much more complex tasks + to be completed, such as assembling entire embedded Linux + distributions. + + + + The remainder of this section introduces several concepts + that should be understood in order to better leverage + the power of BitBake. + + +
+ Recipes + + + BitBake Recipes, which are denoted by the file extension + .bb, are the most basic metadata files. + These recipe files provide BitBake with the following: + + Descriptive information about the package + The version of the recipe + Existing Dependencies + Where the source code resides + Whether the source code requires any patches + How to compile the source code + Where on the target machine to install the + package being compiled + + + + + Within the context of BitBake, or any project utilizing BitBake + as its build system, files with the .bb + extension are referred to as recipes. + + The term "package" is also commonly used to describe recipes. + However, since the same word is used to describe packaged + output from a project, it is best to maintain a single + descriptive term, "recipes". + + +
+ +
+ Configuration Files + + + Configuration files, which are denoted by the + .conf extension, define + various configuration variables that govern the project's build + process. + These files fall into several areas that define + machine configuration options, distribution configuration + options, compiler tuning options, general common + configuration options, and user configuration options. + The main configuration file is the sample + bitbake.conf file, which is + located within the BitBake source tree + conf directory. + +
+ +
+ Classes + + + Class files, which are denoted by the + .bbclass extension, contain + information that is useful to share between metadata files. + The BitBake source tree currently comes with one class metadata file + called base.bbclass. + You can find this file in the + classes directory. + The base.bbclass is special since it + is always included automatically for all recipes + and classes. + This class contains definitions for standard basic tasks such + as fetching, unpacking, configuring (empty by default), + compiling (runs any Makefile present), installing (empty by + default) and packaging (empty by default). + These tasks are often overridden or extended by other classes + added during the project development process. + +
+ +
+ Layers + + + Layers allow you to isolate different types of + customizations from each other. + While you might find it tempting to keep everything in one layer + when working on a single project, the more modular you organize + your metadata, the easier it is to cope with future changes. + + + + To illustrate how you can use layers to keep things modular, + consider customizations you might make to support a specific target machine. + These types of customizations typically reside in a special layer, + rather than a general layer, called a Board Specific Package (BSP) Layer. + Furthermore, the machine customizations should be isolated from + recipes and metadata that support a new GUI environment, for + example. + This situation gives you a couple of layers: one for the machine + configurations and one for the GUI environment. + It is important to understand, however, that the BSP layer can still + make machine-specific additions to recipes within + the GUI environment layer without polluting the GUI layer itself + with those machine-specific changes. + You can accomplish this through a recipe that is a BitBake append + (.bbappend) file. + +
+ +
+ Append Files + + + Append files, which are files that have the + .bbappend file extension, add or + extend build information to an existing + recipe file. + + + + BitBake expects every append file to have a corresponding recipe file. + Furthermore, the append file and corresponding recipe file + must use the same root filename. + The filenames can differ only in the file type suffix used + (e.g. formfactor_0.0.bb and + formfactor_0.0.bbappend). + + + + Information in append files overrides the information in the + similarly-named recipe file. + + + + When you name an append file, you can use the + wildcard character (%) to allow for matching recipe names. + For example, suppose you have an append file named + as follows: + + busybox_1.21.%.bbappend + + That append file would match any busybox_1.21.x.bb + version of the recipe. + So, the append file would match the following recipe names: + + busybox_1.21.1.bb + busybox_1.21.2.bb + busybox_1.21.3.bb + + If the busybox recipe was updated to + busybox_1.3.0.bb, the append name would not + match. + However, if you named the append file + busybox_1.%.bbappend, then you would have a match. + +
+
+ +
+ Obtaining BitBake + + + You can obtain BitBake several different ways: + + Cloning BitBake: + Using Git to clone the BitBake source code repository + is the recommended method for obtaining BitBake. + Cloning the repository makes it easy to get bug fixes + and have access to stable branches and the master + branch. + Once you have cloned BitBake, you should use + the latest stable + branch for development since the master branch is for + BitBake development and might contain less stable changes. + + You usually need a version of BitBake + that matches the metadata you are using. + The metadata is generally backwards compatible but + not forward compatible. + Here is an example that clones the BitBake repository: + + $ git clone git://git.openembedded.org/bitbake + + This command clones the BitBake Git repository into a + directory called bitbake. + Alternatively, you can + designate a directory after the + git clone command + if you want to call the new directory something + other than bitbake. + Here is an example that names the directory + bbdev: + + $ git clone git://git.openembedded.org/bitbake bbdev + + Installation using your Distribution + Package Management System: + This method is not + recommended because the BitBake version that is + provided by your distribution, in most cases, + is several + releases behind a snapshot of the BitBake repository. + + Taking a snapshot of BitBake: + Downloading a snapshot of BitBake from the + source code repository gives you access to a known + branch or release of BitBake. + + Cloning the Git repository, as described earlier, + is the preferred method for getting BitBake. + Cloning the repository makes it easier to update as + patches are added to the stable branches. + + The following example downloads a snapshot of + BitBake version 1.17.0: + + $ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz + $ tar zxpvf bitbake-1.17.0.tar.gz + + After extraction of the tarball using the tar utility, + you have a directory entitled + bitbake-1.17.0. + + + +
+ +
+ The BitBake Command + + + The bitbake command is the primary interface + to the BitBake tool. + This section presents the BitBake command syntax and provides + several execution examples. + + +
+ Usage and syntax + + + Following is the usage and syntax for BitBake: + + $ bitbake -h + Usage: bitbake [options] [recipename/target ...] + + Executes the specified task (default is 'build') for a given set of target recipes (.bb files). + It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which + will provide the layer, BBFILES and other configuration information. + + Options: + --version show program's version number and exit + -h, --help show this help message and exit + -b BUILDFILE, --buildfile=BUILDFILE + Execute tasks from a specific .bb recipe directly. + WARNING: Does not handle any dependencies from other + recipes. + -k, --continue Continue as much as possible after an error. While the + target that failed and anything depending on it cannot + be built, as much as possible will be built before + stopping. + -a, --tryaltconfigs Continue with builds by trying to use alternative + providers where possible. + -f, --force Force the specified targets/task to run (invalidating + any existing stamp file). + -c CMD, --cmd=CMD Specify the task to execute. The exact options + available depend on the metadata. Some examples might + be 'compile' or 'populate_sysroot' or 'listtasks' may + give a list of the tasks available. + -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP + Invalidate the stamp for the specified task such as + 'compile' and then run the default task for the + specified target(s). + -r PREFILE, --read=PREFILE + Read the specified file before bitbake.conf. + -R POSTFILE, --postread=POSTFILE + Read the specified file after bitbake.conf. + -v, --verbose Output more log message data to the terminal. + -D, --debug Increase the debug level. You can specify this more + than once. + -n, --dry-run Don't execute, just go through the motions. + -S DUMP_SIGNATURES, --dump-signatures=DUMP_SIGNATURES + Dump out the signature construction information, with + no task execution. Parameters are passed to the + signature handling code, use 'none' if no specific + handler is required. + -p, --parse-only Quit after parsing the BB recipes. + -s, --show-versions Show current and preferred versions of all recipes. + -e, --environment Show the global or per-package environment complete + with information about where variables were + set/changed. + -g, --graphviz Save dependency tree information for the specified + targets in the dot syntax. + -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED + Assume these dependencies don't exist and are already + provided (equivalent to ASSUME_PROVIDED). Useful to + make dependency graphs more appealing + -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS + Show debug logging for the specified logging domains + -P, --profile Profile the command and save reports. + -u UI, --ui=UI The user interface to use (e.g. knotty, hob, depexp). + -t SERVERTYPE, --servertype=SERVERTYPE + Choose which server to use, process or xmlrpc. + --revisions-changed Set the exit code depending on whether upstream + floating revisions have changed or not. + --server-only Run bitbake without a UI, only starting a server + (cooker) process. + -B BIND, --bind=BIND The name/address for the bitbake server to bind to. + --no-setscene Do not run any setscene tasks. sstate will be ignored + and everything needed, built. + --remote-server=REMOTE_SERVER + Connect to the specified server. + -m, --kill-server Terminate the remote server. + --observe-only Connect to a server as an observing-only client. + --status-only Check the status of the remote bitbake server. + + +
+ +
+ Examples + + + This section presents some examples showing how to use BitBake. + + +
+ Executing a Task Against a Single Recipe + + + Executing tasks for a single recipe file is relatively simple. + You specify the file in question, and BitBake parses + it and executes the specified task. + If you do not specify a task, BitBake executes the default + task, which is "build”. + BitBake obeys inter-task dependencies when doing + so. + + + + The following command runs the build task, which is + the default task, on the foo_1.0.bb + recipe file: + + $ bitbake -b foo_1.0.bb + + The following command runs the clean task on the + foo.bb recipe file: + + $ bitbake -b foo.bb -c clean + + + The "-b" option explicitly does not handle recipe + dependencies. + Other than for debugging purposes, it is instead + recommended that you use the syntax presented in the + next section. + + +
+ +
+ Executing Tasks Against a Set of Recipe Files + + + There are a number of additional complexities introduced + when one wants to manage multiple .bb + files. + Clearly there needs to be a way to tell BitBake what + files are available, and of those, which you + want to execute. + There also needs to be a way for each recipe + to express its dependencies, both for build-time and + runtime. + There must be a way for you to express recipe preferences + when multiple recipes provide the same functionality, or when + there are multiple versions of a recipe. + + + + The bitbake command, when not using + "--buildfile" or "-b" only accepts a "PROVIDES". + You cannot provide anything else. + By default, a recipe file generally "PROVIDES" its + "packagename" as shown in the following example: + + $ bitbake foo + + This next example "PROVIDES" the package name and also uses + the "-c" option to tell BitBake to just execute the + do_clean task: + + $ bitbake -c clean foo + + +
+ +
+ Generating Dependency Graphs + + + BitBake is able to generate dependency graphs using + the dot syntax. + You can convert these graphs into images using the + dot tool from + Graphviz. + + + + When you generate a dependency graph, BitBake writes four files + to the current working directory: + + package-depends.dot: + Shows BitBake's knowledge of dependencies between + runtime targets. + + pn-depends.dot: + Shows dependencies between build-time targets + (i.e. recipes). + + task-depends.dot: + Shows dependencies between tasks. + + pn-buildlist: + Shows a simple list of targets that are to be built. + + + + + + To stop depending on common depends, use the "-I" depend + option and BitBake omits them from the graph. + Leaving this information out can produce more readable graphs. + This way, you can remove from the graph + DEPENDS from inherited classes + such as base.bbclass. + + + + Here are two examples that create dependency graphs. + The second example omits depends common in OpenEmbedded from + the graph: + + $ bitbake -g foo + + $ bitbake -g -I virtual/kernel -I eglibc foo + + +
+
+
+ -- cgit v1.2.3-54-g00ecf