1 files changed, 571 insertions, 0 deletions

diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst
new file mode 100644
index 0000000000..3c24e4e2cc
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst
@@ -0,0 +1,571 @@
+========
+Overview
+========
+
+Welcome to the BitBake User Manual. This manual provides information on
+the BitBake tool. The information attempts to be as independent as
+possible regarding systems that use BitBake, such as OpenEmbedded and
+the Yocto Project. In some cases, scenarios or examples within the
+context of a build system are used in the manual to help with
+understanding. For these cases, the manual clearly states the context.
+
+.. _intro:
+
+Introduction
+============
+
+Fundamentally, BitBake is a generic task execution engine that allows
+shell and Python tasks to be run efficiently and in parallel while
+working within complex inter-task dependency constraints. One of
+BitBake's main users, OpenEmbedded, takes this core and builds embedded
+Linux software stacks using a task-oriented approach.
+
+Conceptually, BitBake is similar to GNU Make in some regards but has
+significant differences:
+
+-  BitBake executes tasks according to provided metadata that builds up
+   the tasks. Metadata is stored in recipe (``.bb``) and related recipe
+   "append" (``.bbappend``) files, configuration (``.conf``) and
+   underlying include (``.inc``) files, and in class (``.bbclass``)
+   files. The metadata provides BitBake with instructions on what tasks
+   to run and the dependencies between those tasks.
+
+-  BitBake includes a fetcher library for obtaining source code from
+   various places such as local files, source control systems, or
+   websites.
+
+-  The instructions for each unit to be built (e.g. a piece of software)
+   are known as "recipe" files and contain all the information about the
+   unit (dependencies, source file locations, checksums, description and
+   so on).
+
+-  BitBake includes a client/server abstraction and can be used from a
+   command line or used as a service over XML-RPC and has several
+   different user interfaces.
+
+History and Goals
+=================
+
+BitBake was originally a part of the OpenEmbedded project. It was
+inspired by the Portage package management system used by the Gentoo
+Linux distribution. On December 7, 2004, OpenEmbedded project team
+member Chris Larson split the project into two distinct pieces:
+
+-  BitBake, a generic task executor
+
+-  OpenEmbedded, a metadata set utilized by BitBake
+
+Today, BitBake is the primary basis of the
+`OpenEmbedded <http://www.openembedded.org/>`__ project, which is being
+used to build and maintain Linux distributions such as the `Angstrom
+Distribution <http://www.angstrom-distribution.org/>`__, and which is
+also being used as the build tool for Linux projects such as the `Yocto
+Project <http://www.yoctoproject.org>`__.
+
+Prior to BitBake, no other build tool adequately met the needs of an
+aspiring embedded Linux distribution. All of the build systems used by
+traditional desktop Linux distributions lacked important functionality,
+and none of the ad hoc Buildroot-based systems, prevalent in the
+embedded space, were scalable or maintainable.
+
+Some important original goals for BitBake were:
+
+-  Handle cross-compilation.
+
+-  Handle inter-package dependencies (build time on target architecture,
+   build time on native architecture, and runtime).
+
+-  Support running any number of tasks within a given package,
+   including, but not limited to, fetching upstream sources, unpacking
+   them, patching them, configuring them, and so forth.
+
+-  Be Linux distribution agnostic for both build and target systems.
+
+-  Be architecture agnostic.
+
+-  Support multiple build and target operating systems (e.g. Cygwin, the
+   BSDs, and so forth).
+
+-  Be self-contained, rather than tightly integrated into the build
+   machine's root filesystem.
+
+-  Handle conditional metadata on the target architecture, operating
+   system, distribution, and machine.
+
+-  Be easy to use the tools to supply local metadata and packages
+   against which to operate.
+
+-  Be easy to use BitBake to collaborate between multiple projects for
+   their builds.
+
+-  Provide an inheritance mechanism to share common metadata between
+   many packages.
+
+Over time it became apparent that some further requirements were
+necessary:
+
+-  Handle variants of a base recipe (e.g. native, sdk, and multilib).
+
+-  Split metadata into layers and allow layers to enhance or override
+   other layers.
+
+-  Allow representation of a given set of input variables to a task as a
+   checksum. Based on that checksum, allow acceleration of builds with
+   prebuilt components.
+
+BitBake satisfies all the original requirements and many more with
+extensions being made to the basic functionality to reflect the
+additional requirements. Flexibility and power have always been the
+priorities. BitBake is highly extensible and supports embedded Python
+code and execution of any arbitrary tasks.
+
+.. _Concepts:
+
+Concepts
+========
+
+BitBake is a program written in the Python language. At the highest
+level, BitBake interprets metadata, decides what tasks are required to
+run, and executes those tasks. Similar to GNU Make, BitBake controls how
+software is built. GNU Make achieves its control through "makefiles",
+while BitBake uses "recipes".
+
+BitBake extends the capabilities of a simple tool like GNU Make by
+allowing for the definition of much more complex tasks, such as
+assembling entire embedded Linux distributions.
+
+The remainder of this section introduces several concepts that should be
+understood in order to better leverage the power of BitBake.
+
+Recipes
+-------
+
+BitBake Recipes, which are denoted by the file extension ``.bb``, are
+the most basic metadata files. These recipe files provide BitBake with
+the following:
+
+-  Descriptive information about the package (author, homepage, license,
+   and so on)
+
+-  The version of the recipe
+
+-  Existing dependencies (both build and runtime dependencies)
+
+-  Where the source code resides and how to fetch it
+
+-  Whether the source code requires any patches, where to find them, and
+   how to apply them
+
+-  How to configure and compile the source code
+
+-  How to assemble the generated artifacts into one or more installable
+   packages
+
+-  Where on the target machine to install the package or packages
+   created
+
+Within the context of BitBake, or any project utilizing BitBake as its
+build system, files with the ``.bb`` extension are referred to as
+recipes.
+
+.. note::
+
+   The term "package" is also commonly used to describe recipes.
+   However, since the same word is used to describe packaged output from
+   a project, it is best to maintain a single descriptive term -
+   "recipes". Put another way, a single "recipe" file is quite capable
+   of generating a number of related but separately installable
+   "packages". In fact, that ability is fairly common.
+
+Configuration Files
+-------------------
+
+Configuration files, which are denoted by the ``.conf`` extension,
+define various configuration variables that govern the project's build
+process. These files fall into several areas that define machine
+configuration, distribution configuration, possible compiler tuning,
+general common configuration, and user configuration. The main
+configuration file is the sample ``bitbake.conf`` file, which is located
+within the BitBake source tree ``conf`` directory.
+
+Classes
+-------
+
+Class files, which are denoted by the ``.bbclass`` extension, contain
+information that is useful to share between metadata files. The BitBake
+source tree currently comes with one class metadata file called
+``base.bbclass``. You can find this file in the ``classes`` directory.
+The ``base.bbclass`` class files is special since it is always included
+automatically for all recipes and classes. This class contains
+definitions for standard basic tasks such as fetching, unpacking,
+configuring (empty by default), compiling (runs any Makefile present),
+installing (empty by default) and packaging (empty by default). These
+tasks are often overridden or extended by other classes added during the
+project development process.
+
+Layers
+------
+
+Layers allow you to isolate different types of customizations from each
+other. While you might find it tempting to keep everything in one layer
+when working on a single project, the more modular your metadata, the
+easier it is to cope with future changes.
+
+To illustrate how you can use layers to keep things modular, consider
+customizations you might make to support a specific target machine.
+These types of customizations typically reside in a special layer,
+rather than a general layer, called a Board Support Package (BSP) layer.
+Furthermore, the machine customizations should be isolated from recipes
+and metadata that support a new GUI environment, for example. This
+situation gives you a couple of layers: one for the machine
+configurations and one for the GUI environment. It is important to
+understand, however, that the BSP layer can still make machine-specific
+additions to recipes within the GUI environment layer without polluting
+the GUI layer itself with those machine-specific changes. You can
+accomplish this through a recipe that is a BitBake append
+(``.bbappend``) file.
+
+.. _append-bbappend-files:
+
+Append Files
+------------
+
+Append files, which are files that have the ``.bbappend`` file
+extension, extend or override information in an existing recipe file.
+
+BitBake expects every append file to have a corresponding recipe file.
+Furthermore, the append file and corresponding recipe file must use the
+same root filename. The filenames can differ only in the file type
+suffix used (e.g. ``formfactor_0.0.bb`` and
+``formfactor_0.0.bbappend``).
+
+Information in append files extends or overrides the information in the
+underlying, similarly-named recipe files.
+
+When you name an append file, you can use the "``%``" wildcard character
+to allow for matching recipe names. For example, suppose you have an
+append file named as follows: busybox_1.21.%.bbappend That append file
+would match any ``busybox_1.21.``\ x\ ``.bb`` version of the recipe. So,
+the append file would match the following recipe names:
+busybox_1.21.1.bb busybox_1.21.2.bb busybox_1.21.3.bb
+
+.. note::
+
+   The use of the "
+   %
+   " character is limited in that it only works directly in front of the
+   .bbappend
+   portion of the append file's name. You cannot use the wildcard
+   character in any other location of the name.
+
+If the ``busybox`` recipe was updated to ``busybox_1.3.0.bb``, the
+append name would not match. However, if you named the append file
+``busybox_1.%.bbappend``, then you would have a match.
+
+In the most general case, you could name the append file something as
+simple as ``busybox_%.bbappend`` to be entirely version independent.
+
+Obtaining BitBake
+=================
+
+You can obtain BitBake several different ways:
+
+-  *Cloning BitBake:* Using Git to clone the BitBake source code
+   repository is the recommended method for obtaining BitBake. Cloning
+   the repository makes it easy to get bug fixes and have access to
+   stable branches and the master branch. Once you have cloned BitBake,
+   you should use the latest stable branch for development since the
+   master branch is for BitBake development and might contain less
+   stable changes.
+
+   You usually need a version of BitBake that matches the metadata you
+   are using. The metadata is generally backwards compatible but not
+   forward compatible.
+
+   Here is an example that clones the BitBake repository: $ git clone
+   git://git.openembedded.org/bitbake This command clones the BitBake
+   Git repository into a directory called ``bitbake``. Alternatively,
+   you can designate a directory after the ``git clone`` command if you
+   want to call the new directory something other than ``bitbake``. Here
+   is an example that names the directory ``bbdev``: $ git clone
+   git://git.openembedded.org/bitbake bbdev
+
+-  *Installation using your Distribution Package Management System:*
+   This method is not recommended because the BitBake version that is
+   provided by your distribution, in most cases, is several releases
+   behind a snapshot of the BitBake repository.
+
+-  *Taking a snapshot of BitBake:* Downloading a snapshot of BitBake
+   from the source code repository gives you access to a known branch or
+   release of BitBake.
+
+   .. note::
+
+      Cloning the Git repository, as described earlier, is the preferred
+      method for getting BitBake. Cloning the repository makes it easier
+      to update as patches are added to the stable branches.
+
+   The following example downloads a snapshot of BitBake version 1.17.0:
+   $ wget
+   http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz $
+   tar zxpvf bitbake-1.17.0.tar.gz After extraction of the tarball using
+   the tar utility, you have a directory entitled ``bitbake-1.17.0``.
+
+-  *Using the BitBake that Comes With Your Build Checkout:* A final
+   possibility for getting a copy of BitBake is that it already comes
+   with your checkout of a larger BitBake-based build system, such as
+   Poky. Rather than manually checking out individual layers and gluing
+   them together yourself, you can check out an entire build system. The
+   checkout will already include a version of BitBake that has been
+   thoroughly tested for compatibility with the other components. For
+   information on how to check out a particular BitBake-based build
+   system, consult that build system's supporting documentation.
+
+.. _bitbake-user-manual-command:
+
+The BitBake Command
+===================
+
+The ``bitbake`` command is the primary interface to the BitBake tool.
+This section presents the BitBake command syntax and provides several
+execution examples.
+
+Usage and syntax
+----------------
+
+Following is the usage and syntax for BitBake: $ bitbake -h Usage:
+bitbake [options] [recipename/target recipe:do_task ...] Executes the
+specified task (default is 'build') for a given set of target recipes
+(.bb files). It is assumed there is a conf/bblayers.conf available in
+cwd or in BBPATH which will provide the layer, BBFILES and other
+configuration information. Options: --version show program's version
+number and exit -h, --help show this help message and exit -b BUILDFILE,
+--buildfile=BUILDFILE Execute tasks from a specific .bb recipe directly.
+WARNING: Does not handle any dependencies from other recipes. -k,
+--continue Continue as much as possible after an error. While the target
+that failed and anything depending on it cannot be built, as much as
+possible will be built before stopping. -f, --force Force the specified
+targets/task to run (invalidating any existing stamp file). -c CMD,
+--cmd=CMD Specify the task to execute. The exact options available
+depend on the metadata. Some examples might be 'compile' or
+'populate_sysroot' or 'listtasks' may give a list of the tasks
+available. -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
+Invalidate the stamp for the specified task such as 'compile' and then
+run the default task for the specified target(s). -r PREFILE,
+--read=PREFILE Read the specified file before bitbake.conf. -R POSTFILE,
+--postread=POSTFILE Read the specified file after bitbake.conf. -v,
+--verbose Enable tracing of shell tasks (with 'set -x'). Also print
+bb.note(...) messages to stdout (in addition to writing them to
+${T}/log.do_<task>). -D, --debug Increase the debug level. You can
+specify this more than once. -D sets the debug level to 1, where only
+bb.debug(1, ...) messages are printed to stdout; -DD sets the debug
+level to 2, where both bb.debug(1, ...) and bb.debug(2, ...) messages
+are printed; etc. Without -D, no debug messages are printed. Note that
+-D only affects output to stdout. All debug messages are written to
+${T}/log.do_taskname, regardless of the debug level. -q, --quiet Output
+less log message data to the terminal. You can specify this more than
+once. -n, --dry-run Don't execute, just go through the motions. -S
+SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER Dump out the
+signature construction information, with no task execution. The
+SIGNATURE_HANDLER parameter is passed to the handler. Two common values
+are none and printdiff but the handler may define more/less. none means
+only dump the signature, printdiff means compare the dumped signature
+with the cached one. -p, --parse-only Quit after parsing the BB recipes.
+-s, --show-versions Show current and preferred versions of all recipes.
+-e, --environment Show the global or per-recipe environment complete
+with information about where variables were set/changed. -g, --graphviz
+Save dependency tree information for the specified targets in the dot
+syntax. -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
+Assume these dependencies don't exist and are already provided
+(equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more
+appealing -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS Show debug
+logging for the specified logging domains -P, --profile Profile the
+command and save reports. -u UI, --ui=UI The user interface to use
+(knotty, ncurses or taskexp - default knotty). --token=XMLRPCTOKEN
+Specify the connection token to be used when connecting to a remote
+server. --revisions-changed Set the exit code depending on whether
+upstream floating revisions have changed or not. --server-only Run
+bitbake without a UI, only starting a server (cooker) process. -B BIND,
+--bind=BIND The name/address for the bitbake xmlrpc server to bind to.
+-T SERVER_TIMEOUT, --idle-timeout=SERVER_TIMEOUT Set timeout to unload
+bitbake server due to inactivity, set to -1 means no unload, default:
+Environment variable BB_SERVER_TIMEOUT. --no-setscene Do not run any
+setscene tasks. sstate will be ignored and everything needed, built.
+--setscene-only Only run setscene tasks, don't run any real tasks.
+--remote-server=REMOTE_SERVER Connect to the specified server. -m,
+--kill-server Terminate any running bitbake server. --observe-only
+Connect to a server as an observing-only client. --status-only Check the
+status of the remote bitbake server. -w WRITEEVENTLOG,
+--write-log=WRITEEVENTLOG Writes the event log of the build to a bitbake
+event json file. Use '' (empty string) to assign the name automatically.
+--runall=RUNALL Run the specified task for any recipe in the taskgraph
+of the specified target (even if it wouldn't otherwise have run).
+--runonly=RUNONLY Run only the specified task within the taskgraph of
+the specified targets (and any task dependencies those tasks may have).
+
+.. _bitbake-examples:
+
+Examples
+--------
+
+This section presents some examples showing how to use BitBake.
+
+.. _example-executing-a-task-against-a-single-recipe:
+
+Executing a Task Against a Single Recipe
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Executing tasks for a single recipe file is relatively simple. You
+specify the file in question, and BitBake parses it and executes the
+specified task. If you do not specify a task, BitBake executes the
+default task, which is "build”. BitBake obeys inter-task dependencies
+when doing so.
+
+The following command runs the build task, which is the default task, on
+the ``foo_1.0.bb`` recipe file: $ bitbake -b foo_1.0.bb The following
+command runs the clean task on the ``foo.bb`` recipe file: $ bitbake -b
+foo.bb -c clean
+
+.. note::
+
+   The "-b" option explicitly does not handle recipe dependencies. Other
+   than for debugging purposes, it is instead recommended that you use
+   the syntax presented in the next section.
+
+Executing Tasks Against a Set of Recipe Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are a number of additional complexities introduced when one wants
+to manage multiple ``.bb`` files. Clearly there needs to be a way to
+tell BitBake what files are available and, of those, which you want to
+execute. There also needs to be a way for each recipe to express its
+dependencies, both for build-time and runtime. There must be a way for
+you to express recipe preferences when multiple recipes provide the same
+functionality, or when there are multiple versions of a recipe.
+
+The ``bitbake`` command, when not using "--buildfile" or "-b" only
+accepts a "PROVIDES". You cannot provide anything else. By default, a
+recipe file generally "PROVIDES" its "packagename" as shown in the
+following example: $ bitbake foo This next example "PROVIDES" the
+package name and also uses the "-c" option to tell BitBake to just
+execute the ``do_clean`` task: $ bitbake -c clean foo
+
+Executing a List of Task and Recipe Combinations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The BitBake command line supports specifying different tasks for
+individual targets when you specify multiple targets. For example,
+suppose you had two targets (or recipes) ``myfirstrecipe`` and
+``mysecondrecipe`` and you needed BitBake to run ``taskA`` for the first
+recipe and ``taskB`` for the second recipe: $ bitbake
+myfirstrecipe:do_taskA mysecondrecipe:do_taskB
+
+Generating Dependency Graphs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+BitBake is able to generate dependency graphs using the ``dot`` syntax.
+You can convert these graphs into images using the ``dot`` tool from
+`Graphviz <http://www.graphviz.org>`__.
+
+When you generate a dependency graph, BitBake writes two files to the
+current working directory:
+
+-  *``task-depends.dot``:* Shows dependencies between tasks. These
+   dependencies match BitBake's internal task execution list.
+
+-  *``pn-buildlist``:* Shows a simple list of targets that are to be
+   built.
+
+To stop depending on common depends, use the "-I" depend option and
+BitBake omits them from the graph. Leaving this information out can
+produce more readable graphs. This way, you can remove from the graph
+``DEPENDS`` from inherited classes such as ``base.bbclass``.
+
+Here are two examples that create dependency graphs. The second example
+omits depends common in OpenEmbedded from the graph: $ bitbake -g foo $
+bitbake -g -I virtual/kernel -I eglibc foo
+
+Executing a Multiple Configuration Build
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+BitBake is able to build multiple images or packages using a single
+command where the different targets require different configurations
+(multiple configuration builds). Each target, in this scenario, is
+referred to as a "multiconfig".
+
+To accomplish a multiple configuration build, you must define each
+target's configuration separately using a parallel configuration file in
+the build directory. The location for these multiconfig configuration
+files is specific. They must reside in the current build directory in a
+sub-directory of ``conf`` named ``multiconfig``. Following is an example
+for two separate targets:
+
+The reason for this required file hierarchy is because the ``BBPATH``
+variable is not constructed until the layers are parsed. Consequently,
+using the configuration file as a pre-configuration file is not possible
+unless it is located in the current working directory.
+
+Minimally, each configuration file must define the machine and the
+temporary directory BitBake uses for the build. Suggested practice
+dictates that you do not overlap the temporary directories used during
+the builds.
+
+Aside from separate configuration files for each target, you must also
+enable BitBake to perform multiple configuration builds. Enabling is
+accomplished by setting the
+```BBMULTICONFIG`` <#var-bb-BBMULTICONFIG>`__ variable in the
+``local.conf`` configuration file. As an example, suppose you had
+configuration files for ``target1`` and ``target2`` defined in the build
+directory. The following statement in the ``local.conf`` file both
+enables BitBake to perform multiple configuration builds and specifies
+the two extra multiconfigs: BBMULTICONFIG = "target1 target2"
+
+Once the target configuration files are in place and BitBake has been
+enabled to perform multiple configuration builds, use the following
+command form to start the builds: $ bitbake [mc:multiconfigname:]target
+[[[mc:multiconfigname:]target] ... ] Here is an example for two extra
+multiconfigs: ``target1`` and ``target2``: $ bitbake mc::target
+mc:target1:target mc:target2:target
+
+.. _bb-enabling-multiple-configuration-build-dependencies:
+
+Enabling Multiple Configuration Build Dependencies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sometimes dependencies can exist between targets (multiconfigs) in a
+multiple configuration build. For example, suppose that in order to
+build an image for a particular architecture, the root filesystem of
+another build for a different architecture needs to exist. In other
+words, the image for the first multiconfig depends on the root
+filesystem of the second multiconfig. This dependency is essentially
+that the task in the recipe that builds one multiconfig is dependent on
+the completion of the task in the recipe that builds another
+multiconfig.
+
+To enable dependencies in a multiple configuration build, you must
+declare the dependencies in the recipe using the following statement
+form: task_or_package[mcdepends] =
+"mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
+To better show how to use this statement, consider an example with two
+multiconfigs: ``target1`` and ``target2``: image_task[mcdepends] =
+"mc:target1:target2:image2:rootfs_task" In this example, the
+from_multiconfig is "target1" and the to_multiconfig is "target2". The
+task on which the image whose recipe contains image_task depends on the
+completion of the rootfs_task used to build out image2, which is
+associated with the "target2" multiconfig.
+
+Once you set up this dependency, you can build the "target1" multiconfig
+using a BitBake command as follows: $ bitbake mc:target1:image1 This
+command executes all the tasks needed to create image1 for the "target1"
+multiconfig. Because of the dependency, BitBake also executes through
+the rootfs_task for the "target2" multiconfig build.
+
+Having a recipe depend on the root filesystem of another build might not
+seem that useful. Consider this change to the statement in the image1
+recipe: image_task[mcdepends] = "mc:target1:target2:image2:image_task"
+In this case, BitBake must create image2 for the "target2" build since
+the "target1" build depends on it.
+
+Because "target1" and "target2" are enabled for multiple configuration
+builds and have separate configuration files, BitBake places the
+artifacts for each build in the respective temporary build directories.