Tasks can either be a shell task or a Python task. For shell tasks, BitBake writes a shell script to ${WORKDIR}/temp/run.do_taskname.pid and then executes the script. The generated shell script contains all the exported variables, and the shell functions with all variables expanded. Output from the shell script goes to the file ${WORKDIR}/temp/log.do_taskname.pid. Looking at the expanded shell functions in the run file and the output in the log files is a useful debugging technique. For Python tasks, BitBake executes the task internally and logs information to the controlling terminal. Future versions of BitBake will write the functions to files similar to the way shell tasks are handled. Logging will be handled in a way similar to shell tasks as well. Once all the tasks have been completed BitBake exits.

BitBake parses configuration files, classes, recipes, and append files. The first thing BitBake does is look for the bitbake.conf file. This file resides in the conf directory, which must be listed in BBPATH. The bitbake.conf file lists other configuration files to include from the conf directory below the directories listed in BBPATH. In general, the most important of these configuration files from a user's perspective is local.conf, which contains the user's customized settings for the build environment. Other notable configuration files are the distribution configuration file and the machine configuration file. These configuration files are normally identified by variables unique to the build systems using BitBake. For example, the Yocto Project uses the DISTRO and MACHINE variables, respectively. After parsing of the configuration files, some standard classes are included. The base.bbclass file is always included. Other classes that are specified in the configuration using the INHERIT variable are also included. BitBake searches for class files in a "classes" subdirectory under the paths in BBPATH in the same way as configuration files. After classes are included, the variable BBFILES is set, usually in local.conf, and defines the list of places to search for recipe and append files. Adding extra content to BBFILES is best achieved through the use of BitBake layers. BitBake parses each recipe and append file located with BBFILES and stores the values of various variables into the datastore. In summary, for each recipe and append file pairing, the configuration plus the base class of variables are set, followed by the data in the recipe file itself, followed by any inherit commands that the recipe file might contain. Because parsing recipe and append files is a time consuming process, a cache, referred to as the "setscene" is kept to speed up subsequent parsing. The setscene is invalid if the timestamps of a recipe changes, any of the include files change, configuration files change, or class files on which the recipe file depends change.

Prior to parsing configuration files, Bitbake looks at certain variables, including: BB_ENV_WHITELIST BB_PRESERVE_ENV BB_ENV_EXTRAWHITE BB_ORIGENV PREFERRED_VERSION PREFERRED_PROVIDERS The first kind of metadata in BitBake is configuration metadata. This metadata is global, and therefore affects all packages and tasks that are executed. BitBake will first search the current working directory for an optional conf/bblayers.conf configuration file. This file is expected to contain a BBLAYERS variable that is a space delimited list of 'layer' directories. For each directory in this list, a conf/layer.conf file will be searched for and parsed with the LAYERDIR variable being set to the directory where the layer was found. The idea is these files will setup BBPATH and other variables correctly for a given build directory automatically for the user. BitBake will then expect to find conf/bitbake.conf file somewhere in the user specified BBPATH. That configuration file generally has include directives to pull in any other metadata (generally files specific to architecture, machine, local and so on). Only variable definitions and include directives are allowed in .conf files. The following variables include: BITBAKE_UI BBDEBUG MULTI_PROVIDER_WHITELIST BB_NUMBER_PARSE_THREADS BBPKGS BB_DEFAULT_TASK TOPDIR BB_VERBOSE_LOGS BB_NICE_LEVEL BBFILE_COLLECTIONS ASSUME_PROVIDED BB_DANGLINGAPPENDS_WARNONLY BBINCLUDED BBFILE_PRIORITY BUILDNAME BBMASK

Layers allow you to isolate different types of customizations from each other. While you might find it tempting to keep everything in one layer when working on a single project, the more modular you organize your metadata, the easier it is to cope with future changes. To illustrate how you can use layers to keep things modular, consider machine customizations. These types of customizations typically reside in a special layer, rather than a general layer, called a Board Specific Package (BSP) Layer. Furthermore, the machine customizations should be isolated from recipes and metadata that support a new GUI environment, for example. This situation gives you a couple of layers: one for the machine configurations and one for the GUI environment. It is important to understand, however, that the BSP layer can still make machine-specific additions to recipes within the GUI environment layer without polluting the GUI layer itself with those machine-specific changes. You can accomplish this through a recipe that is a BitBake append (.bbappend) file. There are certain variables specific to layers: LAYERDEPENDS LAYERVERSION

Variables specific to scheduling functionality exist: BB_SCHEDULER BB_SCHEDULERS

BitBake's rudimentary inheritance mechanism is accomplished using classes. As briefly mentioned in the metadata introduction, BitBake parses a class when an inherit directive is encountered, and it is located in the classes directory relative to the directories in BBPATH.

Recipe files, which are files that have the .bb file extension, are logical units of tasks for execution. Normally, that logical unit is a package that needs to be built. BitBake obeys all inter-recipe dependencies. Recipe files must reside in locations found in the BBFILES variable.

Append files, which are files that have the .bbappend file extension, add or extend build information to an existing recipe file. BitBake expects every append file to have a corresponding recipe file. Furthermore, the append file and corresponding recipe file must use the same root filename. The filenames can differ only in the file type suffix used (e.g. formfactor_0.0.bb and formfactor_0.0.bbappend). Information in append files overrides the information in the similarly-named recipe file.

The OpenEmbedded build system uses BitBake to produce images. BitBake consists of several functional areas. This section takes a closer look at each of those areas.

The first stages of building a recipe are to fetch and unpack the source code: figures/ The do_fetch and do_unpack tasks fetch the source files and unpack them into the work directory. By default, everything is accomplished in the build directory, which has a defined structure. Unpacked source files are pointed to by a variable. For example, in the Yocto Project and OpenEmbedded build systems, the S variable points to these source files. Each recipe has an area in the Build Directory where the unpacked source code resides. The name of that directory for any given recipe is defined from several different variables. You can see the variables that define these directories by looking at the figure that shows the structure and variables used in the Yocto Project: TMPDIR PACKAGE_ARCH TARGET_OS PN PV PR WORKDIR S Briefly, the S directory contains the unpacked source files for a recipe. The WORKDIR directory is where all the building goes on for a given recipe.

Once source code is fetched and unpacked, BitBake locates patch files and applies them to the source files: The do_patch task processes recipes by using the SRC_URI variable to locate applicable patch files, which by default are *.patch or *.diff files, or any file if "apply=yes" is specified for the file in SRC_URI. BitBake finds and applies multiple patches for a single recipe in the order in which it finds the patches. Patches are applied to the recipe's source files located in the S directory. For more information on how the source directories are created, see the "Source Fetching" section.

After source code is patched, BitBake executes tasks that configure and compile the source code: This step in the build process consists of three tasks: do_configure: This task configures the source by enabling and disabling any build-time and configuration options for the software being built. Configurations can come from the recipe itself as well as from an inherited class. Additionally, the software itself might configure itself depending on the target for which it is being built. The configurations handled by the do_configure task are specific to source code configuration for the source code being built by the recipe. If you are using the Autotools class (autotools.bbclass), you can add additional configuration options by using the EXTRA_OECONF variable. For information on how this variable works within that class, see the meta/classes/autotools.bbclass file. do_compile: Once a configuration task has been satisfied, BitBake compiles the source using the do_compile task. Compilation occurs in the directory pointed to by the B variable. Realize that the B directory is, by default, the same as the S directory. do_install: Once compilation is done, BitBake executes the do_install task. This task copies files from the B directory and places them in a holding area pointed to by the D variable.

After source code is configured and compiled, the OpenEmbedded build system analyzes the results and splits the output into packages: The do_package and do_packagedata tasks combine to analyze the files found in the D directory and split them into subsets based on available packages and files. The analyzing process involves the following as well as other items: splitting out debugging symbols, looking at shared library dependencies between packages, and looking at package relationships. The do_packagedata task creates package metadata based on the analysis such that the OpenEmbedded build system can generate the final packages. Working, staged, and intermediate results of the analysis and package splitting process use these areas: PKGD PKGDATA_DIR PKGDESTWORK PKGDEST The FILES variable defines the files that go into each package in PACKAGES. If you want details on how this is accomplished in the Yocto Project for example, you can look at the package.bbclass file in a Yocto tree. Depending on the type of packages being created (RPM, DEB, or IPK), the do_package_write_* task creates the actual packages and places them in the Package Feed area, which is ${TMPDIR}/deploy. Support for creating feeds directly from the deploy/* directories does not exist. Creating such feeds usually requires some kind of feed maintenance mechanism that would upload the new packages into an official package feed (e.g. the Ångström distribution). This functionality is highly distribution-specific and thus is not provided out of the box.

Once packages are split and stored in the Package Feeds area, the OpenEmbedded build system uses BitBake to generate the root filesystem image: The image generation process consists of several stages and depends on many variables. The do_rootfs task uses these key variables to help create the list of packages to actually install: IMAGE_INSTALL: Lists out the base set of packages to install from the Package Feeds area. PACKAGE_EXCLUDE: Specifies packages that should not be installed. IMAGE_FEATURES: Specifies features to include in the image. Most of these features map to additional packages for installation. PACKAGE_CLASSES: Specifies the package backend to use and consequently helps determine where to locate packages within the Package Feeds area. IMAGE_LINGUAS: Determines the language(s) for which additional language support packages are installed. Package installation is under control of the package manager (e.g. smart/rpm, opkg, or apt/dpkg) regardless of whether or not package management is enabled for the target. At the end of the process, if package management is not enabled for the target, the package manager's data files are deleted from the root filesystem. During image generation, the build system attempts to run all post-installation scripts. Any that fail to run on the build host are run on the target when the target system is first booted. If you are using a read-only root filesystem, all the post installation scripts must succeed during the package installation phase since the root filesystem cannot be written into. During Optimization, optimizing processes are run across the image. These processes include mklibs and prelink. The mklibs process optimizes the size of the libraries. A prelink process optimizes the dynamic linking of shared libraries to reduce start up time of executables. Part of the image generation process includes compressing the root filesystem image. Compression is accomplished through several optimization routines designed to reduce the overall size of the image. After the root filesystem has been constructed, the image generation process turns everything into an image file or a set of image files. The formats used for the root filesystem depend on the IMAGE_FSTYPES variable. The entire image generation process is run under Pseudo. Running under Pseudo ensures that the files in the root filesystem have correct ownership.

The OpenEmbedded build system uses BitBake to generate the Software Development Kit (SDK) installer script: Like image generation, the SDK script process consists of several stages and depends on many variables. The do_populate_sdk task uses these key variables to help create the list of packages to actually install. The do_populate_sdk task handles two parts: a target part and a host part. The target part is the part built for the target hardware and includes libraries and headers. The host part is the part of the SDK that runs on the SDKMACHINE. Once both parts are constructed, the do_populate_sdk task performs some cleanup on both parts. After the cleanup, the task creates a cross-development environment setup script and any configuration files that might be needed. The final output of the task is the Cross-development toolchain installation script (.sh file), which includes the environment setup script.