From 35734315dde80207aaaee3cf5081dcfbc2b4bf2c Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Thu, 5 Apr 2018 09:55:23 -0700 Subject: ref-manual: Deleted the "ref-bitbake.xml" chapter This information was merged into the BitBake User Manual. (From yocto-docs rev: eb68d4429aed652e4ca10c1ab55d3a815d453d6f) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/ref-manual/ref-bitbake.xml | 479 ------------------------------- 1 file changed, 479 deletions(-) delete mode 100644 documentation/ref-manual/ref-bitbake.xml (limited to 'documentation') diff --git a/documentation/ref-manual/ref-bitbake.xml b/documentation/ref-manual/ref-bitbake.xml deleted file mode 100644 index 9e3e9cf35d..0000000000 --- a/documentation/ref-manual/ref-bitbake.xml +++ /dev/null @@ -1,479 +0,0 @@ - %poky; ] > - - - - BitBake - - - BitBake is a program written in Python that interprets the - Metadata used by - the OpenEmbedded build system. - At some point, developers wonder what actually happens when you enter: - - $ bitbake core-image-sato - - - - - This chapter provides an overview of what happens behind the scenes from BitBake's perspective. - - - - BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships. - As such, it has no real knowledge of what the tasks being executed actually do. - BitBake just considers a list of tasks with dependencies and handles - Metadata - consisting of variables in a certain format that get passed to the tasks. - - -
- Parsing - - - BitBake parses configuration files, classes, and .bb files. - - - - The first thing BitBake does is look for the - bitbake.conf file. - This file resides in the - Source Directory - within the meta/conf/ directory. - BitBake finds it by examining its - BBPATH environment - variable and looking for the meta/conf/ - directory. - - - - The bitbake.conf file lists other configuration - files to include from a conf/ - directory below the directories listed in BBPATH. - In general, the most important configuration file from a user's perspective - is local.conf, which contains a user's customized - settings for the OpenEmbedded build environment. - Other notable configuration files are the distribution - configuration file (set by the - DISTRO variable) - and the machine configuration file - (set by the - MACHINE variable). - The DISTRO and MACHINE BitBake environment - variables are both usually set in - the local.conf file. - Valid distribution - configuration files are available in the meta/conf/distro/ directory - and valid machine configuration - files in the meta/conf/machine/ directory. - Within the meta/conf/machine/include/ - directory are various tune-*.inc configuration files that provide common - "tuning" settings specific to and shared between particular architectures and machines. - - - - After the parsing of the configuration files, some standard classes are included. - The base.bbclass file is always included. - Other classes that are specified in the configuration using the - INHERIT - variable are also included. - Class files are searched for in a classes subdirectory - under the paths in BBPATH in the same way as - configuration files. - - - - After classes are included, the variable - BBFILES - is set, usually in - local.conf, and defines the list of places to search for - .bb files. - By default, the BBFILES variable specifies the - meta/recipes-*/ directory within Poky. - Adding extra content to BBFILES is best achieved through the use of - BitBake layers as described in the - "Understanding and Creating Layers" - section of the Yocto Project Development Tasks Manual. - - - - BitBake parses each .bb file in BBFILES and - stores the values of various variables. - In summary, for each .bb - file the configuration plus the base class of variables are set, followed - by the data in the .bb file - itself, followed by any inherit commands that - .bb file might contain. - - - - Because parsing .bb files is a time - consuming process, a cache is kept to speed up subsequent parsing. - This cache is invalid if the timestamp of the .bb - file itself changes, or if the timestamps of any of the include, - configuration files or class files on which the - .bb file depends change. - - - - - You need to be aware of how BitBake parses curly braces. - If a recipe uses a closing curly brace within the function and - the character has no leading spaces, BitBake produces a parsing - error. - If you use a pair of curly brace in a shell function, the - closing curly brace must not be located at the start of the line - without leading spaces. - - - - Here is an example that causes BitBake to produce a parsing - error: - - fakeroot create_shar() { - cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh - usage() - { - echo "test" - ###### The following "}" at the start of the line causes a parsing error ###### - } - EOF - } - - Writing the recipe this way avoids the error: - - fakeroot create_shar() { - cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh - usage() - { - echo "test" - ######The following "}" with a leading space at the start of the line avoids the error ###### - } - EOF - } - - - -
- -
- Preferences and Providers - - - Once all the .bb files have been - parsed, BitBake starts to build the target (core-image-sato - in the previous section's example) and looks for providers of that target. - Once a provider is selected, BitBake resolves all the dependencies for - the target. - In the case of core-image-sato, it would lead to - packagegroup-core-x11-sato, - which in turn leads to recipes like matchbox-terminal, - pcmanfm and gthumb. - These recipes in turn depend on glibc and the toolchain. - - - - Sometimes a target might have multiple providers. - A common example is "virtual/kernel", which is provided by each kernel package. - Each machine often selects the best kernel provider by using a line similar to the - following in the machine configuration file: - - - - PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" - - - - The default PREFERRED_PROVIDER - is the provider with the same name as the target. - - - - Understanding how providers are chosen is made complicated by the fact - that multiple versions might exist. - BitBake defaults to the highest version of a provider. - Version comparisons are made using the same method as Debian. - You can use the - PREFERRED_VERSION - variable to specify a particular version (usually in the distro configuration). - You can influence the order by using the - DEFAULT_PREFERENCE - variable. - By default, files have a preference of "0". - Setting the DEFAULT_PREFERENCE to "-1" makes the - package unlikely to be used unless it is explicitly referenced. - Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used. - PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting. - DEFAULT_PREFERENCE is often used to mark newer and more experimental package - versions until they have undergone sufficient testing to be considered stable. - - - - In summary, BitBake has created a list of providers, which is prioritized, for each target. - -
- -
- Dependencies - - - Each target BitBake builds consists of multiple tasks such as - fetch, unpack, - patch, configure, - and compile. - For best performance on multi-core systems, BitBake considers each task as an independent - entity with its own set of dependencies. - - - - Dependencies are defined through several variables. - You can find information about variables BitBake uses in the - BitBake documentation, which is found in the - bitbake/doc/manual directory within the - Source Directory. - At a basic level, it is sufficient to know that BitBake uses the - DEPENDS and - RDEPENDS - variables when calculating dependencies. - -
- -
- The Task List - - - Based on the generated list of providers and the dependency information, - BitBake can now calculate exactly what tasks it needs to run and in what - order it needs to run them. - The build now starts with BitBake forking off threads up to the limit set in the - BB_NUMBER_THREADS variable. - BitBake continues to fork threads as long as there are tasks ready to run, - those tasks have all their dependencies met, and the thread threshold has not been - exceeded. - - - - It is worth noting that you can greatly speed up the build time by properly setting - the BB_NUMBER_THREADS variable. - See the - "Building Images" - section in the Yocto Project Quick Start for more information. - - - - As each task completes, a timestamp is written to the directory specified by the - STAMP variable. - On subsequent runs, BitBake looks within the build/tmp/stamps - directory and does not rerun - tasks that are already completed unless a timestamp is found to be invalid. - Currently, invalid timestamps are only considered on a per - .bb file basis. - So, for example, if the configure stamp has a timestamp greater than the - compile timestamp for a given target, then the compile task would rerun. - Running the compile task again, however, has no effect on other providers - that depend on that target. - This behavior could change or become configurable in future versions of BitBake. - - - - Some tasks are marked as "nostamp" tasks. - No timestamp file is created when these tasks are run. - Consequently, "nostamp" tasks are always rerun. - -
- -
- Running a Task - - - Tasks can either be a shell task or a Python task. - For shell tasks, BitBake writes a shell script to - ${WORKDIR}/temp/run.do_taskname.pid and then executes the script. - The generated shell script contains all the exported variables, and the shell functions - with all variables expanded. - Output from the shell script goes to the file ${WORKDIR}/temp/log.do_taskname.pid. - Looking at the expanded shell functions in the run file and the output in the log files - is a useful debugging technique. - - - - For Python tasks, BitBake executes the task internally and logs information to the - controlling terminal. - Future versions of BitBake will write the functions to files similar to the way - shell tasks are handled. - Logging will be handled in a way similar to shell tasks as well. - - - - Once all the tasks have been completed BitBake exits. - - - - When running a task, BitBake tightly controls the execution environment - of the build tasks to make sure unwanted contamination from the build machine - cannot influence the build. - Consequently, if you do want something to get passed into the build - task's environment, you must take a few steps: - - Tell BitBake to load what you want from the environment - into the data store. - You can do so through the BB_ENV_EXTRAWHITE - variable. - For example, assume you want to prevent the build system from - accessing your $HOME/.ccache directory. - The following command tells BitBake to load - CCACHE_DIR from the environment into the data - store: - - export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR" - - Tell BitBake to export what you have loaded into the - environment store to the task environment of every running task. - Loading something from the environment into the data store - (previous step) only makes it available in the datastore. - To export it to the task environment of every running task, - use a command similar to the following in your - local.conf or distro configuration file: - - export CCACHE_DIR - - - - - - A side effect of the previous steps is that BitBake records the variable - as a dependency of the build process in things like the shared state - checksums. - If doing so results in unnecessary rebuilds of tasks, you can whitelist the - variable so that the shared state code ignores the dependency when it creates - checksums. - For information on this process, see the - BB_HASHBASE_WHITELIST example in the - "Checksums (Signatures)" - section in the Yocto Project Concepts Manual. - -
- -
- BitBake Command Line - - - Following is the BitBake help output: - - - -$ bitbake --help -Usage: bitbake [options] [recipename/target ...] - - Executes the specified task (default is 'build') for a given set of target recipes (.bb files). - It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which - will provide the layer, BBFILES and other configuration information. - -Options: - --version show program's version number and exit - -h, --help show this help message and exit - -b BUILDFILE, --buildfile=BUILDFILE - Execute tasks from a specific .bb recipe directly. - WARNING: Does not handle any dependencies from other - recipes. - -k, --continue Continue as much as possible after an error. While the - target that failed and anything depending on it cannot - be built, as much as possible will be built before - stopping. - -a, --tryaltconfigs Continue with builds by trying to use alternative - providers where possible. - -f, --force Force the specified targets/task to run (invalidating - any existing stamp file). - -c CMD, --cmd=CMD Specify the task to execute. The exact options - available depend on the metadata. Some examples might - be 'compile' or 'populate_sysroot' or 'listtasks' may - give a list of the tasks available. - -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP - Invalidate the stamp for the specified task such as - 'compile' and then run the default task for the - specified target(s). - -r PREFILE, --read=PREFILE - Read the specified file before bitbake.conf. - -R POSTFILE, --postread=POSTFILE - Read the specified file after bitbake.conf. - -v, --verbose Output more log message data to the terminal. - -D, --debug Increase the debug level. You can specify this more - than once. - -n, --dry-run Don't execute, just go through the motions. - -S, --dump-signatures - Don't execute, just dump out the signature - construction information. - -p, --parse-only Quit after parsing the BB recipes. - -s, --show-versions Show current and preferred versions of all recipes. - -e, --environment Show the global or per-package environment complete - with information about where variables were - set/changed. - -g, --graphviz Save dependency tree information for the specified - targets in the dot syntax. - -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED - Assume these dependencies don't exist and are already - provided (equivalent to ASSUME_PROVIDED). Useful to - make dependency graphs more appealing - -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS - Show debug logging for the specified logging domains - -P, --profile Profile the command and save reports. - -u UI, --ui=UI The user interface to use (e.g. knotty and taskexp). - -t SERVERTYPE, --servertype=SERVERTYPE - Choose which server to use, process or xmlrpc. - --revisions-changed Set the exit code depending on whether upstream - floating revisions have changed or not. - --server-only Run bitbake without a UI, only starting a server - (cooker) process. - -B BIND, --bind=BIND The name/address for the bitbake server to bind to. - --no-setscene Do not run any setscene tasks. sstate will be ignored - and everything needed, built. - --remote-server=REMOTE_SERVER - Connect to the specified server. - -m, --kill-server Terminate the remote server. - --observe-only Connect to a server as an observing-only client. - -
- -
- Fetchers - - - BitBake also contains a set of "fetcher" modules that allow - retrieval of source code from various types of sources. - For example, BitBake can get source code from a disk with the metadata, from websites, - from remote shell accounts, or from Source Code Management (SCM) systems - like cvs/subversion/git. - - - - Fetchers are usually triggered by entries in - SRC_URI. - You can find information about the options and formats of entries for specific - fetchers in the BitBake manual located in the - bitbake/doc/manual directory of the - Source Directory. - - - - One useful feature for certain Source Code Manager (SCM) fetchers - is the ability to "auto-update" when the upstream SCM changes - version. - Since this ability requires certain functionality from the SCM, - not all systems support it. - Currently Subversion, Bazaar and to a limited extent, Git support - the ability to "auto-update". - This feature works using the SRCREV - variable. - See the - "Using an External SCM" - section in the Yocto Project Development Tasks Manual for more - information. - - -
- - - -- cgit v1.2.3-54-g00ecf