From 972dcfcdbfe75dcfeb777150c136576cf1a71e99 Mon Sep 17 00:00:00 2001 From: Tudor Florea Date: Fri, 9 Oct 2015 22:59:03 +0200 Subject: initial commit for Enea Linux 5.0 arm Signed-off-by: Tudor Florea --- documentation/ref-manual/ref-bitbake.xml | 472 +++++++++++++++++++++++++++++++ 1 file changed, 472 insertions(+) create mode 100644 documentation/ref-manual/ref-bitbake.xml (limited to 'documentation/ref-manual/ref-bitbake.xml') diff --git a/documentation/ref-manual/ref-bitbake.xml b/documentation/ref-manual/ref-bitbake.xml new file mode 100644 index 0000000000..30aff6431f --- /dev/null +++ b/documentation/ref-manual/ref-bitbake.xml @@ -0,0 +1,472 @@ + %poky; ] > + + + + BitBake + + + BitBake is a program written in Python that interprets the + Metadata used by + the OpenEmbedded build system. + At some point, developers wonder what actually happens when you enter: + + $ bitbake core-image-sato + + + + + This chapter provides an overview of what happens behind the scenes from BitBake's perspective. + + + + BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships. + As such, it has no real knowledge of what the tasks being executed actually do. + BitBake just considers a list of tasks with dependencies and handles + Metadata + consisting of variables in a certain format that get passed to the tasks. + + +
+ Parsing + + + BitBake parses configuration files, classes, and .bb files. + + + + The first thing BitBake does is look for the bitbake.conf file. + This file resides in the + Source Directory + within the meta/conf/ directory. + BitBake finds it by examining its + BBPATH environment + variable and looking for the meta/conf/ + directory. + + + + The bitbake.conf file lists other configuration + files to include from a conf/ + directory below the directories listed in BBPATH. + In general, the most important configuration file from a user's perspective + is local.conf, which contains a user's customized + settings for the OpenEmbedded build environment. + Other notable configuration files are the distribution + configuration file (set by the + DISTRO variable) + and the machine configuration file + (set by the + MACHINE variable). + The DISTRO and MACHINE BitBake environment + variables are both usually set in + the local.conf file. + Valid distribution + configuration files are available in the meta/conf/distro/ directory + and valid machine configuration + files in the meta/conf/machine/ directory. + Within the meta/conf/machine/include/ + directory are various tune-*.inc configuration files that provide common + "tuning" settings specific to and shared between particular architectures and machines. + + + + After the parsing of the configuration files, some standard classes are included. + The base.bbclass file is always included. + Other classes that are specified in the configuration using the + INHERIT + variable are also included. + Class files are searched for in a classes subdirectory + under the paths in BBPATH in the same way as + configuration files. + + + + After classes are included, the variable + BBFILES + is set, usually in + local.conf, and defines the list of places to search for + .bb files. + By default, the BBFILES variable specifies the + meta/recipes-*/ directory within Poky. + Adding extra content to BBFILES is best achieved through the use of + BitBake layers as described in the + "Understanding and + Creating Layers" section of the Yocto Project Development Manual. + + + + BitBake parses each .bb file in BBFILES and + stores the values of various variables. + In summary, for each .bb + file the configuration plus the base class of variables are set, followed + by the data in the .bb file + itself, followed by any inherit commands that + .bb file might contain. + + + + Because parsing .bb files is a time + consuming process, a cache is kept to speed up subsequent parsing. + This cache is invalid if the timestamp of the .bb + file itself changes, or if the timestamps of any of the include, + configuration files or class files on which the + .bb file depends change. + + + + + You need to be aware of how BitBake parses curly braces. + If a recipe uses a closing curly brace within the function and + the character has no leading spaces, BitBake produces a parsing + error. + If you use a pair of curly brace in a shell function, the + closing curly brace must not be located at the start of the line + without leading spaces. + + + + Here is an example that causes BitBake to produce a parsing + error: + + fakeroot create_shar() { + cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh + usage() + { + echo "test" + ###### The following "}" at the start of the line causes a parsing error ###### + } + EOF + } + + Writing the recipe this way avoids the error: + + fakeroot create_shar() { + cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh + usage() + { + echo "test" + ######The following "}" with a leading space at the start of the line avoids the error ###### + } + EOF + } + + + +
+ +
+ Preferences and Providers + + + Once all the .bb files have been + parsed, BitBake starts to build the target (core-image-sato + in the previous section's example) and looks for providers of that target. + Once a provider is selected, BitBake resolves all the dependencies for + the target. + In the case of core-image-sato, it would lead to + packagegroup-core-x11-sato, + which in turn leads to recipes like matchbox-terminal, + pcmanfm and gthumb. + These recipes in turn depend on glibc and the toolchain. + + + + Sometimes a target might have multiple providers. + A common example is "virtual/kernel", which is provided by each kernel package. + Each machine often selects the best kernel provider by using a line similar to the + following in the machine configuration file: + + + + PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" + + + + The default PREFERRED_PROVIDER + is the provider with the same name as the target. + + + + Understanding how providers are chosen is made complicated by the fact + that multiple versions might exist. + BitBake defaults to the highest version of a provider. + Version comparisons are made using the same method as Debian. + You can use the + PREFERRED_VERSION + variable to specify a particular version (usually in the distro configuration). + You can influence the order by using the + DEFAULT_PREFERENCE + variable. + By default, files have a preference of "0". + Setting the DEFAULT_PREFERENCE to "-1" makes the + package unlikely to be used unless it is explicitly referenced. + Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used. + PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting. + DEFAULT_PREFERENCE is often used to mark newer and more experimental package + versions until they have undergone sufficient testing to be considered stable. + + + + In summary, BitBake has created a list of providers, which is prioritized, for each target. + +
+ +
+ Dependencies + + + Each target BitBake builds consists of multiple tasks such as + fetch, unpack, + patch, configure, + and compile. + For best performance on multi-core systems, BitBake considers each task as an independent + entity with its own set of dependencies. + + + + Dependencies are defined through several variables. + You can find information about variables BitBake uses in the BitBake documentation, + which is found in the bitbake/doc/manual directory within the + Source Directory. + At a basic level, it is sufficient to know that BitBake uses the + DEPENDS and + RDEPENDS variables when + calculating dependencies. + +
+ +
+ The Task List + + + Based on the generated list of providers and the dependency information, + BitBake can now calculate exactly what tasks it needs to run and in what + order it needs to run them. + The build now starts with BitBake forking off threads up to the limit set in the + BB_NUMBER_THREADS variable. + BitBake continues to fork threads as long as there are tasks ready to run, + those tasks have all their dependencies met, and the thread threshold has not been + exceeded. + + + + It is worth noting that you can greatly speed up the build time by properly setting + the BB_NUMBER_THREADS variable. + See the + "Building an Image" + section in the Yocto Project Quick Start for more information. + + + + As each task completes, a timestamp is written to the directory specified by the + STAMP variable. + On subsequent runs, BitBake looks within the build/tmp/stamps + directory and does not rerun + tasks that are already completed unless a timestamp is found to be invalid. + Currently, invalid timestamps are only considered on a per + .bb file basis. + So, for example, if the configure stamp has a timestamp greater than the + compile timestamp for a given target, then the compile task would rerun. + Running the compile task again, however, has no effect on other providers + that depend on that target. + This behavior could change or become configurable in future versions of BitBake. + + + + Some tasks are marked as "nostamp" tasks. + No timestamp file is created when these tasks are run. + Consequently, "nostamp" tasks are always rerun. + +
+ +
+ Running a Task + + + Tasks can either be a shell task or a Python task. + For shell tasks, BitBake writes a shell script to + ${WORKDIR}/temp/run.do_taskname.pid and then executes the script. + The generated shell script contains all the exported variables, and the shell functions + with all variables expanded. + Output from the shell script goes to the file ${WORKDIR}/temp/log.do_taskname.pid. + Looking at the expanded shell functions in the run file and the output in the log files + is a useful debugging technique. + + + + For Python tasks, BitBake executes the task internally and logs information to the + controlling terminal. + Future versions of BitBake will write the functions to files similar to the way + shell tasks are handled. + Logging will be handled in a way similar to shell tasks as well. + + + + Once all the tasks have been completed BitBake exits. + + + + When running a task, BitBake tightly controls the execution environment + of the build tasks to make sure unwanted contamination from the build machine + cannot influence the build. + Consequently, if you do want something to get passed into the build + task's environment, you must take a few steps: + + Tell BitBake to load what you want from the environment + into the data store. + You can do so through the BB_ENV_EXTRAWHITE + variable. + For example, assume you want to prevent the build system from + accessing your $HOME/.ccache directory. + The following command tells BitBake to load + CCACHE_DIR from the environment into the data + store: + + export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR" + + Tell BitBake to export what you have loaded into the + environment store to the task environment of every running task. + Loading something from the environment into the data store + (previous step) only makes it available in the datastore. + To export it to the task environment of every running task, + use a command similar to the following in your + local.conf or distro configuration file: + + export CCACHE_DIR + + + + + + A side effect of the previous steps is that BitBake records the variable + as a dependency of the build process in things like the shared state + checksums. + If doing so results in unnecessary rebuilds of tasks, you can whitelist the + variable so that the shared state code ignores the dependency when it creates + checksums. + For information on this process, see the BB_HASHBASE_WHITELIST + example in the "Checksums (Signatures)" section. + +
+ +
+ BitBake Command Line + + + Following is the BitBake help output: + + + +$ bitbake --help +Usage: bitbake [options] [recipename/target ...] + + Executes the specified task (default is 'build') for a given set of target recipes (.bb files). + It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which + will provide the layer, BBFILES and other configuration information. + +Options: + --version show program's version number and exit + -h, --help show this help message and exit + -b BUILDFILE, --buildfile=BUILDFILE + Execute tasks from a specific .bb recipe directly. + WARNING: Does not handle any dependencies from other + recipes. + -k, --continue Continue as much as possible after an error. While the + target that failed and anything depending on it cannot + be built, as much as possible will be built before + stopping. + -a, --tryaltconfigs Continue with builds by trying to use alternative + providers where possible. + -f, --force Force the specified targets/task to run (invalidating + any existing stamp file). + -c CMD, --cmd=CMD Specify the task to execute. The exact options + available depend on the metadata. Some examples might + be 'compile' or 'populate_sysroot' or 'listtasks' may + give a list of the tasks available. + -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP + Invalidate the stamp for the specified task such as + 'compile' and then run the default task for the + specified target(s). + -r PREFILE, --read=PREFILE + Read the specified file before bitbake.conf. + -R POSTFILE, --postread=POSTFILE + Read the specified file after bitbake.conf. + -v, --verbose Output more log message data to the terminal. + -D, --debug Increase the debug level. You can specify this more + than once. + -n, --dry-run Don't execute, just go through the motions. + -S, --dump-signatures + Don't execute, just dump out the signature + construction information. + -p, --parse-only Quit after parsing the BB recipes. + -s, --show-versions Show current and preferred versions of all recipes. + -e, --environment Show the global or per-package environment complete + with information about where variables were + set/changed. + -g, --graphviz Save dependency tree information for the specified + targets in the dot syntax. + -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED + Assume these dependencies don't exist and are already + provided (equivalent to ASSUME_PROVIDED). Useful to + make dependency graphs more appealing + -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS + Show debug logging for the specified logging domains + -P, --profile Profile the command and save reports. + -u UI, --ui=UI The user interface to use (e.g. knotty, hob, depexp). + -t SERVERTYPE, --servertype=SERVERTYPE + Choose which server to use, process or xmlrpc. + --revisions-changed Set the exit code depending on whether upstream + floating revisions have changed or not. + --server-only Run bitbake without a UI, only starting a server + (cooker) process. + -B BIND, --bind=BIND The name/address for the bitbake server to bind to. + --no-setscene Do not run any setscene tasks. sstate will be ignored + and everything needed, built. + --remote-server=REMOTE_SERVER + Connect to the specified server. + -m, --kill-server Terminate the remote server. + --observe-only Connect to a server as an observing-only client. + +
+ +
+ Fetchers + + + BitBake also contains a set of "fetcher" modules that allow + retrieval of source code from various types of sources. + For example, BitBake can get source code from a disk with the metadata, from websites, + from remote shell accounts, or from Source Code Management (SCM) systems + like cvs/subversion/git. + + + + Fetchers are usually triggered by entries in + SRC_URI. + You can find information about the options and formats of entries for specific + fetchers in the BitBake manual located in the + bitbake/doc/manual directory of the + Source Directory. + + + + One useful feature for certain Source Code Manager (SCM) fetchers is the ability to + "auto-update" when the upstream SCM changes version. + Since this ability requires certain functionality from the SCM, not all + systems support it. + Currently Subversion, Bazaar and to a limited extent, Git support the ability to "auto-update". + This feature works using the SRCREV + variable. + See the + "Using an External SCM" section + in the Yocto Project Development Manual for more information. + + +
+ + + -- cgit v1.2.3-54-g00ecf