From 32a860851c849edd44328d608376b3c27889a63e Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Fri, 19 Aug 2011 10:11:41 -0700 Subject: documentation/poky-ref-manual/ref-bitbake.xml: scrubbed for Poky I swapped the Yocto Project for Poky throughout. Some other general edits were done. I put in a reference to a spot to learn how to best set the BB_NUMBER_THREADS variable. (From yocto-docs rev: 2543a915957014a59e6c49a1cdde270725d47af3) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/poky-ref-manual/ref-bitbake.xml | 170 +++++++++++++++----------- 1 file changed, 97 insertions(+), 73 deletions(-) (limited to 'documentation/poky-ref-manual') diff --git a/documentation/poky-ref-manual/ref-bitbake.xml b/documentation/poky-ref-manual/ref-bitbake.xml index 24e4a068d1..2cf3704214 100644 --- a/documentation/poky-ref-manual/ref-bitbake.xml +++ b/documentation/poky-ref-manual/ref-bitbake.xml @@ -6,23 +6,23 @@ Reference: BitBake - BitBake is a program written in Python that interprets the metadata that makes up Poky. - At some point, people wonder what actually happens when you enter: - + BitBake is a program written in Python that interprets the metadata that makes up the Yocto Project. + At some point, developers wonder what actually happens when you enter: + $ bitbake core-image-sato - + This appendix provides an overview of what happens behind the scenes from BitBake's perspective. - + BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships. As such, it has no real knowledge of what the tasks being executed actually do. BitBake just considers a list of tasks with dependencies and handles metadata that consists of variables in a certain format that get passed to the tasks. - +
Parsing @@ -33,61 +33,64 @@ The first thing BitBake does is look for the bitbake.conf file. - Poky keeps this file in meta/conf/. + The Yocto Project keeps this file in the Yocto Project file's meta/conf/ + directory. BitBake finds it by examining the BBPATH environment variable and looking for the meta/conf/ directory. - In Poky, bitbake.conf lists other configuration + In the Yocto Project, bitbake.conf lists other configuration files to include from a conf/ directory below the directories listed in BBPATH. - In general the most important configuration file from a user's perspective + In general, the most important configuration file from a user's perspective is local.conf, which contains a user's customized - settings for Poky. + settings for the Yocto Project build environment. Other notable configuration files are the distribution - configuration file (set by the - DISTRO variable) and the machine configuration file - (set by the MACHINE - variable). - The DISTRO and MACHINE environment variables are both usually set in + configuration file (set by the + DISTRO variable) + and the machine configuration file + (set by the + MACHINE variable). + The DISTRO and MACHINE environment + variables are both usually set in the local.conf file. Valid distribution - configuration files are available in the - meta/conf/distro/ directory and valid machine configuration - files in the meta/conf/machine/ - directory. + configuration files are available in the meta/conf/distro/ directory + and valid machine configuration + files in the meta/conf/machine/ directory. Within the meta/conf/machine/include/ directory are various tune-*.inc configuration files that provide common "tuning" settings specific to and shared between particular architectures and machines. - After the parsing of the configuration files some standard classes are included. + After the parsing of the configuration files, some standard classes are included. The base.bbclass file is always included. Other classes that are specified in the configuration using the - INHERIT + INHERIT variable are also inculded. - Class files are searched for in a classes subdirectory + Class files are searched for in a classes subdirectory under the paths in BBPATH in the same way as configuration files. - After classes are included, the - variable BBFILES + After classes are included, the variable + BBFILES is set, usually in local.conf, and defines the list of places to search for .bb files. - By default, the BBFILES variable specifies the meta/recipes-*/ - directory within Poky. - Adding extra content to BBFILES is best achieved through the use of BitBake - "layers". + By default, the BBFILES variable specifies the + meta/recipes-*/ directory within Poky. + Adding extra content to BBFILES is best achieved through the use of + BitBake layers as described in the + BitBake Layers section. - BitBake parses each .bb file in BBFILES and + BitBake parses each .bb file in BBFILES and stores the values of various variables. In summary, for each .bb file the configuration plus the base class of variables are set, followed @@ -111,13 +114,14 @@ Once all the .bb files have been - parsed, BitBake starts to build the target (core-image-sato in the previous section's - example) and looks for providers of that target. + parsed, BitBake starts to build the target (core-image-sato + in the previous section's example) and looks for providers of that target. Once a provider is selected, BitBake resolves all the dependencies for the target. - In the case of "core-image-sato", it would lead to task-base.bb, - which in turn leads to packages like Contacts, - Dates and BusyBox. + In the case of core-image-sato, it would lead to + task-base.bb, + which in turn leads to packages like Contacts, + Dates and BusyBox. These packages in turn depend on glibc and the toolchain. @@ -128,12 +132,12 @@ following in the machine configuration file: - -PREFERRED_PROVIDER_virtual/kernel = "linux-rp" - + + PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" + - The default PREFERRED_PROVIDER + The default PREFERRED_PROVIDER is the provider with the same name as the target. @@ -142,17 +146,18 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp" that multiple versions might exist. BitBake defaults to the highest version of a provider. Version comparisons are made using the same method as Debian. - You can use the PREFERRED_VERSION + You can use the + PREFERRED_VERSION variable to specify a particular version (usually in the distro configuration). You can influence the order by using the - DEFAULT_PREFERENCE + DEFAULT_PREFERENCE variable. By default, files have a preference of "0". - Setting the DEFAULT_PREFERENCE to "-1" makes the package unlikely to be used unless it is - explicitly referenced. - Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used. - PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting. - DEFAULT_PREFERENCE is often used to mark newer and more experimental package + Setting the DEFAULT_PREFERENCE to "-1" makes the + package unlikely to be used unless it is explicitly referenced. + Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used. + PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting. + DEFAULT_PREFERENCE is often used to mark newer and more experimental package versions until they have undergone sufficient testing to be considered stable. @@ -165,8 +170,10 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp" Dependencies - Each target BitBake builds consists of multiple tasks such as fetch, unpack, patch, configure, - and compile. + Each target BitBake builds consists of multiple tasks such as + fetch, unpack, + patch, configure, + and compile. For best performance on multi-core systems, BitBake considers each task as an independent entity with its own set of dependencies. @@ -175,12 +182,11 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp" Dependencies are defined through several variables. You can find information about variables BitBake uses in the BitBake manual. - At a basic level it is sufficient to know that BitBake uses the - DEPENDS and - RDEPENDS variables when + At a basic level, it is sufficient to know that BitBake uses the + DEPENDS and + RDEPENDS variables when calculating dependencies. -
@@ -191,39 +197,49 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp" BitBake can now calculate exactly what tasks it needs to run and in what order it needs to run them. The build now starts with BitBake forking off threads up to the limit set in the - BB_NUMBER_THREADS variable. + BB_NUMBER_THREADS variable. BitBake continues to fork threads as long as there are tasks ready to run, those tasks have all their dependencies met, and the thread threshold has not been exceeded. + + It is worth noting that you can greatly speed up the build time by properly setting + the BB_NUMBER_THREADS variable. + See the + + Building an Image section in the + + Yocto Project Quick Start for more information. + + As each task completes, a timestamp is written to the directory specified by the - STAMPS variable (usually + STAMPS variable (usually build/tmp/stamps/*/). - On subsequent runs, BitBake looks at the STAMPS directory and does not rerun + On subsequent runs, BitBake looks at the STAMPS directory and does not rerun tasks that are already completed unless a timestamp is found to be invalid. Currently, invalid timestamps are only considered on a per .bb file basis. So, for example, if the configure stamp has a timestamp greater than the - compile timestamp for a given target then the compile task would rerun. + compile timestamp for a given target, then the compile task would rerun. Running the compile task again, however, has no effect on other providers that depend on that target. This behavior could change or become configurable in future versions of BitBake. - + Some tasks are marked as "nostamp" tasks. No timestamp file is created when these tasks are run. Consequently, "nostamp" tasks are always rerun. - +
Running a Task - Tasks can either be a shell task or a python task. + Tasks can either be a shell task or a Python task. For shell tasks, BitBake writes a shell script to ${WORKDIR}/temp/run.do_taskname.pid and then executes the script. The generated shell script contains all the exported variables, and the shell functions @@ -268,7 +284,8 @@ Options: -h, --help show this help message and exit -b BUILDFILE, --buildfile=BUILDFILE execute the task against this .bb file, rather than a - package from BBFILES. + package from BBFILES. Does not handle any + dependencies. -k, --continue continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these @@ -276,8 +293,6 @@ Options: -a, --tryaltconfigs continue with builds by trying to use alternative providers where possible. -f, --force force run of specified cmd, regardless of stamp status - -i, --interactive drop into the interactive mode also called the BitBake - shell. -c CMD, --cmd=CMD Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call @@ -285,26 +300,35 @@ Options: what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks - -r FILE, --read=FILE read the specified file before bitbake.conf + -r PREFILE, --read=PREFILE + read the specified file before bitbake.conf + -R POSTFILE, --postread=POSTFILE + read the specified file after bitbake.conf -v, --verbose output more chit-chat to the terminal -D, --debug Increase the debug level. You can specify this more than once. -n, --dry-run don't execute, just go through the motions + -S, --dump-signatures + don't execute, just dump out the signature + construction information -p, --parse-only quit after parsing the BB files (developers only) - -d, --disable-psyco disable using the psyco just-in-time compiler (not - recommended) -s, --show-versions show current and preferred versions of all packages -e, --environment show the global or per-package environment (this is what used to be bbread) -g, --graphviz emit the dependency trees of the specified packages in the dot syntax - -I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS - Stop processing at the given list of dependencies when - generating dependency graphs. This can help to make - the graph more appealing + -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED + Assume these dependencies don't exist and are already + provided (equivalent to ASSUME_PROVIDED). Useful to + make dependency graphs more appealing -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS Show debug logging for the specified logging domains -P, --profile profile the command and print a report + -u UI, --ui=UI userinterface to use + -t SERVERTYPE, --servertype=SERVERTYPE + Choose which server to use, none, process or xmlrpc + --revisions-changed Set the exit code depending on whether upstream + floating revisions have changed or not
@@ -321,22 +345,22 @@ Options: Fetchers are usually triggered by entries in - SRC_URI. + SRC_URI. You can find information about the options and formats of entries for specific fetchers in the BitBake manual. - One useful feature for certain SCM fetchers is the ability to + One useful feature for certain Source Code Manager (SCM) fetchers is the ability to "auto-update" when the upstream SCM changes version. Since this ability requires certain functionality from the SCM, not all systems support it. Currently Subversion, Bazaar and to a limited extent, Git support the ability to "auto-update". - This feature works using the SRCREV + This feature works using the SRCREV variable. See the - Developing within Poky with an External SCM-based Package - section for more information. + Development Within Yocto Project for a Package that Uses + an External SCM section for more information. -- cgit v1.2.3-54-g00ecf