From 565d5abd870dba547760904f636bda368b171e48 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Wed, 13 Jun 2012 13:51:09 -0700 Subject: documentation/poky-ref-manual/technical-details.xml: 1.1.2 variables and updates First pass at implementing the poky.ent variables. Also corrected and updated some text areas. (From yocto-docs rev: a81007b29ece7a4edcebaebdb999f42de72f0ebf) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../poky-ref-manual/technical-details.xml | 141 ++++++++++----------- 1 file changed, 68 insertions(+), 73 deletions(-) diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml index b279289643..f045b8f552 100644 --- a/documentation/poky-ref-manual/technical-details.xml +++ b/documentation/poky-ref-manual/technical-details.xml @@ -1,5 +1,7 @@ +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[ %poky; ] > + Technical Details @@ -31,10 +33,8 @@ Configuration data acts as the glue to bind everything together. For more information on data, see the - - Yocto Project Terms section in - - The Yocto Project Development Manual. + "Yocto Project Terms" + section in the Yocto Project Development Manual. @@ -45,8 +45,7 @@ Following are some brief details on these core components. For more detailed information on these components see the - 'Reference: Directory Structure' - appendix. + "Reference: Directory Structure" appendix.
@@ -76,7 +75,7 @@ BitBake chooses the one selected by the distribution configuration. You can get more details about how BitBake chooses between different target versions and providers in the - Preferences and Providers section. + "Preferences and Providers" section. @@ -115,7 +114,7 @@ The term "package" can also be used to describe recipes. However, since the same word is used for the packaged output from the Yocto Project (i.e. .ipk or .deb files), - this document avoids using the term "package" when refering to recipes. + this document avoids using the term "package" when referring to recipes.
@@ -127,7 +126,7 @@ between metadata files. An example is the Autotools class, which contains common settings for any application that Autotools uses. - The Reference: Classes appendix provides details + The "Reference: Classes" appendix provides details about common classes and how to use them. @@ -163,7 +162,7 @@ As mentioned in the previous paragraph, building from scratch ensures that everything is current and starts from a known state. However, building from scratch also takes much longer as it generally means - rebuiding things that don't necessarily need rebuilt. + rebuilding things that don't necessarily need rebuilt. @@ -239,7 +238,7 @@ affect the output for target packages. Also, the build process has the objective of making native/cross packages relocatable. The checksum therefore needs to exclude WORKDIR. - The simplistic approach for excluding the worknig directory is to set + The simplistic approach for excluding the working directory is to set WORKDIR to some fixed value and create the checksum for the "run" script. @@ -299,77 +298,73 @@ Thus far, this section has limited discussion to the direct inputs into a task. - Information based on direct inputs is referred to as the "basehash" in the code. - However, there is still the question of a task's indirect inputs, the things that - were already built and present in the build directory. - The checksum (or signature) for a particular task needs to add the hashes of all the - tasks on which the particular task depends. - Choosing which dependencies to add is a policy decision. - However, the effect is to generate a master checksum that combines the - basehash and the hashes of the task's dependencies. + Information based on direct inputs is referred to as the "basehash" in the + code. + However, there is still the question of a task's indirect inputs - the + things that were already built and present in the build directory. + The checksum (or signature) for a particular task needs to add the hashes + of all the tasks on which the particular task depends. + Choosing which dependencies to add is a policy decision. + However, the effect is to generate a master checksum that combines the basehash + and the hashes of the task's dependencies. - While figuring out the dependencies and creating these checksums is good, - what does the Yocto Project build system do with the checksum information? - The build system uses a signature handler that is responsible for - processing the checksum information. - By default, there is a dummy "noop" signature handler enabled in BitBake. - This means that behaviour is unchanged from previous versions. - OECore uses the "basic" signature handler through this setting in the - bitbake.conf file: - - BB_SIGNATURE_HANDLER ?= "basic" - - Also within the BitBake configuration file, we can give BitBake - some extra information to help it handle this information. - The following statements effectively result in a list of global - variable dependency excludes - variables never included in - any checksum: + At the code level, there are a variety of ways both the basehash and the + dependent task hashes can be influenced. + Within the BitBake configuration file, we can give BitBake some extra information + to help it construct the basehash. + The following statements effectively result in a list of global variable + dependency excludes - variables never included in any checksum: - BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH" - BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS" - BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER" - BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET" - BB_HASHTASK_WHITELIST += "(.*-cross$|.*-native$|.*-cross-initial$| \ - .*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)" + BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH" + BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS" + BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER" + BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET" - This example is actually where WORKDIR - is excluded since WORKDIR is constructed as a - path within TMPDIR, which is on the whitelist. + The previous example actually excludes + WORKDIR + since it is actually constructed as a path within + TMPDIR, which is on + the whitelist. - The BB_HASHTASK_WHITELIST covers dependent tasks and - excludes certain kinds of tasks from the dependency chains. - The effect of the previous example is to isolate the native, target, - and cross-components. - So, for example, toolchain changes do not force a rebuild of the whole system. + The rules for deciding which hashes of dependent tasks to include through + dependency chains are more complex and are generally accomplished with a + python function. + The code in meta/lib/oe/sstatesig.py shows two examples + of this and also illustrates how you can insert your own policy into the system + if so desired. + This file defines the two basic signature generators OE-Core + uses: "OEBasic" and "OEBasicHash". + By default, there is a dummy "noop" signature handler enabled in BitBake. + This means that behavior is unchanged from previous versions. + OE-Core uses the "OEBasic" signature handler by default + through this setting in the bitbake.conf file: + + BB_SIGNATURE_HANDLER ?= "OEBasic" + + The "OEBasicHash" BB_SIGNATURE_HANDLER is the same as the + "OEBasic" version but adds the task hash to the stamp files. + This results in any metadata change that changes the task hash, automatically + causing the task to be run again. + This removes the need to bump PR + values and changes to metadata automatically ripple across the build. + Currently, this behavior is not the default behavior for OE-Core + but is the default in poky. - The end result of the "basic" handler is to make some dependency and - hash information available to the build. - This includes: + It is also worth noting that the end result of these signature generators is to + make some dependency and hash information available to the build. + This information includes: - BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe - BB_BASEHASH_<filename:taskname> - the base hashes for each dependent task - BBHASHDEPS_<filename:taskname> - The task dependencies for each task - BB_TASKHASH - the hash of the currently running task + BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe + BB_BASEHASH_<filename:taskname> - the base hashes for each dependent task + BBHASHDEPS_<filename:taskname> - The task dependencies for each task + BB_TASKHASH - the hash of the currently running task - There is also a "basichash" BB_SIGNATURE_HANDLER, - which is the same as the basic version but adds the task hash to the stamp files. - This results in any metadata change that changes the task hash, - automatically causing the task to be run again. - This removes the need to bump PR - values and changes to metadata automatically ripple across the build. - Currently, this behavior is not the default behavior. - However, it is likely that the Yocto Project team will go forward with this - behavior in the future since all the functionality exists. - The reason for the delay is the potential impact to the distribution feed - creation as they need increasing PR fields - and the Yocto Project currently lacks a mechanism to automate incrementing - this field. @@ -431,7 +426,7 @@ do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" - This method, as well as the following example, also works for mutliple directories. + This method, as well as the following example, also works for multiple directories. do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}" do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}" @@ -553,7 +548,7 @@ Once you are aware of such a change, you can take steps to invalidate the cache and force the task to run. The step to take is as simple as changing a function's comments in the source code. - For example, to invalidate package shared state files, change the comment statments + For example, to invalidate package shared state files, change the comment statements of do_package or the comments of one of the functions it calls. The change is purely cosmetic, but it causes the checksum to be recalculated and forces the task to be run again. @@ -562,7 +557,7 @@ For an example of a commit that makes a cosmetic change to invalidate a shared state, see this - commit. + commit. -- cgit v1.2.3-54-g00ecf