summaryrefslogtreecommitdiffstats
path: root/documentation/poky-ref-manual/technical-details.xml
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2012-03-15 16:37:25 -0600
committerRichard Purdie <richard.purdie@linuxfoundation.org>2012-03-23 14:24:48 +0000
commit3b95307126a92f2ac8b940fa876ec5f170332ffd (patch)
treef00b2aac6ac7d1ff2a40eb97f752643de1495a9b /documentation/poky-ref-manual/technical-details.xml
parentf99aa9453e93f4aa0a1f9382f2c28acb33df6d40 (diff)
downloadpoky-3b95307126a92f2ac8b940fa876ec5f170332ffd.tar.gz
documentation/poky-ref-manual/technical-details.xml: Fix Checksums section
Implemented Richard Purdie's changes for the "Checksums (Signatures) section. I also performed a spell-check on the chapter and found several rogue words that I corrected. Fixes [YOCTO #2024] Reported-by: Richard Purdie <richard.purdie@intel.com> (From yocto-docs rev: 3025ee64bc0f046abe87f95f1e8b9092a6079798) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation/poky-ref-manual/technical-details.xml')
-rw-r--r--documentation/poky-ref-manual/technical-details.xml132
1 files changed, 64 insertions, 68 deletions
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml
index af62ee4f35..19ce9ab224 100644
--- a/documentation/poky-ref-manual/technical-details.xml
+++ b/documentation/poky-ref-manual/technical-details.xml
@@ -117,7 +117,7 @@
117 The term "package" can also be used to describe recipes. 117 The term "package" can also be used to describe recipes.
118 However, since the same word is used for the packaged output from the Yocto 118 However, since the same word is used for the packaged output from the Yocto
119 Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), 119 Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
120 this document avoids using the term "package" when refering to recipes. 120 this document avoids using the term "package" when referring to recipes.
121 </para> 121 </para>
122 </section> 122 </section>
123 123
@@ -165,7 +165,7 @@
165 As mentioned in the previous paragraph, building from scratch ensures that 165 As mentioned in the previous paragraph, building from scratch ensures that
166 everything is current and starts from a known state. 166 everything is current and starts from a known state.
167 However, building from scratch also takes much longer as it generally means 167 However, building from scratch also takes much longer as it generally means
168 rebuiding things that don't necessarily need rebuilt. 168 rebuilding things that don't necessarily need rebuilt.
169 </para> 169 </para>
170 170
171 <para> 171 <para>
@@ -241,7 +241,7 @@
241 affect the output for target packages. 241 affect the output for target packages.
242 Also, the build process has the objective of making native/cross packages relocatable. 242 Also, the build process has the objective of making native/cross packages relocatable.
243 The checksum therefore needs to exclude <filename>WORKDIR</filename>. 243 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
244 The simplistic approach for excluding the worknig directory is to set 244 The simplistic approach for excluding the working directory is to set
245 <filename>WORKDIR</filename> to some fixed value and create the checksum 245 <filename>WORKDIR</filename> to some fixed value and create the checksum
246 for the "run" script. 246 for the "run" script.
247 </para> 247 </para>
@@ -301,77 +301,73 @@
301 301
302 <para> 302 <para>
303 Thus far, this section has limited discussion to the direct inputs into a task. 303 Thus far, this section has limited discussion to the direct inputs into a task.
304 Information based on direct inputs is referred to as the "basehash" in the code. 304 Information based on direct inputs is referred to as the "basehash" in the
305 However, there is still the question of a task's indirect inputs, the things that 305 code.
306 were already built and present in the build directory. 306 However, there is still the question of a task's indirect inputs - the
307 The checksum (or signature) for a particular task needs to add the hashes of all the 307 things that were already built and present in the build directory.
308 tasks on which the particular task depends. 308 The checksum (or signature) for a particular task needs to add the hashes
309 Choosing which dependencies to add is a policy decision. 309 of all the tasks on which the particular task depends.
310 However, the effect is to generate a master checksum that combines the 310 Choosing which dependencies to add is a policy decision.
311 basehash and the hashes of the task's dependencies. 311 However, the effect is to generate a master checksum that combines the basehash
312 </para> 312 and the hashes of the task's dependencies.
313 313 </para>
314 <para> 314
315 While figuring out the dependencies and creating these checksums is good, 315 <para>
316 what does the Yocto Project build system do with the checksum information? 316 At the code level, there are a variety of ways both the basehash and the
317 The build system uses a signature handler that is responsible for 317 dependent task hashes can be influenced.
318 processing the checksum information. 318 Within the BitBake configuration file, we can give BitBake some extra information
319 By default, there is a dummy "noop" signature handler enabled in BitBake. 319 to help it construct the basehash.
320 This means that behaviour is unchanged from previous versions. 320 The following statements effectively result in a list of global variable
321 OECore uses the "basic" signature handler through this setting in the 321 dependency excludes - variables never included in any checksum:
322 <filename>bitbake.conf</filename> file:
323 <literallayout class='monospaced'> 322 <literallayout class='monospaced'>
324 BB_SIGNATURE_HANDLER ?= "basic" 323 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH"
324 BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
325 BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
326 BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
325 </literallayout> 327 </literallayout>
326 Also within the BitBake configuration file, we can give BitBake 328 The previous example actually excludes
327 some extra information to help it handle this information. 329 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
328 The following statements effectively result in a list of global 330 since it is actually constructed as a path within
329 variable dependency excludes - variables never included in 331 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
330 any checksum: 332 the whitelist.
333 </para>
334
335 <para>
336 The rules for deciding which hashes of dependent tasks to include through
337 dependency chains are more complex and are generally accomplished with a
338 python function.
339 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
340 of this and also illustrates how you can insert your own policy into the system
341 if so desired.
342 This file defines the two basic signature generators <filename>OE-Core</filename>
343 uses: "OEBasic" and "OEBasicHash".
344 By default, there is a dummy "noop" signature handler enabled in BitBake.
345 This means that behavior is unchanged from previous versions.
346 <filename>OE-Core</filename> uses the "OEBasic" signature handler by default
347 through this setting in the <filename>bitbake.conf</filename> file:
331 <literallayout class='monospaced'> 348 <literallayout class='monospaced'>
332 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH" 349 BB_SIGNATURE_HANDLER ?= "OEBasic"
333 BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
334 BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
335 BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
336 BB_HASHTASK_WHITELIST += "(.*-cross$|.*-native$|.*-cross-initial$| \
337 .*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
338 </literallayout> 350 </literallayout>
339 This example is actually where <filename>WORKDIR</filename> 351 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
340 is excluded since <filename>WORKDIR</filename> is constructed as a 352 "OEBasic" version but adds the task hash to the stamp files.
341 path within <filename>TMPDIR</filename>, which is on the whitelist. 353 This results in any metadata change that changes the task hash, automatically
354 causing the task to be run again.
355 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
356 values and changes to metadata automatically ripple across the build.
357 Currently, this behavior is not the default behavior for <filename>OE-Core</filename>
358 but is the default in <filename>poky</filename>.
342 </para> 359 </para>
343 360
344 <para> 361 <para>
345 The <filename>BB_HASHTASK_WHITELIST</filename> covers dependent tasks and 362 It is also worth noting that the end result of these signature generators is to
346 excludes certain kinds of tasks from the dependency chains. 363 make some dependency and hash information available to the build.
347 The effect of the previous example is to isolate the native, target, 364 This information includes:
348 and cross-components.
349 So, for example, toolchain changes do not force a rebuild of the whole system.
350 </para>
351
352 <para>
353 The end result of the "basic" handler is to make some dependency and
354 hash information available to the build.
355 This includes:
356 <literallayout class='monospaced'> 365 <literallayout class='monospaced'>
357 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe 366 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe
358 BB_BASEHASH_&lt;filename:taskname&gt; - the base hashes for each dependent task 367 BB_BASEHASH_&lt;filename:taskname&gt; - the base hashes for each dependent task
359 BBHASHDEPS_&lt;filename:taskname&gt; - The task dependencies for each task 368 BBHASHDEPS_&lt;filename:taskname&gt; - The task dependencies for each task
360 BB_TASKHASH - the hash of the currently running task 369 BB_TASKHASH - the hash of the currently running task
361 </literallayout> 370 </literallayout>
362 There is also a "basichash" <filename>BB_SIGNATURE_HANDLER</filename>,
363 which is the same as the basic version but adds the task hash to the stamp files.
364 This results in any metadata change that changes the task hash,
365 automatically causing the task to be run again.
366 This removes the need to bump <filename>PR</filename>
367 values and changes to metadata automatically ripple across the build.
368 Currently, this behavior is not the default behavior.
369 However, it is likely that the Yocto Project team will go forward with this
370 behavior in the future since all the functionality exists.
371 The reason for the delay is the potential impact to the distribution feed
372 creation as they need increasing <filename>PR</filename> fields
373 and the Yocto Project currently lacks a mechanism to automate incrementing
374 this field.
375 </para> 371 </para>
376 </section> 372 </section>
377 373
@@ -433,7 +429,7 @@
433 <literallayout class='monospaced'> 429 <literallayout class='monospaced'>
434 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" 430 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
435 </literallayout> 431 </literallayout>
436 This method, as well as the following example, also works for mutliple directories. 432 This method, as well as the following example, also works for multiple directories.
437 <literallayout class='monospaced'> 433 <literallayout class='monospaced'>
438 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}" 434 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
439 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}" 435 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
@@ -555,7 +551,7 @@
555 Once you are aware of such a change, you can take steps to invalidate the cache 551 Once you are aware of such a change, you can take steps to invalidate the cache
556 and force the task to run. 552 and force the task to run.
557 The step to take is as simple as changing a function's comments in the source code. 553 The step to take is as simple as changing a function's comments in the source code.
558 For example, to invalidate package shared state files, change the comment statments 554 For example, to invalidate package shared state files, change the comment statements
559 of <filename>do_package</filename> or the comments of one of the functions it calls. 555 of <filename>do_package</filename> or the comments of one of the functions it calls.
560 The change is purely cosmetic, but it causes the checksum to be recalculated and 556 The change is purely cosmetic, but it causes the checksum to be recalculated and
561 forces the task to be run again. 557 forces the task to be run again.
@@ -576,7 +572,7 @@
576 <para> 572 <para>
577 This section describes the mechanism by which the Yocto Project build system 573 This section describes the mechanism by which the Yocto Project build system
578 tracks changes to licensing text. 574 tracks changes to licensing text.
579 The section also describes how to enable commercially licensed receipes, 575 The section also describes how to enable commercially licensed recipes,
580 which by default are disabled. 576 which by default are disabled.
581 </para> 577 </para>
582 578