summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html')
-rw-r--r--documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html164
1 files changed, 0 insertions, 164 deletions
diff --git a/documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html b/documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html
deleted file mode 100644
index 5dccce93b9..0000000000
--- a/documentation/ref-manual/eclipse/html/poky-ref-manual/checksums.html
+++ /dev/null
@@ -1,164 +0,0 @@
1<html>
2<head>
3<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
4<title>3.2.2. Checksums (Signatures)</title>
5<link rel="stylesheet" type="text/css" href="../book.css">
6<meta name="generator" content="DocBook XSL Stylesheets V1.76.1">
7<link rel="home" href="index.html" title="The Yocto Project Reference Manual">
8<link rel="up" href="shared-state-cache.html" title="3.2. Shared State Cache">
9<link rel="prev" href="overall-architecture.html" title="3.2.1. Overall Architecture">
10<link rel="next" href="shared-state.html" title="3.2.3. Shared State">
11</head>
12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="section" title="3.2.2. Checksums (Signatures)">
13<div class="titlepage"><div><div><h3 class="title">
14<a name="checksums"></a>3.2.2. Checksums (Signatures)</h3></div></div></div>
15<p>
16 The shared state code uses a checksum, which is a unique signature of a task's
17 inputs, to determine if a task needs to be run again.
18 Because it is a change in a task's inputs that triggers a rerun, the process
19 needs to detect all the inputs to a given task.
20 For shell tasks, this turns out to be fairly easy because
21 the build process generates a "run" shell script for each task and
22 it is possible to create a checksum that gives you a good idea of when
23 the task's data changes.
24 </p>
25<p>
26 To complicate the problem, there are things that should not be included in
27 the checksum.
28 First, there is the actual specific build path of a given task -
29 the <code class="filename">WORKDIR</code>.
30 It does not matter if the working directory changes because it should not
31 affect the output for target packages.
32 Also, the build process has the objective of making native/cross packages relocatable.
33 The checksum therefore needs to exclude <code class="filename">WORKDIR</code>.
34 The simplistic approach for excluding the working directory is to set
35 <code class="filename">WORKDIR</code> to some fixed value and create the checksum
36 for the "run" script.
37 </p>
38<p>
39 Another problem results from the "run" scripts containing functions that
40 might or might not get called.
41 The incremental build solution contains code that figures out dependencies
42 between shell functions.
43 This code is used to prune the "run" scripts down to the minimum set,
44 thereby alleviating this problem and making the "run" scripts much more
45 readable as a bonus.
46 </p>
47<p>
48 So far we have solutions for shell scripts.
49 What about python tasks?
50 The same approach applies even though these tasks are more difficult.
51 The process needs to figure out what variables a python function accesses
52 and what functions it calls.
53 Again, the incremental build solution contains code that first figures out
54 the variable and function dependencies, and then creates a checksum for the data
55 used as the input to the task.
56 </p>
57<p>
58 Like the <code class="filename">WORKDIR</code> case, situations exist where dependencies
59 should be ignored.
60 For these cases, you can instruct the build process to ignore a dependency
61 by using a line like the following:
62 </p>
63<pre class="literallayout">
64 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
65 </pre>
66<p>
67 This example ensures that the <code class="filename">PACKAGE_ARCHS</code> variable does not
68 depend on the value of <code class="filename">MACHINE</code>, even if it does reference it.
69 </p>
70<p>
71 Equally, there are cases where we need to add dependencies BitBake is not able to find.
72 You can accomplish this by using a line like the following:
73 </p>
74<pre class="literallayout">
75 PACKAGE_ARCHS[vardeps] = "MACHINE"
76 </pre>
77<p>
78 This example explicitly adds the <code class="filename">MACHINE</code> variable as a
79 dependency for <code class="filename">PACKAGE_ARCHS</code>.
80 </p>
81<p>
82 Consider a case with inline python, for example, where BitBake is not
83 able to figure out dependencies.
84 When running in debug mode (i.e. using <code class="filename">-DDD</code>), BitBake
85 produces output when it discovers something for which it cannot figure out
86 dependencies.
87 The Yocto Project team has currently not managed to cover those dependencies
88 in detail and is aware of the need to fix this situation.
89 </p>
90<p>
91 Thus far, this section has limited discussion to the direct inputs into a task.
92 Information based on direct inputs is referred to as the "basehash" in the
93 code.
94 However, there is still the question of a task's indirect inputs - the
95 things that were already built and present in the Build Directory.
96 The checksum (or signature) for a particular task needs to add the hashes
97 of all the tasks on which the particular task depends.
98 Choosing which dependencies to add is a policy decision.
99 However, the effect is to generate a master checksum that combines the basehash
100 and the hashes of the task's dependencies.
101 </p>
102<p>
103 At the code level, there are a variety of ways both the basehash and the
104 dependent task hashes can be influenced.
105 Within the BitBake configuration file, we can give BitBake some extra information
106 to help it construct the basehash.
107 The following statements effectively result in a list of global variable
108 dependency excludes - variables never included in any checksum:
109 </p>
110<pre class="literallayout">
111 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH"
112 BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
113 BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
114 BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
115 </pre>
116<p>
117 The previous example actually excludes
118 <a class="link" href="ref-variables-glos.html#var-WORKDIR" title="WORKDIR"><code class="filename">WORKDIR</code></a>
119 since it is actually constructed as a path within
120 <a class="link" href="ref-variables-glos.html#var-TMPDIR" title="TMPDIR"><code class="filename">TMPDIR</code></a>, which is on
121 the whitelist.
122 </p>
123<p>
124 The rules for deciding which hashes of dependent tasks to include through
125 dependency chains are more complex and are generally accomplished with a
126 python function.
127 The code in <code class="filename">meta/lib/oe/sstatesig.py</code> shows two examples
128 of this and also illustrates how you can insert your own policy into the system
129 if so desired.
130 This file defines the two basic signature generators <code class="filename">OE-Core</code>
131 uses: "OEBasic" and "OEBasicHash".
132 By default, there is a dummy "noop" signature handler enabled in BitBake.
133 This means that behavior is unchanged from previous versions.
134 <code class="filename">OE-Core</code> uses the "OEBasic" signature handler by default
135 through this setting in the <code class="filename">bitbake.conf</code> file:
136 </p>
137<pre class="literallayout">
138 BB_SIGNATURE_HANDLER ?= "OEBasic"
139 </pre>
140<p>
141 The "OEBasicHash" <code class="filename">BB_SIGNATURE_HANDLER</code> is the same as the
142 "OEBasic" version but adds the task hash to the stamp files.
143 This results in any metadata change that changes the task hash, automatically
144 causing the task to be run again.
145 This removes the need to bump <a class="link" href="ref-variables-glos.html#var-PR" title="PR"><code class="filename">PR</code></a>
146 values and changes to metadata automatically ripple across the build.
147 Currently, this behavior is not the default behavior for <code class="filename">OE-Core</code>
148 but is the default in <code class="filename">poky</code>.
149 </p>
150<p>
151 It is also worth noting that the end result of these signature generators is to
152 make some dependency and hash information available to the build.
153 This information includes:
154 </p>
155<pre class="literallayout">
156 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe
157 BB_BASEHASH_&lt;filename:taskname&gt; - the base hashes for each dependent task
158 BBHASHDEPS_&lt;filename:taskname&gt; - The task dependencies for each task
159 BB_TASKHASH - the hash of the currently running task
160 </pre>
161<p>
162 </p>
163</div></body>
164</html>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-27 21:34:00 +0000