diff options
author | Scott Rifenbark <srifenbark@gmail.com> | 2018-01-29 15:18:03 -0800 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2018-02-14 15:25:29 +0000 |
commit | ae06e04cd225d2c2147ca355e2dd39b4f6cf6775 (patch) | |
tree | c920e85262a91e7626279e7dcbbd56a299919f49 /documentation/getting-started/eclipse/html/getting-started/overview-checksums.html | |
parent | ebc7de094881dd8f2450aa4fdf548f2e9c835df1 (diff) | |
download | poky-ae06e04cd225d2c2147ca355e2dd39b4f6cf6775.tar.gz |
documentation: Created new "Getting Started" manual.
Creation involved removing the overview-manual and replacing it
with the getting-started manual. All links to the string
"&YOCTO_DOCS_OVERVIEW_URL" had to be replaced with
"&YOCTO_DOCS_GS_URL" across the entire YP manual set. I renamed
files used to create the manual with prefixes suited for the
new manual name, which is "Getting Started With Yocto Project".
The style sheet for the new manual needed updating to display the
new .PNG image for the title page. The mega-manual file had to
be updated to include the files. The mega-manual.sed file had
to be updated to include the new manual and not use the overview
manual.
(From yocto-docs rev: 6c7abf9192390121000f577d6c98f259d290d15d)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation/getting-started/eclipse/html/getting-started/overview-checksums.html')
-rw-r--r-- | documentation/getting-started/eclipse/html/getting-started/overview-checksums.html | 209 |
1 files changed, 209 insertions, 0 deletions
diff --git a/documentation/getting-started/eclipse/html/getting-started/overview-checksums.html b/documentation/getting-started/eclipse/html/getting-started/overview-checksums.html new file mode 100644 index 0000000000..09ad110f71 --- /dev/null +++ b/documentation/getting-started/eclipse/html/getting-started/overview-checksums.html | |||
@@ -0,0 +1,209 @@ | |||
1 | <html> | ||
2 | <head> | ||
3 | <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> | ||
4 | <title>3.3.2. Checksums (Signatures)</title> | ||
5 | <link rel="stylesheet" type="text/css" href="../book.css"> | ||
6 | <meta name="generator" content="DocBook XSL Stylesheets V1.76.1"> | ||
7 | <link rel="home" href="index.html" title="Getting Started With Yocto Project"> | ||
8 | <link rel="up" href="shared-state-cache.html" title="3.3. Shared State Cache"> | ||
9 | <link rel="prev" href="overall-architecture.html" title="3.3.1. Overall Architecture"> | ||
10 | <link rel="next" href="shared-state.html" title="3.3.3. Shared State"> | ||
11 | </head> | ||
12 | <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="section" title="3.3.2. Checksums (Signatures)"> | ||
13 | <div class="titlepage"><div><div><h3 class="title"> | ||
14 | <a name="overview-checksums"></a>3.3.2. Checksums (Signatures)</h3></div></div></div> | ||
15 | <p> | ||
16 | The shared state code uses a checksum, which is a unique | ||
17 | signature of a task's inputs, to determine if a task needs to | ||
18 | be run again. | ||
19 | Because it is a change in a task's inputs that triggers a | ||
20 | rerun, the process needs to detect all the inputs to a given | ||
21 | task. | ||
22 | For shell tasks, this turns out to be fairly easy because | ||
23 | the build process generates a "run" shell script for each task | ||
24 | and it is possible to create a checksum that gives you a good | ||
25 | idea of when the task's data changes. | ||
26 | </p> | ||
27 | <p> | ||
28 | To complicate the problem, there are things that should not be | ||
29 | included in the checksum. | ||
30 | First, there is the actual specific build path of a given | ||
31 | task - the | ||
32 | <a class="link" href="../ref-manual/var-WORKDIR.html" target="_self"><code class="filename">WORKDIR</code></a>. | ||
33 | It does not matter if the work directory changes because it | ||
34 | should not affect the output for target packages. | ||
35 | Also, the build process has the objective of making native | ||
36 | or cross packages relocatable. | ||
37 | </p> | ||
38 | <div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"> | ||
39 | <h3 class="title">Note</h3> | ||
40 | Both native and cross packages run on the build host. | ||
41 | However, cross packages generate output for the target | ||
42 | architecture. | ||
43 | </div> | ||
44 | <p> | ||
45 | The checksum therefore needs to exclude | ||
46 | <code class="filename">WORKDIR</code>. | ||
47 | The simplistic approach for excluding the work directory is to | ||
48 | set <code class="filename">WORKDIR</code> to some fixed value and | ||
49 | create the checksum for the "run" script. | ||
50 | </p> | ||
51 | <p> | ||
52 | Another problem results from the "run" scripts containing | ||
53 | functions that might or might not get called. | ||
54 | The incremental build solution contains code that figures out | ||
55 | dependencies between shell functions. | ||
56 | This code is used to prune the "run" scripts down to the | ||
57 | minimum set, thereby alleviating this problem and making the | ||
58 | "run" scripts much more readable as a bonus. | ||
59 | </p> | ||
60 | <p> | ||
61 | So far we have solutions for shell scripts. | ||
62 | What about Python tasks? | ||
63 | The same approach applies even though these tasks are more | ||
64 | difficult. | ||
65 | The process needs to figure out what variables a Python | ||
66 | function accesses and what functions it calls. | ||
67 | Again, the incremental build solution contains code that first | ||
68 | figures out the variable and function dependencies, and then | ||
69 | creates a checksum for the data used as the input to the task. | ||
70 | </p> | ||
71 | <p> | ||
72 | Like the <code class="filename">WORKDIR</code> case, situations exist | ||
73 | where dependencies should be ignored. | ||
74 | For these cases, you can instruct the build process to | ||
75 | ignore a dependency by using a line like the following: | ||
76 | </p> | ||
77 | <pre class="literallayout"> | ||
78 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" | ||
79 | </pre> | ||
80 | <p> | ||
81 | This example ensures that the | ||
82 | <a class="link" href="../ref-manual/var-PACKAGE_ARCHS.html" target="_self"><code class="filename">PACKAGE_ARCHS</code></a> | ||
83 | variable does not depend on the value of | ||
84 | <a class="link" href="../ref-manual/var-MACHINE.html" target="_self"><code class="filename">MACHINE</code></a>, | ||
85 | even if it does reference it. | ||
86 | </p> | ||
87 | <p> | ||
88 | Equally, there are cases where we need to add dependencies | ||
89 | BitBake is not able to find. | ||
90 | You can accomplish this by using a line like the following: | ||
91 | </p> | ||
92 | <pre class="literallayout"> | ||
93 | PACKAGE_ARCHS[vardeps] = "MACHINE" | ||
94 | </pre> | ||
95 | <p> | ||
96 | This example explicitly adds the <code class="filename">MACHINE</code> | ||
97 | variable as a dependency for | ||
98 | <code class="filename">PACKAGE_ARCHS</code>. | ||
99 | </p> | ||
100 | <p> | ||
101 | Consider a case with in-line Python, for example, where | ||
102 | BitBake is not able to figure out dependencies. | ||
103 | When running in debug mode (i.e. using | ||
104 | <code class="filename">-DDD</code>), BitBake produces output when it | ||
105 | discovers something for which it cannot figure out dependencies. | ||
106 | The Yocto Project team has currently not managed to cover | ||
107 | those dependencies in detail and is aware of the need to fix | ||
108 | this situation. | ||
109 | </p> | ||
110 | <p> | ||
111 | Thus far, this section has limited discussion to the direct | ||
112 | inputs into a task. | ||
113 | Information based on direct inputs is referred to as the | ||
114 | "basehash" in the code. | ||
115 | However, there is still the question of a task's indirect | ||
116 | inputs - the things that were already built and present in the | ||
117 | <a class="link" href="../ref-manual/build-directory.html" target="_self">Build Directory</a>. | ||
118 | The checksum (or signature) for a particular task needs to add | ||
119 | the hashes of all the tasks on which the particular task | ||
120 | depends. | ||
121 | Choosing which dependencies to add is a policy decision. | ||
122 | However, the effect is to generate a master checksum that | ||
123 | combines the basehash and the hashes of the task's | ||
124 | dependencies. | ||
125 | </p> | ||
126 | <p> | ||
127 | At the code level, there are a variety of ways both the | ||
128 | basehash and the dependent task hashes can be influenced. | ||
129 | Within the BitBake configuration file, we can give BitBake | ||
130 | some extra information to help it construct the basehash. | ||
131 | The following statement effectively results in a list of | ||
132 | global variable dependency excludes - variables never | ||
133 | included in any checksum: | ||
134 | </p> | ||
135 | <pre class="literallayout"> | ||
136 | BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \ | ||
137 | SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \ | ||
138 | USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \ | ||
139 | PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \ | ||
140 | CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX" | ||
141 | </pre> | ||
142 | <p> | ||
143 | The previous example excludes | ||
144 | <a class="link" href="../ref-manual/var-WORKDIR.html" target="_self"><code class="filename">WORKDIR</code></a> | ||
145 | since that variable is actually constructed as a path within | ||
146 | <a class="link" href="../ref-manual/var-TMPDIR.html" target="_self"><code class="filename">TMPDIR</code></a>, | ||
147 | which is on the whitelist. | ||
148 | </p> | ||
149 | <p> | ||
150 | The rules for deciding which hashes of dependent tasks to | ||
151 | include through dependency chains are more complex and are | ||
152 | generally accomplished with a Python function. | ||
153 | The code in <code class="filename">meta/lib/oe/sstatesig.py</code> shows | ||
154 | two examples of this and also illustrates how you can insert | ||
155 | your own policy into the system if so desired. | ||
156 | This file defines the two basic signature generators | ||
157 | <a class="link" href="../ref-manual/oe-core.html" target="_self">OE-Core</a> | ||
158 | uses: "OEBasic" and "OEBasicHash". | ||
159 | By default, there is a dummy "noop" signature handler enabled | ||
160 | in BitBake. | ||
161 | This means that behavior is unchanged from previous versions. | ||
162 | OE-Core uses the "OEBasicHash" signature handler by default | ||
163 | through this setting in the <code class="filename">bitbake.conf</code> | ||
164 | file: | ||
165 | </p> | ||
166 | <pre class="literallayout"> | ||
167 | BB_SIGNATURE_HANDLER ?= "OEBasicHash" | ||
168 | </pre> | ||
169 | <p> | ||
170 | The "OEBasicHash" <code class="filename">BB_SIGNATURE_HANDLER</code> | ||
171 | is the same as the "OEBasic" version but adds the task hash to | ||
172 | the stamp files. | ||
173 | This results in any | ||
174 | <a class="link" href="../ref-manual/metadata.html" target="_self">Metadata</a> | ||
175 | change that changes the task hash, automatically | ||
176 | causing the task to be run again. | ||
177 | This removes the need to bump | ||
178 | <a class="link" href="../ref-manual/var-PR.html" target="_self"><code class="filename">PR</code></a> | ||
179 | values, and changes to Metadata automatically ripple across | ||
180 | the build. | ||
181 | </p> | ||
182 | <p> | ||
183 | It is also worth noting that the end result of these | ||
184 | signature generators is to make some dependency and hash | ||
185 | information available to the build. | ||
186 | This information includes: | ||
187 | </p> | ||
188 | <div class="itemizedlist"><ul class="itemizedlist" type="disc"> | ||
189 | <li class="listitem"><p> | ||
190 | <code class="filename">BB_BASEHASH_task-</code><em class="replaceable"><code>taskname</code></em>: | ||
191 | The base hashes for each task in the recipe. | ||
192 | </p></li> | ||
193 | <li class="listitem"><p> | ||
194 | <code class="filename">BB_BASEHASH_</code><em class="replaceable"><code>filename</code></em><code class="filename">:</code><em class="replaceable"><code>taskname</code></em>: | ||
195 | The base hashes for each dependent task. | ||
196 | </p></li> | ||
197 | <li class="listitem"><p> | ||
198 | <code class="filename">BBHASHDEPS_</code><em class="replaceable"><code>filename</code></em><code class="filename">:</code><em class="replaceable"><code>taskname</code></em>: | ||
199 | The task dependencies for each task. | ||
200 | </p></li> | ||
201 | <li class="listitem"><p> | ||
202 | <code class="filename">BB_TASKHASH</code>: | ||
203 | The hash of the currently running task. | ||
204 | </p></li> | ||
205 | </ul></div> | ||
206 | <p> | ||
207 | </p> | ||
208 | </div></body> | ||
209 | </html> | ||