summaryrefslogtreecommitdiffstats
path: root/documentation/poky-ref-manual/technical-details.xml
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2012-12-07 17:29:51 -0600
committerRichard Purdie <richard.purdie@linuxfoundation.org>2012-12-11 16:17:56 +0000
commit73ffb8298b545a1a1fb96bc5952b7365c4c43bfd (patch)
tree9c6473cb4e3c5dd8370c2d417c304e1a302fa643 /documentation/poky-ref-manual/technical-details.xml
parentacb3f72afaa28ba5d23ca6e5cdf9f1162ea656a3 (diff)
downloadpoky-73ffb8298b545a1a1fb96bc5952b7365c4c43bfd.tar.gz
Documentation: poky-ref-manual - Removed all trailing whitespace.
(From yocto-docs rev: 564a28c2501034ea7e2eb16afc43dfaf931b6f6f) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation/poky-ref-manual/technical-details.xml')
-rw-r--r--documentation/poky-ref-manual/technical-details.xml434
1 files changed, 217 insertions, 217 deletions
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml
index 042a96ed87..72689a0e22 100644
--- a/documentation/poky-ref-manual/technical-details.xml
+++ b/documentation/poky-ref-manual/technical-details.xml
@@ -6,7 +6,7 @@
6<title>Technical Details</title> 6<title>Technical Details</title>
7 7
8 <para> 8 <para>
9 This chapter provides technical details for various parts of the Yocto Project. 9 This chapter provides technical details for various parts of the Yocto Project.
10 Currently, topics include Yocto Project components and shared state (sstate) cache. 10 Currently, topics include Yocto Project components and shared state (sstate) cache.
11 </para> 11 </para>
12 12
@@ -14,21 +14,21 @@
14 <title>Yocto Project Components</title> 14 <title>Yocto Project Components</title>
15 15
16 <para> 16 <para>
17 The BitBake task executor together with various types of configuration files form the 17 The BitBake task executor together with various types of configuration files form the
18 OpenEmbedded Core. 18 OpenEmbedded Core.
19 This section overviews the BitBake task executor and the 19 This section overviews the BitBake task executor and the
20 configuration files by describing what they are used for and how they interact. 20 configuration files by describing what they are used for and how they interact.
21 </para> 21 </para>
22 22
23 <para> 23 <para>
24 BitBake handles the parsing and execution of the data files. 24 BitBake handles the parsing and execution of the data files.
25 The data itself is of various types: 25 The data itself is of various types:
26 <itemizedlist> 26 <itemizedlist>
27 <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular 27 <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular
28 pieces of software</para></listitem> 28 pieces of software</para></listitem>
29 <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build 29 <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build
30 information (e.g. how to build a Linux kernel).</para></listitem> 30 information (e.g. how to build a Linux kernel).</para></listitem>
31 <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings, 31 <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings,
32 policy decisions, etc. 32 policy decisions, etc.
33 Configuration data acts as the glue to bind everything together.</para></listitem> 33 Configuration data acts as the glue to bind everything together.</para></listitem>
34 </itemizedlist> 34 </itemizedlist>
@@ -37,17 +37,17 @@
37 section in the Yocto Project Development Manual. 37 section in the Yocto Project Development Manual.
38 </para> 38 </para>
39 39
40 <para> 40 <para>
41 BitBake knows how to combine multiple data sources together and refers to each data source 41 BitBake knows how to combine multiple data sources together and refers to each data source
42 as a layer. 42 as a layer.
43 For information on layers, see the 43 For information on layers, see the
44 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and 44 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
45 Creating Layers</ulink>" section of the Yocto Project Development Manual. 45 Creating Layers</ulink>" section of the Yocto Project Development Manual.
46 </para> 46 </para>
47 47
48 <para> 48 <para>
49 Following are some brief details on these core components. 49 Following are some brief details on these core components.
50 For more detailed information on these components see the 50 For more detailed information on these components see the
51 "<link linkend='ref-structure'>Directory Structure</link>" chapter. 51 "<link linkend='ref-structure'>Directory Structure</link>" chapter.
52 </para> 52 </para>
53 53
@@ -57,7 +57,7 @@
57 <para> 57 <para>
58 BitBake is the tool at the heart of the OpenEmbedded build system and is responsible 58 BitBake is the tool at the heart of the OpenEmbedded build system and is responsible
59 for parsing the metadata, generating a list of tasks from it, 59 for parsing the metadata, generating a list of tasks from it,
60 and then executing those tasks. 60 and then executing those tasks.
61 To see a list of the options BitBake supports, use the following help command: 61 To see a list of the options BitBake supports, use the following help command:
62 <literallayout class='monospaced'> 62 <literallayout class='monospaced'>
63 $ bitbake --help 63 $ bitbake --help
@@ -66,8 +66,8 @@
66 66
67 <para> 67 <para>
68 The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where 68 The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
69 <filename>packagename</filename> is the name of the package you want to build 69 <filename>packagename</filename> is the name of the package you want to build
70 (referred to as the "target" in this manual). 70 (referred to as the "target" in this manual).
71 The target often equates to the first part of a <filename>.bb</filename> filename. 71 The target often equates to the first part of a <filename>.bb</filename> filename.
72 So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you 72 So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you
73 might type the following: 73 might type the following:
@@ -76,15 +76,15 @@
76 </literallayout> 76 </literallayout>
77 Several different versions of <filename>matchbox-desktop</filename> might exist. 77 Several different versions of <filename>matchbox-desktop</filename> might exist.
78 BitBake chooses the one selected by the distribution configuration. 78 BitBake chooses the one selected by the distribution configuration.
79 You can get more details about how BitBake chooses between different 79 You can get more details about how BitBake chooses between different
80 target versions and providers in the 80 target versions and providers in the
81 "<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section. 81 "<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section.
82 </para> 82 </para>
83 83
84 <para> 84 <para>
85 BitBake also tries to execute any dependent tasks first. 85 BitBake also tries to execute any dependent tasks first.
86 So for example, before building <filename>matchbox-desktop</filename>, BitBake 86 So for example, before building <filename>matchbox-desktop</filename>, BitBake
87 would build a cross compiler and <filename>eglibc</filename> if they had not already 87 would build a cross compiler and <filename>eglibc</filename> if they had not already
88 been built. 88 been built.
89 <note>This release of the Yocto Project does not support the <filename>glibc</filename> 89 <note>This release of the Yocto Project does not support the <filename>glibc</filename>
90 GNU version of the Unix standard C library. By default, the OpenEmbedded build system 90 GNU version of the Unix standard C library. By default, the OpenEmbedded build system
@@ -92,12 +92,12 @@
92 </para> 92 </para>
93 93
94 <para> 94 <para>
95 A useful BitBake option to consider is the <filename>-k</filename> or 95 A useful BitBake option to consider is the <filename>-k</filename> or
96 <filename>--continue</filename> option. 96 <filename>--continue</filename> option.
97 This option instructs BitBake to try and continue processing the job as much 97 This option instructs BitBake to try and continue processing the job as much
98 as possible even after encountering an error. 98 as possible even after encountering an error.
99 When an error occurs, the target that 99 When an error occurs, the target that
100 failed and those that depend on it cannot be remade. 100 failed and those that depend on it cannot be remade.
101 However, when you use this option other dependencies can still be processed. 101 However, when you use this option other dependencies can still be processed.
102 </para> 102 </para>
103 </section> 103 </section>
@@ -106,17 +106,17 @@
106 <title>Metadata (Recipes)</title> 106 <title>Metadata (Recipes)</title>
107 107
108 <para> 108 <para>
109 The <filename>.bb</filename> files are usually referred to as "recipes." 109 The <filename>.bb</filename> files are usually referred to as "recipes."
110 In general, a recipe contains information about a single piece of software. 110 In general, a recipe contains information about a single piece of software.
111 The information includes the location from which to download the source patches 111 The information includes the location from which to download the source patches
112 (if any are needed), which special configuration options to apply, 112 (if any are needed), which special configuration options to apply,
113 how to compile the source files, and how to package the compiled output. 113 how to compile the source files, and how to package the compiled output.
114 </para> 114 </para>
115 115
116 <para> 116 <para>
117 The term "package" can also be used to describe recipes. 117 The term "package" can also be used to describe recipes.
118 However, since the same word is used for the packaged output from the OpenEmbedded 118 However, since the same word is used for the packaged output from the OpenEmbedded
119 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), 119 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
120 this document avoids using the term "package" when referring to recipes. 120 this document avoids using the term "package" when referring to recipes.
121 </para> 121 </para>
122 </section> 122 </section>
@@ -126,7 +126,7 @@
126 126
127 <para> 127 <para>
128 Class files (<filename>.bbclass</filename>) contain information that is useful to share 128 Class files (<filename>.bbclass</filename>) contain information that is useful to share
129 between metadata files. 129 between metadata files.
130 An example is the Autotools class, which contains 130 An example is the Autotools class, which contains
131 common settings for any application that Autotools uses. 131 common settings for any application that Autotools uses.
132 The "<link linkend='ref-classes'>Classes</link>" chapter provides details 132 The "<link linkend='ref-classes'>Classes</link>" chapter provides details
@@ -139,8 +139,8 @@
139 139
140 <para> 140 <para>
141 The configuration files (<filename>.conf</filename>) define various configuration variables 141 The configuration files (<filename>.conf</filename>) define various configuration variables
142 that govern the OpenEmbedded build process. 142 that govern the OpenEmbedded build process.
143 These files fall into several areas that define machine configuration options, 143 These files fall into several areas that define machine configuration options,
144 distribution configuration options, compiler tuning options, general common configuration 144 distribution configuration options, compiler tuning options, general common configuration
145 options and user configuration options (<filename>local.conf</filename>, which is found 145 options and user configuration options (<filename>local.conf</filename>, which is found
146 in the <ulink url='build-directory'>Build Directory</ulink>). 146 in the <ulink url='build-directory'>Build Directory</ulink>).
@@ -152,19 +152,19 @@
152 <title>Shared State Cache</title> 152 <title>Shared State Cache</title>
153 153
154 <para> 154 <para>
155 By design, the OpenEmbedded build system builds everything from scratch unless 155 By design, the OpenEmbedded build system builds everything from scratch unless
156 BitBake can determine that parts don't need to be rebuilt. 156 BitBake can determine that parts don't need to be rebuilt.
157 Fundamentally, building from scratch is attractive as it means all parts are 157 Fundamentally, building from scratch is attractive as it means all parts are
158 built fresh and there is no possibility of stale data causing problems. 158 built fresh and there is no possibility of stale data causing problems.
159 When developers hit problems, they typically default back to building from scratch 159 When developers hit problems, they typically default back to building from scratch
160 so they know the state of things from the start. 160 so they know the state of things from the start.
161 </para> 161 </para>
162 162
163 <para> 163 <para>
164 Building an image from scratch is both an advantage and a disadvantage to the process. 164 Building an image from scratch is both an advantage and a disadvantage to the process.
165 As mentioned in the previous paragraph, building from scratch ensures that 165 As mentioned in the previous paragraph, building from scratch ensures that
166 everything is current and starts from a known state. 166 everything is current and starts from a known state.
167 However, building from scratch also takes much longer as it generally means 167 However, building from scratch also takes much longer as it generally means
168 rebuilding things that don't necessarily need rebuilt. 168 rebuilding things that don't necessarily need rebuilt.
169 </para> 169 </para>
170 170
@@ -181,15 +181,15 @@
181 </para> 181 </para>
182 182
183 <para> 183 <para>
184 For the first question, the build system detects changes in the "inputs" to a given task by 184 For the first question, the build system detects changes in the "inputs" to a given task by
185 creating a checksum (or signature) of the task's inputs. 185 creating a checksum (or signature) of the task's inputs.
186 If the checksum changes, the system assumes the inputs have changed and the task needs to be 186 If the checksum changes, the system assumes the inputs have changed and the task needs to be
187 rerun. 187 rerun.
188 For the second question, the shared state (sstate) code tracks which tasks add which output 188 For the second question, the shared state (sstate) code tracks which tasks add which output
189 to the build process. 189 to the build process.
190 This means the output from a given task can be removed, upgraded or otherwise manipulated. 190 This means the output from a given task can be removed, upgraded or otherwise manipulated.
191 The third question is partly addressed by the solution for the second question 191 The third question is partly addressed by the solution for the second question
192 assuming the build system can fetch the sstate objects from remote locations and 192 assuming the build system can fetch the sstate objects from remote locations and
193 install them if they are deemed to be valid. 193 install them if they are deemed to be valid.
194 </para> 194 </para>
195 195
@@ -202,18 +202,18 @@
202 <title>Overall Architecture</title> 202 <title>Overall Architecture</title>
203 203
204 <para> 204 <para>
205 When determining what parts of the system need to be built, BitBake 205 When determining what parts of the system need to be built, BitBake
206 uses a per-task basis and does not use a per-recipe basis. 206 uses a per-task basis and does not use a per-recipe basis.
207 You might wonder why using a per-task basis is preferred over a per-recipe basis. 207 You might wonder why using a per-task basis is preferred over a per-recipe basis.
208 To help explain, consider having the IPK packaging backend enabled and then switching to DEB. 208 To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
209 In this case, <filename>do_install</filename> and <filename>do_package</filename> 209 In this case, <filename>do_install</filename> and <filename>do_package</filename>
210 output are still valid. 210 output are still valid.
211 However, with a per-recipe approach, the build would not include the 211 However, with a per-recipe approach, the build would not include the
212 <filename>.deb</filename> files. 212 <filename>.deb</filename> files.
213 Consequently, you would have to invalidate the whole build and rerun it. 213 Consequently, you would have to invalidate the whole build and rerun it.
214 Rerunning everything is not the best situation. 214 Rerunning everything is not the best situation.
215 Also in this case, the core must be "taught" much about specific tasks. 215 Also in this case, the core must be "taught" much about specific tasks.
216 This methodology does not scale well and does not allow users to easily add new tasks 216 This methodology does not scale well and does not allow users to easily add new tasks
217 in layers or as external recipes without touching the packaged-staging core. 217 in layers or as external recipes without touching the packaged-staging core.
218 </para> 218 </para>
219 </section> 219 </section>
@@ -222,37 +222,37 @@
222 <title>Checksums (Signatures)</title> 222 <title>Checksums (Signatures)</title>
223 223
224 <para> 224 <para>
225 The shared state code uses a checksum, which is a unique signature of a task's 225 The shared state code uses a checksum, which is a unique signature of a task's
226 inputs, to determine if a task needs to be run again. 226 inputs, to determine if a task needs to be run again.
227 Because it is a change in a task's inputs that triggers a rerun, the process 227 Because it is a change in a task's inputs that triggers a rerun, the process
228 needs to detect all the inputs to a given task. 228 needs to detect all the inputs to a given task.
229 For shell tasks, this turns out to be fairly easy because 229 For shell tasks, this turns out to be fairly easy because
230 the build process generates a "run" shell script for each task and 230 the build process generates a "run" shell script for each task and
231 it is possible to create a checksum that gives you a good idea of when 231 it is possible to create a checksum that gives you a good idea of when
232 the task's data changes. 232 the task's data changes.
233 </para> 233 </para>
234 234
235 <para> 235 <para>
236 To complicate the problem, there are things that should not be included in 236 To complicate the problem, there are things that should not be included in
237 the checksum. 237 the checksum.
238 First, there is the actual specific build path of a given task - 238 First, there is the actual specific build path of a given task -
239 the <filename>WORKDIR</filename>. 239 the <filename>WORKDIR</filename>.
240 It does not matter if the working directory changes because it should not 240 It does not matter if the working directory changes because it should not
241 affect the output for target packages. 241 affect the output for target packages.
242 Also, the build process has the objective of making native/cross packages relocatable. 242 Also, the build process has the objective of making native/cross packages relocatable.
243 The checksum therefore needs to exclude <filename>WORKDIR</filename>. 243 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
244 The simplistic approach for excluding the working directory is to set 244 The simplistic approach for excluding the working directory is to set
245 <filename>WORKDIR</filename> to some fixed value and create the checksum 245 <filename>WORKDIR</filename> to some fixed value and create the checksum
246 for the "run" script. 246 for the "run" script.
247 </para> 247 </para>
248 248
249 <para> 249 <para>
250 Another problem results from the "run" scripts containing functions that 250 Another problem results from the "run" scripts containing functions that
251 might or might not get called. 251 might or might not get called.
252 The incremental build solution contains code that figures out dependencies 252 The incremental build solution contains code that figures out dependencies
253 between shell functions. 253 between shell functions.
254 This code is used to prune the "run" scripts down to the minimum set, 254 This code is used to prune the "run" scripts down to the minimum set,
255 thereby alleviating this problem and making the "run" scripts much more 255 thereby alleviating this problem and making the "run" scripts much more
256 readable as a bonus. 256 readable as a bonus.
257 </para> 257 </para>
258 258
@@ -260,62 +260,62 @@
260 So far we have solutions for shell scripts. 260 So far we have solutions for shell scripts.
261 What about python tasks? 261 What about python tasks?
262 The same approach applies even though these tasks are more difficult. 262 The same approach applies even though these tasks are more difficult.
263 The process needs to figure out what variables a python function accesses 263 The process needs to figure out what variables a python function accesses
264 and what functions it calls. 264 and what functions it calls.
265 Again, the incremental build solution contains code that first figures out 265 Again, the incremental build solution contains code that first figures out
266 the variable and function dependencies, and then creates a checksum for the data 266 the variable and function dependencies, and then creates a checksum for the data
267 used as the input to the task. 267 used as the input to the task.
268 </para> 268 </para>
269 269
270 <para> 270 <para>
271 Like the <filename>WORKDIR</filename> case, situations exist where dependencies 271 Like the <filename>WORKDIR</filename> case, situations exist where dependencies
272 should be ignored. 272 should be ignored.
273 For these cases, you can instruct the build process to ignore a dependency 273 For these cases, you can instruct the build process to ignore a dependency
274 by using a line like the following: 274 by using a line like the following:
275 <literallayout class='monospaced'> 275 <literallayout class='monospaced'>
276 PACKAGE_ARCHS[vardepsexclude] = "MACHINE" 276 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
277 </literallayout> 277 </literallayout>
278 This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not 278 This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not
279 depend on the value of <filename>MACHINE</filename>, even if it does reference it. 279 depend on the value of <filename>MACHINE</filename>, even if it does reference it.
280 </para> 280 </para>
281 281
282 <para> 282 <para>
283 Equally, there are cases where we need to add dependencies BitBake is not able to find. 283 Equally, there are cases where we need to add dependencies BitBake is not able to find.
284 You can accomplish this by using a line like the following: 284 You can accomplish this by using a line like the following:
285 <literallayout class='monospaced'> 285 <literallayout class='monospaced'>
286 PACKAGE_ARCHS[vardeps] = "MACHINE" 286 PACKAGE_ARCHS[vardeps] = "MACHINE"
287 </literallayout> 287 </literallayout>
288 This example explicitly adds the <filename>MACHINE</filename> variable as a 288 This example explicitly adds the <filename>MACHINE</filename> variable as a
289 dependency for <filename>PACKAGE_ARCHS</filename>. 289 dependency for <filename>PACKAGE_ARCHS</filename>.
290 </para> 290 </para>
291 291
292 <para> 292 <para>
293 Consider a case with inline python, for example, where BitBake is not 293 Consider a case with inline python, for example, where BitBake is not
294 able to figure out dependencies. 294 able to figure out dependencies.
295 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake 295 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
296 produces output when it discovers something for which it cannot figure out 296 produces output when it discovers something for which it cannot figure out
297 dependencies. 297 dependencies.
298 The Yocto Project team has currently not managed to cover those dependencies 298 The Yocto Project team has currently not managed to cover those dependencies
299 in detail and is aware of the need to fix this situation. 299 in detail and is aware of the need to fix this situation.
300 </para> 300 </para>
301 301
302 <para> 302 <para>
303 Thus far, this section has limited discussion to the direct inputs into a task. 303 Thus far, this section has limited discussion to the direct inputs into a task.
304 Information based on direct inputs is referred to as the "basehash" in the 304 Information based on direct inputs is referred to as the "basehash" in the
305 code. 305 code.
306 However, there is still the question of a task's indirect inputs - the 306 However, there is still the question of a task's indirect inputs - the
307 things that were already built and present in the Build Directory. 307 things that were already built and present in the Build Directory.
308 The checksum (or signature) for a particular task needs to add the hashes 308 The checksum (or signature) for a particular task needs to add the hashes
309 of all the tasks on which the particular task depends. 309 of all the tasks on which the particular task depends.
310 Choosing which dependencies to add is a policy decision. 310 Choosing which dependencies to add is a policy decision.
311 However, the effect is to generate a master checksum that combines the basehash 311 However, the effect is to generate a master checksum that combines the basehash
312 and the hashes of the task's dependencies. 312 and the hashes of the task's dependencies.
313 </para> 313 </para>
314 314
315 <para> 315 <para>
316 At the code level, there are a variety of ways both the basehash and the 316 At the code level, there are a variety of ways both the basehash and the
317 dependent task hashes can be influenced. 317 dependent task hashes can be influenced.
318 Within the BitBake configuration file, we can give BitBake some extra information 318 Within the BitBake configuration file, we can give BitBake some extra information
319 to help it construct the basehash. 319 to help it construct the basehash.
320 The following statements effectively result in a list of global variable 320 The following statements effectively result in a list of global variable
321 dependency excludes - variables never included in any checksum: 321 dependency excludes - variables never included in any checksum:
@@ -325,42 +325,42 @@
325 BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER" 325 BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
326 BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET" 326 BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
327 </literallayout> 327 </literallayout>
328 The previous example actually excludes 328 The previous example actually excludes
329 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> 329 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
330 since it is actually constructed as a path within 330 since it is actually constructed as a path within
331 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on 331 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
332 the whitelist. 332 the whitelist.
333 </para> 333 </para>
334 334
335 <para> 335 <para>
336 The rules for deciding which hashes of dependent tasks to include through 336 The rules for deciding which hashes of dependent tasks to include through
337 dependency chains are more complex and are generally accomplished with a 337 dependency chains are more complex and are generally accomplished with a
338 python function. 338 python function.
339 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples 339 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
340 of this and also illustrates how you can insert your own policy into the system 340 of this and also illustrates how you can insert your own policy into the system
341 if so desired. 341 if so desired.
342 This file defines the two basic signature generators <filename>OE-Core</filename> 342 This file defines the two basic signature generators <filename>OE-Core</filename>
343 uses: "OEBasic" and "OEBasicHash". 343 uses: "OEBasic" and "OEBasicHash".
344 By default, there is a dummy "noop" signature handler enabled in BitBake. 344 By default, there is a dummy "noop" signature handler enabled in BitBake.
345 This means that behavior is unchanged from previous versions. 345 This means that behavior is unchanged from previous versions.
346 <filename>OE-Core</filename> uses the "OEBasic" signature handler by default 346 <filename>OE-Core</filename> uses the "OEBasic" signature handler by default
347 through this setting in the <filename>bitbake.conf</filename> file: 347 through this setting in the <filename>bitbake.conf</filename> file:
348 <literallayout class='monospaced'> 348 <literallayout class='monospaced'>
349 BB_SIGNATURE_HANDLER ?= "OEBasic" 349 BB_SIGNATURE_HANDLER ?= "OEBasic"
350 </literallayout> 350 </literallayout>
351 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the 351 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
352 "OEBasic" version but adds the task hash to the stamp files. 352 "OEBasic" version but adds the task hash to the stamp files.
353 This results in any metadata change that changes the task hash, automatically 353 This results in any metadata change that changes the task hash, automatically
354 causing the task to be run again. 354 causing the task to be run again.
355 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link> 355 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
356 values and changes to metadata automatically ripple across the build. 356 values and changes to metadata automatically ripple across the build.
357 Currently, this behavior is not the default behavior for <filename>OE-Core</filename> 357 Currently, this behavior is not the default behavior for <filename>OE-Core</filename>
358 but is the default in <filename>poky</filename>. 358 but is the default in <filename>poky</filename>.
359 </para> 359 </para>
360 360
361 <para> 361 <para>
362 It is also worth noting that the end result of these signature generators is to 362 It is also worth noting that the end result of these signature generators is to
363 make some dependency and hash information available to the build. 363 make some dependency and hash information available to the build.
364 This information includes: 364 This information includes:
365 <literallayout class='monospaced'> 365 <literallayout class='monospaced'>
366 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe 366 BB_BASEHASH_task-&lt;taskname&gt; - the base hashes for each task in the recipe
@@ -375,15 +375,15 @@
375 <title>Shared State</title> 375 <title>Shared State</title>
376 376
377 <para> 377 <para>
378 Checksums and dependencies, as discussed in the previous section, solve half the 378 Checksums and dependencies, as discussed in the previous section, solve half the
379 problem. 379 problem.
380 The other part of the problem is being able to use checksum information during the build 380 The other part of the problem is being able to use checksum information during the build
381 and being able to reuse or rebuild specific components. 381 and being able to reuse or rebuild specific components.
382 </para> 382 </para>
383 383
384 <para> 384 <para>
385 The shared state class (<filename>sstate.bbclass</filename>) 385 The shared state class (<filename>sstate.bbclass</filename>)
386 is a relatively generic implementation of how to "capture" a snapshot of a given task. 386 is a relatively generic implementation of how to "capture" a snapshot of a given task.
387 The idea is that the build process does not care about the source of a task's output. 387 The idea is that the build process does not care about the source of a task's output.
388 Output could be freshly built or it could be downloaded and unpacked from 388 Output could be freshly built or it could be downloaded and unpacked from
389 somewhere - the build process doesn't need to worry about its source. 389 somewhere - the build process doesn't need to worry about its source.
@@ -392,17 +392,17 @@
392 <para> 392 <para>
393 There are two types of output, one is just about creating a directory 393 There are two types of output, one is just about creating a directory
394 in <filename>WORKDIR</filename>. 394 in <filename>WORKDIR</filename>.
395 A good example is the output of either <filename>do_install</filename> or 395 A good example is the output of either <filename>do_install</filename> or
396 <filename>do_package</filename>. 396 <filename>do_package</filename>.
397 The other type of output occurs when a set of data is merged into a shared directory 397 The other type of output occurs when a set of data is merged into a shared directory
398 tree such as the sysroot. 398 tree such as the sysroot.
399 </para> 399 </para>
400 400
401 <para> 401 <para>
402 The Yocto Project team has tried to keep the details of the implementation hidden in 402 The Yocto Project team has tried to keep the details of the implementation hidden in
403 <filename>sstate.bbclass</filename>. 403 <filename>sstate.bbclass</filename>.
404 From a user's perspective, adding shared state wrapping to a task 404 From a user's perspective, adding shared state wrapping to a task
405 is as simple as this <filename>do_deploy</filename> example taken from 405 is as simple as this <filename>do_deploy</filename> example taken from
406 <filename>do_deploy.bbclass</filename>: 406 <filename>do_deploy.bbclass</filename>:
407 <literallayout class='monospaced'> 407 <literallayout class='monospaced'>
408 DEPLOYDIR = "${WORKDIR}/deploy-${PN}" 408 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
@@ -418,13 +418,13 @@
418 </literallayout> 418 </literallayout>
419 In the example, we add some extra flags to the task, a name field ("deploy"), an 419 In the example, we add some extra flags to the task, a name field ("deploy"), an
420 input directory where the task sends data, and the output 420 input directory where the task sends data, and the output
421 directory where the data from the task should eventually be copied. 421 directory where the data from the task should eventually be copied.
422 We also add a <filename>_setscene</filename> variant of the task and add the task 422 We also add a <filename>_setscene</filename> variant of the task and add the task
423 name to the <filename>SSTATETASKS</filename> list. 423 name to the <filename>SSTATETASKS</filename> list.
424 </para> 424 </para>
425 425
426 <para> 426 <para>
427 If you have a directory whose contents you need to preserve, you can do this with 427 If you have a directory whose contents you need to preserve, you can do this with
428 a line like the following: 428 a line like the following:
429 <literallayout class='monospaced'> 429 <literallayout class='monospaced'>
430 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" 430 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
@@ -441,21 +441,21 @@
441 </para> 441 </para>
442 442
443 <para> 443 <para>
444 Behind the scenes, the shared state code works by looking in 444 Behind the scenes, the shared state code works by looking in
445 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and 445 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
446 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link> 446 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
447 for shared state files. 447 for shared state files.
448 Here is an example: 448 Here is an example:
449 <literallayout class='monospaced'> 449 <literallayout class='monospaced'>
450 SSTATE_MIRRORS ?= "\ 450 SSTATE_MIRRORS ?= "\
451 file://.* http://someserver.tld/share/sstate/PATH \n \ 451 file://.* http://someserver.tld/share/sstate/PATH \n \
452 file://.* file:///some/local/dir/sstate/PATH" 452 file://.* file:///some/local/dir/sstate/PATH"
453 </literallayout> 453 </literallayout>
454 <note> 454 <note>
455 The shared state directory (<filename>SSTATE_DIR</filename>) is 455 The shared state directory (<filename>SSTATE_DIR</filename>) is
456 organized into two-character subdirectories, where the subdirectory 456 organized into two-character subdirectories, where the subdirectory
457 names are based on the first two characters of the hash. 457 names are based on the first two characters of the hash.
458 If the shared state directory structure for a mirror has the 458 If the shared state directory structure for a mirror has the
459 same structure as <filename>SSTATE_DIR</filename>, you must 459 same structure as <filename>SSTATE_DIR</filename>, you must
460 specify "PATH" as part of the URI to enable the build system 460 specify "PATH" as part of the URI to enable the build system
461 to map to the appropriate subdirectory. 461 to map to the appropriate subdirectory.
@@ -465,8 +465,8 @@
465 <para> 465 <para>
466 The shared state package validity can be detected just by looking at the 466 The shared state package validity can be detected just by looking at the
467 filename since the filename contains the task checksum (or signature) as 467 filename since the filename contains the task checksum (or signature) as
468 described earlier in this section. 468 described earlier in this section.
469 If a valid shared state package is found, the build process downloads it 469 If a valid shared state package is found, the build process downloads it
470 and uses it to accelerate the task. 470 and uses it to accelerate the task.
471 </para> 471 </para>
472 472
@@ -474,19 +474,19 @@
474 The build processes uses the <filename>*_setscene</filename> tasks 474 The build processes uses the <filename>*_setscene</filename> tasks
475 for the task acceleration phase. 475 for the task acceleration phase.
476 BitBake goes through this phase before the main execution code and tries 476 BitBake goes through this phase before the main execution code and tries
477 to accelerate any tasks for which it can find shared state packages. 477 to accelerate any tasks for which it can find shared state packages.
478 If a shared state package for a task is available, the shared state 478 If a shared state package for a task is available, the shared state
479 package is used. 479 package is used.
480 This means the task and any tasks on which it is dependent are not 480 This means the task and any tasks on which it is dependent are not
481 executed. 481 executed.
482 </para> 482 </para>
483 483
484 <para> 484 <para>
485 As a real world example, the aim is when building an IPK-based image, 485 As a real world example, the aim is when building an IPK-based image,
486 only the <filename>do_package_write_ipk</filename> tasks would have their 486 only the <filename>do_package_write_ipk</filename> tasks would have their
487 shared state packages fetched and extracted. 487 shared state packages fetched and extracted.
488 Since the sysroot is not used, it would never get extracted. 488 Since the sysroot is not used, it would never get extracted.
489 This is another reason why a task-based approach is preferred over a 489 This is another reason why a task-based approach is preferred over a
490 recipe-based approach, which would have to install the output from every task. 490 recipe-based approach, which would have to install the output from every task.
491 </para> 491 </para>
492 </section> 492 </section>
@@ -495,9 +495,9 @@
495 <title>Tips and Tricks</title> 495 <title>Tips and Tricks</title>
496 496
497 <para> 497 <para>
498 The code in the build system that supports incremental builds is not 498 The code in the build system that supports incremental builds is not
499 simple code. 499 simple code.
500 This section presents some tips and tricks that help you work around 500 This section presents some tips and tricks that help you work around
501 issues related to shared state code. 501 issues related to shared state code.
502 </para> 502 </para>
503 503
@@ -505,25 +505,25 @@
505 <title>Debugging</title> 505 <title>Debugging</title>
506 506
507 <para> 507 <para>
508 When things go wrong, debugging needs to be straightforward. 508 When things go wrong, debugging needs to be straightforward.
509 Because of this, the Yocto Project team included strong debugging 509 Because of this, the Yocto Project team included strong debugging
510 tools: 510 tools:
511 <itemizedlist> 511 <itemizedlist>
512 <listitem><para>Whenever a shared state package is written out, so is a 512 <listitem><para>Whenever a shared state package is written out, so is a
513 corresponding <filename>.siginfo</filename> file. 513 corresponding <filename>.siginfo</filename> file.
514 This practice results in a pickled python database of all 514 This practice results in a pickled python database of all
515 the metadata that went into creating the hash for a given shared state 515 the metadata that went into creating the hash for a given shared state
516 package.</para></listitem> 516 package.</para></listitem>
517 <listitem><para>If BitBake is run with the <filename>--dump-signatures</filename> 517 <listitem><para>If BitBake is run with the <filename>--dump-signatures</filename>
518 (or <filename>-S</filename>) option, BitBake dumps out 518 (or <filename>-S</filename>) option, BitBake dumps out
519 <filename>.siginfo</filename> files in 519 <filename>.siginfo</filename> files in
520 the stamp directory for every task it would have executed instead of 520 the stamp directory for every task it would have executed instead of
521 building the specified target package.</para></listitem> 521 building the specified target package.</para></listitem>
522 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that 522 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
523 can process these <filename>.siginfo</filename> files. 523 can process these <filename>.siginfo</filename> files.
524 If one file is specified, it will dump out the dependency 524 If one file is specified, it will dump out the dependency
525 information in the file. 525 information in the file.
526 If two files are specified, it will compare the two files and dump out 526 If two files are specified, it will compare the two files and dump out
527 the differences between the two. 527 the differences between the two.
528 This allows the question of "What changed between X and Y?" to be 528 This allows the question of "What changed between X and Y?" to be
529 answered easily.</para></listitem> 529 answered easily.</para></listitem>
@@ -538,41 +538,41 @@
538 The shared state code uses checksums and shared state 538 The shared state code uses checksums and shared state
539 cache to avoid unnecessarily rebuilding tasks. 539 cache to avoid unnecessarily rebuilding tasks.
540 As with all schemes, this one has some drawbacks. 540 As with all schemes, this one has some drawbacks.
541 It is possible that you could make implicit changes that are not factored 541 It is possible that you could make implicit changes that are not factored
542 into the checksum calculation, but do affect a task's output. 542 into the checksum calculation, but do affect a task's output.
543 A good example is perhaps when a tool changes its output. 543 A good example is perhaps when a tool changes its output.
544 Let's say that the output of <filename>rpmdeps</filename> needed to change. 544 Let's say that the output of <filename>rpmdeps</filename> needed to change.
545 The result of the change should be that all the "package", "package_write_rpm", 545 The result of the change should be that all the "package", "package_write_rpm",
546 and "package_deploy-rpm" shared state cache items would become invalid. 546 and "package_deploy-rpm" shared state cache items would become invalid.
547 But, because this is a change that is external to the code and therefore implicit, 547 But, because this is a change that is external to the code and therefore implicit,
548 the associated shared state cache items do not become invalidated. 548 the associated shared state cache items do not become invalidated.
549 In this case, the build process would use the cached items rather than running the 549 In this case, the build process would use the cached items rather than running the
550 task again. 550 task again.
551 Obviously, these types of implicit changes can cause problems. 551 Obviously, these types of implicit changes can cause problems.
552 </para> 552 </para>
553 553
554 <para> 554 <para>
555 To avoid these problems during the build, you need to understand the effects of any 555 To avoid these problems during the build, you need to understand the effects of any
556 change you make. 556 change you make.
557 Note that any changes you make directly to a function automatically are factored into 557 Note that any changes you make directly to a function automatically are factored into
558 the checksum calculation and thus, will invalidate the associated area of sstate cache. 558 the checksum calculation and thus, will invalidate the associated area of sstate cache.
559 You need to be aware of any implicit changes that are not obvious changes to the 559 You need to be aware of any implicit changes that are not obvious changes to the
560 code and could affect the output of a given task. 560 code and could affect the output of a given task.
561 Once you are aware of such a change, you can take steps to invalidate the cache 561 Once you are aware of such a change, you can take steps to invalidate the cache
562 and force the task to run. 562 and force the task to run.
563 The step to take is as simple as changing a function's comments in the source code. 563 The step to take is as simple as changing a function's comments in the source code.
564 For example, to invalidate package shared state files, change the comment statements 564 For example, to invalidate package shared state files, change the comment statements
565 of <filename>do_package</filename> or the comments of one of the functions it calls. 565 of <filename>do_package</filename> or the comments of one of the functions it calls.
566 The change is purely cosmetic, but it causes the checksum to be recalculated and 566 The change is purely cosmetic, but it causes the checksum to be recalculated and
567 forces the task to be run again. 567 forces the task to be run again.
568 </para> 568 </para>
569 569
570 <note> 570 <note>
571 For an example of a commit that makes a cosmetic change to invalidate 571 For an example of a commit that makes a cosmetic change to invalidate
572 a shared state, see this 572 a shared state, see this
573 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. 573 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
574 </note> 574 </note>
575 </section> 575 </section>
576 </section> 576 </section>
577</section> 577</section>
578 578
@@ -580,24 +580,24 @@
580 <title>x32</title> 580 <title>x32</title>
581 581
582 <para> 582 <para>
583 x32 is a new processor-specific Application Binary Interface (psABI) for x86_64. 583 x32 is a new processor-specific Application Binary Interface (psABI) for x86_64.
584 An ABI defines the calling conventions between functions in a processing environment. 584 An ABI defines the calling conventions between functions in a processing environment.
585 The interface determines what registers are used and what the sizes are for various C data types. 585 The interface determines what registers are used and what the sizes are for various C data types.
586 </para> 586 </para>
587 587
588 <para> 588 <para>
589 Some processing environments prefer using 32-bit applications even when running 589 Some processing environments prefer using 32-bit applications even when running
590 on Intel 64-bit platforms. 590 on Intel 64-bit platforms.
591 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms. 591 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
592 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources, 592 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
593 leaving the system underutilized. 593 leaving the system underutilized.
594 Now consider the x86_64 psABI. 594 Now consider the x86_64 psABI.
595 This ABI is newer and uses 64-bits for data sizes and program pointers. 595 This ABI is newer and uses 64-bits for data sizes and program pointers.
596 The extra bits increase the footprint size of the programs, libraries, 596 The extra bits increase the footprint size of the programs, libraries,
597 and also increases the memory and file system size requirements. 597 and also increases the memory and file system size requirements.
598 Executing under the x32 psABI enables user programs to utilize CPU and system resources 598 Executing under the x32 psABI enables user programs to utilize CPU and system resources
599 more efficiently while keeping the memory footprint of the applications low. 599 more efficiently while keeping the memory footprint of the applications low.
600 Extra bits are used for registers but not for addressing mechanisms. 600 Extra bits are used for registers but not for addressing mechanisms.
601 </para> 601 </para>
602 602
603 <section id='support'> 603 <section id='support'>
@@ -608,14 +608,14 @@
608 release supports current development specifications of x32 psABI. 608 release supports current development specifications of x32 psABI.
609 As of this release of the Yocto Project, x32 psABI support exists as follows: 609 As of this release of the Yocto Project, x32 psABI support exists as follows:
610 <itemizedlist> 610 <itemizedlist>
611 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets. 611 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
612 </para></listitem> 612 </para></listitem>
613 <listitem><para>You can use the x32 psABI support through the <filename>meta-x32</filename> 613 <listitem><para>You can use the x32 psABI support through the <filename>meta-x32</filename>
614 layer on top of the OE-core/Yocto layer.</para></listitem> 614 layer on top of the OE-core/Yocto layer.</para></listitem>
615 <listitem><para>The toolchain from the <filename>experimental/meta-x32</filename> layer 615 <listitem><para>The toolchain from the <filename>experimental/meta-x32</filename> layer
616 is used for building x32 psABI program binaries.</para></listitem> 616 is used for building x32 psABI program binaries.</para></listitem>
617 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem> 617 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
618 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and 618 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
619 <filename>core-image-sato</filename> images.</para></listitem> 619 <filename>core-image-sato</filename> images.</para></listitem>
620 </itemizedlist> 620 </itemizedlist>
621 </para> 621 </para>
@@ -625,18 +625,18 @@
625 <title>Future Development and Limitations</title> 625 <title>Future Development and Limitations</title>
626 626
627 <para> 627 <para>
628 As of this Yocto Project release, the x32 psABI kernel and library interfaces 628 As of this Yocto Project release, the x32 psABI kernel and library interfaces
629 specifications are not finalized. 629 specifications are not finalized.
630 </para> 630 </para>
631 631
632 <para> 632 <para>
633 Future Plans for the x32 psABI in the Yocto Project include the following: 633 Future Plans for the x32 psABI in the Yocto Project include the following:
634 <itemizedlist> 634 <itemizedlist>
635 <listitem><para>Enhance and fix the few remaining recipes so they 635 <listitem><para>Enhance and fix the few remaining recipes so they
636 work with and support x32 toolchains.</para></listitem> 636 work with and support x32 toolchains.</para></listitem>
637 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem> 637 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
638 <listitem><para>Support larger images.</para></listitem> 638 <listitem><para>Support larger images.</para></listitem>
639 <listitem><para>Integrate x32 recipes, toolchain, and kernel changes from 639 <listitem><para>Integrate x32 recipes, toolchain, and kernel changes from
640 <filename>experimental/meta-x32</filename> into OE-core.</para></listitem> 640 <filename>experimental/meta-x32</filename> into OE-core.</para></listitem>
641 </itemizedlist> 641 </itemizedlist>
642 </para> 642 </para>
@@ -650,7 +650,7 @@
650 Yocto Project, you can follow these steps to use the x32 spABI: 650 Yocto Project, you can follow these steps to use the x32 spABI:
651 <itemizedlist> 651 <itemizedlist>
652 <listitem><para>Add the <filename>experimental/meta-x32</filename> layer to your local 652 <listitem><para>Add the <filename>experimental/meta-x32</filename> layer to your local
653 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. 653 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
654 You can find the <filename>experimental/meta-x32</filename> source repository at 654 You can find the <filename>experimental/meta-x32</filename> source repository at
655 <ulink url='&YOCTO_GIT_URL;'></ulink>.</para></listitem> 655 <ulink url='&YOCTO_GIT_URL;'></ulink>.</para></listitem>
656 <listitem><para>Edit your <filename>conf/bblayers.conf</filename> file so that it includes 656 <listitem><para>Edit your <filename>conf/bblayers.conf</filename> file so that it includes
@@ -677,8 +677,8 @@
677 or 'INVALID'), True) or 'lib'}" 677 or 'INVALID'), True) or 'lib'}"
678 #MACHINE = "atom-pc" 678 #MACHINE = "atom-pc"
679 #DEFAULTTUNE = "core2-64-x32" 679 #DEFAULTTUNE = "core2-64-x32"
680 </literallayout></para></listitem> 680 </literallayout></para></listitem>
681 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI. 681 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
682 Here is an example: 682 Here is an example:
683 <literallayout class='monospaced'> 683 <literallayout class='monospaced'>
684 $ bitake core-image-sato 684 $ bitake core-image-sato
@@ -696,15 +696,15 @@
696 <title>Licenses</title> 696 <title>Licenses</title>
697 697
698 <para> 698 <para>
699 This section describes the mechanism by which the OpenEmbedded build system 699 This section describes the mechanism by which the OpenEmbedded build system
700 tracks changes to licensing text. 700 tracks changes to licensing text.
701 The section also describes how to enable commercially licensed recipes, 701 The section also describes how to enable commercially licensed recipes,
702 which by default are disabled. 702 which by default are disabled.
703 </para> 703 </para>
704 704
705 <para> 705 <para>
706 For information that can help you maintain compliance with various open 706 For information that can help you maintain compliance with various open
707 source licensing during the lifecycle of the product, see the 707 source licensing during the lifecycle of the product, see the
708 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section 708 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
709 in the Yocto Project Development Manual. 709 in the Yocto Project Development Manual.
710 </para> 710 </para>
@@ -713,8 +713,8 @@
713 <title>Tracking License Changes</title> 713 <title>Tracking License Changes</title>
714 714
715 <para> 715 <para>
716 The license of an upstream project might change in the future. 716 The license of an upstream project might change in the future.
717 In order to prevent these changes going unnoticed, the 717 In order to prevent these changes going unnoticed, the
718 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename> 718 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
719 variable tracks changes to the license text. The checksums are validated at the end of the 719 variable tracks changes to the license text. The checksums are validated at the end of the
720 configure step, and if the checksums do not match, the build will fail. 720 configure step, and if the checksums do not match, the build will fail.
@@ -736,15 +736,15 @@
736 </para> 736 </para>
737 737
738 <para> 738 <para>
739 The build system uses the 739 The build system uses the
740 <filename><link linkend='var-S'>S</link></filename> variable as the 740 <filename><link linkend='var-S'>S</link></filename> variable as the
741 default directory used when searching files listed in 741 default directory used when searching files listed in
742 <filename>LIC_FILES_CHKSUM</filename>. 742 <filename>LIC_FILES_CHKSUM</filename>.
743 The previous example employs the default directory. 743 The previous example employs the default directory.
744 </para> 744 </para>
745 745
746 <para> 746 <para>
747 You can also use relative paths as shown in the following example: 747 You can also use relative paths as shown in the following example:
748 <literallayout class='monospaced'> 748 <literallayout class='monospaced'>
749 LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\ 749 LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\
750 md5=bb14ed3c4cda583abc85401304b5cd4e" 750 md5=bb14ed3c4cda583abc85401304b5cd4e"
@@ -753,14 +753,14 @@
753 </para> 753 </para>
754 754
755 <para> 755 <para>
756 In this example, the first line locates a file in 756 In this example, the first line locates a file in
757 <filename>${S}/src/ls.c</filename>. 757 <filename>${S}/src/ls.c</filename>.
758 The second line refers to a file in 758 The second line refers to a file in
759 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, which is the parent 759 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, which is the parent
760 of <filename><link linkend='var-S'>S</link></filename>. 760 of <filename><link linkend='var-S'>S</link></filename>.
761 </para> 761 </para>
762 <para> 762 <para>
763 Note that this variable is mandatory for all recipes, unless the 763 Note that this variable is mandatory for all recipes, unless the
764 <filename>LICENSE</filename> variable is set to "CLOSED". 764 <filename>LICENSE</filename> variable is set to "CLOSED".
765 </para> 765 </para>
766 </section> 766 </section>
@@ -768,48 +768,48 @@
768 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax"> 768 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
769 <title>Explanation of Syntax</title> 769 <title>Explanation of Syntax</title>
770 <para> 770 <para>
771 As mentioned in the previous section, the 771 As mentioned in the previous section, the
772 <filename>LIC_FILES_CHKSUM</filename> variable lists all the 772 <filename>LIC_FILES_CHKSUM</filename> variable lists all the
773 important files that contain the license text for the source code. 773 important files that contain the license text for the source code.
774 It is possible to specify a checksum for an entire file, or a specific section of a 774 It is possible to specify a checksum for an entire file, or a specific section of a
775 file (specified by beginning and ending line numbers with the "beginline" and "endline" 775 file (specified by beginning and ending line numbers with the "beginline" and "endline"
776 parameters, respectively). 776 parameters, respectively).
777 The latter is useful for source files with a license notice header, 777 The latter is useful for source files with a license notice header,
778 README documents, and so forth. 778 README documents, and so forth.
779 If you do not use the "beginline" parameter, then it is assumed that the text begins on the 779 If you do not use the "beginline" parameter, then it is assumed that the text begins on the
780 first line of the file. 780 first line of the file.
781 Similarly, if you do not use the "endline" parameter, it is assumed that the license text 781 Similarly, if you do not use the "endline" parameter, it is assumed that the license text
782 ends with the last line of the file. 782 ends with the last line of the file.
783 </para> 783 </para>
784 784
785 <para> 785 <para>
786 The "md5" parameter stores the md5 checksum of the license text. 786 The "md5" parameter stores the md5 checksum of the license text.
787 If the license text changes in any way as compared to this parameter 787 If the license text changes in any way as compared to this parameter
788 then a mismatch occurs. 788 then a mismatch occurs.
789 This mismatch triggers a build failure and notifies the developer. 789 This mismatch triggers a build failure and notifies the developer.
790 Notification allows the developer to review and address the license text changes. 790 Notification allows the developer to review and address the license text changes.
791 Also note that if a mismatch occurs during the build, the correct md5 791 Also note that if a mismatch occurs during the build, the correct md5
792 checksum is placed in the build log and can be easily copied to the recipe. 792 checksum is placed in the build log and can be easily copied to the recipe.
793 </para> 793 </para>
794 794
795 <para> 795 <para>
796 There is no limit to how many files you can specify using the 796 There is no limit to how many files you can specify using the
797 <filename>LIC_FILES_CHKSUM</filename> variable. 797 <filename>LIC_FILES_CHKSUM</filename> variable.
798 Generally, however, every project requires a few specifications for license tracking. 798 Generally, however, every project requires a few specifications for license tracking.
799 Many projects have a "COPYING" file that stores the license information for all the source 799 Many projects have a "COPYING" file that stores the license information for all the source
800 code files. 800 code files.
801 This practice allows you to just track the "COPYING" file as long as it is kept up to date. 801 This practice allows you to just track the "COPYING" file as long as it is kept up to date.
802 </para> 802 </para>
803 803
804 <tip> 804 <tip>
805 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match 805 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
806 error and displays the correct "md5" parameter value during the build. 806 error and displays the correct "md5" parameter value during the build.
807 The correct parameter is also captured in the build log. 807 The correct parameter is also captured in the build log.
808 </tip> 808 </tip>
809 809
810 <tip> 810 <tip>
811 If the whole file contains only license text, you do not need to use the "beginline" and 811 If the whole file contains only license text, you do not need to use the "beginline" and
812 "endline" parameters. 812 "endline" parameters.
813 </tip> 813 </tip>
814 </section> 814 </section>
815 </section> 815 </section>
@@ -820,10 +820,10 @@
820 <para> 820 <para>
821 By default, the OpenEmbedded build system disables 821 By default, the OpenEmbedded build system disables
822 components that have commercial or other special licensing 822 components that have commercial or other special licensing
823 requirements. 823 requirements.
824 Such requirements are defined on a 824 Such requirements are defined on a
825 recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable 825 recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable
826 definition in the affected recipe. 826 definition in the affected recipe.
827 For instance, the 827 For instance, the
828 <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> 828 <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
829 recipe contains the following statement: 829 recipe contains the following statement:
@@ -839,7 +839,7 @@
839 definition to be enabled and included in an image, it 839 definition to be enabled and included in an image, it
840 needs to have a matching entry in the global 840 needs to have a matching entry in the global
841 <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable 841 <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable
842 typically defined in your <filename>local.conf</filename> file. 842 typically defined in your <filename>local.conf</filename> file.
843 For example, to enable 843 For example, to enable
844 the <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> 844 the <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
845 package, you could add either the string 845 package, you could add either the string
@@ -867,7 +867,7 @@
867 the initial underscore character or characters. 867 the initial underscore character or characters.
868 A partial string will match 868 A partial string will match
869 any license that contains the given string as the first 869 any license that contains the given string as the first
870 portion of its license. 870 portion of its license.
871 For example, the following 871 For example, the following
872 whitelist string will also match both of the packages 872 whitelist string will also match both of the packages
873 previously mentioned as well as any other packages that have 873 previously mentioned as well as any other packages that have
@@ -879,7 +879,7 @@
879 879
880 <section id="license-flag-matching"> 880 <section id="license-flag-matching">
881 <title>License Flag Matching</title> 881 <title>License Flag Matching</title>
882 882
883 <para> 883 <para>
884 The definition of 'matching' in reference to a 884 The definition of 'matching' in reference to a
885 recipe's <filename>LICENSE_FLAGS</filename> setting is simple. 885 recipe's <filename>LICENSE_FLAGS</filename> setting is simple.
@@ -891,7 +891,7 @@
891 Before a flag 891 Before a flag
892 defined by a particular recipe is tested against the 892 defined by a particular recipe is tested against the
893 contents of the <filename>LICENSE_FLAGS_WHITELIST</filename> variable, the 893 contents of the <filename>LICENSE_FLAGS_WHITELIST</filename> variable, the
894 string <filename>_${PN}</filename> (with 894 string <filename>_${PN}</filename> (with
895 <link linkend='var-PN'><filename>PN</filename></link> expanded of course) is 895 <link linkend='var-PN'><filename>PN</filename></link> expanded of course) is
896 appended to the flag, thus automatically making each 896 appended to the flag, thus automatically making each
897 <filename>LICENSE_FLAGS</filename> value recipe-specific. 897 <filename>LICENSE_FLAGS</filename> value recipe-specific.
@@ -907,7 +907,7 @@
907 You can broaden the match by 907 You can broaden the match by
908 putting any "_"-separated beginning subset of a 908 putting any "_"-separated beginning subset of a
909 <filename>LICENSE_FLAGS</filename> flag in the whitelist, which will also 909 <filename>LICENSE_FLAGS</filename> flag in the whitelist, which will also
910 match. 910 match.
911 For example, simply specifying "commercial" in 911 For example, simply specifying "commercial" in
912 the whitelist would match any expanded <filename>LICENSE_FLAGS</filename> 912 the whitelist would match any expanded <filename>LICENSE_FLAGS</filename>
913 definition starting with "commercial" such as 913 definition starting with "commercial" such as
@@ -923,7 +923,7 @@
923 <para> 923 <para>
924 Broadening the match allows for a range of specificity for the items 924 Broadening the match allows for a range of specificity for the items
925 in the whitelist, from more general to perfectly 925 in the whitelist, from more general to perfectly
926 specific. 926 specific.
927 So you have the choice of exhaustively 927 So you have the choice of exhaustively
928 enumerating each license flag in the whitelist to 928 enumerating each license flag in the whitelist to
929 allow only those specific recipes into the image, or 929 allow only those specific recipes into the image, or
@@ -935,7 +935,7 @@
935 <para> 935 <para>
936 This scheme works even if the flag already 936 This scheme works even if the flag already
937 has <filename>_${PN}</filename> appended - the extra <filename>_${PN}</filename> is 937 has <filename>_${PN}</filename> appended - the extra <filename>_${PN}</filename> is
938 redundant, but does not affect the outcome. 938 redundant, but does not affect the outcome.
939 For example, a license flag of "commercial_1.2_foo" would 939 For example, a license flag of "commercial_1.2_foo" would
940 turn into "commercial_1.2_foo_foo" and would match 940 turn into "commercial_1.2_foo_foo" and would match
941 both the general "commercial" and the specific 941 both the general "commercial" and the specific
@@ -944,8 +944,8 @@
944 "commercial_1.2_foo_foo" and "commercial_1.2", which 944 "commercial_1.2_foo_foo" and "commercial_1.2", which
945 does not make much sense regarding use in the whitelist. 945 does not make much sense regarding use in the whitelist.
946 </para> 946 </para>
947 947
948 <para> 948 <para>
949 For a versioned string, you could instead specify 949 For a versioned string, you could instead specify
950 "commercial_foo_1.2", which would turn into 950 "commercial_foo_1.2", which would turn into
951 "commercial_foo_1.2_foo". 951 "commercial_foo_1.2_foo".
@@ -993,8 +993,8 @@
993 </para> 993 </para>
994 994
995 <para> 995 <para>
996 Specifying audio and video plug-ins as part of the 996 Specifying audio and video plug-ins as part of the
997 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and 997 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
998 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements 998 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
999 or commercial qt components as part of 999 or commercial qt components as part of
1000 the <filename>COMMERCIAL_QT</filename> statement (along 1000 the <filename>COMMERCIAL_QT</filename> statement (along
@@ -1006,6 +1006,6 @@
1006 </section> 1006 </section>
1007</section> 1007</section>
1008</chapter> 1008</chapter>
1009<!-- 1009<!--
1010vim: expandtab tw=80 ts=4 1010vim: expandtab tw=80 ts=4
1011--> 1011-->