diff options
Diffstat (limited to 'documentation/poky-ref-manual/technical-details.xml')
-rw-r--r-- | documentation/poky-ref-manual/technical-details.xml | 434 |
1 files changed, 217 insertions, 217 deletions
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml index 042a96ed87..72689a0e22 100644 --- a/documentation/poky-ref-manual/technical-details.xml +++ b/documentation/poky-ref-manual/technical-details.xml | |||
@@ -6,7 +6,7 @@ | |||
6 | <title>Technical Details</title> | 6 | <title>Technical Details</title> |
7 | 7 | ||
8 | <para> | 8 | <para> |
9 | This chapter provides technical details for various parts of the Yocto Project. | 9 | This chapter provides technical details for various parts of the Yocto Project. |
10 | Currently, topics include Yocto Project components and shared state (sstate) cache. | 10 | Currently, topics include Yocto Project components and shared state (sstate) cache. |
11 | </para> | 11 | </para> |
12 | 12 | ||
@@ -14,21 +14,21 @@ | |||
14 | <title>Yocto Project Components</title> | 14 | <title>Yocto Project Components</title> |
15 | 15 | ||
16 | <para> | 16 | <para> |
17 | The BitBake task executor together with various types of configuration files form the | 17 | The BitBake task executor together with various types of configuration files form the |
18 | OpenEmbedded Core. | 18 | OpenEmbedded Core. |
19 | This section overviews the BitBake task executor and the | 19 | This section overviews the BitBake task executor and the |
20 | configuration files by describing what they are used for and how they interact. | 20 | configuration files by describing what they are used for and how they interact. |
21 | </para> | 21 | </para> |
22 | 22 | ||
23 | <para> | 23 | <para> |
24 | BitBake handles the parsing and execution of the data files. | 24 | BitBake handles the parsing and execution of the data files. |
25 | The data itself is of various types: | 25 | The data itself is of various types: |
26 | <itemizedlist> | 26 | <itemizedlist> |
27 | <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular | 27 | <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular |
28 | pieces of software</para></listitem> | 28 | pieces of software</para></listitem> |
29 | <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build | 29 | <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build |
30 | information (e.g. how to build a Linux kernel).</para></listitem> | 30 | information (e.g. how to build a Linux kernel).</para></listitem> |
31 | <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings, | 31 | <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings, |
32 | policy decisions, etc. | 32 | policy decisions, etc. |
33 | Configuration data acts as the glue to bind everything together.</para></listitem> | 33 | Configuration data acts as the glue to bind everything together.</para></listitem> |
34 | </itemizedlist> | 34 | </itemizedlist> |
@@ -37,17 +37,17 @@ | |||
37 | section in the Yocto Project Development Manual. | 37 | section in the Yocto Project Development Manual. |
38 | </para> | 38 | </para> |
39 | 39 | ||
40 | <para> | 40 | <para> |
41 | BitBake knows how to combine multiple data sources together and refers to each data source | 41 | BitBake knows how to combine multiple data sources together and refers to each data source |
42 | as a layer. | 42 | as a layer. |
43 | For information on layers, see the | 43 | For information on layers, see the |
44 | "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and | 44 | "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and |
45 | Creating Layers</ulink>" section of the Yocto Project Development Manual. | 45 | Creating Layers</ulink>" section of the Yocto Project Development Manual. |
46 | </para> | 46 | </para> |
47 | 47 | ||
48 | <para> | 48 | <para> |
49 | Following are some brief details on these core components. | 49 | Following are some brief details on these core components. |
50 | For more detailed information on these components see the | 50 | For more detailed information on these components see the |
51 | "<link linkend='ref-structure'>Directory Structure</link>" chapter. | 51 | "<link linkend='ref-structure'>Directory Structure</link>" chapter. |
52 | </para> | 52 | </para> |
53 | 53 | ||
@@ -57,7 +57,7 @@ | |||
57 | <para> | 57 | <para> |
58 | BitBake is the tool at the heart of the OpenEmbedded build system and is responsible | 58 | BitBake is the tool at the heart of the OpenEmbedded build system and is responsible |
59 | for parsing the metadata, generating a list of tasks from it, | 59 | for parsing the metadata, generating a list of tasks from it, |
60 | and then executing those tasks. | 60 | and then executing those tasks. |
61 | To see a list of the options BitBake supports, use the following help command: | 61 | To see a list of the options BitBake supports, use the following help command: |
62 | <literallayout class='monospaced'> | 62 | <literallayout class='monospaced'> |
63 | $ bitbake --help | 63 | $ bitbake --help |
@@ -66,8 +66,8 @@ | |||
66 | 66 | ||
67 | <para> | 67 | <para> |
68 | The most common usage for BitBake is <filename>bitbake <packagename></filename>, where | 68 | The most common usage for BitBake is <filename>bitbake <packagename></filename>, where |
69 | <filename>packagename</filename> is the name of the package you want to build | 69 | <filename>packagename</filename> is the name of the package you want to build |
70 | (referred to as the "target" in this manual). | 70 | (referred to as the "target" in this manual). |
71 | The target often equates to the first part of a <filename>.bb</filename> filename. | 71 | The target often equates to the first part of a <filename>.bb</filename> filename. |
72 | So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you | 72 | So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you |
73 | might type the following: | 73 | might type the following: |
@@ -76,15 +76,15 @@ | |||
76 | </literallayout> | 76 | </literallayout> |
77 | Several different versions of <filename>matchbox-desktop</filename> might exist. | 77 | Several different versions of <filename>matchbox-desktop</filename> might exist. |
78 | BitBake chooses the one selected by the distribution configuration. | 78 | BitBake chooses the one selected by the distribution configuration. |
79 | You can get more details about how BitBake chooses between different | 79 | You can get more details about how BitBake chooses between different |
80 | target versions and providers in the | 80 | target versions and providers in the |
81 | "<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section. | 81 | "<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section. |
82 | </para> | 82 | </para> |
83 | 83 | ||
84 | <para> | 84 | <para> |
85 | BitBake also tries to execute any dependent tasks first. | 85 | BitBake also tries to execute any dependent tasks first. |
86 | So for example, before building <filename>matchbox-desktop</filename>, BitBake | 86 | So for example, before building <filename>matchbox-desktop</filename>, BitBake |
87 | would build a cross compiler and <filename>eglibc</filename> if they had not already | 87 | would build a cross compiler and <filename>eglibc</filename> if they had not already |
88 | been built. | 88 | been built. |
89 | <note>This release of the Yocto Project does not support the <filename>glibc</filename> | 89 | <note>This release of the Yocto Project does not support the <filename>glibc</filename> |
90 | GNU version of the Unix standard C library. By default, the OpenEmbedded build system | 90 | GNU version of the Unix standard C library. By default, the OpenEmbedded build system |
@@ -92,12 +92,12 @@ | |||
92 | </para> | 92 | </para> |
93 | 93 | ||
94 | <para> | 94 | <para> |
95 | A useful BitBake option to consider is the <filename>-k</filename> or | 95 | A useful BitBake option to consider is the <filename>-k</filename> or |
96 | <filename>--continue</filename> option. | 96 | <filename>--continue</filename> option. |
97 | This option instructs BitBake to try and continue processing the job as much | 97 | This option instructs BitBake to try and continue processing the job as much |
98 | as possible even after encountering an error. | 98 | as possible even after encountering an error. |
99 | When an error occurs, the target that | 99 | When an error occurs, the target that |
100 | failed and those that depend on it cannot be remade. | 100 | failed and those that depend on it cannot be remade. |
101 | However, when you use this option other dependencies can still be processed. | 101 | However, when you use this option other dependencies can still be processed. |
102 | </para> | 102 | </para> |
103 | </section> | 103 | </section> |
@@ -106,17 +106,17 @@ | |||
106 | <title>Metadata (Recipes)</title> | 106 | <title>Metadata (Recipes)</title> |
107 | 107 | ||
108 | <para> | 108 | <para> |
109 | The <filename>.bb</filename> files are usually referred to as "recipes." | 109 | The <filename>.bb</filename> files are usually referred to as "recipes." |
110 | In general, a recipe contains information about a single piece of software. | 110 | In general, a recipe contains information about a single piece of software. |
111 | The information includes the location from which to download the source patches | 111 | The information includes the location from which to download the source patches |
112 | (if any are needed), which special configuration options to apply, | 112 | (if any are needed), which special configuration options to apply, |
113 | how to compile the source files, and how to package the compiled output. | 113 | how to compile the source files, and how to package the compiled output. |
114 | </para> | 114 | </para> |
115 | 115 | ||
116 | <para> | 116 | <para> |
117 | The term "package" can also be used to describe recipes. | 117 | The term "package" can also be used to describe recipes. |
118 | However, since the same word is used for the packaged output from the OpenEmbedded | 118 | However, since the same word is used for the packaged output from the OpenEmbedded |
119 | build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), | 119 | build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), |
120 | this document avoids using the term "package" when referring to recipes. | 120 | this document avoids using the term "package" when referring to recipes. |
121 | </para> | 121 | </para> |
122 | </section> | 122 | </section> |
@@ -126,7 +126,7 @@ | |||
126 | 126 | ||
127 | <para> | 127 | <para> |
128 | Class files (<filename>.bbclass</filename>) contain information that is useful to share | 128 | Class files (<filename>.bbclass</filename>) contain information that is useful to share |
129 | between metadata files. | 129 | between metadata files. |
130 | An example is the Autotools class, which contains | 130 | An example is the Autotools class, which contains |
131 | common settings for any application that Autotools uses. | 131 | common settings for any application that Autotools uses. |
132 | The "<link linkend='ref-classes'>Classes</link>" chapter provides details | 132 | The "<link linkend='ref-classes'>Classes</link>" chapter provides details |
@@ -139,8 +139,8 @@ | |||
139 | 139 | ||
140 | <para> | 140 | <para> |
141 | The configuration files (<filename>.conf</filename>) define various configuration variables | 141 | The configuration files (<filename>.conf</filename>) define various configuration variables |
142 | that govern the OpenEmbedded build process. | 142 | that govern the OpenEmbedded build process. |
143 | These files fall into several areas that define machine configuration options, | 143 | These files fall into several areas that define machine configuration options, |
144 | distribution configuration options, compiler tuning options, general common configuration | 144 | distribution configuration options, compiler tuning options, general common configuration |
145 | options and user configuration options (<filename>local.conf</filename>, which is found | 145 | options and user configuration options (<filename>local.conf</filename>, which is found |
146 | in the <ulink url='build-directory'>Build Directory</ulink>). | 146 | in the <ulink url='build-directory'>Build Directory</ulink>). |
@@ -152,19 +152,19 @@ | |||
152 | <title>Shared State Cache</title> | 152 | <title>Shared State Cache</title> |
153 | 153 | ||
154 | <para> | 154 | <para> |
155 | By design, the OpenEmbedded build system builds everything from scratch unless | 155 | By design, the OpenEmbedded build system builds everything from scratch unless |
156 | BitBake can determine that parts don't need to be rebuilt. | 156 | BitBake can determine that parts don't need to be rebuilt. |
157 | Fundamentally, building from scratch is attractive as it means all parts are | 157 | Fundamentally, building from scratch is attractive as it means all parts are |
158 | built fresh and there is no possibility of stale data causing problems. | 158 | built fresh and there is no possibility of stale data causing problems. |
159 | When developers hit problems, they typically default back to building from scratch | 159 | When developers hit problems, they typically default back to building from scratch |
160 | so they know the state of things from the start. | 160 | so they know the state of things from the start. |
161 | </para> | 161 | </para> |
162 | 162 | ||
163 | <para> | 163 | <para> |
164 | Building an image from scratch is both an advantage and a disadvantage to the process. | 164 | Building an image from scratch is both an advantage and a disadvantage to the process. |
165 | As mentioned in the previous paragraph, building from scratch ensures that | 165 | As mentioned in the previous paragraph, building from scratch ensures that |
166 | everything is current and starts from a known state. | 166 | everything is current and starts from a known state. |
167 | However, building from scratch also takes much longer as it generally means | 167 | However, building from scratch also takes much longer as it generally means |
168 | rebuilding things that don't necessarily need rebuilt. | 168 | rebuilding things that don't necessarily need rebuilt. |
169 | </para> | 169 | </para> |
170 | 170 | ||
@@ -181,15 +181,15 @@ | |||
181 | </para> | 181 | </para> |
182 | 182 | ||
183 | <para> | 183 | <para> |
184 | For the first question, the build system detects changes in the "inputs" to a given task by | 184 | For the first question, the build system detects changes in the "inputs" to a given task by |
185 | creating a checksum (or signature) of the task's inputs. | 185 | creating a checksum (or signature) of the task's inputs. |
186 | If the checksum changes, the system assumes the inputs have changed and the task needs to be | 186 | If the checksum changes, the system assumes the inputs have changed and the task needs to be |
187 | rerun. | 187 | rerun. |
188 | For the second question, the shared state (sstate) code tracks which tasks add which output | 188 | For the second question, the shared state (sstate) code tracks which tasks add which output |
189 | to the build process. | 189 | to the build process. |
190 | This means the output from a given task can be removed, upgraded or otherwise manipulated. | 190 | This means the output from a given task can be removed, upgraded or otherwise manipulated. |
191 | The third question is partly addressed by the solution for the second question | 191 | The third question is partly addressed by the solution for the second question |
192 | assuming the build system can fetch the sstate objects from remote locations and | 192 | assuming the build system can fetch the sstate objects from remote locations and |
193 | install them if they are deemed to be valid. | 193 | install them if they are deemed to be valid. |
194 | </para> | 194 | </para> |
195 | 195 | ||
@@ -202,18 +202,18 @@ | |||
202 | <title>Overall Architecture</title> | 202 | <title>Overall Architecture</title> |
203 | 203 | ||
204 | <para> | 204 | <para> |
205 | When determining what parts of the system need to be built, BitBake | 205 | When determining what parts of the system need to be built, BitBake |
206 | uses a per-task basis and does not use a per-recipe basis. | 206 | uses a per-task basis and does not use a per-recipe basis. |
207 | You might wonder why using a per-task basis is preferred over a per-recipe basis. | 207 | You might wonder why using a per-task basis is preferred over a per-recipe basis. |
208 | To help explain, consider having the IPK packaging backend enabled and then switching to DEB. | 208 | To help explain, consider having the IPK packaging backend enabled and then switching to DEB. |
209 | In this case, <filename>do_install</filename> and <filename>do_package</filename> | 209 | In this case, <filename>do_install</filename> and <filename>do_package</filename> |
210 | output are still valid. | 210 | output are still valid. |
211 | However, with a per-recipe approach, the build would not include the | 211 | However, with a per-recipe approach, the build would not include the |
212 | <filename>.deb</filename> files. | 212 | <filename>.deb</filename> files. |
213 | Consequently, you would have to invalidate the whole build and rerun it. | 213 | Consequently, you would have to invalidate the whole build and rerun it. |
214 | Rerunning everything is not the best situation. | 214 | Rerunning everything is not the best situation. |
215 | Also in this case, the core must be "taught" much about specific tasks. | 215 | Also in this case, the core must be "taught" much about specific tasks. |
216 | This methodology does not scale well and does not allow users to easily add new tasks | 216 | This methodology does not scale well and does not allow users to easily add new tasks |
217 | in layers or as external recipes without touching the packaged-staging core. | 217 | in layers or as external recipes without touching the packaged-staging core. |
218 | </para> | 218 | </para> |
219 | </section> | 219 | </section> |
@@ -222,37 +222,37 @@ | |||
222 | <title>Checksums (Signatures)</title> | 222 | <title>Checksums (Signatures)</title> |
223 | 223 | ||
224 | <para> | 224 | <para> |
225 | The shared state code uses a checksum, which is a unique signature of a task's | 225 | The shared state code uses a checksum, which is a unique signature of a task's |
226 | inputs, to determine if a task needs to be run again. | 226 | inputs, to determine if a task needs to be run again. |
227 | Because it is a change in a task's inputs that triggers a rerun, the process | 227 | Because it is a change in a task's inputs that triggers a rerun, the process |
228 | needs to detect all the inputs to a given task. | 228 | needs to detect all the inputs to a given task. |
229 | For shell tasks, this turns out to be fairly easy because | 229 | For shell tasks, this turns out to be fairly easy because |
230 | the build process generates a "run" shell script for each task and | 230 | the build process generates a "run" shell script for each task and |
231 | it is possible to create a checksum that gives you a good idea of when | 231 | it is possible to create a checksum that gives you a good idea of when |
232 | the task's data changes. | 232 | the task's data changes. |
233 | </para> | 233 | </para> |
234 | 234 | ||
235 | <para> | 235 | <para> |
236 | To complicate the problem, there are things that should not be included in | 236 | To complicate the problem, there are things that should not be included in |
237 | the checksum. | 237 | the checksum. |
238 | First, there is the actual specific build path of a given task - | 238 | First, there is the actual specific build path of a given task - |
239 | the <filename>WORKDIR</filename>. | 239 | the <filename>WORKDIR</filename>. |
240 | It does not matter if the working directory changes because it should not | 240 | It does not matter if the working directory changes because it should not |
241 | affect the output for target packages. | 241 | affect the output for target packages. |
242 | Also, the build process has the objective of making native/cross packages relocatable. | 242 | Also, the build process has the objective of making native/cross packages relocatable. |
243 | The checksum therefore needs to exclude <filename>WORKDIR</filename>. | 243 | The checksum therefore needs to exclude <filename>WORKDIR</filename>. |
244 | The simplistic approach for excluding the working directory is to set | 244 | The simplistic approach for excluding the working directory is to set |
245 | <filename>WORKDIR</filename> to some fixed value and create the checksum | 245 | <filename>WORKDIR</filename> to some fixed value and create the checksum |
246 | for the "run" script. | 246 | for the "run" script. |
247 | </para> | 247 | </para> |
248 | 248 | ||
249 | <para> | 249 | <para> |
250 | Another problem results from the "run" scripts containing functions that | 250 | Another problem results from the "run" scripts containing functions that |
251 | might or might not get called. | 251 | might or might not get called. |
252 | The incremental build solution contains code that figures out dependencies | 252 | The incremental build solution contains code that figures out dependencies |
253 | between shell functions. | 253 | between shell functions. |
254 | This code is used to prune the "run" scripts down to the minimum set, | 254 | This code is used to prune the "run" scripts down to the minimum set, |
255 | thereby alleviating this problem and making the "run" scripts much more | 255 | thereby alleviating this problem and making the "run" scripts much more |
256 | readable as a bonus. | 256 | readable as a bonus. |
257 | </para> | 257 | </para> |
258 | 258 | ||
@@ -260,62 +260,62 @@ | |||
260 | So far we have solutions for shell scripts. | 260 | So far we have solutions for shell scripts. |
261 | What about python tasks? | 261 | What about python tasks? |
262 | The same approach applies even though these tasks are more difficult. | 262 | The same approach applies even though these tasks are more difficult. |
263 | The process needs to figure out what variables a python function accesses | 263 | The process needs to figure out what variables a python function accesses |
264 | and what functions it calls. | 264 | and what functions it calls. |
265 | Again, the incremental build solution contains code that first figures out | 265 | Again, the incremental build solution contains code that first figures out |
266 | the variable and function dependencies, and then creates a checksum for the data | 266 | the variable and function dependencies, and then creates a checksum for the data |
267 | used as the input to the task. | 267 | used as the input to the task. |
268 | </para> | 268 | </para> |
269 | 269 | ||
270 | <para> | 270 | <para> |
271 | Like the <filename>WORKDIR</filename> case, situations exist where dependencies | 271 | Like the <filename>WORKDIR</filename> case, situations exist where dependencies |
272 | should be ignored. | 272 | should be ignored. |
273 | For these cases, you can instruct the build process to ignore a dependency | 273 | For these cases, you can instruct the build process to ignore a dependency |
274 | by using a line like the following: | 274 | by using a line like the following: |
275 | <literallayout class='monospaced'> | 275 | <literallayout class='monospaced'> |
276 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" | 276 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" |
277 | </literallayout> | 277 | </literallayout> |
278 | This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not | 278 | This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not |
279 | depend on the value of <filename>MACHINE</filename>, even if it does reference it. | 279 | depend on the value of <filename>MACHINE</filename>, even if it does reference it. |
280 | </para> | 280 | </para> |
281 | 281 | ||
282 | <para> | 282 | <para> |
283 | Equally, there are cases where we need to add dependencies BitBake is not able to find. | 283 | Equally, there are cases where we need to add dependencies BitBake is not able to find. |
284 | You can accomplish this by using a line like the following: | 284 | You can accomplish this by using a line like the following: |
285 | <literallayout class='monospaced'> | 285 | <literallayout class='monospaced'> |
286 | PACKAGE_ARCHS[vardeps] = "MACHINE" | 286 | PACKAGE_ARCHS[vardeps] = "MACHINE" |
287 | </literallayout> | 287 | </literallayout> |
288 | This example explicitly adds the <filename>MACHINE</filename> variable as a | 288 | This example explicitly adds the <filename>MACHINE</filename> variable as a |
289 | dependency for <filename>PACKAGE_ARCHS</filename>. | 289 | dependency for <filename>PACKAGE_ARCHS</filename>. |
290 | </para> | 290 | </para> |
291 | 291 | ||
292 | <para> | 292 | <para> |
293 | Consider a case with inline python, for example, where BitBake is not | 293 | Consider a case with inline python, for example, where BitBake is not |
294 | able to figure out dependencies. | 294 | able to figure out dependencies. |
295 | When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake | 295 | When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake |
296 | produces output when it discovers something for which it cannot figure out | 296 | produces output when it discovers something for which it cannot figure out |
297 | dependencies. | 297 | dependencies. |
298 | The Yocto Project team has currently not managed to cover those dependencies | 298 | The Yocto Project team has currently not managed to cover those dependencies |
299 | in detail and is aware of the need to fix this situation. | 299 | in detail and is aware of the need to fix this situation. |
300 | </para> | 300 | </para> |
301 | 301 | ||
302 | <para> | 302 | <para> |
303 | Thus far, this section has limited discussion to the direct inputs into a task. | 303 | Thus far, this section has limited discussion to the direct inputs into a task. |
304 | Information based on direct inputs is referred to as the "basehash" in the | 304 | Information based on direct inputs is referred to as the "basehash" in the |
305 | code. | 305 | code. |
306 | However, there is still the question of a task's indirect inputs - the | 306 | However, there is still the question of a task's indirect inputs - the |
307 | things that were already built and present in the Build Directory. | 307 | things that were already built and present in the Build Directory. |
308 | The checksum (or signature) for a particular task needs to add the hashes | 308 | The checksum (or signature) for a particular task needs to add the hashes |
309 | of all the tasks on which the particular task depends. | 309 | of all the tasks on which the particular task depends. |
310 | Choosing which dependencies to add is a policy decision. | 310 | Choosing which dependencies to add is a policy decision. |
311 | However, the effect is to generate a master checksum that combines the basehash | 311 | However, the effect is to generate a master checksum that combines the basehash |
312 | and the hashes of the task's dependencies. | 312 | and the hashes of the task's dependencies. |
313 | </para> | 313 | </para> |
314 | 314 | ||
315 | <para> | 315 | <para> |
316 | At the code level, there are a variety of ways both the basehash and the | 316 | At the code level, there are a variety of ways both the basehash and the |
317 | dependent task hashes can be influenced. | 317 | dependent task hashes can be influenced. |
318 | Within the BitBake configuration file, we can give BitBake some extra information | 318 | Within the BitBake configuration file, we can give BitBake some extra information |
319 | to help it construct the basehash. | 319 | to help it construct the basehash. |
320 | The following statements effectively result in a list of global variable | 320 | The following statements effectively result in a list of global variable |
321 | dependency excludes - variables never included in any checksum: | 321 | dependency excludes - variables never included in any checksum: |
@@ -325,42 +325,42 @@ | |||
325 | BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER" | 325 | BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER" |
326 | BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET" | 326 | BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET" |
327 | </literallayout> | 327 | </literallayout> |
328 | The previous example actually excludes | 328 | The previous example actually excludes |
329 | <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> | 329 | <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> |
330 | since it is actually constructed as a path within | 330 | since it is actually constructed as a path within |
331 | <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on | 331 | <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on |
332 | the whitelist. | 332 | the whitelist. |
333 | </para> | 333 | </para> |
334 | 334 | ||
335 | <para> | 335 | <para> |
336 | The rules for deciding which hashes of dependent tasks to include through | 336 | The rules for deciding which hashes of dependent tasks to include through |
337 | dependency chains are more complex and are generally accomplished with a | 337 | dependency chains are more complex and are generally accomplished with a |
338 | python function. | 338 | python function. |
339 | The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples | 339 | The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples |
340 | of this and also illustrates how you can insert your own policy into the system | 340 | of this and also illustrates how you can insert your own policy into the system |
341 | if so desired. | 341 | if so desired. |
342 | This file defines the two basic signature generators <filename>OE-Core</filename> | 342 | This file defines the two basic signature generators <filename>OE-Core</filename> |
343 | uses: "OEBasic" and "OEBasicHash". | 343 | uses: "OEBasic" and "OEBasicHash". |
344 | By default, there is a dummy "noop" signature handler enabled in BitBake. | 344 | By default, there is a dummy "noop" signature handler enabled in BitBake. |
345 | This means that behavior is unchanged from previous versions. | 345 | This means that behavior is unchanged from previous versions. |
346 | <filename>OE-Core</filename> uses the "OEBasic" signature handler by default | 346 | <filename>OE-Core</filename> uses the "OEBasic" signature handler by default |
347 | through this setting in the <filename>bitbake.conf</filename> file: | 347 | through this setting in the <filename>bitbake.conf</filename> file: |
348 | <literallayout class='monospaced'> | 348 | <literallayout class='monospaced'> |
349 | BB_SIGNATURE_HANDLER ?= "OEBasic" | 349 | BB_SIGNATURE_HANDLER ?= "OEBasic" |
350 | </literallayout> | 350 | </literallayout> |
351 | The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the | 351 | The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the |
352 | "OEBasic" version but adds the task hash to the stamp files. | 352 | "OEBasic" version but adds the task hash to the stamp files. |
353 | This results in any metadata change that changes the task hash, automatically | 353 | This results in any metadata change that changes the task hash, automatically |
354 | causing the task to be run again. | 354 | causing the task to be run again. |
355 | This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link> | 355 | This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link> |
356 | values and changes to metadata automatically ripple across the build. | 356 | values and changes to metadata automatically ripple across the build. |
357 | Currently, this behavior is not the default behavior for <filename>OE-Core</filename> | 357 | Currently, this behavior is not the default behavior for <filename>OE-Core</filename> |
358 | but is the default in <filename>poky</filename>. | 358 | but is the default in <filename>poky</filename>. |
359 | </para> | 359 | </para> |
360 | 360 | ||
361 | <para> | 361 | <para> |
362 | It is also worth noting that the end result of these signature generators is to | 362 | It is also worth noting that the end result of these signature generators is to |
363 | make some dependency and hash information available to the build. | 363 | make some dependency and hash information available to the build. |
364 | This information includes: | 364 | This information includes: |
365 | <literallayout class='monospaced'> | 365 | <literallayout class='monospaced'> |
366 | BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe | 366 | BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe |
@@ -375,15 +375,15 @@ | |||
375 | <title>Shared State</title> | 375 | <title>Shared State</title> |
376 | 376 | ||
377 | <para> | 377 | <para> |
378 | Checksums and dependencies, as discussed in the previous section, solve half the | 378 | Checksums and dependencies, as discussed in the previous section, solve half the |
379 | problem. | 379 | problem. |
380 | The other part of the problem is being able to use checksum information during the build | 380 | The other part of the problem is being able to use checksum information during the build |
381 | and being able to reuse or rebuild specific components. | 381 | and being able to reuse or rebuild specific components. |
382 | </para> | 382 | </para> |
383 | 383 | ||
384 | <para> | 384 | <para> |
385 | The shared state class (<filename>sstate.bbclass</filename>) | 385 | The shared state class (<filename>sstate.bbclass</filename>) |
386 | is a relatively generic implementation of how to "capture" a snapshot of a given task. | 386 | is a relatively generic implementation of how to "capture" a snapshot of a given task. |
387 | The idea is that the build process does not care about the source of a task's output. | 387 | The idea is that the build process does not care about the source of a task's output. |
388 | Output could be freshly built or it could be downloaded and unpacked from | 388 | Output could be freshly built or it could be downloaded and unpacked from |
389 | somewhere - the build process doesn't need to worry about its source. | 389 | somewhere - the build process doesn't need to worry about its source. |
@@ -392,17 +392,17 @@ | |||
392 | <para> | 392 | <para> |
393 | There are two types of output, one is just about creating a directory | 393 | There are two types of output, one is just about creating a directory |
394 | in <filename>WORKDIR</filename>. | 394 | in <filename>WORKDIR</filename>. |
395 | A good example is the output of either <filename>do_install</filename> or | 395 | A good example is the output of either <filename>do_install</filename> or |
396 | <filename>do_package</filename>. | 396 | <filename>do_package</filename>. |
397 | The other type of output occurs when a set of data is merged into a shared directory | 397 | The other type of output occurs when a set of data is merged into a shared directory |
398 | tree such as the sysroot. | 398 | tree such as the sysroot. |
399 | </para> | 399 | </para> |
400 | 400 | ||
401 | <para> | 401 | <para> |
402 | The Yocto Project team has tried to keep the details of the implementation hidden in | 402 | The Yocto Project team has tried to keep the details of the implementation hidden in |
403 | <filename>sstate.bbclass</filename>. | 403 | <filename>sstate.bbclass</filename>. |
404 | From a user's perspective, adding shared state wrapping to a task | 404 | From a user's perspective, adding shared state wrapping to a task |
405 | is as simple as this <filename>do_deploy</filename> example taken from | 405 | is as simple as this <filename>do_deploy</filename> example taken from |
406 | <filename>do_deploy.bbclass</filename>: | 406 | <filename>do_deploy.bbclass</filename>: |
407 | <literallayout class='monospaced'> | 407 | <literallayout class='monospaced'> |
408 | DEPLOYDIR = "${WORKDIR}/deploy-${PN}" | 408 | DEPLOYDIR = "${WORKDIR}/deploy-${PN}" |
@@ -418,13 +418,13 @@ | |||
418 | </literallayout> | 418 | </literallayout> |
419 | In the example, we add some extra flags to the task, a name field ("deploy"), an | 419 | In the example, we add some extra flags to the task, a name field ("deploy"), an |
420 | input directory where the task sends data, and the output | 420 | input directory where the task sends data, and the output |
421 | directory where the data from the task should eventually be copied. | 421 | directory where the data from the task should eventually be copied. |
422 | We also add a <filename>_setscene</filename> variant of the task and add the task | 422 | We also add a <filename>_setscene</filename> variant of the task and add the task |
423 | name to the <filename>SSTATETASKS</filename> list. | 423 | name to the <filename>SSTATETASKS</filename> list. |
424 | </para> | 424 | </para> |
425 | 425 | ||
426 | <para> | 426 | <para> |
427 | If you have a directory whose contents you need to preserve, you can do this with | 427 | If you have a directory whose contents you need to preserve, you can do this with |
428 | a line like the following: | 428 | a line like the following: |
429 | <literallayout class='monospaced'> | 429 | <literallayout class='monospaced'> |
430 | do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" | 430 | do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" |
@@ -441,21 +441,21 @@ | |||
441 | </para> | 441 | </para> |
442 | 442 | ||
443 | <para> | 443 | <para> |
444 | Behind the scenes, the shared state code works by looking in | 444 | Behind the scenes, the shared state code works by looking in |
445 | <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and | 445 | <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and |
446 | <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link> | 446 | <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link> |
447 | for shared state files. | 447 | for shared state files. |
448 | Here is an example: | 448 | Here is an example: |
449 | <literallayout class='monospaced'> | 449 | <literallayout class='monospaced'> |
450 | SSTATE_MIRRORS ?= "\ | 450 | SSTATE_MIRRORS ?= "\ |
451 | file://.* http://someserver.tld/share/sstate/PATH \n \ | 451 | file://.* http://someserver.tld/share/sstate/PATH \n \ |
452 | file://.* file:///some/local/dir/sstate/PATH" | 452 | file://.* file:///some/local/dir/sstate/PATH" |
453 | </literallayout> | 453 | </literallayout> |
454 | <note> | 454 | <note> |
455 | The shared state directory (<filename>SSTATE_DIR</filename>) is | 455 | The shared state directory (<filename>SSTATE_DIR</filename>) is |
456 | organized into two-character subdirectories, where the subdirectory | 456 | organized into two-character subdirectories, where the subdirectory |
457 | names are based on the first two characters of the hash. | 457 | names are based on the first two characters of the hash. |
458 | If the shared state directory structure for a mirror has the | 458 | If the shared state directory structure for a mirror has the |
459 | same structure as <filename>SSTATE_DIR</filename>, you must | 459 | same structure as <filename>SSTATE_DIR</filename>, you must |
460 | specify "PATH" as part of the URI to enable the build system | 460 | specify "PATH" as part of the URI to enable the build system |
461 | to map to the appropriate subdirectory. | 461 | to map to the appropriate subdirectory. |
@@ -465,8 +465,8 @@ | |||
465 | <para> | 465 | <para> |
466 | The shared state package validity can be detected just by looking at the | 466 | The shared state package validity can be detected just by looking at the |
467 | filename since the filename contains the task checksum (or signature) as | 467 | filename since the filename contains the task checksum (or signature) as |
468 | described earlier in this section. | 468 | described earlier in this section. |
469 | If a valid shared state package is found, the build process downloads it | 469 | If a valid shared state package is found, the build process downloads it |
470 | and uses it to accelerate the task. | 470 | and uses it to accelerate the task. |
471 | </para> | 471 | </para> |
472 | 472 | ||
@@ -474,19 +474,19 @@ | |||
474 | The build processes uses the <filename>*_setscene</filename> tasks | 474 | The build processes uses the <filename>*_setscene</filename> tasks |
475 | for the task acceleration phase. | 475 | for the task acceleration phase. |
476 | BitBake goes through this phase before the main execution code and tries | 476 | BitBake goes through this phase before the main execution code and tries |
477 | to accelerate any tasks for which it can find shared state packages. | 477 | to accelerate any tasks for which it can find shared state packages. |
478 | If a shared state package for a task is available, the shared state | 478 | If a shared state package for a task is available, the shared state |
479 | package is used. | 479 | package is used. |
480 | This means the task and any tasks on which it is dependent are not | 480 | This means the task and any tasks on which it is dependent are not |
481 | executed. | 481 | executed. |
482 | </para> | 482 | </para> |
483 | 483 | ||
484 | <para> | 484 | <para> |
485 | As a real world example, the aim is when building an IPK-based image, | 485 | As a real world example, the aim is when building an IPK-based image, |
486 | only the <filename>do_package_write_ipk</filename> tasks would have their | 486 | only the <filename>do_package_write_ipk</filename> tasks would have their |
487 | shared state packages fetched and extracted. | 487 | shared state packages fetched and extracted. |
488 | Since the sysroot is not used, it would never get extracted. | 488 | Since the sysroot is not used, it would never get extracted. |
489 | This is another reason why a task-based approach is preferred over a | 489 | This is another reason why a task-based approach is preferred over a |
490 | recipe-based approach, which would have to install the output from every task. | 490 | recipe-based approach, which would have to install the output from every task. |
491 | </para> | 491 | </para> |
492 | </section> | 492 | </section> |
@@ -495,9 +495,9 @@ | |||
495 | <title>Tips and Tricks</title> | 495 | <title>Tips and Tricks</title> |
496 | 496 | ||
497 | <para> | 497 | <para> |
498 | The code in the build system that supports incremental builds is not | 498 | The code in the build system that supports incremental builds is not |
499 | simple code. | 499 | simple code. |
500 | This section presents some tips and tricks that help you work around | 500 | This section presents some tips and tricks that help you work around |
501 | issues related to shared state code. | 501 | issues related to shared state code. |
502 | </para> | 502 | </para> |
503 | 503 | ||
@@ -505,25 +505,25 @@ | |||
505 | <title>Debugging</title> | 505 | <title>Debugging</title> |
506 | 506 | ||
507 | <para> | 507 | <para> |
508 | When things go wrong, debugging needs to be straightforward. | 508 | When things go wrong, debugging needs to be straightforward. |
509 | Because of this, the Yocto Project team included strong debugging | 509 | Because of this, the Yocto Project team included strong debugging |
510 | tools: | 510 | tools: |
511 | <itemizedlist> | 511 | <itemizedlist> |
512 | <listitem><para>Whenever a shared state package is written out, so is a | 512 | <listitem><para>Whenever a shared state package is written out, so is a |
513 | corresponding <filename>.siginfo</filename> file. | 513 | corresponding <filename>.siginfo</filename> file. |
514 | This practice results in a pickled python database of all | 514 | This practice results in a pickled python database of all |
515 | the metadata that went into creating the hash for a given shared state | 515 | the metadata that went into creating the hash for a given shared state |
516 | package.</para></listitem> | 516 | package.</para></listitem> |
517 | <listitem><para>If BitBake is run with the <filename>--dump-signatures</filename> | 517 | <listitem><para>If BitBake is run with the <filename>--dump-signatures</filename> |
518 | (or <filename>-S</filename>) option, BitBake dumps out | 518 | (or <filename>-S</filename>) option, BitBake dumps out |
519 | <filename>.siginfo</filename> files in | 519 | <filename>.siginfo</filename> files in |
520 | the stamp directory for every task it would have executed instead of | 520 | the stamp directory for every task it would have executed instead of |
521 | building the specified target package.</para></listitem> | 521 | building the specified target package.</para></listitem> |
522 | <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that | 522 | <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that |
523 | can process these <filename>.siginfo</filename> files. | 523 | can process these <filename>.siginfo</filename> files. |
524 | If one file is specified, it will dump out the dependency | 524 | If one file is specified, it will dump out the dependency |
525 | information in the file. | 525 | information in the file. |
526 | If two files are specified, it will compare the two files and dump out | 526 | If two files are specified, it will compare the two files and dump out |
527 | the differences between the two. | 527 | the differences between the two. |
528 | This allows the question of "What changed between X and Y?" to be | 528 | This allows the question of "What changed between X and Y?" to be |
529 | answered easily.</para></listitem> | 529 | answered easily.</para></listitem> |
@@ -538,41 +538,41 @@ | |||
538 | The shared state code uses checksums and shared state | 538 | The shared state code uses checksums and shared state |
539 | cache to avoid unnecessarily rebuilding tasks. | 539 | cache to avoid unnecessarily rebuilding tasks. |
540 | As with all schemes, this one has some drawbacks. | 540 | As with all schemes, this one has some drawbacks. |
541 | It is possible that you could make implicit changes that are not factored | 541 | It is possible that you could make implicit changes that are not factored |
542 | into the checksum calculation, but do affect a task's output. | 542 | into the checksum calculation, but do affect a task's output. |
543 | A good example is perhaps when a tool changes its output. | 543 | A good example is perhaps when a tool changes its output. |
544 | Let's say that the output of <filename>rpmdeps</filename> needed to change. | 544 | Let's say that the output of <filename>rpmdeps</filename> needed to change. |
545 | The result of the change should be that all the "package", "package_write_rpm", | 545 | The result of the change should be that all the "package", "package_write_rpm", |
546 | and "package_deploy-rpm" shared state cache items would become invalid. | 546 | and "package_deploy-rpm" shared state cache items would become invalid. |
547 | But, because this is a change that is external to the code and therefore implicit, | 547 | But, because this is a change that is external to the code and therefore implicit, |
548 | the associated shared state cache items do not become invalidated. | 548 | the associated shared state cache items do not become invalidated. |
549 | In this case, the build process would use the cached items rather than running the | 549 | In this case, the build process would use the cached items rather than running the |
550 | task again. | 550 | task again. |
551 | Obviously, these types of implicit changes can cause problems. | 551 | Obviously, these types of implicit changes can cause problems. |
552 | </para> | 552 | </para> |
553 | 553 | ||
554 | <para> | 554 | <para> |
555 | To avoid these problems during the build, you need to understand the effects of any | 555 | To avoid these problems during the build, you need to understand the effects of any |
556 | change you make. | 556 | change you make. |
557 | Note that any changes you make directly to a function automatically are factored into | 557 | Note that any changes you make directly to a function automatically are factored into |
558 | the checksum calculation and thus, will invalidate the associated area of sstate cache. | 558 | the checksum calculation and thus, will invalidate the associated area of sstate cache. |
559 | You need to be aware of any implicit changes that are not obvious changes to the | 559 | You need to be aware of any implicit changes that are not obvious changes to the |
560 | code and could affect the output of a given task. | 560 | code and could affect the output of a given task. |
561 | Once you are aware of such a change, you can take steps to invalidate the cache | 561 | Once you are aware of such a change, you can take steps to invalidate the cache |
562 | and force the task to run. | 562 | and force the task to run. |
563 | The step to take is as simple as changing a function's comments in the source code. | 563 | The step to take is as simple as changing a function's comments in the source code. |
564 | For example, to invalidate package shared state files, change the comment statements | 564 | For example, to invalidate package shared state files, change the comment statements |
565 | of <filename>do_package</filename> or the comments of one of the functions it calls. | 565 | of <filename>do_package</filename> or the comments of one of the functions it calls. |
566 | The change is purely cosmetic, but it causes the checksum to be recalculated and | 566 | The change is purely cosmetic, but it causes the checksum to be recalculated and |
567 | forces the task to be run again. | 567 | forces the task to be run again. |
568 | </para> | 568 | </para> |
569 | 569 | ||
570 | <note> | 570 | <note> |
571 | For an example of a commit that makes a cosmetic change to invalidate | 571 | For an example of a commit that makes a cosmetic change to invalidate |
572 | a shared state, see this | 572 | a shared state, see this |
573 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. | 573 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. |
574 | </note> | 574 | </note> |
575 | </section> | 575 | </section> |
576 | </section> | 576 | </section> |
577 | </section> | 577 | </section> |
578 | 578 | ||
@@ -580,24 +580,24 @@ | |||
580 | <title>x32</title> | 580 | <title>x32</title> |
581 | 581 | ||
582 | <para> | 582 | <para> |
583 | x32 is a new processor-specific Application Binary Interface (psABI) for x86_64. | 583 | x32 is a new processor-specific Application Binary Interface (psABI) for x86_64. |
584 | An ABI defines the calling conventions between functions in a processing environment. | 584 | An ABI defines the calling conventions between functions in a processing environment. |
585 | The interface determines what registers are used and what the sizes are for various C data types. | 585 | The interface determines what registers are used and what the sizes are for various C data types. |
586 | </para> | 586 | </para> |
587 | 587 | ||
588 | <para> | 588 | <para> |
589 | Some processing environments prefer using 32-bit applications even when running | 589 | Some processing environments prefer using 32-bit applications even when running |
590 | on Intel 64-bit platforms. | 590 | on Intel 64-bit platforms. |
591 | Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms. | 591 | Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms. |
592 | The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources, | 592 | The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources, |
593 | leaving the system underutilized. | 593 | leaving the system underutilized. |
594 | Now consider the x86_64 psABI. | 594 | Now consider the x86_64 psABI. |
595 | This ABI is newer and uses 64-bits for data sizes and program pointers. | 595 | This ABI is newer and uses 64-bits for data sizes and program pointers. |
596 | The extra bits increase the footprint size of the programs, libraries, | 596 | The extra bits increase the footprint size of the programs, libraries, |
597 | and also increases the memory and file system size requirements. | 597 | and also increases the memory and file system size requirements. |
598 | Executing under the x32 psABI enables user programs to utilize CPU and system resources | 598 | Executing under the x32 psABI enables user programs to utilize CPU and system resources |
599 | more efficiently while keeping the memory footprint of the applications low. | 599 | more efficiently while keeping the memory footprint of the applications low. |
600 | Extra bits are used for registers but not for addressing mechanisms. | 600 | Extra bits are used for registers but not for addressing mechanisms. |
601 | </para> | 601 | </para> |
602 | 602 | ||
603 | <section id='support'> | 603 | <section id='support'> |
@@ -608,14 +608,14 @@ | |||
608 | release supports current development specifications of x32 psABI. | 608 | release supports current development specifications of x32 psABI. |
609 | As of this release of the Yocto Project, x32 psABI support exists as follows: | 609 | As of this release of the Yocto Project, x32 psABI support exists as follows: |
610 | <itemizedlist> | 610 | <itemizedlist> |
611 | <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets. | 611 | <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets. |
612 | </para></listitem> | 612 | </para></listitem> |
613 | <listitem><para>You can use the x32 psABI support through the <filename>meta-x32</filename> | 613 | <listitem><para>You can use the x32 psABI support through the <filename>meta-x32</filename> |
614 | layer on top of the OE-core/Yocto layer.</para></listitem> | 614 | layer on top of the OE-core/Yocto layer.</para></listitem> |
615 | <listitem><para>The toolchain from the <filename>experimental/meta-x32</filename> layer | 615 | <listitem><para>The toolchain from the <filename>experimental/meta-x32</filename> layer |
616 | is used for building x32 psABI program binaries.</para></listitem> | 616 | is used for building x32 psABI program binaries.</para></listitem> |
617 | <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem> | 617 | <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem> |
618 | <listitem><para>You can create and boot <filename>core-image-minimal</filename> and | 618 | <listitem><para>You can create and boot <filename>core-image-minimal</filename> and |
619 | <filename>core-image-sato</filename> images.</para></listitem> | 619 | <filename>core-image-sato</filename> images.</para></listitem> |
620 | </itemizedlist> | 620 | </itemizedlist> |
621 | </para> | 621 | </para> |
@@ -625,18 +625,18 @@ | |||
625 | <title>Future Development and Limitations</title> | 625 | <title>Future Development and Limitations</title> |
626 | 626 | ||
627 | <para> | 627 | <para> |
628 | As of this Yocto Project release, the x32 psABI kernel and library interfaces | 628 | As of this Yocto Project release, the x32 psABI kernel and library interfaces |
629 | specifications are not finalized. | 629 | specifications are not finalized. |
630 | </para> | 630 | </para> |
631 | 631 | ||
632 | <para> | 632 | <para> |
633 | Future Plans for the x32 psABI in the Yocto Project include the following: | 633 | Future Plans for the x32 psABI in the Yocto Project include the following: |
634 | <itemizedlist> | 634 | <itemizedlist> |
635 | <listitem><para>Enhance and fix the few remaining recipes so they | 635 | <listitem><para>Enhance and fix the few remaining recipes so they |
636 | work with and support x32 toolchains.</para></listitem> | 636 | work with and support x32 toolchains.</para></listitem> |
637 | <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem> | 637 | <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem> |
638 | <listitem><para>Support larger images.</para></listitem> | 638 | <listitem><para>Support larger images.</para></listitem> |
639 | <listitem><para>Integrate x32 recipes, toolchain, and kernel changes from | 639 | <listitem><para>Integrate x32 recipes, toolchain, and kernel changes from |
640 | <filename>experimental/meta-x32</filename> into OE-core.</para></listitem> | 640 | <filename>experimental/meta-x32</filename> into OE-core.</para></listitem> |
641 | </itemizedlist> | 641 | </itemizedlist> |
642 | </para> | 642 | </para> |
@@ -650,7 +650,7 @@ | |||
650 | Yocto Project, you can follow these steps to use the x32 spABI: | 650 | Yocto Project, you can follow these steps to use the x32 spABI: |
651 | <itemizedlist> | 651 | <itemizedlist> |
652 | <listitem><para>Add the <filename>experimental/meta-x32</filename> layer to your local | 652 | <listitem><para>Add the <filename>experimental/meta-x32</filename> layer to your local |
653 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. | 653 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. |
654 | You can find the <filename>experimental/meta-x32</filename> source repository at | 654 | You can find the <filename>experimental/meta-x32</filename> source repository at |
655 | <ulink url='&YOCTO_GIT_URL;'></ulink>.</para></listitem> | 655 | <ulink url='&YOCTO_GIT_URL;'></ulink>.</para></listitem> |
656 | <listitem><para>Edit your <filename>conf/bblayers.conf</filename> file so that it includes | 656 | <listitem><para>Edit your <filename>conf/bblayers.conf</filename> file so that it includes |
@@ -677,8 +677,8 @@ | |||
677 | or 'INVALID'), True) or 'lib'}" | 677 | or 'INVALID'), True) or 'lib'}" |
678 | #MACHINE = "atom-pc" | 678 | #MACHINE = "atom-pc" |
679 | #DEFAULTTUNE = "core2-64-x32" | 679 | #DEFAULTTUNE = "core2-64-x32" |
680 | </literallayout></para></listitem> | 680 | </literallayout></para></listitem> |
681 | <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI. | 681 | <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI. |
682 | Here is an example: | 682 | Here is an example: |
683 | <literallayout class='monospaced'> | 683 | <literallayout class='monospaced'> |
684 | $ bitake core-image-sato | 684 | $ bitake core-image-sato |
@@ -696,15 +696,15 @@ | |||
696 | <title>Licenses</title> | 696 | <title>Licenses</title> |
697 | 697 | ||
698 | <para> | 698 | <para> |
699 | This section describes the mechanism by which the OpenEmbedded build system | 699 | This section describes the mechanism by which the OpenEmbedded build system |
700 | tracks changes to licensing text. | 700 | tracks changes to licensing text. |
701 | The section also describes how to enable commercially licensed recipes, | 701 | The section also describes how to enable commercially licensed recipes, |
702 | which by default are disabled. | 702 | which by default are disabled. |
703 | </para> | 703 | </para> |
704 | 704 | ||
705 | <para> | 705 | <para> |
706 | For information that can help you maintain compliance with various open | 706 | For information that can help you maintain compliance with various open |
707 | source licensing during the lifecycle of the product, see the | 707 | source licensing during the lifecycle of the product, see the |
708 | "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section | 708 | "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section |
709 | in the Yocto Project Development Manual. | 709 | in the Yocto Project Development Manual. |
710 | </para> | 710 | </para> |
@@ -713,8 +713,8 @@ | |||
713 | <title>Tracking License Changes</title> | 713 | <title>Tracking License Changes</title> |
714 | 714 | ||
715 | <para> | 715 | <para> |
716 | The license of an upstream project might change in the future. | 716 | The license of an upstream project might change in the future. |
717 | In order to prevent these changes going unnoticed, the | 717 | In order to prevent these changes going unnoticed, the |
718 | <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename> | 718 | <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename> |
719 | variable tracks changes to the license text. The checksums are validated at the end of the | 719 | variable tracks changes to the license text. The checksums are validated at the end of the |
720 | configure step, and if the checksums do not match, the build will fail. | 720 | configure step, and if the checksums do not match, the build will fail. |
@@ -736,15 +736,15 @@ | |||
736 | </para> | 736 | </para> |
737 | 737 | ||
738 | <para> | 738 | <para> |
739 | The build system uses the | 739 | The build system uses the |
740 | <filename><link linkend='var-S'>S</link></filename> variable as the | 740 | <filename><link linkend='var-S'>S</link></filename> variable as the |
741 | default directory used when searching files listed in | 741 | default directory used when searching files listed in |
742 | <filename>LIC_FILES_CHKSUM</filename>. | 742 | <filename>LIC_FILES_CHKSUM</filename>. |
743 | The previous example employs the default directory. | 743 | The previous example employs the default directory. |
744 | </para> | 744 | </para> |
745 | 745 | ||
746 | <para> | 746 | <para> |
747 | You can also use relative paths as shown in the following example: | 747 | You can also use relative paths as shown in the following example: |
748 | <literallayout class='monospaced'> | 748 | <literallayout class='monospaced'> |
749 | LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\ | 749 | LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\ |
750 | md5=bb14ed3c4cda583abc85401304b5cd4e" | 750 | md5=bb14ed3c4cda583abc85401304b5cd4e" |
@@ -753,14 +753,14 @@ | |||
753 | </para> | 753 | </para> |
754 | 754 | ||
755 | <para> | 755 | <para> |
756 | In this example, the first line locates a file in | 756 | In this example, the first line locates a file in |
757 | <filename>${S}/src/ls.c</filename>. | 757 | <filename>${S}/src/ls.c</filename>. |
758 | The second line refers to a file in | 758 | The second line refers to a file in |
759 | <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, which is the parent | 759 | <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, which is the parent |
760 | of <filename><link linkend='var-S'>S</link></filename>. | 760 | of <filename><link linkend='var-S'>S</link></filename>. |
761 | </para> | 761 | </para> |
762 | <para> | 762 | <para> |
763 | Note that this variable is mandatory for all recipes, unless the | 763 | Note that this variable is mandatory for all recipes, unless the |
764 | <filename>LICENSE</filename> variable is set to "CLOSED". | 764 | <filename>LICENSE</filename> variable is set to "CLOSED". |
765 | </para> | 765 | </para> |
766 | </section> | 766 | </section> |
@@ -768,48 +768,48 @@ | |||
768 | <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax"> | 768 | <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax"> |
769 | <title>Explanation of Syntax</title> | 769 | <title>Explanation of Syntax</title> |
770 | <para> | 770 | <para> |
771 | As mentioned in the previous section, the | 771 | As mentioned in the previous section, the |
772 | <filename>LIC_FILES_CHKSUM</filename> variable lists all the | 772 | <filename>LIC_FILES_CHKSUM</filename> variable lists all the |
773 | important files that contain the license text for the source code. | 773 | important files that contain the license text for the source code. |
774 | It is possible to specify a checksum for an entire file, or a specific section of a | 774 | It is possible to specify a checksum for an entire file, or a specific section of a |
775 | file (specified by beginning and ending line numbers with the "beginline" and "endline" | 775 | file (specified by beginning and ending line numbers with the "beginline" and "endline" |
776 | parameters, respectively). | 776 | parameters, respectively). |
777 | The latter is useful for source files with a license notice header, | 777 | The latter is useful for source files with a license notice header, |
778 | README documents, and so forth. | 778 | README documents, and so forth. |
779 | If you do not use the "beginline" parameter, then it is assumed that the text begins on the | 779 | If you do not use the "beginline" parameter, then it is assumed that the text begins on the |
780 | first line of the file. | 780 | first line of the file. |
781 | Similarly, if you do not use the "endline" parameter, it is assumed that the license text | 781 | Similarly, if you do not use the "endline" parameter, it is assumed that the license text |
782 | ends with the last line of the file. | 782 | ends with the last line of the file. |
783 | </para> | 783 | </para> |
784 | 784 | ||
785 | <para> | 785 | <para> |
786 | The "md5" parameter stores the md5 checksum of the license text. | 786 | The "md5" parameter stores the md5 checksum of the license text. |
787 | If the license text changes in any way as compared to this parameter | 787 | If the license text changes in any way as compared to this parameter |
788 | then a mismatch occurs. | 788 | then a mismatch occurs. |
789 | This mismatch triggers a build failure and notifies the developer. | 789 | This mismatch triggers a build failure and notifies the developer. |
790 | Notification allows the developer to review and address the license text changes. | 790 | Notification allows the developer to review and address the license text changes. |
791 | Also note that if a mismatch occurs during the build, the correct md5 | 791 | Also note that if a mismatch occurs during the build, the correct md5 |
792 | checksum is placed in the build log and can be easily copied to the recipe. | 792 | checksum is placed in the build log and can be easily copied to the recipe. |
793 | </para> | 793 | </para> |
794 | 794 | ||
795 | <para> | 795 | <para> |
796 | There is no limit to how many files you can specify using the | 796 | There is no limit to how many files you can specify using the |
797 | <filename>LIC_FILES_CHKSUM</filename> variable. | 797 | <filename>LIC_FILES_CHKSUM</filename> variable. |
798 | Generally, however, every project requires a few specifications for license tracking. | 798 | Generally, however, every project requires a few specifications for license tracking. |
799 | Many projects have a "COPYING" file that stores the license information for all the source | 799 | Many projects have a "COPYING" file that stores the license information for all the source |
800 | code files. | 800 | code files. |
801 | This practice allows you to just track the "COPYING" file as long as it is kept up to date. | 801 | This practice allows you to just track the "COPYING" file as long as it is kept up to date. |
802 | </para> | 802 | </para> |
803 | 803 | ||
804 | <tip> | 804 | <tip> |
805 | If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match | 805 | If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match |
806 | error and displays the correct "md5" parameter value during the build. | 806 | error and displays the correct "md5" parameter value during the build. |
807 | The correct parameter is also captured in the build log. | 807 | The correct parameter is also captured in the build log. |
808 | </tip> | 808 | </tip> |
809 | 809 | ||
810 | <tip> | 810 | <tip> |
811 | If the whole file contains only license text, you do not need to use the "beginline" and | 811 | If the whole file contains only license text, you do not need to use the "beginline" and |
812 | "endline" parameters. | 812 | "endline" parameters. |
813 | </tip> | 813 | </tip> |
814 | </section> | 814 | </section> |
815 | </section> | 815 | </section> |
@@ -820,10 +820,10 @@ | |||
820 | <para> | 820 | <para> |
821 | By default, the OpenEmbedded build system disables | 821 | By default, the OpenEmbedded build system disables |
822 | components that have commercial or other special licensing | 822 | components that have commercial or other special licensing |
823 | requirements. | 823 | requirements. |
824 | Such requirements are defined on a | 824 | Such requirements are defined on a |
825 | recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable | 825 | recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable |
826 | definition in the affected recipe. | 826 | definition in the affected recipe. |
827 | For instance, the | 827 | For instance, the |
828 | <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> | 828 | <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> |
829 | recipe contains the following statement: | 829 | recipe contains the following statement: |
@@ -839,7 +839,7 @@ | |||
839 | definition to be enabled and included in an image, it | 839 | definition to be enabled and included in an image, it |
840 | needs to have a matching entry in the global | 840 | needs to have a matching entry in the global |
841 | <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable | 841 | <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable |
842 | typically defined in your <filename>local.conf</filename> file. | 842 | typically defined in your <filename>local.conf</filename> file. |
843 | For example, to enable | 843 | For example, to enable |
844 | the <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> | 844 | the <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> |
845 | package, you could add either the string | 845 | package, you could add either the string |
@@ -867,7 +867,7 @@ | |||
867 | the initial underscore character or characters. | 867 | the initial underscore character or characters. |
868 | A partial string will match | 868 | A partial string will match |
869 | any license that contains the given string as the first | 869 | any license that contains the given string as the first |
870 | portion of its license. | 870 | portion of its license. |
871 | For example, the following | 871 | For example, the following |
872 | whitelist string will also match both of the packages | 872 | whitelist string will also match both of the packages |
873 | previously mentioned as well as any other packages that have | 873 | previously mentioned as well as any other packages that have |
@@ -879,7 +879,7 @@ | |||
879 | 879 | ||
880 | <section id="license-flag-matching"> | 880 | <section id="license-flag-matching"> |
881 | <title>License Flag Matching</title> | 881 | <title>License Flag Matching</title> |
882 | 882 | ||
883 | <para> | 883 | <para> |
884 | The definition of 'matching' in reference to a | 884 | The definition of 'matching' in reference to a |
885 | recipe's <filename>LICENSE_FLAGS</filename> setting is simple. | 885 | recipe's <filename>LICENSE_FLAGS</filename> setting is simple. |
@@ -891,7 +891,7 @@ | |||
891 | Before a flag | 891 | Before a flag |
892 | defined by a particular recipe is tested against the | 892 | defined by a particular recipe is tested against the |
893 | contents of the <filename>LICENSE_FLAGS_WHITELIST</filename> variable, the | 893 | contents of the <filename>LICENSE_FLAGS_WHITELIST</filename> variable, the |
894 | string <filename>_${PN}</filename> (with | 894 | string <filename>_${PN}</filename> (with |
895 | <link linkend='var-PN'><filename>PN</filename></link> expanded of course) is | 895 | <link linkend='var-PN'><filename>PN</filename></link> expanded of course) is |
896 | appended to the flag, thus automatically making each | 896 | appended to the flag, thus automatically making each |
897 | <filename>LICENSE_FLAGS</filename> value recipe-specific. | 897 | <filename>LICENSE_FLAGS</filename> value recipe-specific. |
@@ -907,7 +907,7 @@ | |||
907 | You can broaden the match by | 907 | You can broaden the match by |
908 | putting any "_"-separated beginning subset of a | 908 | putting any "_"-separated beginning subset of a |
909 | <filename>LICENSE_FLAGS</filename> flag in the whitelist, which will also | 909 | <filename>LICENSE_FLAGS</filename> flag in the whitelist, which will also |
910 | match. | 910 | match. |
911 | For example, simply specifying "commercial" in | 911 | For example, simply specifying "commercial" in |
912 | the whitelist would match any expanded <filename>LICENSE_FLAGS</filename> | 912 | the whitelist would match any expanded <filename>LICENSE_FLAGS</filename> |
913 | definition starting with "commercial" such as | 913 | definition starting with "commercial" such as |
@@ -923,7 +923,7 @@ | |||
923 | <para> | 923 | <para> |
924 | Broadening the match allows for a range of specificity for the items | 924 | Broadening the match allows for a range of specificity for the items |
925 | in the whitelist, from more general to perfectly | 925 | in the whitelist, from more general to perfectly |
926 | specific. | 926 | specific. |
927 | So you have the choice of exhaustively | 927 | So you have the choice of exhaustively |
928 | enumerating each license flag in the whitelist to | 928 | enumerating each license flag in the whitelist to |
929 | allow only those specific recipes into the image, or | 929 | allow only those specific recipes into the image, or |
@@ -935,7 +935,7 @@ | |||
935 | <para> | 935 | <para> |
936 | This scheme works even if the flag already | 936 | This scheme works even if the flag already |
937 | has <filename>_${PN}</filename> appended - the extra <filename>_${PN}</filename> is | 937 | has <filename>_${PN}</filename> appended - the extra <filename>_${PN}</filename> is |
938 | redundant, but does not affect the outcome. | 938 | redundant, but does not affect the outcome. |
939 | For example, a license flag of "commercial_1.2_foo" would | 939 | For example, a license flag of "commercial_1.2_foo" would |
940 | turn into "commercial_1.2_foo_foo" and would match | 940 | turn into "commercial_1.2_foo_foo" and would match |
941 | both the general "commercial" and the specific | 941 | both the general "commercial" and the specific |
@@ -944,8 +944,8 @@ | |||
944 | "commercial_1.2_foo_foo" and "commercial_1.2", which | 944 | "commercial_1.2_foo_foo" and "commercial_1.2", which |
945 | does not make much sense regarding use in the whitelist. | 945 | does not make much sense regarding use in the whitelist. |
946 | </para> | 946 | </para> |
947 | 947 | ||
948 | <para> | 948 | <para> |
949 | For a versioned string, you could instead specify | 949 | For a versioned string, you could instead specify |
950 | "commercial_foo_1.2", which would turn into | 950 | "commercial_foo_1.2", which would turn into |
951 | "commercial_foo_1.2_foo". | 951 | "commercial_foo_1.2_foo". |
@@ -993,8 +993,8 @@ | |||
993 | </para> | 993 | </para> |
994 | 994 | ||
995 | <para> | 995 | <para> |
996 | Specifying audio and video plug-ins as part of the | 996 | Specifying audio and video plug-ins as part of the |
997 | <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and | 997 | <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and |
998 | <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements | 998 | <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements |
999 | or commercial qt components as part of | 999 | or commercial qt components as part of |
1000 | the <filename>COMMERCIAL_QT</filename> statement (along | 1000 | the <filename>COMMERCIAL_QT</filename> statement (along |
@@ -1006,6 +1006,6 @@ | |||
1006 | </section> | 1006 | </section> |
1007 | </section> | 1007 | </section> |
1008 | </chapter> | 1008 | </chapter> |
1009 | <!-- | 1009 | <!-- |
1010 | vim: expandtab tw=80 ts=4 | 1010 | vim: expandtab tw=80 ts=4 |
1011 | --> | 1011 | --> |