summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/technical-details.xml')
-rw-r--r--documentation/ref-manual/technical-details.xml1409
1 files changed, 1409 insertions, 0 deletions
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
new file mode 100644
index 0000000000..d34be750e3
--- /dev/null
+++ b/documentation/ref-manual/technical-details.xml
@@ -0,0 +1,1409 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='technical-details'>
6<title>Technical Details</title>
7
8 <para>
9 This chapter provides technical details for various parts of the
10 Yocto Project.
11 Currently, topics include Yocto Project components,
12 cross-toolchain generation, shared state (sstate) cache,
13 x32, Wayland support, and Licenses.
14 </para>
15
16<section id='usingpoky-components'>
17 <title>Yocto Project Components</title>
18
19 <para>
20 The
21 <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>
22 task executor together with various types of configuration files form
23 the OpenEmbedded Core.
24 This section overviews these components by describing their use and
25 how they interact.
26 </para>
27
28 <para>
29 BitBake handles the parsing and execution of the data files.
30 The data itself is of various types:
31 <itemizedlist>
32 <listitem><para><emphasis>Recipes:</emphasis> Provides details
33 about particular pieces of software.
34 </para></listitem>
35 <listitem><para><emphasis>Class Data:</emphasis> Abstracts
36 common build information (e.g. how to build a Linux kernel).
37 </para></listitem>
38 <listitem><para><emphasis>Configuration Data:</emphasis> Defines
39 machine-specific settings, policy decisions, and so forth.
40 Configuration data acts as the glue to bind everything
41 together.
42 </para></listitem>
43 </itemizedlist>
44 </para>
45
46 <para>
47 BitBake knows how to combine multiple data sources together and refers
48 to each data source as a layer.
49 For information on layers, see the
50 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
51 Creating Layers</ulink>" section of the Yocto Project Development Manual.
52 </para>
53
54 <para>
55 Following are some brief details on these core components.
56 For additional information on how these components interact during
57 a build, see the
58 "<link linkend='closer-look'>A Closer Look at the Yocto Project Development Environment</link>"
59 Chapter.
60 </para>
61
62 <section id='usingpoky-components-bitbake'>
63 <title>BitBake</title>
64
65 <para>
66 BitBake is the tool at the heart of the OpenEmbedded build system
67 and is responsible for parsing the
68 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
69 generating a list of tasks from it, and then executing those tasks.
70 </para>
71
72 <para>
73 This section briefly introduces BitBake.
74 If you want more information on BitBake, see the
75 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.
76 </para>
77
78 <para>
79 To see a list of the options BitBake supports, use either of
80 the following commands:
81 <literallayout class='monospaced'>
82 $ bitbake -h
83 $ bitbake --help
84 </literallayout>
85 </para>
86
87 <para>
88 The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
89 <filename>packagename</filename> is the name of the package you want to build
90 (referred to as the "target" in this manual).
91 The target often equates to the first part of a recipe's filename
92 (e.g. "foo" for a recipe named
93 <filename>foo_1.3.0-r0.bb</filename>).
94 So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
95 might type the following:
96 <literallayout class='monospaced'>
97 $ bitbake matchbox-desktop
98 </literallayout>
99 Several different versions of <filename>matchbox-desktop</filename> might exist.
100 BitBake chooses the one selected by the distribution configuration.
101 You can get more details about how BitBake chooses between different
102 target versions and providers in the
103 "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-providers'>Preferences and Providers</ulink>"
104 section of the BitBake User Manual.
105 </para>
106
107 <para>
108 BitBake also tries to execute any dependent tasks first.
109 So for example, before building <filename>matchbox-desktop</filename>, BitBake
110 would build a cross compiler and <filename>eglibc</filename> if they had not already
111 been built.
112 <note>This release of the Yocto Project does not support the <filename>glibc</filename>
113 GNU version of the Unix standard C library. By default, the OpenEmbedded build system
114 builds with <filename>eglibc</filename>.</note>
115 </para>
116
117 <para>
118 A useful BitBake option to consider is the <filename>-k</filename> or
119 <filename>--continue</filename> option.
120 This option instructs BitBake to try and continue processing the job
121 as long as possible even after encountering an error.
122 When an error occurs, the target that
123 failed and those that depend on it cannot be remade.
124 However, when you use this option other dependencies can still be
125 processed.
126 </para>
127 </section>
128
129 <section id='usingpoky-components-metadata'>
130 <title>Metadata (Recipes)</title>
131
132 <para>
133 Files that have the <filename>.bb</filename> suffix are "recipes"
134 files.
135 In general, a recipe contains information about a single piece of
136 software.
137 This information includes the location from which to download the
138 unaltered source, any source patches to be applied to that source
139 (if needed), which special configuration options to apply,
140 how to compile the source files, and how to package the compiled
141 output.
142 </para>
143
144 <para>
145 The term "package" is sometimes used to refer to recipes. However,
146 since the word "package" is used for the packaged output from the OpenEmbedded
147 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
148 this document avoids using the term "package" when referring to recipes.
149 </para>
150 </section>
151
152 <section id='usingpoky-components-classes'>
153 <title>Classes</title>
154
155 <para>
156 Class files (<filename>.bbclass</filename>) contain information that
157 is useful to share between
158 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
159 An example is the
160 <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
161 class, which contains common settings for any application that
162 Autotools uses.
163 The "<link linkend='ref-classes'>Classes</link>" chapter provides
164 details about classes and how to use them.
165 </para>
166 </section>
167
168 <section id='usingpoky-components-configuration'>
169 <title>Configuration</title>
170
171 <para>
172 The configuration files (<filename>.conf</filename>) define various configuration variables
173 that govern the OpenEmbedded build process.
174 These files fall into several areas that define machine configuration options,
175 distribution configuration options, compiler tuning options, general common configuration
176 options, and user configuration options in <filename>local.conf</filename>, which is found
177 in the
178 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
179 </para>
180 </section>
181</section>
182
183<section id="cross-development-toolchain-generation">
184 <title>Cross-Development Toolchain Generation</title>
185
186 <para>
187 The Yocto Project does most of the work for you when it comes to
188 creating
189 <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
190 This section provides some technical background on how
191 cross-development toolchains are created and used.
192 For more information on toolchains, you can also see the
193 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
194 </para>
195
196 <para>
197 In the Yocto Project development environment, cross-development
198 toolchains are used to build the image and applications that run on the
199 target hardware.
200 With just a few commands, the OpenEmbedded build system creates
201 these necessary toolchains for you.
202 </para>
203
204 <para>
205 The following figure shows a high-level build environment regarding
206 toolchain construction and use.
207 </para>
208
209 <para>
210 <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
211 </para>
212
213 <para>
214 Most of the work occurs on the Build Host.
215 This is the machine used to build images and generally work within the
216 the Yocto Project environment.
217 When you run BitBake to create an image, the OpenEmbedded build system
218 uses the host <filename>gcc</filename> compiler to bootstrap a
219 cross-compiler named <filename>gcc-cross</filename>.
220 The <filename>gcc-cross</filename> compiler is what BitBake uses to
221 compile source files when creating the target image.
222 You can think of <filename>gcc-cross</filename> simply as an
223 automatically generated cross-compiler that is used internally within
224 BitBake only.
225 </para>
226
227 <para>
228 The chain of events that occurs when <filename>gcc-cross</filename> is
229 bootstrapped is as follows:
230 <literallayout class='monospaced'>
231 gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> eglibc-initial -> eglibc -> gcc-cross -> gcc-runtime
232 </literallayout>
233 <itemizedlist>
234 <listitem><para><filename>gcc</filename>:
235 The build host's GNU Compiler Collection (GCC).
236 </para></listitem>
237 <listitem><para><filename>binutils-cross</filename>:
238 The bare minimum binary utilities needed in order to run
239 the <filename>gcc-cross-initial</filename> phase of the
240 bootstrap operation.
241 </para></listitem>
242 <listitem><para><filename>gcc-cross-initial</filename>:
243 An early stage of the bootstrap process for creating
244 the cross-compiler.
245 This stage builds enough of the <filename>gcc-cross</filename>,
246 the C library, and other pieces needed to finish building the
247 final cross-compiler in later stages.
248 This tool is a "native" package (i.e. it is designed to run on
249 the build host).
250 </para></listitem>
251 <listitem><para><filename>linux-libc-headers</filename>:
252 Headers needed for the cross-compiler.
253 </para></listitem>
254 <listitem><para><filename>eglibc-initial</filename>:
255 An initial version of the Embedded GLIBC needed to bootstrap
256 <filename>eglibc</filename>.
257 </para></listitem>
258 <listitem><para><filename>gcc-cross</filename>:
259 The final stage of the bootstrap process for the
260 cross-compiler.
261 This stage results in the actual cross-compiler that
262 BitBake uses when it builds an image for a targeted
263 device.
264 <note>
265 If you are replacing this cross compiler toolchain
266 with a custom version, you must replace
267 <filename>gcc-cross</filename>.
268 </note>
269 This tool is also a "native" package (i.e. it is
270 designed to run on the build host).
271 </para></listitem>
272 <listitem><para><filename>gcc-runtime</filename>:
273 Runtime libraries resulting from the toolchain bootstrapping
274 process.
275 This tool produces a binary that consists of the
276 runtime libraries need for the targeted device.
277 </para></listitem>
278 </itemizedlist>
279 </para>
280
281 <para>
282 You can use the OpenEmbedded build system to build an installer for
283 the relocatable SDK used to develop applications.
284 When you run the installer, it installs the toolchain, which contains
285 the development tools (e.g., the
286 <filename>gcc-cross-canadian</filename>),
287 <filename>binutils-cross-canadian</filename>, and other
288 <filename>nativesdk-*</filename> tools you need to cross-compile and
289 test your software.
290 The figure shows the commands you use to easily build out this
291 toolchain.
292 This cross-development toolchain is built to execute on the
293 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
294 which might or might not be the same
295 machine as the Build Host.
296 <note>
297 If your target architecture is supported by the Yocto Project,
298 you can take advantage of pre-built images that ship with the
299 Yocto Project and already contain cross-development toolchain
300 installers.
301 </note>
302 </para>
303
304 <para>
305 Here is the bootstrap process for the relocatable toolchain:
306 <literallayout class='monospaced'>
307 gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers -> eglibc-initial -> nativesdk-eglibc -> gcc-crosssdk -> gcc-cross-canadian
308 </literallayout>
309 <itemizedlist>
310 <listitem><para><filename>gcc</filename>:
311 The build host's GNU Compiler Collection (GCC).
312 </para></listitem>
313 <listitem><para><filename>binutils-crosssdk</filename>:
314 The bare minimum binary utilities needed in order to run
315 the <filename>gcc-crosssdk-initial</filename> phase of the
316 bootstrap operation.
317 </para></listitem>
318 <listitem><para><filename>gcc-crosssdk-initial</filename>:
319 An early stage of the bootstrap process for creating
320 the cross-compiler.
321 This stage builds enough of the
322 <filename>gcc-crosssdk</filename> and supporting pieces so that
323 the final stage of the bootstrap process can produce the
324 finished cross-compiler.
325 This tool is a "native" binary that runs on the build host.
326 </para></listitem>
327 <listitem><para><filename>linux-libc-headers</filename>:
328 Headers needed for the cross-compiler.
329 </para></listitem>
330 <listitem><para><filename>eglibc-initial</filename>:
331 An initial version of the Embedded GLIBC needed to bootstrap
332 <filename>nativesdk-eglibc</filename>.
333 </para></listitem>
334 <listitem><para><filename>nativesdk-eglibc</filename>:
335 The Embedded GLIBC needed to bootstrap the
336 <filename>gcc-crosssdk</filename>.
337 </para></listitem>
338 <listitem><para><filename>gcc-crosssdk</filename>:
339 The final stage of the bootstrap process for the
340 relocatable cross-compiler.
341 The <filename>gcc-crosssdk</filename> is a transitory compiler
342 and never leaves the build host.
343 Its purpose is to help in the bootstrap process to create the
344 eventual relocatable <filename>gcc-cross-canadian</filename>
345 compiler, which is relocatable.
346 This tool is also a "native" package (i.e. it is
347 designed to run on the build host).
348 </para></listitem>
349 <listitem><para><filename>gcc-cross-canadian</filename>:
350 The final relocatable cross-compiler.
351 When run on the
352 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
353 this tool
354 produces executable code that runs on the target device.
355 Only one cross-canadian compiler is produced per architecture
356 since they can be targeted at different processor optimizations
357 using configurations passed to the compiler through the
358 compile commands.
359 This circumvents the need for multiple compilers and thus
360 reduces the size of the toolchains.
361 </para></listitem>
362 </itemizedlist>
363 </para>
364
365 <note>
366 For information on advantages gained when building a
367 cross-development toolchain installer, see the
368 "<ulink url='&YOCTO_DOCS_ADT_URL;#optionally-building-a-toolchain-installer'>Optionally Building a Toolchain Installer</ulink>"
369 section in the Yocto Project Application Developer's Guide.
370 </note>
371</section>
372
373<section id="shared-state-cache">
374 <title>Shared State Cache</title>
375
376 <para>
377 By design, the OpenEmbedded build system builds everything from scratch unless
378 BitBake can determine that parts do not need to be rebuilt.
379 Fundamentally, building from scratch is attractive as it means all parts are
380 built fresh and there is no possibility of stale data causing problems.
381 When developers hit problems, they typically default back to building from scratch
382 so they know the state of things from the start.
383 </para>
384
385 <para>
386 Building an image from scratch is both an advantage and a disadvantage to the process.
387 As mentioned in the previous paragraph, building from scratch ensures that
388 everything is current and starts from a known state.
389 However, building from scratch also takes much longer as it generally means
390 rebuilding things that do not necessarily need to be rebuilt.
391 </para>
392
393 <para>
394 The Yocto Project implements shared state code that supports incremental builds.
395 The implementation of the shared state code answers the following questions that
396 were fundamental roadblocks within the OpenEmbedded incremental build support system:
397 <itemizedlist>
398 <listitem><para>What pieces of the system have changed and what pieces have
399 not changed?</para></listitem>
400 <listitem><para>How are changed pieces of software removed and replaced?</para></listitem>
401 <listitem><para>How are pre-built components that do not need to be rebuilt from scratch
402 used when they are available?</para></listitem>
403 </itemizedlist>
404 </para>
405
406 <para>
407 For the first question, the build system detects changes in the "inputs" to a given task by
408 creating a checksum (or signature) of the task's inputs.
409 If the checksum changes, the system assumes the inputs have changed and the task needs to be
410 rerun.
411 For the second question, the shared state (sstate) code tracks which tasks add which output
412 to the build process.
413 This means the output from a given task can be removed, upgraded or otherwise manipulated.
414 The third question is partly addressed by the solution for the second question
415 assuming the build system can fetch the sstate objects from remote locations and
416 install them if they are deemed to be valid.
417 </para>
418
419 <note>
420 The OpenEmbedded build system does not maintain
421 <link linkend='var-PR'><filename>PR</filename></link> information
422 as part of the shared state packages.
423 Consequently, considerations exist that affect maintaining shared
424 state feeds.
425 For information on how the OpenEmbedded build system
426 works with packages and can
427 track incrementing <filename>PR</filename> information, see the
428 "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
429 section.
430 </note>
431
432 <para>
433 The rest of this section goes into detail about the overall incremental build
434 architecture, the checksums (signatures), shared state, and some tips and tricks.
435 </para>
436
437 <section id='overall-architecture'>
438 <title>Overall Architecture</title>
439
440 <para>
441 When determining what parts of the system need to be built, BitBake
442 works on a per-task basis rather than a per-recipe basis.
443 You might wonder why using a per-task basis is preferred over a per-recipe basis.
444 To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
445 In this case, <filename>do_install</filename> and <filename>do_package</filename>
446 outputs are still valid.
447 However, with a per-recipe approach, the build would not include the
448 <filename>.deb</filename> files.
449 Consequently, you would have to invalidate the whole build and rerun it.
450 Rerunning everything is not the best solution.
451 Also, in this case, the core must be "taught" much about specific tasks.
452 This methodology does not scale well and does not allow users to easily add new tasks
453 in layers or as external recipes without touching the packaged-staging core.
454 </para>
455 </section>
456
457 <section id='checksums'>
458 <title>Checksums (Signatures)</title>
459
460 <para>
461 The shared state code uses a checksum, which is a unique signature of a task's
462 inputs, to determine if a task needs to be run again.
463 Because it is a change in a task's inputs that triggers a rerun, the process
464 needs to detect all the inputs to a given task.
465 For shell tasks, this turns out to be fairly easy because
466 the build process generates a "run" shell script for each task and
467 it is possible to create a checksum that gives you a good idea of when
468 the task's data changes.
469 </para>
470
471 <para>
472 To complicate the problem, there are things that should not be included in
473 the checksum.
474 First, there is the actual specific build path of a given task -
475 the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
476 It does not matter if the work directory changes because it should not
477 affect the output for target packages.
478 Also, the build process has the objective of making native or cross packages relocatable.
479 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
480 The simplistic approach for excluding the work directory is to set
481 <filename>WORKDIR</filename> to some fixed value and create the checksum
482 for the "run" script.
483 </para>
484
485 <para>
486 Another problem results from the "run" scripts containing functions that
487 might or might not get called.
488 The incremental build solution contains code that figures out dependencies
489 between shell functions.
490 This code is used to prune the "run" scripts down to the minimum set,
491 thereby alleviating this problem and making the "run" scripts much more
492 readable as a bonus.
493 </para>
494
495 <para>
496 So far we have solutions for shell scripts.
497 What about Python tasks?
498 The same approach applies even though these tasks are more difficult.
499 The process needs to figure out what variables a Python function accesses
500 and what functions it calls.
501 Again, the incremental build solution contains code that first figures out
502 the variable and function dependencies, and then creates a checksum for the data
503 used as the input to the task.
504 </para>
505
506 <para>
507 Like the <filename>WORKDIR</filename> case, situations exist where dependencies
508 should be ignored.
509 For these cases, you can instruct the build process to ignore a dependency
510 by using a line like the following:
511 <literallayout class='monospaced'>
512 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
513 </literallayout>
514 This example ensures that the <filename>PACKAGE_ARCHS</filename>
515 variable does not
516 depend on the value of
517 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>,
518 even if it does reference it.
519 </para>
520
521 <para>
522 Equally, there are cases where we need to add dependencies BitBake is not able to find.
523 You can accomplish this by using a line like the following:
524 <literallayout class='monospaced'>
525 PACKAGE_ARCHS[vardeps] = "MACHINE"
526 </literallayout>
527 This example explicitly adds the <filename>MACHINE</filename> variable as a
528 dependency for <filename>PACKAGE_ARCHS</filename>.
529 </para>
530
531 <para>
532 Consider a case with in-line Python, for example, where BitBake is not
533 able to figure out dependencies.
534 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
535 produces output when it discovers something for which it cannot figure out
536 dependencies.
537 The Yocto Project team has currently not managed to cover those dependencies
538 in detail and is aware of the need to fix this situation.
539 </para>
540
541 <para>
542 Thus far, this section has limited discussion to the direct inputs into a task.
543 Information based on direct inputs is referred to as the "basehash" in the
544 code.
545 However, there is still the question of a task's indirect inputs - the
546 things that were already built and present in the
547 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
548 The checksum (or signature) for a particular task needs to add the hashes
549 of all the tasks on which the particular task depends.
550 Choosing which dependencies to add is a policy decision.
551 However, the effect is to generate a master checksum that combines the basehash
552 and the hashes of the task's dependencies.
553 </para>
554
555 <para>
556 At the code level, there are a variety of ways both the basehash and the
557 dependent task hashes can be influenced.
558 Within the BitBake configuration file, we can give BitBake some extra information
559 to help it construct the basehash.
560 The following statement effectively results in a list of global variable
561 dependency excludes - variables never included in any checksum:
562 <literallayout class='monospaced'>
563 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
564 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
565 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
566 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
567 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
568 </literallayout>
569 The previous example excludes
570 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
571 since that variable is actually constructed as a path within
572 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
573 the whitelist.
574 </para>
575
576 <para>
577 The rules for deciding which hashes of dependent tasks to include through
578 dependency chains are more complex and are generally accomplished with a
579 Python function.
580 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
581 of this and also illustrates how you can insert your own policy into the system
582 if so desired.
583 This file defines the two basic signature generators <filename>OE-Core</filename>
584 uses: "OEBasic" and "OEBasicHash".
585 By default, there is a dummy "noop" signature handler enabled in BitBake.
586 This means that behavior is unchanged from previous versions.
587 <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
588 through this setting in the <filename>bitbake.conf</filename> file:
589 <literallayout class='monospaced'>
590 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
591 </literallayout>
592 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
593 "OEBasic" version but adds the task hash to the stamp files.
594 This results in any
595 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
596 change that changes the task hash, automatically
597 causing the task to be run again.
598 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
599 values, and changes to Metadata automatically ripple across the build.
600 </para>
601
602 <para>
603 It is also worth noting that the end result of these signature generators is to
604 make some dependency and hash information available to the build.
605 This information includes:
606 <itemizedlist>
607 <listitem><para><filename>BB_BASEHASH_task-&lt;taskname&gt;</filename>:
608 The base hashes for each task in the recipe.
609 </para></listitem>
610 <listitem><para><filename>BB_BASEHASH_&lt;filename:taskname&gt;</filename>:
611 The base hashes for each dependent task.
612 </para></listitem>
613 <listitem><para><filename>BBHASHDEPS_&lt;filename:taskname&gt;</filename>:
614 The task dependencies for each task.
615 </para></listitem>
616 <listitem><para><filename>BB_TASKHASH</filename>:
617 The hash of the currently running task.
618 </para></listitem>
619 </itemizedlist>
620 </para>
621 </section>
622
623 <section id='shared-state'>
624 <title>Shared State</title>
625
626 <para>
627 Checksums and dependencies, as discussed in the previous section, solve half the
628 problem of supporting a shared state.
629 The other part of the problem is being able to use checksum information during the build
630 and being able to reuse or rebuild specific components.
631 </para>
632
633 <para>
634 The
635 <link linkend='ref-classes-sstate'><filename>sstate</filename></link>
636 class is a relatively generic implementation of how to "capture"
637 a snapshot of a given task.
638 The idea is that the build process does not care about the source of a task's output.
639 Output could be freshly built or it could be downloaded and unpacked from
640 somewhere - the build process does not need to worry about its origin.
641 </para>
642
643 <para>
644 There are two types of output, one is just about creating a directory
645 in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
646 A good example is the output of either <filename>do_install</filename> or
647 <filename>do_package</filename>.
648 The other type of output occurs when a set of data is merged into a shared directory
649 tree such as the sysroot.
650 </para>
651
652 <para>
653 The Yocto Project team has tried to keep the details of the
654 implementation hidden in <filename>sstate</filename> class.
655 From a user's perspective, adding shared state wrapping to a task
656 is as simple as this <filename>do_deploy</filename> example taken
657 from the
658 <link linkend='ref-classes-deploy'><filename>deploy</filename></link>
659 class:
660 <literallayout class='monospaced'>
661 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
662 SSTATETASKS += "do_deploy"
663 do_deploy[sstate-name] = "deploy"
664 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
665 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
666
667 python do_deploy_setscene () {
668 sstate_setscene(d)
669 }
670 addtask do_deploy_setscene
671 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
672 </literallayout>
673 In this example, we add some extra flags to the task, a name field ("deploy"), an
674 input directory where the task sends data, and the output
675 directory where the data from the task should eventually be copied.
676 We also add a <filename>_setscene</filename> variant of the task and add the task
677 name to the <filename>SSTATETASKS</filename> list.
678 </para>
679
680 <para>
681 If you have a directory whose contents you need to preserve, you can do this with
682 a line like the following:
683 <literallayout class='monospaced'>
684 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
685 </literallayout>
686 This method, as well as the following example, also works for multiple directories.
687 <literallayout class='monospaced'>
688 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
689 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
690 do_package[sstate-lockfile] = "${PACKAGELOCK}"
691 </literallayout>
692 These methods also include the ability to take a lockfile when manipulating
693 shared state directory structures since some cases are sensitive to file
694 additions or removals.
695 </para>
696
697 <para>
698 Behind the scenes, the shared state code works by looking in
699 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
700 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
701 for shared state files.
702 Here is an example:
703 <literallayout class='monospaced'>
704 SSTATE_MIRRORS ?= "\
705 file://.* http://someserver.tld/share/sstate/PATH \n \
706 file://.* file:///some/local/dir/sstate/PATH"
707 </literallayout>
708 <note>
709 The shared state directory (<filename>SSTATE_DIR</filename>) is
710 organized into two-character subdirectories, where the subdirectory
711 names are based on the first two characters of the hash.
712 If the shared state directory structure for a mirror has the
713 same structure as <filename>SSTATE_DIR</filename>, you must
714 specify "PATH" as part of the URI to enable the build system
715 to map to the appropriate subdirectory.
716 </note>
717 </para>
718
719 <para>
720 The shared state package validity can be detected just by looking at the
721 filename since the filename contains the task checksum (or signature) as
722 described earlier in this section.
723 If a valid shared state package is found, the build process downloads it
724 and uses it to accelerate the task.
725 </para>
726
727 <para>
728 The build processes use the <filename>*_setscene</filename> tasks
729 for the task acceleration phase.
730 BitBake goes through this phase before the main execution code and tries
731 to accelerate any tasks for which it can find shared state packages.
732 If a shared state package for a task is available, the shared state
733 package is used.
734 This means the task and any tasks on which it is dependent are not
735 executed.
736 </para>
737
738 <para>
739 As a real world example, the aim is when building an IPK-based image,
740 only the <filename>do_package_write_ipk</filename> tasks would have their
741 shared state packages fetched and extracted.
742 Since the sysroot is not used, it would never get extracted.
743 This is another reason why a task-based approach is preferred over a
744 recipe-based approach, which would have to install the output from every task.
745 </para>
746 </section>
747
748 <section id='tips-and-tricks'>
749 <title>Tips and Tricks</title>
750
751 <para>
752 The code in the build system that supports incremental builds is not
753 simple code.
754 This section presents some tips and tricks that help you work around
755 issues related to shared state code.
756 </para>
757
758 <section id='debugging'>
759 <title>Debugging</title>
760
761 <para>
762 When things go wrong, debugging needs to be straightforward.
763 Because of this, the Yocto Project includes strong debugging
764 tools:
765 <itemizedlist>
766 <listitem><para>Whenever a shared state package is written out, so is a
767 corresponding <filename>.siginfo</filename> file.
768 This practice results in a pickled Python database of all
769 the metadata that went into creating the hash for a given shared state
770 package.</para></listitem>
771 <listitem><para>If you run BitBake with the <filename>--dump-signatures</filename>
772 (or <filename>-S</filename>) option, BitBake dumps out
773 <filename>.siginfo</filename> files in
774 the stamp directory for every task it would have executed instead of
775 building the specified target package.</para></listitem>
776 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
777 can process <filename>.siginfo</filename> files.
778 If you specify one of these files, BitBake dumps out the dependency
779 information in the file.
780 If you specify two files, BitBake compares the two files and dumps out
781 the differences between the two.
782 This more easily helps answer the question of "What
783 changed between X and Y?"</para></listitem>
784 </itemizedlist>
785 </para>
786 </section>
787
788 <section id='invalidating-shared-state'>
789 <title>Invalidating Shared State</title>
790
791 <para>
792 The OpenEmbedded build system uses checksums and shared state
793 cache to avoid unnecessarily rebuilding tasks.
794 Collectively, this scheme is known as "shared state code."
795 </para>
796
797 <para>
798 As with all schemes, this one has some drawbacks.
799 It is possible that you could make implicit changes to your
800 code that the checksum calculations do not take into
801 account.
802 These implicit changes affect a task's output but do not trigger
803 the shared state code into rebuilding a recipe.
804 Consider an example during which a tool changes its output.
805 Assume that the output of <filename>rpmdeps</filename> changes.
806 The result of the change should be that all the
807 <filename>package</filename> and
808 <filename>package_write_rpm</filename> shared state cache
809 items become invalid.
810 However, because the change to the output is
811 external to the code and therefore implicit,
812 the associated shared state cache items do not become
813 invalidated.
814 In this case, the build process uses the cached items rather
815 than running the task again.
816 Obviously, these types of implicit changes can cause problems.
817 </para>
818
819 <para>
820 To avoid these problems during the build, you need to
821 understand the effects of any changes you make.
822 Realize that changes you make directly to a function
823 are automatically factored into the checksum calculation.
824 Thus, these explicit changes invalidate the associated area of
825 shared state cache.
826 However, you need to be aware of any implicit changes that
827 are not obvious changes to the code and could affect the output
828 of a given task.
829 </para>
830
831 <para>
832 When you identify an implicit change, you can easily take steps
833 to invalidate the cache and force the tasks to run.
834 The steps you can take are as simple as changing a function's
835 comments in the source code.
836 For example, to invalidate package shared state files, change
837 the comment statements of <filename>do_package</filename> or
838 the comments of one of the functions it calls.
839 Even though the change is purely cosmetic, it causes the
840 checksum to be recalculated and forces the OpenEmbedded build
841 system to run the task again.
842 </para>
843
844 <note>
845 For an example of a commit that makes a cosmetic change to
846 invalidate shared state, see this
847 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
848 </note>
849 </section>
850 </section>
851</section>
852
853<section id='x32'>
854 <title>x32</title>
855
856 <para>
857 x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
858 An ABI defines the calling conventions between functions in a processing environment.
859 The interface determines what registers are used and what the sizes are for various C data types.
860 </para>
861
862 <para>
863 Some processing environments prefer using 32-bit applications even when running
864 on Intel 64-bit platforms.
865 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
866 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
867 leaving the system underutilized.
868 Now consider the x86_64 psABI.
869 This ABI is newer and uses 64-bits for data sizes and program pointers.
870 The extra bits increase the footprint size of the programs, libraries,
871 and also increases the memory and file system size requirements.
872 Executing under the x32 psABI enables user programs to utilize CPU and system resources
873 more efficiently while keeping the memory footprint of the applications low.
874 Extra bits are used for registers but not for addressing mechanisms.
875 </para>
876
877 <section id='support'>
878 <title>Support</title>
879
880 <para>
881 This Yocto Project release supports the final specifications of x32
882 psABI.
883 Support for x32 psABI exists as follows:
884 <itemizedlist>
885 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
886 </para></listitem>
887 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
888 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
889 <filename>core-image-sato</filename> images.</para></listitem>
890 </itemizedlist>
891 </para>
892 </section>
893
894 <section id='completing-x32'>
895 <title>Completing x32</title>
896
897 <para>
898 Future Plans for the x32 psABI in the Yocto Project include the following:
899 <itemizedlist>
900 <listitem><para>Enhance and fix the few remaining recipes so they
901 work with and support x32 toolchains.</para></listitem>
902 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
903 <listitem><para>Support larger images.</para></listitem>
904 </itemizedlist>
905 </para>
906 </section>
907
908 <section id='using-x32-right-now'>
909 <title>Using x32 Right Now</title>
910
911 <para>
912 Follow these steps to use the x32 spABI:
913 <itemizedlist>
914 <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
915 machines by editing the <filename>conf/local.conf</filename> like this:
916 <literallayout class='monospaced'>
917 MACHINE = "qemux86-64"
918 DEFAULTTUNE = "x86-64-x32"
919 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
920 or 'INVALID'), True) or 'lib'}"
921 #MACHINE = "genericx86"
922 #DEFAULTTUNE = "core2-64-x32"
923 </literallayout></para></listitem>
924 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
925 Here is an example:
926 <literallayout class='monospaced'>
927 $ bitbake core-image-sato
928 </literallayout></para></listitem>
929 <listitem><para>As usual, run your image using QEMU:
930 <literallayout class='monospaced'>
931 $ runqemu qemux86-64 core-image-sato
932 </literallayout></para></listitem>
933 </itemizedlist>
934 </para>
935 </section>
936</section>
937
938<section id="wayland">
939 <title>Wayland</title>
940
941 <para>
942 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
943 is a computer display server protocol that
944 provides a method for compositing window managers to communicate
945 directly with applications and video hardware and expects them to
946 communicate with input hardware using other libraries.
947 Using Wayland with supporting targets can result in better control
948 over graphics frame rendering than an application might otherwise
949 achieve.
950 </para>
951
952 <para>
953 The Yocto Project provides the Wayland protocol libraries and the
954 reference
955 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
956 compositor as part of its release.
957 This section describes what you need to do to implement Wayland and
958 use the compositor when building an image for a supporting target.
959 </para>
960
961 <section id="wayland-support">
962 <title>Support</title>
963
964 <para>
965 The Wayland protocol libraries and the reference Weston compositor
966 ship as integrated packages in the <filename>meta</filename> layer
967 of the
968 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
969 Specifically, you can find the recipes that build both Wayland
970 and Weston at <filename>meta/recipes-graphics/wayland</filename>.
971 </para>
972
973 <para>
974 You can build both the Wayland and Weston packages for use only
975 with targets that accept the
976 <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
977 which is also known as Mesa DRI.
978 This implies that you cannot build and use the packages if your
979 target uses, for example, the
980 <trademark class='registered'>Intel</trademark> Embedded Media and
981 Graphics Driver (<trademark class='registered'>Intel</trademark>
982 EMGD) that overrides Mesa DRI.
983 </para>
984
985 <note>
986 Due to lack of EGL support, Weston 1.0.3 will not run directly on
987 the emulated QEMU hardware.
988 However, this version of Weston will run under X emulation without
989 issues.
990 </note>
991 </section>
992
993 <section id="enabling-wayland-in-an-image">
994 <title>Enabling Wayland in an Image</title>
995
996 <para>
997 To enable Wayland, you need to enable it to be built and enable
998 it to be included in the image.
999 </para>
1000
1001 <section id="enable-building">
1002 <title>Building</title>
1003
1004 <para>
1005 To cause Mesa to build the <filename>wayland-egl</filename>
1006 platform and Weston to build Wayland with Kernel Mode
1007 Setting
1008 (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
1009 support, include the "wayland" flag in the
1010 <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
1011 statement in your <filename>local.conf</filename> file:
1012 <literallayout class='monospaced'>
1013 DISTRO_FEATURES_append = " wayland"
1014 </literallayout>
1015 </para>
1016
1017 <note>
1018 If X11 has been enabled elsewhere, Weston will build Wayland
1019 with X11 support
1020 </note>
1021 </section>
1022
1023 <section id="enable-installation-in-an-image">
1024 <title>Installing</title>
1025
1026 <para>
1027 To install the Wayland feature into an image, you must
1028 include the following
1029 <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
1030 statement in your <filename>local.conf</filename> file:
1031 <literallayout class='monospaced'>
1032 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
1033 </literallayout>
1034 </para>
1035 </section>
1036 </section>
1037
1038 <section id="running-weston">
1039 <title>Running Weston</title>
1040
1041 <para>
1042 To run Weston inside X11, enabling it as described earlier and
1043 building a Sato image is sufficient.
1044 If you are running your image under Sato, a Weston Launcher appears
1045 in the "Utility" category.
1046 </para>
1047
1048 <para>
1049 Alternatively, you can run Weston through the command-line
1050 interpretor (CLI), which is better suited for development work.
1051 To run Weston under the CLI, you need to do the following after
1052 your image is built:
1053 <orderedlist>
1054 <listitem><para>Run these commands to export
1055 <filename>XDG_RUNTIME_DIR</filename>:
1056 <literallayout class='monospaced'>
1057 mkdir -p /tmp/$USER-weston
1058 chmod 0700 /tmp/$USER-weston
1059 export XDG_RUNTIME_DIR=/tmp/$USER-weston
1060 </literallayout></para></listitem>
1061 <listitem><para>Launch Weston in the shell:
1062 <literallayout class='monospaced'>
1063 weston
1064 </literallayout></para></listitem>
1065 </orderedlist>
1066 </para>
1067 </section>
1068</section>
1069
1070<section id="licenses">
1071 <title>Licenses</title>
1072
1073 <para>
1074 This section describes the mechanism by which the OpenEmbedded build system
1075 tracks changes to licensing text.
1076 The section also describes how to enable commercially licensed recipes,
1077 which by default are disabled.
1078 </para>
1079
1080 <para>
1081 For information that can help you maintain compliance with various open
1082 source licensing during the lifecycle of the product, see the
1083 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
1084 in the Yocto Project Development Manual.
1085 </para>
1086
1087 <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
1088 <title>Tracking License Changes</title>
1089
1090 <para>
1091 The license of an upstream project might change in the future.
1092 In order to prevent these changes going unnoticed, the
1093 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
1094 variable tracks changes to the license text. The checksums are validated at the end of the
1095 configure step, and if the checksums do not match, the build will fail.
1096 </para>
1097
1098 <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
1099 <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
1100
1101 <para>
1102 The <filename>LIC_FILES_CHKSUM</filename>
1103 variable contains checksums of the license text in the source code for the recipe.
1104 Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
1105 <literallayout class='monospaced'>
1106 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
1107 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
1108 file://licfile2.txt;endline=50;md5=zzzz \
1109 ..."
1110 </literallayout>
1111 </para>
1112
1113 <para>
1114 The build system uses the
1115 <filename><link linkend='var-S'>S</link></filename> variable as
1116 the default directory when searching files listed in
1117 <filename>LIC_FILES_CHKSUM</filename>.
1118 The previous example employs the default directory.
1119 </para>
1120
1121 <para>
1122 Consider this next example:
1123 <literallayout class='monospaced'>
1124 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
1125 md5=bb14ed3c4cda583abc85401304b5cd4e"
1126 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
1127 </literallayout>
1128 </para>
1129
1130 <para>
1131 The first line locates a file in
1132 <filename>${S}/src/ls.c</filename>.
1133 The second line refers to a file in
1134 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
1135 </para>
1136 <para>
1137 Note that <filename>LIC_FILES_CHKSUM</filename> variable is
1138 mandatory for all recipes, unless the
1139 <filename>LICENSE</filename> variable is set to "CLOSED".
1140 </para>
1141 </section>
1142
1143 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
1144 <title>Explanation of Syntax</title>
1145 <para>
1146 As mentioned in the previous section, the
1147 <filename>LIC_FILES_CHKSUM</filename> variable lists all the
1148 important files that contain the license text for the source code.
1149 It is possible to specify a checksum for an entire file, or a specific section of a
1150 file (specified by beginning and ending line numbers with the "beginline" and "endline"
1151 parameters, respectively).
1152 The latter is useful for source files with a license notice header,
1153 README documents, and so forth.
1154 If you do not use the "beginline" parameter, then it is assumed that the text begins on the
1155 first line of the file.
1156 Similarly, if you do not use the "endline" parameter, it is assumed that the license text
1157 ends with the last line of the file.
1158 </para>
1159
1160 <para>
1161 The "md5" parameter stores the md5 checksum of the license text.
1162 If the license text changes in any way as compared to this parameter
1163 then a mismatch occurs.
1164 This mismatch triggers a build failure and notifies the developer.
1165 Notification allows the developer to review and address the license text changes.
1166 Also note that if a mismatch occurs during the build, the correct md5
1167 checksum is placed in the build log and can be easily copied to the recipe.
1168 </para>
1169
1170 <para>
1171 There is no limit to how many files you can specify using the
1172 <filename>LIC_FILES_CHKSUM</filename> variable.
1173 Generally, however, every project requires a few specifications for license tracking.
1174 Many projects have a "COPYING" file that stores the license information for all the source
1175 code files.
1176 This practice allows you to just track the "COPYING" file as long as it is kept up to date.
1177 </para>
1178
1179 <tip>
1180 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
1181 error and displays the correct "md5" parameter value during the build.
1182 The correct parameter is also captured in the build log.
1183 </tip>
1184
1185 <tip>
1186 If the whole file contains only license text, you do not need to use the "beginline" and
1187 "endline" parameters.
1188 </tip>
1189 </section>
1190 </section>
1191
1192 <section id="enabling-commercially-licensed-recipes">
1193 <title>Enabling Commercially Licensed Recipes</title>
1194
1195 <para>
1196 By default, the OpenEmbedded build system disables
1197 components that have commercial or other special licensing
1198 requirements.
1199 Such requirements are defined on a
1200 recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable
1201 definition in the affected recipe.
1202 For instance, the
1203 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1204 recipe contains the following statement:
1205 <literallayout class='monospaced'>
1206 LICENSE_FLAGS = "commercial"
1207 </literallayout>
1208 Here is a slightly more complicated example that contains both an
1209 explicit recipe name and version (after variable expansion):
1210 <literallayout class='monospaced'>
1211 LICENSE_FLAGS = "license_${PN}_${PV}"
1212 </literallayout>
1213 In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
1214 definition to be enabled and included in an image, it
1215 needs to have a matching entry in the global
1216 <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable
1217 typically defined in your <filename>local.conf</filename> file.
1218 For example, to enable
1219 the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1220 package, you could add either the string
1221 "commercial_gst-plugins-ugly" or the more general string
1222 "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
1223 See the
1224 "<link linkend='license-flag-matching'>License Flag Matching</link>" section
1225 for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
1226 Here is the example:
1227 <literallayout class='monospaced'>
1228 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
1229 </literallayout>
1230 Likewise, to additionally enable the package built from the recipe containing
1231 <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
1232 that the actual recipe name was <filename>emgd_1.10.bb</filename>,
1233 the following string would enable that package as well as
1234 the original <filename>gst-plugins-ugly</filename> package:
1235 <literallayout class='monospaced'>
1236 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
1237 </literallayout>
1238 As a convenience, you do not need to specify the complete license string
1239 in the whitelist for every package.
1240 You can use an abbreviated form, which consists
1241 of just the first portion or portions of the license string before
1242 the initial underscore character or characters.
1243 A partial string will match
1244 any license that contains the given string as the first
1245 portion of its license.
1246 For example, the following
1247 whitelist string will also match both of the packages
1248 previously mentioned as well as any other packages that have
1249 licenses starting with "commercial" or "license".
1250 <literallayout class='monospaced'>
1251 LICENSE_FLAGS_WHITELIST = "commercial license"
1252 </literallayout>
1253 </para>
1254
1255 <section id="license-flag-matching">
1256 <title>License Flag Matching</title>
1257
1258 <para>
1259 License flag matching allows you to control what recipes the
1260 OpenEmbedded build system includes in the build.
1261 Fundamentally, the build system attempts to match
1262 <filename>LICENSE_FLAGS</filename> strings found in
1263 recipes against <filename>LICENSE_FLAGS_WHITELIST</filename>
1264 strings found in the whitelist.
1265 A match causes the build system to include a recipe in the
1266 build, while failure to find a match causes the build system to
1267 exclude a recipe.
1268 </para>
1269
1270 <para>
1271 In general, license flag matching is simple.
1272 However, understanding some concepts will help you
1273 correctly and effectively use matching.
1274 </para>
1275
1276 <para>
1277 Before a flag
1278 defined by a particular recipe is tested against the
1279 contents of the whitelist, the expanded string
1280 <filename>_${PN}</filename> is appended to the flag.
1281 This expansion makes each <filename>LICENSE_FLAGS</filename>
1282 value recipe-specific.
1283 After expansion, the string is then matched against the
1284 whitelist.
1285 Thus, specifying
1286 <filename>LICENSE_FLAGS = "commercial"</filename>
1287 in recipe "foo", for example, results in the string
1288 <filename>"commercial_foo"</filename>.
1289 And, to create a match, that string must appear in the
1290 whitelist.
1291 </para>
1292
1293 <para>
1294 Judicious use of the <filename>LICENSE_FLAGS</filename>
1295 strings and the contents of the
1296 <filename>LICENSE_FLAGS_WHITELIST</filename> variable
1297 allows you a lot of flexibility for including or excluding
1298 recipes based on licensing.
1299 For example, you can broaden the matching capabilities by
1300 using license flags string subsets in the whitelist.
1301 <note>When using a string subset, be sure to use the part of
1302 the expanded string that precedes the appended underscore
1303 character (e.g. <filename>usethispart_1.3</filename>,
1304 <filename>usethispart_1.4</filename>, and so forth).
1305 </note>
1306 For example, simply specifying the string "commercial" in
1307 the whitelist matches any expanded
1308 <filename>LICENSE_FLAGS</filename> definition that starts with
1309 the string "commercial" such as "commercial_foo" and
1310 "commercial_bar", which are the strings the build system
1311 automatically generates for hypothetical recipes named
1312 "foo" and "bar" assuming those recipes simply specify the
1313 following:
1314 <literallayout class='monospaced'>
1315 LICENSE_FLAGS = "commercial"
1316 </literallayout>
1317 Thus, you can choose to exhaustively
1318 enumerate each license flag in the whitelist and
1319 allow only specific recipes into the image, or
1320 you can use a string subset that causes a broader range of
1321 matches to allow a range of recipes into the image.
1322 </para>
1323
1324 <para>
1325 This scheme works even if the
1326 <filename>LICENSE_FLAGS</filename> string already
1327 has <filename>_${PN}</filename> appended.
1328 For example, the build system turns the license flag
1329 "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
1330 match both the general "commercial" and the specific
1331 "commercial_1.2_foo" strings found in the whitelist, as
1332 expected.
1333 </para>
1334
1335 <para>
1336 Here are some other scenarios:
1337 <itemizedlist>
1338 <listitem><para>You can specify a versioned string in the
1339 recipe such as "commercial_foo_1.2" in a "foo" recipe.
1340 The build system expands this string to
1341 "commercial_foo_1.2_foo".
1342 Combine this license flag with a whitelist that has
1343 the string "commercial" and you match the flag along
1344 with any other flag that starts with the string
1345 "commercial".</para></listitem>
1346 <listitem><para>Under the same circumstances, you can
1347 use "commercial_foo" in the whitelist and the
1348 build system not only matches "commercial_foo_1.2" but
1349 also matches any license flag with the string
1350 "commercial_foo", regardless of the version.
1351 </para></listitem>
1352 <listitem><para>You can be very specific and use both the
1353 package and version parts in the whitelist (e.g.
1354 "commercial_foo_1.2") to specifically match a
1355 versioned recipe.</para></listitem>
1356 </itemizedlist>
1357 </para>
1358 </section>
1359
1360 <section id="other-variables-related-to-commercial-licenses">
1361 <title>Other Variables Related to Commercial Licenses</title>
1362
1363 <para>
1364 Other helpful variables related to commercial
1365 license handling exist and are defined in the
1366 <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
1367 <literallayout class='monospaced'>
1368 COMMERCIAL_AUDIO_PLUGINS ?= ""
1369 COMMERCIAL_VIDEO_PLUGINS ?= ""
1370 COMMERCIAL_QT = ""
1371 </literallayout>
1372 If you want to enable these components, you can do so by making sure you have
1373 statements similar to the following
1374 in your <filename>local.conf</filename> configuration file:
1375 <literallayout class='monospaced'>
1376 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
1377 gst-plugins-ugly-mpegaudioparse"
1378 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
1379 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
1380 COMMERCIAL_QT ?= "qmmp"
1381 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
1382 </literallayout>
1383 Of course, you could also create a matching whitelist
1384 for those components using the more general "commercial"
1385 in the whitelist, but that would also enable all the
1386 other packages with <filename>LICENSE_FLAGS</filename> containing
1387 "commercial", which you may or may not want:
1388 <literallayout class='monospaced'>
1389 LICENSE_FLAGS_WHITELIST = "commercial"
1390 </literallayout>
1391 </para>
1392
1393 <para>
1394 Specifying audio and video plug-ins as part of the
1395 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
1396 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
1397 or commercial Qt components as part of
1398 the <filename>COMMERCIAL_QT</filename> statement (along
1399 with the enabling <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
1400 plug-ins or components into built images, thus adding
1401 support for media formats or components.
1402 </para>
1403 </section>
1404 </section>
1405</section>
1406</chapter>
1407<!--
1408vim: expandtab tw=80 ts=4
1409-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-19 22:43:11 +0000