summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/technical-details.xml')
-rw-r--r--documentation/ref-manual/technical-details.xml1419
1 files changed, 1419 insertions, 0 deletions
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
new file mode 100644
index 0000000..8bba42b
--- /dev/null
+++ b/documentation/ref-manual/technical-details.xml
@@ -0,0 +1,1419 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='technical-details'>
6<title>Technical Details</title>
7
8 <para>
9 This chapter provides technical details for various parts of the
10 Yocto Project.
11 Currently, topics include Yocto Project components,
12 cross-toolchain generation, shared state (sstate) cache,
13 x32, Wayland support, and Licenses.
14 </para>
15
16<section id='usingpoky-components'>
17 <title>Yocto Project Components</title>
18
19 <para>
20 The
21 <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>
22 task executor together with various types of configuration files form
23 the OpenEmbedded Core.
24 This section overviews these components by describing their use and
25 how they interact.
26 </para>
27
28 <para>
29 BitBake handles the parsing and execution of the data files.
30 The data itself is of various types:
31 <itemizedlist>
32 <listitem><para><emphasis>Recipes:</emphasis> Provides details
33 about particular pieces of software.
34 </para></listitem>
35 <listitem><para><emphasis>Class Data:</emphasis> Abstracts
36 common build information (e.g. how to build a Linux kernel).
37 </para></listitem>
38 <listitem><para><emphasis>Configuration Data:</emphasis> Defines
39 machine-specific settings, policy decisions, and so forth.
40 Configuration data acts as the glue to bind everything
41 together.
42 </para></listitem>
43 </itemizedlist>
44 </para>
45
46 <para>
47 BitBake knows how to combine multiple data sources together and refers
48 to each data source as a layer.
49 For information on layers, see the
50 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
51 Creating Layers</ulink>" section of the Yocto Project Development Manual.
52 </para>
53
54 <para>
55 Following are some brief details on these core components.
56 For additional information on how these components interact during
57 a build, see the
58 "<link linkend='closer-look'>A Closer Look at the Yocto Project Development Environment</link>"
59 Chapter.
60 </para>
61
62 <section id='usingpoky-components-bitbake'>
63 <title>BitBake</title>
64
65 <para>
66 BitBake is the tool at the heart of the OpenEmbedded build system
67 and is responsible for parsing the
68 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
69 generating a list of tasks from it, and then executing those tasks.
70 </para>
71
72 <para>
73 This section briefly introduces BitBake.
74 If you want more information on BitBake, see the
75 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.
76 </para>
77
78 <para>
79 To see a list of the options BitBake supports, use either of
80 the following commands:
81 <literallayout class='monospaced'>
82 $ bitbake -h
83 $ bitbake --help
84 </literallayout>
85 </para>
86
87 <para>
88 The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
89 <filename>packagename</filename> is the name of the package you want to build
90 (referred to as the "target" in this manual).
91 The target often equates to the first part of a recipe's filename
92 (e.g. "foo" for a recipe named
93 <filename>foo_1.3.0-r0.bb</filename>).
94 So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
95 might type the following:
96 <literallayout class='monospaced'>
97 $ bitbake matchbox-desktop
98 </literallayout>
99 Several different versions of <filename>matchbox-desktop</filename> might exist.
100 BitBake chooses the one selected by the distribution configuration.
101 You can get more details about how BitBake chooses between different
102 target versions and providers in the
103 "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences'>Preferences</ulink>"
104 section of the BitBake User Manual.
105 </para>
106
107 <para>
108 BitBake also tries to execute any dependent tasks first.
109 So for example, before building <filename>matchbox-desktop</filename>, BitBake
110 would build a cross compiler and <filename>eglibc</filename> if they had not already
111 been built.
112 <note>This release of the Yocto Project does not support the <filename>glibc</filename>
113 GNU version of the Unix standard C library. By default, the OpenEmbedded build system
114 builds with <filename>eglibc</filename>.</note>
115 </para>
116
117 <para>
118 A useful BitBake option to consider is the <filename>-k</filename> or
119 <filename>--continue</filename> option.
120 This option instructs BitBake to try and continue processing the job
121 as long as possible even after encountering an error.
122 When an error occurs, the target that
123 failed and those that depend on it cannot be remade.
124 However, when you use this option other dependencies can still be
125 processed.
126 </para>
127 </section>
128
129 <section id='usingpoky-components-metadata'>
130 <title>Metadata (Recipes)</title>
131
132 <para>
133 Files that have the <filename>.bb</filename> suffix are "recipes"
134 files.
135 In general, a recipe contains information about a single piece of
136 software.
137 This information includes the location from which to download the
138 unaltered source, any source patches to be applied to that source
139 (if needed), which special configuration options to apply,
140 how to compile the source files, and how to package the compiled
141 output.
142 </para>
143
144 <para>
145 The term "package" is sometimes used to refer to recipes. However,
146 since the word "package" is used for the packaged output from the OpenEmbedded
147 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
148 this document avoids using the term "package" when referring to recipes.
149 </para>
150 </section>
151
152 <section id='usingpoky-components-classes'>
153 <title>Classes</title>
154
155 <para>
156 Class files (<filename>.bbclass</filename>) contain information that
157 is useful to share between
158 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
159 An example is the
160 <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
161 class, which contains common settings for any application that
162 Autotools uses.
163 The "<link linkend='ref-classes'>Classes</link>" chapter provides
164 details about classes and how to use them.
165 </para>
166 </section>
167
168 <section id='usingpoky-components-configuration'>
169 <title>Configuration</title>
170
171 <para>
172 The configuration files (<filename>.conf</filename>) define various configuration variables
173 that govern the OpenEmbedded build process.
174 These files fall into several areas that define machine configuration options,
175 distribution configuration options, compiler tuning options, general common configuration
176 options, and user configuration options in <filename>local.conf</filename>, which is found
177 in the
178 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
179 </para>
180 </section>
181</section>
182
183<section id="cross-development-toolchain-generation">
184 <title>Cross-Development Toolchain Generation</title>
185
186 <para>
187 The Yocto Project does most of the work for you when it comes to
188 creating
189 <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
190 This section provides some technical background on how
191 cross-development toolchains are created and used.
192 For more information on toolchains, you can also see the
193 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
194 </para>
195
196 <para>
197 In the Yocto Project development environment, cross-development
198 toolchains are used to build the image and applications that run on the
199 target hardware.
200 With just a few commands, the OpenEmbedded build system creates
201 these necessary toolchains for you.
202 </para>
203
204 <para>
205 The following figure shows a high-level build environment regarding
206 toolchain construction and use.
207 </para>
208
209 <para>
210 <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
211 </para>
212
213 <para>
214 Most of the work occurs on the Build Host.
215 This is the machine used to build images and generally work within the
216 the Yocto Project environment.
217 When you run BitBake to create an image, the OpenEmbedded build system
218 uses the host <filename>gcc</filename> compiler to bootstrap a
219 cross-compiler named <filename>gcc-cross</filename>.
220 The <filename>gcc-cross</filename> compiler is what BitBake uses to
221 compile source files when creating the target image.
222 You can think of <filename>gcc-cross</filename> simply as an
223 automatically generated cross-compiler that is used internally within
224 BitBake only.
225 </para>
226
227 <para>
228 The chain of events that occurs when <filename>gcc-cross</filename> is
229 bootstrapped is as follows:
230 <literallayout class='monospaced'>
231 gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> eglibc-initial -> eglibc -> gcc-cross -> gcc-runtime
232 </literallayout>
233 <itemizedlist>
234 <listitem><para><filename>gcc</filename>:
235 The build host's GNU Compiler Collection (GCC).
236 </para></listitem>
237 <listitem><para><filename>binutils-cross</filename>:
238 The bare minimum binary utilities needed in order to run
239 the <filename>gcc-cross-initial</filename> phase of the
240 bootstrap operation.
241 </para></listitem>
242 <listitem><para><filename>gcc-cross-initial</filename>:
243 An early stage of the bootstrap process for creating
244 the cross-compiler.
245 This stage builds enough of the <filename>gcc-cross</filename>,
246 the C library, and other pieces needed to finish building the
247 final cross-compiler in later stages.
248 This tool is a "native" package (i.e. it is designed to run on
249 the build host).
250 </para></listitem>
251 <listitem><para><filename>linux-libc-headers</filename>:
252 Headers needed for the cross-compiler.
253 </para></listitem>
254 <listitem><para><filename>eglibc-initial</filename>:
255 An initial version of the Embedded GLIBC needed to bootstrap
256 <filename>eglibc</filename>.
257 </para></listitem>
258 <listitem><para><filename>gcc-cross</filename>:
259 The final stage of the bootstrap process for the
260 cross-compiler.
261 This stage results in the actual cross-compiler that
262 BitBake uses when it builds an image for a targeted
263 device.
264 <note>
265 If you are replacing this cross compiler toolchain
266 with a custom version, you must replace
267 <filename>gcc-cross</filename>.
268 </note>
269 This tool is also a "native" package (i.e. it is
270 designed to run on the build host).
271 </para></listitem>
272 <listitem><para><filename>gcc-runtime</filename>:
273 Runtime libraries resulting from the toolchain bootstrapping
274 process.
275 This tool produces a binary that consists of the
276 runtime libraries need for the targeted device.
277 </para></listitem>
278 </itemizedlist>
279 </para>
280
281 <para>
282 You can use the OpenEmbedded build system to build an installer for
283 the relocatable SDK used to develop applications.
284 When you run the installer, it installs the toolchain, which contains
285 the development tools (e.g., the
286 <filename>gcc-cross-canadian</filename>),
287 <filename>binutils-cross-canadian</filename>, and other
288 <filename>nativesdk-*</filename> tools you need to cross-compile and
289 test your software.
290 The figure shows the commands you use to easily build out this
291 toolchain.
292 This cross-development toolchain is built to execute on the
293 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
294 which might or might not be the same
295 machine as the Build Host.
296 <note>
297 If your target architecture is supported by the Yocto Project,
298 you can take advantage of pre-built images that ship with the
299 Yocto Project and already contain cross-development toolchain
300 installers.
301 </note>
302 </para>
303
304 <para>
305 Here is the bootstrap process for the relocatable toolchain:
306 <literallayout class='monospaced'>
307 gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers -> eglibc-initial -> nativesdk-eglibc -> gcc-crosssdk -> gcc-cross-canadian
308 </literallayout>
309 <itemizedlist>
310 <listitem><para><filename>gcc</filename>:
311 The build host's GNU Compiler Collection (GCC).
312 </para></listitem>
313 <listitem><para><filename>binutils-crosssdk</filename>:
314 The bare minimum binary utilities needed in order to run
315 the <filename>gcc-crosssdk-initial</filename> phase of the
316 bootstrap operation.
317 </para></listitem>
318 <listitem><para><filename>gcc-crosssdk-initial</filename>:
319 An early stage of the bootstrap process for creating
320 the cross-compiler.
321 This stage builds enough of the
322 <filename>gcc-crosssdk</filename> and supporting pieces so that
323 the final stage of the bootstrap process can produce the
324 finished cross-compiler.
325 This tool is a "native" binary that runs on the build host.
326 </para></listitem>
327 <listitem><para><filename>linux-libc-headers</filename>:
328 Headers needed for the cross-compiler.
329 </para></listitem>
330 <listitem><para><filename>eglibc-initial</filename>:
331 An initial version of the Embedded GLIBC needed to bootstrap
332 <filename>nativesdk-eglibc</filename>.
333 </para></listitem>
334 <listitem><para><filename>nativesdk-eglibc</filename>:
335 The Embedded GLIBC needed to bootstrap the
336 <filename>gcc-crosssdk</filename>.
337 </para></listitem>
338 <listitem><para><filename>gcc-crosssdk</filename>:
339 The final stage of the bootstrap process for the
340 relocatable cross-compiler.
341 The <filename>gcc-crosssdk</filename> is a transitory compiler
342 and never leaves the build host.
343 Its purpose is to help in the bootstrap process to create the
344 eventual relocatable <filename>gcc-cross-canadian</filename>
345 compiler, which is relocatable.
346 This tool is also a "native" package (i.e. it is
347 designed to run on the build host).
348 </para></listitem>
349 <listitem><para><filename>gcc-cross-canadian</filename>:
350 The final relocatable cross-compiler.
351 When run on the
352 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
353 this tool
354 produces executable code that runs on the target device.
355 Only one cross-canadian compiler is produced per architecture
356 since they can be targeted at different processor optimizations
357 using configurations passed to the compiler through the
358 compile commands.
359 This circumvents the need for multiple compilers and thus
360 reduces the size of the toolchains.
361 </para></listitem>
362 </itemizedlist>
363 </para>
364
365 <note>
366 For information on advantages gained when building a
367 cross-development toolchain installer, see the
368 "<ulink url='&YOCTO_DOCS_ADT_URL;#optionally-building-a-toolchain-installer'>Optionally Building a Toolchain Installer</ulink>"
369 section in the Yocto Project Application Developer's Guide.
370 </note>
371</section>
372
373<section id="shared-state-cache">
374 <title>Shared State Cache</title>
375
376 <para>
377 By design, the OpenEmbedded build system builds everything from scratch unless
378 BitBake can determine that parts do not need to be rebuilt.
379 Fundamentally, building from scratch is attractive as it means all parts are
380 built fresh and there is no possibility of stale data causing problems.
381 When developers hit problems, they typically default back to building from scratch
382 so they know the state of things from the start.
383 </para>
384
385 <para>
386 Building an image from scratch is both an advantage and a disadvantage to the process.
387 As mentioned in the previous paragraph, building from scratch ensures that
388 everything is current and starts from a known state.
389 However, building from scratch also takes much longer as it generally means
390 rebuilding things that do not necessarily need to be rebuilt.
391 </para>
392
393 <para>
394 The Yocto Project implements shared state code that supports incremental builds.
395 The implementation of the shared state code answers the following questions that
396 were fundamental roadblocks within the OpenEmbedded incremental build support system:
397 <itemizedlist>
398 <listitem><para>What pieces of the system have changed and what pieces have
399 not changed?</para></listitem>
400 <listitem><para>How are changed pieces of software removed and replaced?</para></listitem>
401 <listitem><para>How are pre-built components that do not need to be rebuilt from scratch
402 used when they are available?</para></listitem>
403 </itemizedlist>
404 </para>
405
406 <para>
407 For the first question, the build system detects changes in the "inputs" to a given task by
408 creating a checksum (or signature) of the task's inputs.
409 If the checksum changes, the system assumes the inputs have changed and the task needs to be
410 rerun.
411 For the second question, the shared state (sstate) code tracks which tasks add which output
412 to the build process.
413 This means the output from a given task can be removed, upgraded or otherwise manipulated.
414 The third question is partly addressed by the solution for the second question
415 assuming the build system can fetch the sstate objects from remote locations and
416 install them if they are deemed to be valid.
417 </para>
418
419 <note>
420 The OpenEmbedded build system does not maintain
421 <link linkend='var-PR'><filename>PR</filename></link> information
422 as part of the shared state packages.
423 Consequently, considerations exist that affect maintaining shared
424 state feeds.
425 For information on how the OpenEmbedded build system
426 works with packages and can
427 track incrementing <filename>PR</filename> information, see the
428 "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
429 section.
430 </note>
431
432 <para>
433 The rest of this section goes into detail about the overall incremental build
434 architecture, the checksums (signatures), shared state, and some tips and tricks.
435 </para>
436
437 <section id='overall-architecture'>
438 <title>Overall Architecture</title>
439
440 <para>
441 When determining what parts of the system need to be built, BitBake
442 works on a per-task basis rather than a per-recipe basis.
443 You might wonder why using a per-task basis is preferred over a per-recipe basis.
444 To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
445 In this case, the
446 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
447 and
448 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
449 task outputs are still valid.
450 However, with a per-recipe approach, the build would not include the
451 <filename>.deb</filename> files.
452 Consequently, you would have to invalidate the whole build and rerun it.
453 Rerunning everything is not the best solution.
454 Also, in this case, the core must be "taught" much about specific tasks.
455 This methodology does not scale well and does not allow users to easily add new tasks
456 in layers or as external recipes without touching the packaged-staging core.
457 </para>
458 </section>
459
460 <section id='checksums'>
461 <title>Checksums (Signatures)</title>
462
463 <para>
464 The shared state code uses a checksum, which is a unique signature of a task's
465 inputs, to determine if a task needs to be run again.
466 Because it is a change in a task's inputs that triggers a rerun, the process
467 needs to detect all the inputs to a given task.
468 For shell tasks, this turns out to be fairly easy because
469 the build process generates a "run" shell script for each task and
470 it is possible to create a checksum that gives you a good idea of when
471 the task's data changes.
472 </para>
473
474 <para>
475 To complicate the problem, there are things that should not be included in
476 the checksum.
477 First, there is the actual specific build path of a given task -
478 the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
479 It does not matter if the work directory changes because it should not
480 affect the output for target packages.
481 Also, the build process has the objective of making native or cross packages relocatable.
482 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
483 The simplistic approach for excluding the work directory is to set
484 <filename>WORKDIR</filename> to some fixed value and create the checksum
485 for the "run" script.
486 </para>
487
488 <para>
489 Another problem results from the "run" scripts containing functions that
490 might or might not get called.
491 The incremental build solution contains code that figures out dependencies
492 between shell functions.
493 This code is used to prune the "run" scripts down to the minimum set,
494 thereby alleviating this problem and making the "run" scripts much more
495 readable as a bonus.
496 </para>
497
498 <para>
499 So far we have solutions for shell scripts.
500 What about Python tasks?
501 The same approach applies even though these tasks are more difficult.
502 The process needs to figure out what variables a Python function accesses
503 and what functions it calls.
504 Again, the incremental build solution contains code that first figures out
505 the variable and function dependencies, and then creates a checksum for the data
506 used as the input to the task.
507 </para>
508
509 <para>
510 Like the <filename>WORKDIR</filename> case, situations exist where dependencies
511 should be ignored.
512 For these cases, you can instruct the build process to ignore a dependency
513 by using a line like the following:
514 <literallayout class='monospaced'>
515 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
516 </literallayout>
517 This example ensures that the
518 <link linkend='var-PACKAGE_ARCHS'><filename>PACKAGE_ARCHS</filename></link>
519 variable does not
520 depend on the value of
521 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>,
522 even if it does reference it.
523 </para>
524
525 <para>
526 Equally, there are cases where we need to add dependencies BitBake is not able to find.
527 You can accomplish this by using a line like the following:
528 <literallayout class='monospaced'>
529 PACKAGE_ARCHS[vardeps] = "MACHINE"
530 </literallayout>
531 This example explicitly adds the <filename>MACHINE</filename> variable as a
532 dependency for <filename>PACKAGE_ARCHS</filename>.
533 </para>
534
535 <para>
536 Consider a case with in-line Python, for example, where BitBake is not
537 able to figure out dependencies.
538 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
539 produces output when it discovers something for which it cannot figure out
540 dependencies.
541 The Yocto Project team has currently not managed to cover those dependencies
542 in detail and is aware of the need to fix this situation.
543 </para>
544
545 <para>
546 Thus far, this section has limited discussion to the direct inputs into a task.
547 Information based on direct inputs is referred to as the "basehash" in the
548 code.
549 However, there is still the question of a task's indirect inputs - the
550 things that were already built and present in the
551 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
552 The checksum (or signature) for a particular task needs to add the hashes
553 of all the tasks on which the particular task depends.
554 Choosing which dependencies to add is a policy decision.
555 However, the effect is to generate a master checksum that combines the basehash
556 and the hashes of the task's dependencies.
557 </para>
558
559 <para>
560 At the code level, there are a variety of ways both the basehash and the
561 dependent task hashes can be influenced.
562 Within the BitBake configuration file, we can give BitBake some extra information
563 to help it construct the basehash.
564 The following statement effectively results in a list of global variable
565 dependency excludes - variables never included in any checksum:
566 <literallayout class='monospaced'>
567 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
568 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
569 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
570 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
571 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
572 </literallayout>
573 The previous example excludes
574 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
575 since that variable is actually constructed as a path within
576 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
577 the whitelist.
578 </para>
579
580 <para>
581 The rules for deciding which hashes of dependent tasks to include through
582 dependency chains are more complex and are generally accomplished with a
583 Python function.
584 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
585 of this and also illustrates how you can insert your own policy into the system
586 if so desired.
587 This file defines the two basic signature generators <filename>OE-Core</filename>
588 uses: "OEBasic" and "OEBasicHash".
589 By default, there is a dummy "noop" signature handler enabled in BitBake.
590 This means that behavior is unchanged from previous versions.
591 <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
592 through this setting in the <filename>bitbake.conf</filename> file:
593 <literallayout class='monospaced'>
594 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
595 </literallayout>
596 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
597 "OEBasic" version but adds the task hash to the stamp files.
598 This results in any
599 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
600 change that changes the task hash, automatically
601 causing the task to be run again.
602 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
603 values, and changes to Metadata automatically ripple across the build.
604 </para>
605
606 <para>
607 It is also worth noting that the end result of these signature generators is to
608 make some dependency and hash information available to the build.
609 This information includes:
610 <itemizedlist>
611 <listitem><para><filename>BB_BASEHASH_task-&lt;taskname&gt;</filename>:
612 The base hashes for each task in the recipe.
613 </para></listitem>
614 <listitem><para><filename>BB_BASEHASH_&lt;filename:taskname&gt;</filename>:
615 The base hashes for each dependent task.
616 </para></listitem>
617 <listitem><para><filename>BBHASHDEPS_&lt;filename:taskname&gt;</filename>:
618 The task dependencies for each task.
619 </para></listitem>
620 <listitem><para><filename>BB_TASKHASH</filename>:
621 The hash of the currently running task.
622 </para></listitem>
623 </itemizedlist>
624 </para>
625 </section>
626
627 <section id='shared-state'>
628 <title>Shared State</title>
629
630 <para>
631 Checksums and dependencies, as discussed in the previous section, solve half the
632 problem of supporting a shared state.
633 The other part of the problem is being able to use checksum information during the build
634 and being able to reuse or rebuild specific components.
635 </para>
636
637 <para>
638 The
639 <link linkend='ref-classes-sstate'><filename>sstate</filename></link>
640 class is a relatively generic implementation of how to "capture"
641 a snapshot of a given task.
642 The idea is that the build process does not care about the source of a task's output.
643 Output could be freshly built or it could be downloaded and unpacked from
644 somewhere - the build process does not need to worry about its origin.
645 </para>
646
647 <para>
648 There are two types of output, one is just about creating a directory
649 in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
650 A good example is the output of either
651 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
652 or
653 <link linkend='ref-tasks-package'><filename>do_package</filename></link>.
654 The other type of output occurs when a set of data is merged into a shared directory
655 tree such as the sysroot.
656 </para>
657
658 <para>
659 The Yocto Project team has tried to keep the details of the
660 implementation hidden in <filename>sstate</filename> class.
661 From a user's perspective, adding shared state wrapping to a task
662 is as simple as this
663 <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link>
664 example taken from the
665 <link linkend='ref-classes-deploy'><filename>deploy</filename></link>
666 class:
667 <literallayout class='monospaced'>
668 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
669 SSTATETASKS += "do_deploy"
670 do_deploy[sstate-name] = "deploy"
671 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
672 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
673
674 python do_deploy_setscene () {
675 sstate_setscene(d)
676 }
677 addtask do_deploy_setscene
678 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
679 </literallayout>
680 In this example, we add some extra flags to the task, a name field ("deploy"), an
681 input directory where the task sends data, and the output
682 directory where the data from the task should eventually be copied.
683 We also add a <filename>_setscene</filename> variant of the task and add the task
684 name to the <filename>SSTATETASKS</filename> list.
685 </para>
686
687 <para>
688 If you have a directory whose contents you need to preserve, you can do this with
689 a line like the following:
690 <literallayout class='monospaced'>
691 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
692 </literallayout>
693 This method, as well as the following example, also works for multiple directories.
694 <literallayout class='monospaced'>
695 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
696 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
697 do_package[sstate-lockfile] = "${PACKAGELOCK}"
698 </literallayout>
699 These methods also include the ability to take a lockfile when manipulating
700 shared state directory structures since some cases are sensitive to file
701 additions or removals.
702 </para>
703
704 <para>
705 Behind the scenes, the shared state code works by looking in
706 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
707 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
708 for shared state files.
709 Here is an example:
710 <literallayout class='monospaced'>
711 SSTATE_MIRRORS ?= "\
712 file://.* http://someserver.tld/share/sstate/PATH \n \
713 file://.* file:///some/local/dir/sstate/PATH"
714 </literallayout>
715 <note>
716 The shared state directory (<filename>SSTATE_DIR</filename>) is
717 organized into two-character subdirectories, where the subdirectory
718 names are based on the first two characters of the hash.
719 If the shared state directory structure for a mirror has the
720 same structure as <filename>SSTATE_DIR</filename>, you must
721 specify "PATH" as part of the URI to enable the build system
722 to map to the appropriate subdirectory.
723 </note>
724 </para>
725
726 <para>
727 The shared state package validity can be detected just by looking at the
728 filename since the filename contains the task checksum (or signature) as
729 described earlier in this section.
730 If a valid shared state package is found, the build process downloads it
731 and uses it to accelerate the task.
732 </para>
733
734 <para>
735 The build processes use the <filename>*_setscene</filename> tasks
736 for the task acceleration phase.
737 BitBake goes through this phase before the main execution code and tries
738 to accelerate any tasks for which it can find shared state packages.
739 If a shared state package for a task is available, the shared state
740 package is used.
741 This means the task and any tasks on which it is dependent are not
742 executed.
743 </para>
744
745 <para>
746 As a real world example, the aim is when building an IPK-based image,
747 only the
748 <link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>
749 tasks would have their
750 shared state packages fetched and extracted.
751 Since the sysroot is not used, it would never get extracted.
752 This is another reason why a task-based approach is preferred over a
753 recipe-based approach, which would have to install the output from every task.
754 </para>
755 </section>
756
757 <section id='tips-and-tricks'>
758 <title>Tips and Tricks</title>
759
760 <para>
761 The code in the build system that supports incremental builds is not
762 simple code.
763 This section presents some tips and tricks that help you work around
764 issues related to shared state code.
765 </para>
766
767 <section id='debugging'>
768 <title>Debugging</title>
769
770 <para>
771 When things go wrong, debugging needs to be straightforward.
772 Because of this, the Yocto Project includes strong debugging
773 tools:
774 <itemizedlist>
775 <listitem><para>Whenever a shared state package is written out, so is a
776 corresponding <filename>.siginfo</filename> file.
777 This practice results in a pickled Python database of all
778 the metadata that went into creating the hash for a given shared state
779 package.</para></listitem>
780 <listitem><para>If you run BitBake with the <filename>--dump-signatures</filename>
781 (or <filename>-S</filename>) option, BitBake dumps out
782 <filename>.siginfo</filename> files in
783 the stamp directory for every task it would have executed instead of
784 building the specified target package.</para></listitem>
785 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
786 can process <filename>.siginfo</filename> files.
787 If you specify one of these files, BitBake dumps out the dependency
788 information in the file.
789 If you specify two files, BitBake compares the two files and dumps out
790 the differences between the two.
791 This more easily helps answer the question of "What
792 changed between X and Y?"</para></listitem>
793 </itemizedlist>
794 </para>
795 </section>
796
797 <section id='invalidating-shared-state'>
798 <title>Invalidating Shared State</title>
799
800 <para>
801 The OpenEmbedded build system uses checksums and shared state
802 cache to avoid unnecessarily rebuilding tasks.
803 Collectively, this scheme is known as "shared state code."
804 </para>
805
806 <para>
807 As with all schemes, this one has some drawbacks.
808 It is possible that you could make implicit changes to your
809 code that the checksum calculations do not take into
810 account.
811 These implicit changes affect a task's output but do not trigger
812 the shared state code into rebuilding a recipe.
813 Consider an example during which a tool changes its output.
814 Assume that the output of <filename>rpmdeps</filename> changes.
815 The result of the change should be that all the
816 <filename>package</filename> and
817 <filename>package_write_rpm</filename> shared state cache
818 items become invalid.
819 However, because the change to the output is
820 external to the code and therefore implicit,
821 the associated shared state cache items do not become
822 invalidated.
823 In this case, the build process uses the cached items rather
824 than running the task again.
825 Obviously, these types of implicit changes can cause problems.
826 </para>
827
828 <para>
829 To avoid these problems during the build, you need to
830 understand the effects of any changes you make.
831 Realize that changes you make directly to a function
832 are automatically factored into the checksum calculation.
833 Thus, these explicit changes invalidate the associated area of
834 shared state cache.
835 However, you need to be aware of any implicit changes that
836 are not obvious changes to the code and could affect the output
837 of a given task.
838 </para>
839
840 <para>
841 When you identify an implicit change, you can easily take steps
842 to invalidate the cache and force the tasks to run.
843 The steps you can take are as simple as changing a function's
844 comments in the source code.
845 For example, to invalidate package shared state files, change
846 the comment statements of
847 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
848 or the comments of one of the functions it calls.
849 Even though the change is purely cosmetic, it causes the
850 checksum to be recalculated and forces the OpenEmbedded build
851 system to run the task again.
852 </para>
853
854 <note>
855 For an example of a commit that makes a cosmetic change to
856 invalidate shared state, see this
857 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
858 </note>
859 </section>
860 </section>
861</section>
862
863<section id='x32'>
864 <title>x32</title>
865
866 <para>
867 x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
868 An ABI defines the calling conventions between functions in a processing environment.
869 The interface determines what registers are used and what the sizes are for various C data types.
870 </para>
871
872 <para>
873 Some processing environments prefer using 32-bit applications even when running
874 on Intel 64-bit platforms.
875 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
876 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
877 leaving the system underutilized.
878 Now consider the x86_64 psABI.
879 This ABI is newer and uses 64-bits for data sizes and program pointers.
880 The extra bits increase the footprint size of the programs, libraries,
881 and also increases the memory and file system size requirements.
882 Executing under the x32 psABI enables user programs to utilize CPU and system resources
883 more efficiently while keeping the memory footprint of the applications low.
884 Extra bits are used for registers but not for addressing mechanisms.
885 </para>
886
887 <section id='support'>
888 <title>Support</title>
889
890 <para>
891 This Yocto Project release supports the final specifications of x32
892 psABI.
893 Support for x32 psABI exists as follows:
894 <itemizedlist>
895 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
896 </para></listitem>
897 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
898 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
899 <filename>core-image-sato</filename> images.</para></listitem>
900 </itemizedlist>
901 </para>
902 </section>
903
904 <section id='completing-x32'>
905 <title>Completing x32</title>
906
907 <para>
908 Future Plans for the x32 psABI in the Yocto Project include the following:
909 <itemizedlist>
910 <listitem><para>Enhance and fix the few remaining recipes so they
911 work with and support x32 toolchains.</para></listitem>
912 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
913 <listitem><para>Support larger images.</para></listitem>
914 </itemizedlist>
915 </para>
916 </section>
917
918 <section id='using-x32-right-now'>
919 <title>Using x32 Right Now</title>
920
921 <para>
922 Follow these steps to use the x32 spABI:
923 <itemizedlist>
924 <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
925 machines by editing the <filename>conf/local.conf</filename> like this:
926 <literallayout class='monospaced'>
927 MACHINE = "qemux86-64"
928 DEFAULTTUNE = "x86-64-x32"
929 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
930 or 'INVALID'), True) or 'lib'}"
931 #MACHINE = "genericx86"
932 #DEFAULTTUNE = "core2-64-x32"
933 </literallayout></para></listitem>
934 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
935 Here is an example:
936 <literallayout class='monospaced'>
937 $ bitbake core-image-sato
938 </literallayout></para></listitem>
939 <listitem><para>As usual, run your image using QEMU:
940 <literallayout class='monospaced'>
941 $ runqemu qemux86-64 core-image-sato
942 </literallayout></para></listitem>
943 </itemizedlist>
944 </para>
945 </section>
946</section>
947
948<section id="wayland">
949 <title>Wayland</title>
950
951 <para>
952 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
953 is a computer display server protocol that
954 provides a method for compositing window managers to communicate
955 directly with applications and video hardware and expects them to
956 communicate with input hardware using other libraries.
957 Using Wayland with supporting targets can result in better control
958 over graphics frame rendering than an application might otherwise
959 achieve.
960 </para>
961
962 <para>
963 The Yocto Project provides the Wayland protocol libraries and the
964 reference
965 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
966 compositor as part of its release.
967 This section describes what you need to do to implement Wayland and
968 use the compositor when building an image for a supporting target.
969 </para>
970
971 <section id="wayland-support">
972 <title>Support</title>
973
974 <para>
975 The Wayland protocol libraries and the reference Weston compositor
976 ship as integrated packages in the <filename>meta</filename> layer
977 of the
978 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
979 Specifically, you can find the recipes that build both Wayland
980 and Weston at <filename>meta/recipes-graphics/wayland</filename>.
981 </para>
982
983 <para>
984 You can build both the Wayland and Weston packages for use only
985 with targets that accept the
986 <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
987 which is also known as Mesa DRI.
988 This implies that you cannot build and use the packages if your
989 target uses, for example, the
990 <trademark class='registered'>Intel</trademark> Embedded Media and
991 Graphics Driver (<trademark class='registered'>Intel</trademark>
992 EMGD) that overrides Mesa DRI.
993 </para>
994
995 <note>
996 Due to lack of EGL support, Weston 1.0.3 will not run directly on
997 the emulated QEMU hardware.
998 However, this version of Weston will run under X emulation without
999 issues.
1000 </note>
1001 </section>
1002
1003 <section id="enabling-wayland-in-an-image">
1004 <title>Enabling Wayland in an Image</title>
1005
1006 <para>
1007 To enable Wayland, you need to enable it to be built and enable
1008 it to be included in the image.
1009 </para>
1010
1011 <section id="enable-building">
1012 <title>Building</title>
1013
1014 <para>
1015 To cause Mesa to build the <filename>wayland-egl</filename>
1016 platform and Weston to build Wayland with Kernel Mode
1017 Setting
1018 (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
1019 support, include the "wayland" flag in the
1020 <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
1021 statement in your <filename>local.conf</filename> file:
1022 <literallayout class='monospaced'>
1023 DISTRO_FEATURES_append = " wayland"
1024 </literallayout>
1025 </para>
1026
1027 <note>
1028 If X11 has been enabled elsewhere, Weston will build Wayland
1029 with X11 support
1030 </note>
1031 </section>
1032
1033 <section id="enable-installation-in-an-image">
1034 <title>Installing</title>
1035
1036 <para>
1037 To install the Wayland feature into an image, you must
1038 include the following
1039 <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
1040 statement in your <filename>local.conf</filename> file:
1041 <literallayout class='monospaced'>
1042 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
1043 </literallayout>
1044 </para>
1045 </section>
1046 </section>
1047
1048 <section id="running-weston">
1049 <title>Running Weston</title>
1050
1051 <para>
1052 To run Weston inside X11, enabling it as described earlier and
1053 building a Sato image is sufficient.
1054 If you are running your image under Sato, a Weston Launcher appears
1055 in the "Utility" category.
1056 </para>
1057
1058 <para>
1059 Alternatively, you can run Weston through the command-line
1060 interpretor (CLI), which is better suited for development work.
1061 To run Weston under the CLI, you need to do the following after
1062 your image is built:
1063 <orderedlist>
1064 <listitem><para>Run these commands to export
1065 <filename>XDG_RUNTIME_DIR</filename>:
1066 <literallayout class='monospaced'>
1067 mkdir -p /tmp/$USER-weston
1068 chmod 0700 /tmp/$USER-weston
1069 export XDG_RUNTIME_DIR=/tmp/$USER-weston
1070 </literallayout></para></listitem>
1071 <listitem><para>Launch Weston in the shell:
1072 <literallayout class='monospaced'>
1073 weston
1074 </literallayout></para></listitem>
1075 </orderedlist>
1076 </para>
1077 </section>
1078</section>
1079
1080<section id="licenses">
1081 <title>Licenses</title>
1082
1083 <para>
1084 This section describes the mechanism by which the OpenEmbedded build system
1085 tracks changes to licensing text.
1086 The section also describes how to enable commercially licensed recipes,
1087 which by default are disabled.
1088 </para>
1089
1090 <para>
1091 For information that can help you maintain compliance with various open
1092 source licensing during the lifecycle of the product, see the
1093 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
1094 in the Yocto Project Development Manual.
1095 </para>
1096
1097 <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
1098 <title>Tracking License Changes</title>
1099
1100 <para>
1101 The license of an upstream project might change in the future.
1102 In order to prevent these changes going unnoticed, the
1103 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
1104 variable tracks changes to the license text. The checksums are validated at the end of the
1105 configure step, and if the checksums do not match, the build will fail.
1106 </para>
1107
1108 <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
1109 <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
1110
1111 <para>
1112 The <filename>LIC_FILES_CHKSUM</filename>
1113 variable contains checksums of the license text in the source code for the recipe.
1114 Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
1115 <literallayout class='monospaced'>
1116 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
1117 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
1118 file://licfile2.txt;endline=50;md5=zzzz \
1119 ..."
1120 </literallayout>
1121 </para>
1122
1123 <para>
1124 The build system uses the
1125 <filename><link linkend='var-S'>S</link></filename> variable as
1126 the default directory when searching files listed in
1127 <filename>LIC_FILES_CHKSUM</filename>.
1128 The previous example employs the default directory.
1129 </para>
1130
1131 <para>
1132 Consider this next example:
1133 <literallayout class='monospaced'>
1134 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
1135 md5=bb14ed3c4cda583abc85401304b5cd4e"
1136 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
1137 </literallayout>
1138 </para>
1139
1140 <para>
1141 The first line locates a file in
1142 <filename>${S}/src/ls.c</filename>.
1143 The second line refers to a file in
1144 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
1145 </para>
1146 <para>
1147 Note that <filename>LIC_FILES_CHKSUM</filename> variable is
1148 mandatory for all recipes, unless the
1149 <filename>LICENSE</filename> variable is set to "CLOSED".
1150 </para>
1151 </section>
1152
1153 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
1154 <title>Explanation of Syntax</title>
1155 <para>
1156 As mentioned in the previous section, the
1157 <filename>LIC_FILES_CHKSUM</filename> variable lists all the
1158 important files that contain the license text for the source code.
1159 It is possible to specify a checksum for an entire file, or a specific section of a
1160 file (specified by beginning and ending line numbers with the "beginline" and "endline"
1161 parameters, respectively).
1162 The latter is useful for source files with a license notice header,
1163 README documents, and so forth.
1164 If you do not use the "beginline" parameter, then it is assumed that the text begins on the
1165 first line of the file.
1166 Similarly, if you do not use the "endline" parameter, it is assumed that the license text
1167 ends with the last line of the file.
1168 </para>
1169
1170 <para>
1171 The "md5" parameter stores the md5 checksum of the license text.
1172 If the license text changes in any way as compared to this parameter
1173 then a mismatch occurs.
1174 This mismatch triggers a build failure and notifies the developer.
1175 Notification allows the developer to review and address the license text changes.
1176 Also note that if a mismatch occurs during the build, the correct md5
1177 checksum is placed in the build log and can be easily copied to the recipe.
1178 </para>
1179
1180 <para>
1181 There is no limit to how many files you can specify using the
1182 <filename>LIC_FILES_CHKSUM</filename> variable.
1183 Generally, however, every project requires a few specifications for license tracking.
1184 Many projects have a "COPYING" file that stores the license information for all the source
1185 code files.
1186 This practice allows you to just track the "COPYING" file as long as it is kept up to date.
1187 </para>
1188
1189 <tip>
1190 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
1191 error and displays the correct "md5" parameter value during the build.
1192 The correct parameter is also captured in the build log.
1193 </tip>
1194
1195 <tip>
1196 If the whole file contains only license text, you do not need to use the "beginline" and
1197 "endline" parameters.
1198 </tip>
1199 </section>
1200 </section>
1201
1202 <section id="enabling-commercially-licensed-recipes">
1203 <title>Enabling Commercially Licensed Recipes</title>
1204
1205 <para>
1206 By default, the OpenEmbedded build system disables
1207 components that have commercial or other special licensing
1208 requirements.
1209 Such requirements are defined on a
1210 recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable
1211 definition in the affected recipe.
1212 For instance, the
1213 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1214 recipe contains the following statement:
1215 <literallayout class='monospaced'>
1216 LICENSE_FLAGS = "commercial"
1217 </literallayout>
1218 Here is a slightly more complicated example that contains both an
1219 explicit recipe name and version (after variable expansion):
1220 <literallayout class='monospaced'>
1221 LICENSE_FLAGS = "license_${PN}_${PV}"
1222 </literallayout>
1223 In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
1224 definition to be enabled and included in an image, it
1225 needs to have a matching entry in the global
1226 <filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable
1227 typically defined in your <filename>local.conf</filename> file.
1228 For example, to enable
1229 the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1230 package, you could add either the string
1231 "commercial_gst-plugins-ugly" or the more general string
1232 "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
1233 See the
1234 "<link linkend='license-flag-matching'>License Flag Matching</link>" section
1235 for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
1236 Here is the example:
1237 <literallayout class='monospaced'>
1238 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
1239 </literallayout>
1240 Likewise, to additionally enable the package built from the recipe containing
1241 <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
1242 that the actual recipe name was <filename>emgd_1.10.bb</filename>,
1243 the following string would enable that package as well as
1244 the original <filename>gst-plugins-ugly</filename> package:
1245 <literallayout class='monospaced'>
1246 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
1247 </literallayout>
1248 As a convenience, you do not need to specify the complete license string
1249 in the whitelist for every package.
1250 You can use an abbreviated form, which consists
1251 of just the first portion or portions of the license string before
1252 the initial underscore character or characters.
1253 A partial string will match
1254 any license that contains the given string as the first
1255 portion of its license.
1256 For example, the following
1257 whitelist string will also match both of the packages
1258 previously mentioned as well as any other packages that have
1259 licenses starting with "commercial" or "license".
1260 <literallayout class='monospaced'>
1261 LICENSE_FLAGS_WHITELIST = "commercial license"
1262 </literallayout>
1263 </para>
1264
1265 <section id="license-flag-matching">
1266 <title>License Flag Matching</title>
1267
1268 <para>
1269 License flag matching allows you to control what recipes the
1270 OpenEmbedded build system includes in the build.
1271 Fundamentally, the build system attempts to match
1272 <filename>LICENSE_FLAGS</filename> strings found in
1273 recipes against <filename>LICENSE_FLAGS_WHITELIST</filename>
1274 strings found in the whitelist.
1275 A match causes the build system to include a recipe in the
1276 build, while failure to find a match causes the build system to
1277 exclude a recipe.
1278 </para>
1279
1280 <para>
1281 In general, license flag matching is simple.
1282 However, understanding some concepts will help you
1283 correctly and effectively use matching.
1284 </para>
1285
1286 <para>
1287 Before a flag
1288 defined by a particular recipe is tested against the
1289 contents of the whitelist, the expanded string
1290 <filename>_${PN}</filename> is appended to the flag.
1291 This expansion makes each <filename>LICENSE_FLAGS</filename>
1292 value recipe-specific.
1293 After expansion, the string is then matched against the
1294 whitelist.
1295 Thus, specifying
1296 <filename>LICENSE_FLAGS = "commercial"</filename>
1297 in recipe "foo", for example, results in the string
1298 <filename>"commercial_foo"</filename>.
1299 And, to create a match, that string must appear in the
1300 whitelist.
1301 </para>
1302
1303 <para>
1304 Judicious use of the <filename>LICENSE_FLAGS</filename>
1305 strings and the contents of the
1306 <filename>LICENSE_FLAGS_WHITELIST</filename> variable
1307 allows you a lot of flexibility for including or excluding
1308 recipes based on licensing.
1309 For example, you can broaden the matching capabilities by
1310 using license flags string subsets in the whitelist.
1311 <note>When using a string subset, be sure to use the part of
1312 the expanded string that precedes the appended underscore
1313 character (e.g. <filename>usethispart_1.3</filename>,
1314 <filename>usethispart_1.4</filename>, and so forth).
1315 </note>
1316 For example, simply specifying the string "commercial" in
1317 the whitelist matches any expanded
1318 <filename>LICENSE_FLAGS</filename> definition that starts with
1319 the string "commercial" such as "commercial_foo" and
1320 "commercial_bar", which are the strings the build system
1321 automatically generates for hypothetical recipes named
1322 "foo" and "bar" assuming those recipes simply specify the
1323 following:
1324 <literallayout class='monospaced'>
1325 LICENSE_FLAGS = "commercial"
1326 </literallayout>
1327 Thus, you can choose to exhaustively
1328 enumerate each license flag in the whitelist and
1329 allow only specific recipes into the image, or
1330 you can use a string subset that causes a broader range of
1331 matches to allow a range of recipes into the image.
1332 </para>
1333
1334 <para>
1335 This scheme works even if the
1336 <filename>LICENSE_FLAGS</filename> string already
1337 has <filename>_${PN}</filename> appended.
1338 For example, the build system turns the license flag
1339 "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
1340 match both the general "commercial" and the specific
1341 "commercial_1.2_foo" strings found in the whitelist, as
1342 expected.
1343 </para>
1344
1345 <para>
1346 Here are some other scenarios:
1347 <itemizedlist>
1348 <listitem><para>You can specify a versioned string in the
1349 recipe such as "commercial_foo_1.2" in a "foo" recipe.
1350 The build system expands this string to
1351 "commercial_foo_1.2_foo".
1352 Combine this license flag with a whitelist that has
1353 the string "commercial" and you match the flag along
1354 with any other flag that starts with the string
1355 "commercial".</para></listitem>
1356 <listitem><para>Under the same circumstances, you can
1357 use "commercial_foo" in the whitelist and the
1358 build system not only matches "commercial_foo_1.2" but
1359 also matches any license flag with the string
1360 "commercial_foo", regardless of the version.
1361 </para></listitem>
1362 <listitem><para>You can be very specific and use both the
1363 package and version parts in the whitelist (e.g.
1364 "commercial_foo_1.2") to specifically match a
1365 versioned recipe.</para></listitem>
1366 </itemizedlist>
1367 </para>
1368 </section>
1369
1370 <section id="other-variables-related-to-commercial-licenses">
1371 <title>Other Variables Related to Commercial Licenses</title>
1372
1373 <para>
1374 Other helpful variables related to commercial
1375 license handling exist and are defined in the
1376 <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
1377 <literallayout class='monospaced'>
1378 COMMERCIAL_AUDIO_PLUGINS ?= ""
1379 COMMERCIAL_VIDEO_PLUGINS ?= ""
1380 COMMERCIAL_QT = ""
1381 </literallayout>
1382 If you want to enable these components, you can do so by making sure you have
1383 statements similar to the following
1384 in your <filename>local.conf</filename> configuration file:
1385 <literallayout class='monospaced'>
1386 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
1387 gst-plugins-ugly-mpegaudioparse"
1388 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
1389 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
1390 COMMERCIAL_QT ?= "qmmp"
1391 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
1392 </literallayout>
1393 Of course, you could also create a matching whitelist
1394 for those components using the more general "commercial"
1395 in the whitelist, but that would also enable all the
1396 other packages with <filename>LICENSE_FLAGS</filename> containing
1397 "commercial", which you may or may not want:
1398 <literallayout class='monospaced'>
1399 LICENSE_FLAGS_WHITELIST = "commercial"
1400 </literallayout>
1401 </para>
1402
1403 <para>
1404 Specifying audio and video plug-ins as part of the
1405 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
1406 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
1407 or commercial Qt components as part of
1408 the <filename>COMMERCIAL_QT</filename> statement (along
1409 with the enabling <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
1410 plug-ins or components into built images, thus adding
1411 support for media formats or components.
1412 </para>
1413 </section>
1414 </section>
1415</section>
1416</chapter>
1417<!--
1418vim: expandtab tw=80 ts=4
1419-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-11-01 06:03:40 +0000