summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/technical-details.xml')
-rw-r--r--documentation/ref-manual/technical-details.xml1421
1 files changed, 1421 insertions, 0 deletions
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
new file mode 100644
index 0000000000..2df36521c2
--- /dev/null
+++ b/documentation/ref-manual/technical-details.xml
@@ -0,0 +1,1421 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='technical-details'>
6<title>Technical Details</title>
7
8 <para>
9 This chapter provides technical details for various parts of the
10 Yocto Project.
11 Currently, topics include Yocto Project components,
12 cross-toolchain generation, shared state (sstate) cache,
13 x32, Wayland support, and Licenses.
14 </para>
15
16<section id='usingpoky-components'>
17 <title>Yocto Project Components</title>
18
19 <para>
20 The
21 <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>
22 task executor together with various types of configuration files form
23 the OpenEmbedded Core.
24 This section overviews these components by describing their use and
25 how they interact.
26 </para>
27
28 <para>
29 BitBake handles the parsing and execution of the data files.
30 The data itself is of various types:
31 <itemizedlist>
32 <listitem><para><emphasis>Recipes:</emphasis> Provides details
33 about particular pieces of software.
34 </para></listitem>
35 <listitem><para><emphasis>Class Data:</emphasis> Abstracts
36 common build information (e.g. how to build a Linux kernel).
37 </para></listitem>
38 <listitem><para><emphasis>Configuration Data:</emphasis> Defines
39 machine-specific settings, policy decisions, and so forth.
40 Configuration data acts as the glue to bind everything
41 together.
42 </para></listitem>
43 </itemizedlist>
44 </para>
45
46 <para>
47 BitBake knows how to combine multiple data sources together and refers
48 to each data source as a layer.
49 For information on layers, see the
50 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
51 Creating Layers</ulink>" section of the Yocto Project Development Manual.
52 </para>
53
54 <para>
55 Following are some brief details on these core components.
56 For additional information on how these components interact during
57 a build, see the
58 "<link linkend='closer-look'>A Closer Look at the Yocto Project Development Environment</link>"
59 Chapter.
60 </para>
61
62 <section id='usingpoky-components-bitbake'>
63 <title>BitBake</title>
64
65 <para>
66 BitBake is the tool at the heart of the OpenEmbedded build system
67 and is responsible for parsing the
68 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
69 generating a list of tasks from it, and then executing those tasks.
70 </para>
71
72 <para>
73 This section briefly introduces BitBake.
74 If you want more information on BitBake, see the
75 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.
76 </para>
77
78 <para>
79 To see a list of the options BitBake supports, use either of
80 the following commands:
81 <literallayout class='monospaced'>
82 $ bitbake -h
83 $ bitbake --help
84 </literallayout>
85 </para>
86
87 <para>
88 The most common usage for BitBake is <filename>bitbake <replaceable>packagename</replaceable></filename>, where
89 <filename>packagename</filename> is the name of the package you want to build
90 (referred to as the "target" in this manual).
91 The target often equates to the first part of a recipe's filename
92 (e.g. "foo" for a recipe named
93 <filename>foo_1.3.0-r0.bb</filename>).
94 So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
95 might type the following:
96 <literallayout class='monospaced'>
97 $ bitbake matchbox-desktop
98 </literallayout>
99 Several different versions of <filename>matchbox-desktop</filename> might exist.
100 BitBake chooses the one selected by the distribution configuration.
101 You can get more details about how BitBake chooses between different
102 target versions and providers in the
103 "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences'>Preferences</ulink>"
104 section of the BitBake User Manual.
105 </para>
106
107 <para>
108 BitBake also tries to execute any dependent tasks first.
109 So for example, before building <filename>matchbox-desktop</filename>, BitBake
110 would build a cross compiler and <filename>glibc</filename> if they had not already
111 been built.
112 </para>
113
114 <para>
115 A useful BitBake option to consider is the <filename>-k</filename> or
116 <filename>--continue</filename> option.
117 This option instructs BitBake to try and continue processing the job
118 as long as possible even after encountering an error.
119 When an error occurs, the target that
120 failed and those that depend on it cannot be remade.
121 However, when you use this option other dependencies can still be
122 processed.
123 </para>
124 </section>
125
126 <section id='usingpoky-components-metadata'>
127 <title>Metadata (Recipes)</title>
128
129 <para>
130 Files that have the <filename>.bb</filename> suffix are "recipes"
131 files.
132 In general, a recipe contains information about a single piece of
133 software.
134 This information includes the location from which to download the
135 unaltered source, any source patches to be applied to that source
136 (if needed), which special configuration options to apply,
137 how to compile the source files, and how to package the compiled
138 output.
139 </para>
140
141 <para>
142 The term "package" is sometimes used to refer to recipes. However,
143 since the word "package" is used for the packaged output from the OpenEmbedded
144 build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
145 this document avoids using the term "package" when referring to recipes.
146 </para>
147 </section>
148
149 <section id='usingpoky-components-classes'>
150 <title>Classes</title>
151
152 <para>
153 Class files (<filename>.bbclass</filename>) contain information that
154 is useful to share between
155 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
156 An example is the
157 <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
158 class, which contains common settings for any application that
159 Autotools uses.
160 The "<link linkend='ref-classes'>Classes</link>" chapter provides
161 details about classes and how to use them.
162 </para>
163 </section>
164
165 <section id='usingpoky-components-configuration'>
166 <title>Configuration</title>
167
168 <para>
169 The configuration files (<filename>.conf</filename>) define various configuration variables
170 that govern the OpenEmbedded build process.
171 These files fall into several areas that define machine configuration options,
172 distribution configuration options, compiler tuning options, general common configuration
173 options, and user configuration options in <filename>local.conf</filename>, which is found
174 in the
175 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
176 </para>
177 </section>
178</section>
179
180<section id="cross-development-toolchain-generation">
181 <title>Cross-Development Toolchain Generation</title>
182
183 <para>
184 The Yocto Project does most of the work for you when it comes to
185 creating
186 <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
187 This section provides some technical background on how
188 cross-development toolchains are created and used.
189 For more information on toolchains, you can also see the
190 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
191 </para>
192
193 <para>
194 In the Yocto Project development environment, cross-development
195 toolchains are used to build the image and applications that run on the
196 target hardware.
197 With just a few commands, the OpenEmbedded build system creates
198 these necessary toolchains for you.
199 </para>
200
201 <para>
202 The following figure shows a high-level build environment regarding
203 toolchain construction and use.
204 </para>
205
206 <para>
207 <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
208 </para>
209
210 <para>
211 Most of the work occurs on the Build Host.
212 This is the machine used to build images and generally work within the
213 the Yocto Project environment.
214 When you run BitBake to create an image, the OpenEmbedded build system
215 uses the host <filename>gcc</filename> compiler to bootstrap a
216 cross-compiler named <filename>gcc-cross</filename>.
217 The <filename>gcc-cross</filename> compiler is what BitBake uses to
218 compile source files when creating the target image.
219 You can think of <filename>gcc-cross</filename> simply as an
220 automatically generated cross-compiler that is used internally within
221 BitBake only.
222 </para>
223
224 <para>
225 The chain of events that occurs when <filename>gcc-cross</filename> is
226 bootstrapped is as follows:
227 <literallayout class='monospaced'>
228 gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime
229 </literallayout>
230 <itemizedlist>
231 <listitem><para><filename>gcc</filename>:
232 The build host's GNU Compiler Collection (GCC).
233 </para></listitem>
234 <listitem><para><filename>binutils-cross</filename>:
235 The bare minimum binary utilities needed in order to run
236 the <filename>gcc-cross-initial</filename> phase of the
237 bootstrap operation.
238 </para></listitem>
239 <listitem><para><filename>gcc-cross-initial</filename>:
240 An early stage of the bootstrap process for creating
241 the cross-compiler.
242 This stage builds enough of the <filename>gcc-cross</filename>,
243 the C library, and other pieces needed to finish building the
244 final cross-compiler in later stages.
245 This tool is a "native" package (i.e. it is designed to run on
246 the build host).
247 </para></listitem>
248 <listitem><para><filename>linux-libc-headers</filename>:
249 Headers needed for the cross-compiler.
250 </para></listitem>
251 <listitem><para><filename>glibc-initial</filename>:
252 An initial version of the Embedded GLIBC needed to bootstrap
253 <filename>glibc</filename>.
254 </para></listitem>
255 <listitem><para><filename>gcc-cross</filename>:
256 The final stage of the bootstrap process for the
257 cross-compiler.
258 This stage results in the actual cross-compiler that
259 BitBake uses when it builds an image for a targeted
260 device.
261 <note>
262 If you are replacing this cross compiler toolchain
263 with a custom version, you must replace
264 <filename>gcc-cross</filename>.
265 </note>
266 This tool is also a "native" package (i.e. it is
267 designed to run on the build host).
268 </para></listitem>
269 <listitem><para><filename>gcc-runtime</filename>:
270 Runtime libraries resulting from the toolchain bootstrapping
271 process.
272 This tool produces a binary that consists of the
273 runtime libraries need for the targeted device.
274 </para></listitem>
275 </itemizedlist>
276 </para>
277
278 <para>
279 You can use the OpenEmbedded build system to build an installer for
280 the relocatable SDK used to develop applications.
281 When you run the installer, it installs the toolchain, which contains
282 the development tools (e.g., the
283 <filename>gcc-cross-canadian</filename>),
284 <filename>binutils-cross-canadian</filename>, and other
285 <filename>nativesdk-*</filename> tools you need to cross-compile and
286 test your software.
287 The figure shows the commands you use to easily build out this
288 toolchain.
289 This cross-development toolchain is built to execute on the
290 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
291 which might or might not be the same
292 machine as the Build Host.
293 <note>
294 If your target architecture is supported by the Yocto Project,
295 you can take advantage of pre-built images that ship with the
296 Yocto Project and already contain cross-development toolchain
297 installers.
298 </note>
299 </para>
300
301 <para>
302 Here is the bootstrap process for the relocatable toolchain:
303 <literallayout class='monospaced'>
304 gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers ->
305 glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian
306 </literallayout>
307 <itemizedlist>
308 <listitem><para><filename>gcc</filename>:
309 The build host's GNU Compiler Collection (GCC).
310 </para></listitem>
311 <listitem><para><filename>binutils-crosssdk</filename>:
312 The bare minimum binary utilities needed in order to run
313 the <filename>gcc-crosssdk-initial</filename> phase of the
314 bootstrap operation.
315 </para></listitem>
316 <listitem><para><filename>gcc-crosssdk-initial</filename>:
317 An early stage of the bootstrap process for creating
318 the cross-compiler.
319 This stage builds enough of the
320 <filename>gcc-crosssdk</filename> and supporting pieces so that
321 the final stage of the bootstrap process can produce the
322 finished cross-compiler.
323 This tool is a "native" binary that runs on the build host.
324 </para></listitem>
325 <listitem><para><filename>linux-libc-headers</filename>:
326 Headers needed for the cross-compiler.
327 </para></listitem>
328 <listitem><para><filename>glibc-initial</filename>:
329 An initial version of the Embedded GLIBC needed to bootstrap
330 <filename>nativesdk-glibc</filename>.
331 </para></listitem>
332 <listitem><para><filename>nativesdk-glibc</filename>:
333 The Embedded GLIBC needed to bootstrap the
334 <filename>gcc-crosssdk</filename>.
335 </para></listitem>
336 <listitem><para><filename>gcc-crosssdk</filename>:
337 The final stage of the bootstrap process for the
338 relocatable cross-compiler.
339 The <filename>gcc-crosssdk</filename> is a transitory compiler
340 and never leaves the build host.
341 Its purpose is to help in the bootstrap process to create the
342 eventual relocatable <filename>gcc-cross-canadian</filename>
343 compiler, which is relocatable.
344 This tool is also a "native" package (i.e. it is
345 designed to run on the build host).
346 </para></listitem>
347 <listitem><para><filename>gcc-cross-canadian</filename>:
348 The final relocatable cross-compiler.
349 When run on the
350 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
351 this tool
352 produces executable code that runs on the target device.
353 Only one cross-canadian compiler is produced per architecture
354 since they can be targeted at different processor optimizations
355 using configurations passed to the compiler through the
356 compile commands.
357 This circumvents the need for multiple compilers and thus
358 reduces the size of the toolchains.
359 </para></listitem>
360 </itemizedlist>
361 </para>
362
363 <note>
364 For information on advantages gained when building a
365 cross-development toolchain installer, see the
366 "<ulink url='&YOCTO_DOCS_ADT_URL;#optionally-building-a-toolchain-installer'>Optionally Building a Toolchain Installer</ulink>"
367 section in the Yocto Project Application Developer's Guide.
368 </note>
369</section>
370
371<section id="shared-state-cache">
372 <title>Shared State Cache</title>
373
374 <para>
375 By design, the OpenEmbedded build system builds everything from scratch unless
376 BitBake can determine that parts do not need to be rebuilt.
377 Fundamentally, building from scratch is attractive as it means all parts are
378 built fresh and there is no possibility of stale data causing problems.
379 When developers hit problems, they typically default back to building from scratch
380 so they know the state of things from the start.
381 </para>
382
383 <para>
384 Building an image from scratch is both an advantage and a disadvantage to the process.
385 As mentioned in the previous paragraph, building from scratch ensures that
386 everything is current and starts from a known state.
387 However, building from scratch also takes much longer as it generally means
388 rebuilding things that do not necessarily need to be rebuilt.
389 </para>
390
391 <para>
392 The Yocto Project implements shared state code that supports incremental builds.
393 The implementation of the shared state code answers the following questions that
394 were fundamental roadblocks within the OpenEmbedded incremental build support system:
395 <itemizedlist>
396 <listitem><para>What pieces of the system have changed and what pieces have
397 not changed?</para></listitem>
398 <listitem><para>How are changed pieces of software removed and replaced?</para></listitem>
399 <listitem><para>How are pre-built components that do not need to be rebuilt from scratch
400 used when they are available?</para></listitem>
401 </itemizedlist>
402 </para>
403
404 <para>
405 For the first question, the build system detects changes in the "inputs" to a given task by
406 creating a checksum (or signature) of the task's inputs.
407 If the checksum changes, the system assumes the inputs have changed and the task needs to be
408 rerun.
409 For the second question, the shared state (sstate) code tracks which tasks add which output
410 to the build process.
411 This means the output from a given task can be removed, upgraded or otherwise manipulated.
412 The third question is partly addressed by the solution for the second question
413 assuming the build system can fetch the sstate objects from remote locations and
414 install them if they are deemed to be valid.
415 </para>
416
417 <note>
418 The OpenEmbedded build system does not maintain
419 <link linkend='var-PR'><filename>PR</filename></link> information
420 as part of the shared state packages.
421 Consequently, considerations exist that affect maintaining shared
422 state feeds.
423 For information on how the OpenEmbedded build system
424 works with packages and can
425 track incrementing <filename>PR</filename> information, see the
426 "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
427 section.
428 </note>
429
430 <para>
431 The rest of this section goes into detail about the overall incremental build
432 architecture, the checksums (signatures), shared state, and some tips and tricks.
433 </para>
434
435 <section id='overall-architecture'>
436 <title>Overall Architecture</title>
437
438 <para>
439 When determining what parts of the system need to be built, BitBake
440 works on a per-task basis rather than a per-recipe basis.
441 You might wonder why using a per-task basis is preferred over a per-recipe basis.
442 To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
443 In this case, the
444 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
445 and
446 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
447 task outputs are still valid.
448 However, with a per-recipe approach, the build would not include the
449 <filename>.deb</filename> files.
450 Consequently, you would have to invalidate the whole build and rerun it.
451 Rerunning everything is not the best solution.
452 Also, in this case, the core must be "taught" much about specific tasks.
453 This methodology does not scale well and does not allow users to easily add new tasks
454 in layers or as external recipes without touching the packaged-staging core.
455 </para>
456 </section>
457
458 <section id='checksums'>
459 <title>Checksums (Signatures)</title>
460
461 <para>
462 The shared state code uses a checksum, which is a unique signature of a task's
463 inputs, to determine if a task needs to be run again.
464 Because it is a change in a task's inputs that triggers a rerun, the process
465 needs to detect all the inputs to a given task.
466 For shell tasks, this turns out to be fairly easy because
467 the build process generates a "run" shell script for each task and
468 it is possible to create a checksum that gives you a good idea of when
469 the task's data changes.
470 </para>
471
472 <para>
473 To complicate the problem, there are things that should not be included in
474 the checksum.
475 First, there is the actual specific build path of a given task -
476 the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
477 It does not matter if the work directory changes because it should not
478 affect the output for target packages.
479 Also, the build process has the objective of making native or cross packages relocatable.
480 The checksum therefore needs to exclude <filename>WORKDIR</filename>.
481 The simplistic approach for excluding the work directory is to set
482 <filename>WORKDIR</filename> to some fixed value and create the checksum
483 for the "run" script.
484 </para>
485
486 <para>
487 Another problem results from the "run" scripts containing functions that
488 might or might not get called.
489 The incremental build solution contains code that figures out dependencies
490 between shell functions.
491 This code is used to prune the "run" scripts down to the minimum set,
492 thereby alleviating this problem and making the "run" scripts much more
493 readable as a bonus.
494 </para>
495
496 <para>
497 So far we have solutions for shell scripts.
498 What about Python tasks?
499 The same approach applies even though these tasks are more difficult.
500 The process needs to figure out what variables a Python function accesses
501 and what functions it calls.
502 Again, the incremental build solution contains code that first figures out
503 the variable and function dependencies, and then creates a checksum for the data
504 used as the input to the task.
505 </para>
506
507 <para>
508 Like the <filename>WORKDIR</filename> case, situations exist where dependencies
509 should be ignored.
510 For these cases, you can instruct the build process to ignore a dependency
511 by using a line like the following:
512 <literallayout class='monospaced'>
513 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
514 </literallayout>
515 This example ensures that the
516 <link linkend='var-PACKAGE_ARCHS'><filename>PACKAGE_ARCHS</filename></link>
517 variable does not
518 depend on the value of
519 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>,
520 even if it does reference it.
521 </para>
522
523 <para>
524 Equally, there are cases where we need to add dependencies BitBake is not able to find.
525 You can accomplish this by using a line like the following:
526 <literallayout class='monospaced'>
527 PACKAGE_ARCHS[vardeps] = "MACHINE"
528 </literallayout>
529 This example explicitly adds the <filename>MACHINE</filename> variable as a
530 dependency for <filename>PACKAGE_ARCHS</filename>.
531 </para>
532
533 <para>
534 Consider a case with in-line Python, for example, where BitBake is not
535 able to figure out dependencies.
536 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
537 produces output when it discovers something for which it cannot figure out
538 dependencies.
539 The Yocto Project team has currently not managed to cover those dependencies
540 in detail and is aware of the need to fix this situation.
541 </para>
542
543 <para>
544 Thus far, this section has limited discussion to the direct inputs into a task.
545 Information based on direct inputs is referred to as the "basehash" in the
546 code.
547 However, there is still the question of a task's indirect inputs - the
548 things that were already built and present in the
549 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
550 The checksum (or signature) for a particular task needs to add the hashes
551 of all the tasks on which the particular task depends.
552 Choosing which dependencies to add is a policy decision.
553 However, the effect is to generate a master checksum that combines the basehash
554 and the hashes of the task's dependencies.
555 </para>
556
557 <para>
558 At the code level, there are a variety of ways both the basehash and the
559 dependent task hashes can be influenced.
560 Within the BitBake configuration file, we can give BitBake some extra information
561 to help it construct the basehash.
562 The following statement effectively results in a list of global variable
563 dependency excludes - variables never included in any checksum:
564 <literallayout class='monospaced'>
565 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
566 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
567 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
568 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
569 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
570 </literallayout>
571 The previous example excludes
572 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
573 since that variable is actually constructed as a path within
574 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
575 the whitelist.
576 </para>
577
578 <para>
579 The rules for deciding which hashes of dependent tasks to include through
580 dependency chains are more complex and are generally accomplished with a
581 Python function.
582 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
583 of this and also illustrates how you can insert your own policy into the system
584 if so desired.
585 This file defines the two basic signature generators <filename>OE-Core</filename>
586 uses: "OEBasic" and "OEBasicHash".
587 By default, there is a dummy "noop" signature handler enabled in BitBake.
588 This means that behavior is unchanged from previous versions.
589 <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
590 through this setting in the <filename>bitbake.conf</filename> file:
591 <literallayout class='monospaced'>
592 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
593 </literallayout>
594 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
595 "OEBasic" version but adds the task hash to the stamp files.
596 This results in any
597 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
598 change that changes the task hash, automatically
599 causing the task to be run again.
600 This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
601 values, and changes to Metadata automatically ripple across the build.
602 </para>
603
604 <para>
605 It is also worth noting that the end result of these signature generators is to
606 make some dependency and hash information available to the build.
607 This information includes:
608 <itemizedlist>
609 <listitem><para><filename>BB_BASEHASH_task-</filename><replaceable>taskname</replaceable>:
610 The base hashes for each task in the recipe.
611 </para></listitem>
612 <listitem><para><filename>BB_BASEHASH_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
613 The base hashes for each dependent task.
614 </para></listitem>
615 <listitem><para><filename>BBHASHDEPS_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
616 The task dependencies for each task.
617 </para></listitem>
618 <listitem><para><filename>BB_TASKHASH</filename>:
619 The hash of the currently running task.
620 </para></listitem>
621 </itemizedlist>
622 </para>
623 </section>
624
625 <section id='shared-state'>
626 <title>Shared State</title>
627
628 <para>
629 Checksums and dependencies, as discussed in the previous section, solve half the
630 problem of supporting a shared state.
631 The other part of the problem is being able to use checksum information during the build
632 and being able to reuse or rebuild specific components.
633 </para>
634
635 <para>
636 The
637 <link linkend='ref-classes-sstate'><filename>sstate</filename></link>
638 class is a relatively generic implementation of how to "capture"
639 a snapshot of a given task.
640 The idea is that the build process does not care about the source of a task's output.
641 Output could be freshly built or it could be downloaded and unpacked from
642 somewhere - the build process does not need to worry about its origin.
643 </para>
644
645 <para>
646 There are two types of output, one is just about creating a directory
647 in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
648 A good example is the output of either
649 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
650 or
651 <link linkend='ref-tasks-package'><filename>do_package</filename></link>.
652 The other type of output occurs when a set of data is merged into a shared directory
653 tree such as the sysroot.
654 </para>
655
656 <para>
657 The Yocto Project team has tried to keep the details of the
658 implementation hidden in <filename>sstate</filename> class.
659 From a user's perspective, adding shared state wrapping to a task
660 is as simple as this
661 <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link>
662 example taken from the
663 <link linkend='ref-classes-deploy'><filename>deploy</filename></link>
664 class:
665 <literallayout class='monospaced'>
666 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
667 SSTATETASKS += "do_deploy"
668 do_deploy[sstate-name] = "deploy"
669 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
670 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
671
672 python do_deploy_setscene () {
673 sstate_setscene(d)
674 }
675 addtask do_deploy_setscene
676 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
677 </literallayout>
678 In this example, we add some extra flags to the task, a name field ("deploy"), an
679 input directory where the task sends data, and the output
680 directory where the data from the task should eventually be copied.
681 We also add a <filename>_setscene</filename> variant of the task and add the task
682 name to the <filename>SSTATETASKS</filename> list.
683 </para>
684
685 <para>
686 If you have a directory whose contents you need to preserve, you can do this with
687 a line like the following:
688 <literallayout class='monospaced'>
689 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
690 </literallayout>
691 This method, as well as the following example, also works for multiple directories.
692 <literallayout class='monospaced'>
693 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
694 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
695 do_package[sstate-lockfile] = "${PACKAGELOCK}"
696 </literallayout>
697 These methods also include the ability to take a lockfile when manipulating
698 shared state directory structures since some cases are sensitive to file
699 additions or removals.
700 </para>
701
702 <para>
703 Behind the scenes, the shared state code works by looking in
704 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
705 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
706 for shared state files.
707 Here is an example:
708 <literallayout class='monospaced'>
709 SSTATE_MIRRORS ?= "\
710 file://.* http://someserver.tld/share/sstate/PATH \n \
711 file://.* file:///some/local/dir/sstate/PATH"
712 </literallayout>
713 <note>
714 The shared state directory (<filename>SSTATE_DIR</filename>) is
715 organized into two-character subdirectories, where the subdirectory
716 names are based on the first two characters of the hash.
717 If the shared state directory structure for a mirror has the
718 same structure as <filename>SSTATE_DIR</filename>, you must
719 specify "PATH" as part of the URI to enable the build system
720 to map to the appropriate subdirectory.
721 </note>
722 </para>
723
724 <para>
725 The shared state package validity can be detected just by looking at the
726 filename since the filename contains the task checksum (or signature) as
727 described earlier in this section.
728 If a valid shared state package is found, the build process downloads it
729 and uses it to accelerate the task.
730 </para>
731
732 <para>
733 The build processes use the <filename>*_setscene</filename> tasks
734 for the task acceleration phase.
735 BitBake goes through this phase before the main execution code and tries
736 to accelerate any tasks for which it can find shared state packages.
737 If a shared state package for a task is available, the shared state
738 package is used.
739 This means the task and any tasks on which it is dependent are not
740 executed.
741 </para>
742
743 <para>
744 As a real world example, the aim is when building an IPK-based image,
745 only the
746 <link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>
747 tasks would have their
748 shared state packages fetched and extracted.
749 Since the sysroot is not used, it would never get extracted.
750 This is another reason why a task-based approach is preferred over a
751 recipe-based approach, which would have to install the output from every task.
752 </para>
753 </section>
754
755 <section id='tips-and-tricks'>
756 <title>Tips and Tricks</title>
757
758 <para>
759 The code in the build system that supports incremental builds is not
760 simple code.
761 This section presents some tips and tricks that help you work around
762 issues related to shared state code.
763 </para>
764
765 <section id='debugging'>
766 <title>Debugging</title>
767
768 <para>
769 When things go wrong, debugging needs to be straightforward.
770 Because of this, the Yocto Project includes strong debugging
771 tools:
772 <itemizedlist>
773 <listitem><para>Whenever a shared state package is written out, so is a
774 corresponding <filename>.siginfo</filename> file.
775 This practice results in a pickled Python database of all
776 the metadata that went into creating the hash for a given shared state
777 package.</para></listitem>
778 <listitem><para>If you run BitBake with the <filename>--dump-signatures</filename>
779 (or <filename>-S</filename>) option, BitBake dumps out
780 <filename>.siginfo</filename> files in
781 the stamp directory for every task it would have executed instead of
782 building the specified target package.</para></listitem>
783 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
784 can process <filename>.siginfo</filename> files.
785 If you specify one of these files, BitBake dumps out the dependency
786 information in the file.
787 If you specify two files, BitBake compares the two files and dumps out
788 the differences between the two.
789 This more easily helps answer the question of "What
790 changed between X and Y?"</para></listitem>
791 </itemizedlist>
792 </para>
793 </section>
794
795 <section id='invalidating-shared-state'>
796 <title>Invalidating Shared State</title>
797
798 <para>
799 The OpenEmbedded build system uses checksums and shared state
800 cache to avoid unnecessarily rebuilding tasks.
801 Collectively, this scheme is known as "shared state code."
802 </para>
803
804 <para>
805 As with all schemes, this one has some drawbacks.
806 It is possible that you could make implicit changes to your
807 code that the checksum calculations do not take into
808 account.
809 These implicit changes affect a task's output but do not trigger
810 the shared state code into rebuilding a recipe.
811 Consider an example during which a tool changes its output.
812 Assume that the output of <filename>rpmdeps</filename> changes.
813 The result of the change should be that all the
814 <filename>package</filename> and
815 <filename>package_write_rpm</filename> shared state cache
816 items become invalid.
817 However, because the change to the output is
818 external to the code and therefore implicit,
819 the associated shared state cache items do not become
820 invalidated.
821 In this case, the build process uses the cached items rather
822 than running the task again.
823 Obviously, these types of implicit changes can cause problems.
824 </para>
825
826 <para>
827 To avoid these problems during the build, you need to
828 understand the effects of any changes you make.
829 Realize that changes you make directly to a function
830 are automatically factored into the checksum calculation.
831 Thus, these explicit changes invalidate the associated area of
832 shared state cache.
833 However, you need to be aware of any implicit changes that
834 are not obvious changes to the code and could affect the output
835 of a given task.
836 </para>
837
838 <para>
839 When you identify an implicit change, you can easily take steps
840 to invalidate the cache and force the tasks to run.
841 The steps you can take are as simple as changing a function's
842 comments in the source code.
843 For example, to invalidate package shared state files, change
844 the comment statements of
845 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
846 or the comments of one of the functions it calls.
847 Even though the change is purely cosmetic, it causes the
848 checksum to be recalculated and forces the OpenEmbedded build
849 system to run the task again.
850 </para>
851
852 <note>
853 For an example of a commit that makes a cosmetic change to
854 invalidate shared state, see this
855 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
856 </note>
857 </section>
858 </section>
859</section>
860
861<section id='x32'>
862 <title>x32</title>
863
864 <para>
865 x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
866 An ABI defines the calling conventions between functions in a processing environment.
867 The interface determines what registers are used and what the sizes are for various C data types.
868 </para>
869
870 <para>
871 Some processing environments prefer using 32-bit applications even when running
872 on Intel 64-bit platforms.
873 Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
874 The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
875 leaving the system underutilized.
876 Now consider the x86_64 psABI.
877 This ABI is newer and uses 64-bits for data sizes and program pointers.
878 The extra bits increase the footprint size of the programs, libraries,
879 and also increases the memory and file system size requirements.
880 Executing under the x32 psABI enables user programs to utilize CPU and system resources
881 more efficiently while keeping the memory footprint of the applications low.
882 Extra bits are used for registers but not for addressing mechanisms.
883 </para>
884
885 <section id='support'>
886 <title>Support</title>
887
888 <para>
889 This Yocto Project release supports the final specifications of x32
890 psABI.
891 Support for x32 psABI exists as follows:
892 <itemizedlist>
893 <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
894 </para></listitem>
895 <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
896 <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
897 <filename>core-image-sato</filename> images.</para></listitem>
898 </itemizedlist>
899 </para>
900 </section>
901
902 <section id='completing-x32'>
903 <title>Completing x32</title>
904
905 <para>
906 Future Plans for the x32 psABI in the Yocto Project include the following:
907 <itemizedlist>
908 <listitem><para>Enhance and fix the few remaining recipes so they
909 work with and support x32 toolchains.</para></listitem>
910 <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
911 <listitem><para>Support larger images.</para></listitem>
912 </itemizedlist>
913 </para>
914 </section>
915
916 <section id='using-x32-right-now'>
917 <title>Using x32 Right Now</title>
918
919 <para>
920 Follow these steps to use the x32 spABI:
921 <itemizedlist>
922 <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
923 machines by editing the <filename>conf/local.conf</filename> like this:
924 <literallayout class='monospaced'>
925 MACHINE = "qemux86-64"
926 DEFAULTTUNE = "x86-64-x32"
927 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
928 or 'INVALID'), True) or 'lib'}"
929 #MACHINE = "genericx86"
930 #DEFAULTTUNE = "core2-64-x32"
931 </literallayout></para></listitem>
932 <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
933 Here is an example:
934 <literallayout class='monospaced'>
935 $ bitbake core-image-sato
936 </literallayout></para></listitem>
937 <listitem><para>As usual, run your image using QEMU:
938 <literallayout class='monospaced'>
939 $ runqemu qemux86-64 core-image-sato
940 </literallayout></para></listitem>
941 </itemizedlist>
942 </para>
943 </section>
944</section>
945
946<section id="wayland">
947 <title>Wayland</title>
948
949 <para>
950 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
951 is a computer display server protocol that
952 provides a method for compositing window managers to communicate
953 directly with applications and video hardware and expects them to
954 communicate with input hardware using other libraries.
955 Using Wayland with supporting targets can result in better control
956 over graphics frame rendering than an application might otherwise
957 achieve.
958 </para>
959
960 <para>
961 The Yocto Project provides the Wayland protocol libraries and the
962 reference
963 <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
964 compositor as part of its release.
965 This section describes what you need to do to implement Wayland and
966 use the compositor when building an image for a supporting target.
967 </para>
968
969 <section id="wayland-support">
970 <title>Support</title>
971
972 <para>
973 The Wayland protocol libraries and the reference Weston compositor
974 ship as integrated packages in the <filename>meta</filename> layer
975 of the
976 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
977 Specifically, you can find the recipes that build both Wayland
978 and Weston at <filename>meta/recipes-graphics/wayland</filename>.
979 </para>
980
981 <para>
982 You can build both the Wayland and Weston packages for use only
983 with targets that accept the
984 <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
985 which is also known as Mesa DRI.
986 This implies that you cannot build and use the packages if your
987 target uses, for example, the
988 <trademark class='registered'>Intel</trademark> Embedded Media and
989 Graphics Driver (<trademark class='registered'>Intel</trademark>
990 EMGD) that overrides Mesa DRI.
991 </para>
992
993 <note>
994 Due to lack of EGL support, Weston 1.0.3 will not run directly on
995 the emulated QEMU hardware.
996 However, this version of Weston will run under X emulation without
997 issues.
998 </note>
999 </section>
1000
1001 <section id="enabling-wayland-in-an-image">
1002 <title>Enabling Wayland in an Image</title>
1003
1004 <para>
1005 To enable Wayland, you need to enable it to be built and enable
1006 it to be included in the image.
1007 </para>
1008
1009 <section id="enable-building">
1010 <title>Building</title>
1011
1012 <para>
1013 To cause Mesa to build the <filename>wayland-egl</filename>
1014 platform and Weston to build Wayland with Kernel Mode
1015 Setting
1016 (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
1017 support, include the "wayland" flag in the
1018 <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
1019 statement in your <filename>local.conf</filename> file:
1020 <literallayout class='monospaced'>
1021 DISTRO_FEATURES_append = " wayland"
1022 </literallayout>
1023 </para>
1024
1025 <note>
1026 If X11 has been enabled elsewhere, Weston will build Wayland
1027 with X11 support
1028 </note>
1029 </section>
1030
1031 <section id="enable-installation-in-an-image">
1032 <title>Installing</title>
1033
1034 <para>
1035 To install the Wayland feature into an image, you must
1036 include the following
1037 <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
1038 statement in your <filename>local.conf</filename> file:
1039 <literallayout class='monospaced'>
1040 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
1041 </literallayout>
1042 </para>
1043 </section>
1044 </section>
1045
1046 <section id="running-weston">
1047 <title>Running Weston</title>
1048
1049 <para>
1050 To run Weston inside X11, enabling it as described earlier and
1051 building a Sato image is sufficient.
1052 If you are running your image under Sato, a Weston Launcher appears
1053 in the "Utility" category.
1054 </para>
1055
1056 <para>
1057 Alternatively, you can run Weston through the command-line
1058 interpretor (CLI), which is better suited for development work.
1059 To run Weston under the CLI, you need to do the following after
1060 your image is built:
1061 <orderedlist>
1062 <listitem><para>Run these commands to export
1063 <filename>XDG_RUNTIME_DIR</filename>:
1064 <literallayout class='monospaced'>
1065 mkdir -p /tmp/$USER-weston
1066 chmod 0700 /tmp/$USER-weston
1067 export XDG_RUNTIME_DIR=/tmp/$USER-weston
1068 </literallayout></para></listitem>
1069 <listitem><para>Launch Weston in the shell:
1070 <literallayout class='monospaced'>
1071 weston
1072 </literallayout></para></listitem>
1073 </orderedlist>
1074 </para>
1075 </section>
1076</section>
1077
1078<section id="licenses">
1079 <title>Licenses</title>
1080
1081 <para>
1082 This section describes the mechanism by which the OpenEmbedded build system
1083 tracks changes to licensing text.
1084 The section also describes how to enable commercially licensed recipes,
1085 which by default are disabled.
1086 </para>
1087
1088 <para>
1089 For information that can help you maintain compliance with various open
1090 source licensing during the lifecycle of the product, see the
1091 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
1092 in the Yocto Project Development Manual.
1093 </para>
1094
1095 <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
1096 <title>Tracking License Changes</title>
1097
1098 <para>
1099 The license of an upstream project might change in the future.
1100 In order to prevent these changes going unnoticed, the
1101 <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
1102 variable tracks changes to the license text. The checksums are validated at the end of the
1103 configure step, and if the checksums do not match, the build will fail.
1104 </para>
1105
1106 <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
1107 <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
1108
1109 <para>
1110 The <filename>LIC_FILES_CHKSUM</filename>
1111 variable contains checksums of the license text in the source code for the recipe.
1112 Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
1113 <literallayout class='monospaced'>
1114 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
1115 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
1116 file://licfile2.txt;endline=50;md5=zzzz \
1117 ..."
1118 </literallayout>
1119 </para>
1120
1121 <para>
1122 The build system uses the
1123 <filename><link linkend='var-S'>S</link></filename> variable as
1124 the default directory when searching files listed in
1125 <filename>LIC_FILES_CHKSUM</filename>.
1126 The previous example employs the default directory.
1127 </para>
1128
1129 <para>
1130 Consider this next example:
1131 <literallayout class='monospaced'>
1132 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
1133 md5=bb14ed3c4cda583abc85401304b5cd4e"
1134 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
1135 </literallayout>
1136 </para>
1137
1138 <para>
1139 The first line locates a file in
1140 <filename>${S}/src/ls.c</filename>.
1141 The second line refers to a file in
1142 <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
1143 </para>
1144 <para>
1145 Note that <filename>LIC_FILES_CHKSUM</filename> variable is
1146 mandatory for all recipes, unless the
1147 <filename>LICENSE</filename> variable is set to "CLOSED".
1148 </para>
1149 </section>
1150
1151 <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
1152 <title>Explanation of Syntax</title>
1153 <para>
1154 As mentioned in the previous section, the
1155 <filename>LIC_FILES_CHKSUM</filename> variable lists all the
1156 important files that contain the license text for the source code.
1157 It is possible to specify a checksum for an entire file, or a specific section of a
1158 file (specified by beginning and ending line numbers with the "beginline" and "endline"
1159 parameters, respectively).
1160 The latter is useful for source files with a license notice header,
1161 README documents, and so forth.
1162 If you do not use the "beginline" parameter, then it is assumed that the text begins on the
1163 first line of the file.
1164 Similarly, if you do not use the "endline" parameter, it is assumed that the license text
1165 ends with the last line of the file.
1166 </para>
1167
1168 <para>
1169 The "md5" parameter stores the md5 checksum of the license text.
1170 If the license text changes in any way as compared to this parameter
1171 then a mismatch occurs.
1172 This mismatch triggers a build failure and notifies the developer.
1173 Notification allows the developer to review and address the license text changes.
1174 Also note that if a mismatch occurs during the build, the correct md5
1175 checksum is placed in the build log and can be easily copied to the recipe.
1176 </para>
1177
1178 <para>
1179 There is no limit to how many files you can specify using the
1180 <filename>LIC_FILES_CHKSUM</filename> variable.
1181 Generally, however, every project requires a few specifications for license tracking.
1182 Many projects have a "COPYING" file that stores the license information for all the source
1183 code files.
1184 This practice allows you to just track the "COPYING" file as long as it is kept up to date.
1185 </para>
1186
1187 <tip>
1188 If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
1189 error and displays the correct "md5" parameter value during the build.
1190 The correct parameter is also captured in the build log.
1191 </tip>
1192
1193 <tip>
1194 If the whole file contains only license text, you do not need to use the "beginline" and
1195 "endline" parameters.
1196 </tip>
1197 </section>
1198 </section>
1199
1200 <section id="enabling-commercially-licensed-recipes">
1201 <title>Enabling Commercially Licensed Recipes</title>
1202
1203 <para>
1204 By default, the OpenEmbedded build system disables
1205 components that have commercial or other special licensing
1206 requirements.
1207 Such requirements are defined on a
1208 recipe-by-recipe basis through the
1209 <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
1210 variable definition in the affected recipe.
1211 For instance, the
1212 <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1213 recipe contains the following statement:
1214 <literallayout class='monospaced'>
1215 LICENSE_FLAGS = "commercial"
1216 </literallayout>
1217 Here is a slightly more complicated example that contains both an
1218 explicit recipe name and version (after variable expansion):
1219 <literallayout class='monospaced'>
1220 LICENSE_FLAGS = "license_${PN}_${PV}"
1221 </literallayout>
1222 In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
1223 definition to be enabled and included in an image, it
1224 needs to have a matching entry in the global
1225 <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
1226 variable, which is a variable
1227 typically defined in your <filename>local.conf</filename> file.
1228 For example, to enable
1229 the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
1230 package, you could add either the string
1231 "commercial_gst-plugins-ugly" or the more general string
1232 "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
1233 See the
1234 "<link linkend='license-flag-matching'>License Flag Matching</link>" section
1235 for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
1236 Here is the example:
1237 <literallayout class='monospaced'>
1238 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
1239 </literallayout>
1240 Likewise, to additionally enable the package built from the recipe containing
1241 <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
1242 that the actual recipe name was <filename>emgd_1.10.bb</filename>,
1243 the following string would enable that package as well as
1244 the original <filename>gst-plugins-ugly</filename> package:
1245 <literallayout class='monospaced'>
1246 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
1247 </literallayout>
1248 As a convenience, you do not need to specify the complete license string
1249 in the whitelist for every package.
1250 You can use an abbreviated form, which consists
1251 of just the first portion or portions of the license string before
1252 the initial underscore character or characters.
1253 A partial string will match
1254 any license that contains the given string as the first
1255 portion of its license.
1256 For example, the following
1257 whitelist string will also match both of the packages
1258 previously mentioned as well as any other packages that have
1259 licenses starting with "commercial" or "license".
1260 <literallayout class='monospaced'>
1261 LICENSE_FLAGS_WHITELIST = "commercial license"
1262 </literallayout>
1263 </para>
1264
1265 <section id="license-flag-matching">
1266 <title>License Flag Matching</title>
1267
1268 <para>
1269 License flag matching allows you to control what recipes the
1270 OpenEmbedded build system includes in the build.
1271 Fundamentally, the build system attempts to match
1272 <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
1273 strings found in recipes against
1274 <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
1275 strings found in the whitelist.
1276 A match causes the build system to include a recipe in the
1277 build, while failure to find a match causes the build system to
1278 exclude a recipe.
1279 </para>
1280
1281 <para>
1282 In general, license flag matching is simple.
1283 However, understanding some concepts will help you
1284 correctly and effectively use matching.
1285 </para>
1286
1287 <para>
1288 Before a flag
1289 defined by a particular recipe is tested against the
1290 contents of the whitelist, the expanded string
1291 <filename>_${PN}</filename> is appended to the flag.
1292 This expansion makes each <filename>LICENSE_FLAGS</filename>
1293 value recipe-specific.
1294 After expansion, the string is then matched against the
1295 whitelist.
1296 Thus, specifying
1297 <filename>LICENSE_FLAGS = "commercial"</filename>
1298 in recipe "foo", for example, results in the string
1299 <filename>"commercial_foo"</filename>.
1300 And, to create a match, that string must appear in the
1301 whitelist.
1302 </para>
1303
1304 <para>
1305 Judicious use of the <filename>LICENSE_FLAGS</filename>
1306 strings and the contents of the
1307 <filename>LICENSE_FLAGS_WHITELIST</filename> variable
1308 allows you a lot of flexibility for including or excluding
1309 recipes based on licensing.
1310 For example, you can broaden the matching capabilities by
1311 using license flags string subsets in the whitelist.
1312 <note>When using a string subset, be sure to use the part of
1313 the expanded string that precedes the appended underscore
1314 character (e.g. <filename>usethispart_1.3</filename>,
1315 <filename>usethispart_1.4</filename>, and so forth).
1316 </note>
1317 For example, simply specifying the string "commercial" in
1318 the whitelist matches any expanded
1319 <filename>LICENSE_FLAGS</filename> definition that starts with
1320 the string "commercial" such as "commercial_foo" and
1321 "commercial_bar", which are the strings the build system
1322 automatically generates for hypothetical recipes named
1323 "foo" and "bar" assuming those recipes simply specify the
1324 following:
1325 <literallayout class='monospaced'>
1326 LICENSE_FLAGS = "commercial"
1327 </literallayout>
1328 Thus, you can choose to exhaustively
1329 enumerate each license flag in the whitelist and
1330 allow only specific recipes into the image, or
1331 you can use a string subset that causes a broader range of
1332 matches to allow a range of recipes into the image.
1333 </para>
1334
1335 <para>
1336 This scheme works even if the
1337 <filename>LICENSE_FLAGS</filename> string already
1338 has <filename>_${PN}</filename> appended.
1339 For example, the build system turns the license flag
1340 "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
1341 match both the general "commercial" and the specific
1342 "commercial_1.2_foo" strings found in the whitelist, as
1343 expected.
1344 </para>
1345
1346 <para>
1347 Here are some other scenarios:
1348 <itemizedlist>
1349 <listitem><para>You can specify a versioned string in the
1350 recipe such as "commercial_foo_1.2" in a "foo" recipe.
1351 The build system expands this string to
1352 "commercial_foo_1.2_foo".
1353 Combine this license flag with a whitelist that has
1354 the string "commercial" and you match the flag along
1355 with any other flag that starts with the string
1356 "commercial".</para></listitem>
1357 <listitem><para>Under the same circumstances, you can
1358 use "commercial_foo" in the whitelist and the
1359 build system not only matches "commercial_foo_1.2" but
1360 also matches any license flag with the string
1361 "commercial_foo", regardless of the version.
1362 </para></listitem>
1363 <listitem><para>You can be very specific and use both the
1364 package and version parts in the whitelist (e.g.
1365 "commercial_foo_1.2") to specifically match a
1366 versioned recipe.</para></listitem>
1367 </itemizedlist>
1368 </para>
1369 </section>
1370
1371 <section id="other-variables-related-to-commercial-licenses">
1372 <title>Other Variables Related to Commercial Licenses</title>
1373
1374 <para>
1375 Other helpful variables related to commercial
1376 license handling exist and are defined in the
1377 <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
1378 <literallayout class='monospaced'>
1379 COMMERCIAL_AUDIO_PLUGINS ?= ""
1380 COMMERCIAL_VIDEO_PLUGINS ?= ""
1381 COMMERCIAL_QT = ""
1382 </literallayout>
1383 If you want to enable these components, you can do so by making sure you have
1384 statements similar to the following
1385 in your <filename>local.conf</filename> configuration file:
1386 <literallayout class='monospaced'>
1387 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
1388 gst-plugins-ugly-mpegaudioparse"
1389 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
1390 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
1391 COMMERCIAL_QT ?= "qmmp"
1392 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
1393 </literallayout>
1394 Of course, you could also create a matching whitelist
1395 for those components using the more general "commercial"
1396 in the whitelist, but that would also enable all the
1397 other packages with
1398 <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
1399 containing "commercial", which you may or may not want:
1400 <literallayout class='monospaced'>
1401 LICENSE_FLAGS_WHITELIST = "commercial"
1402 </literallayout>
1403 </para>
1404
1405 <para>
1406 Specifying audio and video plug-ins as part of the
1407 <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
1408 <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
1409 or commercial Qt components as part of
1410 the <filename>COMMERCIAL_QT</filename> statement (along
1411 with the enabling <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
1412 plug-ins or components into built images, thus adding
1413 support for media formats or components.
1414 </para>
1415 </section>
1416 </section>
1417</section>
1418</chapter>
1419<!--
1420vim: expandtab tw=80 ts=4
1421-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-26 05:52:12 +0000