summaryrefslogtreecommitdiffstats
path: root/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
diff options
context:
space:
mode:
Diffstat (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst')
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst1728
1 files changed, 1728 insertions, 0 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
new file mode 100644
index 0000000000..790065ef64
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -0,0 +1,1728 @@
1====================
2Syntax and Operators
3====================
4
5BitBake files have their own syntax. The syntax has similarities to
6several other languages but also has some unique features. This section
7describes the available syntax and operators as well as provides
8examples.
9
10Basic Syntax
11============
12
13This section provides some basic syntax examples.
14
15Basic Variable Setting
16----------------------
17
18The following example sets ``VARIABLE`` to "value". This assignment
19occurs immediately as the statement is parsed. It is a "hard"
20assignment. VARIABLE = "value" As expected, if you include leading or
21trailing spaces as part of an assignment, the spaces are retained:
22VARIABLE = " value" VARIABLE = "value " Setting ``VARIABLE`` to "" sets
23it to an empty string, while setting the variable to " " sets it to a
24blank space (i.e. these are not the same values). VARIABLE = "" VARIABLE
25= " "
26
27You can use single quotes instead of double quotes when setting a
28variable's value. Doing so allows you to use values that contain the
29double quote character: VARIABLE = 'I have a " in my value'
30
31.. note::
32
33 Unlike in Bourne shells, single quotes work identically to double
34 quotes in all other ways. They do not suppress variable expansions.
35
36Modifying Existing Variables
37----------------------------
38
39Sometimes you need to modify existing variables. Following are some
40cases where you might find you want to modify an existing variable:
41
42- Customize a recipe that uses the variable.
43
44- Change a variable's default value used in a ``*.bbclass`` file.
45
46- Change the variable in a ``*.bbappend`` file to override the variable
47 in the original recipe.
48
49- Change the variable in a configuration file so that the value
50 overrides an existing configuration.
51
52Changing a variable value can sometimes depend on how the value was
53originally assigned and also on the desired intent of the change. In
54particular, when you append a value to a variable that has a default
55value, the resulting value might not be what you expect. In this case,
56the value you provide might replace the value rather than append to the
57default value.
58
59If after you have changed a variable's value and something unexplained
60occurs, you can use BitBake to check the actual value of the suspect
61variable. You can make these checks for both configuration and recipe
62level changes:
63
64- For configuration changes, use the following: $ bitbake -e This
65 command displays variable values after the configuration files (i.e.
66 ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth)
67 have been parsed.
68
69 .. note::
70
71 Variables that are exported to the environment are preceded by the
72 string "export" in the command's output.
73
74- For recipe changes, use the following: $ bitbake recipe -e \| grep
75 VARIABLE=" This command checks to see if the variable actually makes
76 it into a specific recipe.
77
78Line Joining
79------------
80
81Outside of `functions <#functions>`__, BitBake joins any line ending in
82a backslash character ("\") with the following line before parsing
83statements. The most common use for the "\" character is to split
84variable assignments over multiple lines, as in the following example:
85FOO = "bar \\ baz \\ qaz" Both the "\" character and the newline
86character that follow it are removed when joining lines. Thus, no
87newline characters end up in the value of ``FOO``.
88
89Consider this additional example where the two assignments both assign
90"barbaz" to ``FOO``: FOO = "barbaz" FOO = "bar\\ baz"
91
92.. note::
93
94 BitBake does not interpret escape sequences like "\n" in variable
95 values. For these to have an effect, the value must be passed to some
96 utility that interprets escape sequences, such as
97 printf
98 or
99 echo -n
100 .
101
102Variable Expansion
103------------------
104
105Variables can reference the contents of other variables using a syntax
106that is similar to variable expansion in Bourne shells. The following
107assignments result in A containing "aval" and B evaluating to
108"preavalpost". A = "aval" B = "pre${A}post"
109
110.. note::
111
112 Unlike in Bourne shells, the curly braces are mandatory: Only
113 ${FOO}
114 and not
115 $FOO
116 is recognized as an expansion of
117 FOO
118 .
119
120The "=" operator does not immediately expand variable references in the
121right-hand side. Instead, expansion is deferred until the variable
122assigned to is actually used. The result depends on the current values
123of the referenced variables. The following example should clarify this
124behavior: A = "${B} baz" B = "${C} bar" C = "foo" \*At this point, ${A}
125equals "foo bar baz"\* C = "qux" \*At this point, ${A} equals "qux bar
126baz"\* B = "norf" \*At this point, ${A} equals "norf baz"\* Contrast
127this behavior with the `immediate variable
128expansion <#immediate-variable-expansion>`__ operator (i.e. ":=").
129
130If the variable expansion syntax is used on a variable that does not
131exist, the string is kept as is. For example, given the following
132assignment, ``BAR`` expands to the literal string "${FOO}" as long as
133``FOO`` does not exist. BAR = "${FOO}"
134
135Setting a default value (?=)
136----------------------------
137
138You can use the "?=" operator to achieve a "softer" assignment for a
139variable. This type of assignment allows you to define a variable if it
140is undefined when the statement is parsed, but to leave the value alone
141if the variable has a value. Here is an example: A ?= "aval" If ``A`` is
142set at the time this statement is parsed, the variable retains its
143value. However, if ``A`` is not set, the variable is set to "aval".
144
145.. note::
146
147 This assignment is immediate. Consequently, if multiple "?="
148 assignments to a single variable exist, the first of those ends up
149 getting used.
150
151Setting a weak default value (??=)
152----------------------------------
153
154It is possible to use a "weaker" assignment than in the previous section
155by using the "??=" operator. This assignment behaves identical to "?="
156except that the assignment is made at the end of the parsing process
157rather than immediately. Consequently, when multiple "??=" assignments
158exist, the last one is used. Also, any "=" or "?=" assignment will
159override the value set with "??=". Here is an example: A ??= "somevalue"
160A ??= "someothervalue" If ``A`` is set before the above statements are
161parsed, the variable retains its value. If ``A`` is not set, the
162variable is set to "someothervalue".
163
164Again, this assignment is a "lazy" or "weak" assignment because it does
165not occur until the end of the parsing process.
166
167Immediate variable expansion (:=)
168---------------------------------
169
170The ":=" operator results in a variable's contents being expanded
171immediately, rather than when the variable is actually used: T = "123" A
172:= "test ${T}" T = "456" B := "${T} ${C}" C = "cval" C := "${C}append"
173In this example, ``A`` contains "test 123", even though the final value
174of ``T`` is "456". The variable ``B`` will end up containing "456
175cvalappend". This is because references to undefined variables are
176preserved as is during (immediate)expansion. This is in contrast to GNU
177Make, where undefined variables expand to nothing. The variable ``C``
178contains "cvalappend" since ``${C}`` immediately expands to "cval".
179
180.. _appending-and-prepending:
181
182Appending (+=) and prepending (=+) With Spaces
183----------------------------------------------
184
185Appending and prepending values is common and can be accomplished using
186the "+=" and "=+" operators. These operators insert a space between the
187current value and prepended or appended value.
188
189These operators take immediate effect during parsing. Here are some
190examples: B = "bval" B += "additionaldata" C = "cval" C =+ "test" The
191variable ``B`` contains "bval additionaldata" and ``C`` contains "test
192cval".
193
194.. _appending-and-prepending-without-spaces:
195
196Appending (.=) and Prepending (=.) Without Spaces
197-------------------------------------------------
198
199If you want to append or prepend values without an inserted space, use
200the ".=" and "=." operators.
201
202These operators take immediate effect during parsing. Here are some
203examples: B = "bval" B .= "additionaldata" C = "cval" C =. "test" The
204variable ``B`` contains "bvaladditionaldata" and ``C`` contains
205"testcval".
206
207Appending and Prepending (Override Style Syntax)
208------------------------------------------------
209
210You can also append and prepend a variable's value using an override
211style syntax. When you use this syntax, no spaces are inserted.
212
213These operators differ from the ":=", ".=", "=.", "+=", and "=+"
214operators in that their effects are applied at variable expansion time
215rather than being immediately applied. Here are some examples: B =
216"bval" B_append = " additional data" C = "cval" C_prepend = "additional
217data " D = "dval" D_append = "additional data" The variable ``B``
218becomes "bval additional data" and ``C`` becomes "additional data cval".
219The variable ``D`` becomes "dvaladditional data".
220
221.. note::
222
223 You must control all spacing when you use the override syntax.
224
225It is also possible to append and prepend to shell functions and
226BitBake-style Python functions. See the "`Shell
227Functions <#shell-functions>`__" and "`BitBake-Style Python
228Functions <#bitbake-style-python-functions>`__ sections for examples.
229
230.. _removing-override-style-syntax:
231
232Removal (Override Style Syntax)
233-------------------------------
234
235You can remove values from lists using the removal override style
236syntax. Specifying a value for removal causes all occurrences of that
237value to be removed from the variable.
238
239When you use this syntax, BitBake expects one or more strings.
240Surrounding spaces and spacing are preserved. Here is an example: FOO =
241"123 456 789 123456 123 456 123 456" FOO_remove = "123" FOO_remove =
242"456" FOO2 = " abc def ghi abcdef abc def abc def def" FOO2_remove = "
243\\ def \\ abc \\ ghi \\ " The variable ``FOO`` becomes
244"  789 123456    " and ``FOO2`` becomes "     abcdef      ".
245
246Like "_append" and "_prepend", "_remove" is applied at variable
247expansion time.
248
249Override Style Operation Advantages
250-----------------------------------
251
252An advantage of the override style operations "_append", "_prepend", and
253"_remove" as compared to the "+=" and "=+" operators is that the
254override style operators provide guaranteed operations. For example,
255consider a class ``foo.bbclass`` that needs to add the value "val" to
256the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows:
257inherit foo FOO = "initial" If ``foo.bbclass`` uses the "+=" operator,
258as follows, then the final value of ``FOO`` will be "initial", which is
259not what is desired: FOO += "val" If, on the other hand, ``foo.bbclass``
260uses the "_append" operator, then the final value of ``FOO`` will be
261"initial val", as intended: FOO_append = " val"
262
263.. note::
264
265 It is never necessary to use "+=" together with "_append". The
266 following sequence of assignments appends "barbaz" to
267 FOO
268 :
269 ::
270
271 FOO_append = "bar"
272 FOO_append = "baz"
273
274
275 The only effect of changing the second assignment in the previous
276 example to use "+=" would be to add a space before "baz" in the
277 appended value (due to how the "+=" operator works).
278
279Another advantage of the override style operations is that you can
280combine them with other overrides as described in the "`Conditional
281Syntax (Overrides) <#conditional-syntax-overrides>`__" section.
282
283Variable Flag Syntax
284--------------------
285
286Variable flags are BitBake's implementation of variable properties or
287attributes. It is a way of tagging extra information onto a variable.
288You can find more out about variable flags in general in the "`Variable
289Flags <#variable-flags>`__" section.
290
291You can define, append, and prepend values to variable flags. All the
292standard syntax operations previously mentioned work for variable flags
293except for override style syntax (i.e. "_prepend", "_append", and
294"_remove").
295
296Here are some examples showing how to set variable flags: FOO[a] = "abc"
297FOO[b] = "123" FOO[a] += "456" The variable ``FOO`` has two flags:
298``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123",
299respectively. The ``[a]`` flag becomes "abc 456".
300
301No need exists to pre-define variable flags. You can simply start using
302them. One extremely common application is to attach some brief
303documentation to a BitBake variable as follows: CACHE[doc] = "The
304directory holding the cache of the metadata."
305
306Inline Python Variable Expansion
307--------------------------------
308
309You can use inline Python variable expansion to set variables. Here is
310an example: DATE = "${@time.strftime('%Y%m%d',time.gmtime())}" This
311example results in the ``DATE`` variable being set to the current date.
312
313Probably the most common use of this feature is to extract the value of
314variables from BitBake's internal data dictionary, ``d``. The following
315lines select the values of a package name and its version number,
316respectively: PN =
317"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or
318'defaultpkgname'}" PV =
319"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or
320'1.0'}"
321
322.. note::
323
324 Inline Python expressions work just like variable expansions insofar
325 as the "=" and ":=" operators are concerned. Given the following
326 assignment,
327 foo()
328 is called each time
329 FOO
330 is expanded:
331 ::
332
333 FOO = "${@foo()}"
334
335
336 Contrast this with the following immediate assignment, where
337 foo()
338 is only called once, while the assignment is parsed:
339 ::
340
341 FOO := "${@foo()}"
342
343
344For a different way to set variables with Python code during parsing,
345see the "`Anonymous Python Functions <#anonymous-python-functions>`__"
346section.
347
348Unsetting variables
349-------------------
350
351It is possible to completely remove a variable or a variable flag from
352BitBake's internal data dictionary by using the "unset" keyword. Here is
353an example: unset DATE unset do_fetch[noexec] These two statements
354remove the ``DATE`` and the ``do_fetch[noexec]`` flag.
355
356Providing Pathnames
357-------------------
358
359When specifying pathnames for use with BitBake, do not use the tilde
360("~") character as a shortcut for your home directory. Doing so might
361cause BitBake to not recognize the path since BitBake does not expand
362this character in the same way a shell would.
363
364Instead, provide a fuller path as the following example illustrates:
365BBLAYERS ?= " \\ /home/scott-lenovo/LayerA \\ "
366
367Exporting Variables to the Environment
368======================================
369
370You can export variables to the environment of running tasks by using
371the ``export`` keyword. For example, in the following example, the
372``do_foo`` task prints "value from the environment" when run: export
373ENV_VARIABLE ENV_VARIABLE = "value from the environment" do_foo() {
374bbplain "$ENV_VARIABLE" }
375
376.. note::
377
378 BitBake does not expand
379 $ENV_VARIABLE
380 in this case because it lacks the obligatory
381 {}
382 . Rather,
383 $ENV_VARIABLE
384 is expanded by the shell.
385
386It does not matter whether ``export ENV_VARIABLE`` appears before or
387after assignments to ``ENV_VARIABLE``.
388
389It is also possible to combine ``export`` with setting a value for the
390variable. Here is an example: export ENV_VARIABLE = "variable-value" In
391the output of ``bitbake -e``, variables that are exported to the
392environment are preceded by "export".
393
394Among the variables commonly exported to the environment are ``CC`` and
395``CFLAGS``, which are picked up by many build systems.
396
397Conditional Syntax (Overrides)
398==============================
399
400BitBake uses ```OVERRIDES`` <#var-bb-OVERRIDES>`__ to control what
401variables are overridden after BitBake parses recipes and configuration
402files. This section describes how you can use ``OVERRIDES`` as
403conditional metadata, talks about key expansion in relationship to
404``OVERRIDES``, and provides some examples to help with understanding.
405
406Conditional Metadata
407--------------------
408
409You can use ``OVERRIDES`` to conditionally select a specific version of
410a variable and to conditionally append or prepend the value of a
411variable.
412
413.. note::
414
415 Overrides can only use lower-case characters. Additionally,
416 underscores are not permitted in override names as they are used to
417 separate overrides from each other and from the variable name.
418
419- *Selecting a Variable:* The ``OVERRIDES`` variable is a
420 colon-character-separated list that contains items for which you want
421 to satisfy conditions. Thus, if you have a variable that is
422 conditional on “arm”, and “arm” is in ``OVERRIDES``, then the
423 “arm”-specific version of the variable is used rather than the
424 non-conditional version. Here is an example: OVERRIDES =
425 "architecture:os:machine" TEST = "default" TEST_os = "osspecific"
426 TEST_nooverride = "othercondvalue" In this example, the ``OVERRIDES``
427 variable lists three overrides: "architecture", "os", and "machine".
428 The variable ``TEST`` by itself has a default value of "default". You
429 select the os-specific version of the ``TEST`` variable by appending
430 the "os" override to the variable (i.e.\ ``TEST_os``).
431
432 To better understand this, consider a practical example that assumes
433 an OpenEmbedded metadata-based Linux kernel recipe file. The
434 following lines from the recipe file first set the kernel branch
435 variable ``KBRANCH`` to a default value, then conditionally override
436 that value based on the architecture of the build: KBRANCH =
437 "standard/base" KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
438 KBRANCH_qemumips = "standard/mti-malta32" KBRANCH_qemuppc =
439 "standard/qemuppc" KBRANCH_qemux86 = "standard/common-pc/base"
440 KBRANCH_qemux86-64 = "standard/common-pc-64/base" KBRANCH_qemumips64
441 = "standard/mti-malta64"
442
443- *Appending and Prepending:* BitBake also supports append and prepend
444 operations to variable values based on whether a specific item is
445 listed in ``OVERRIDES``. Here is an example: DEPENDS = "glibc
446 ncurses" OVERRIDES = "machine:local" DEPENDS_append_machine = "
447 libmad" In this example, ``DEPENDS`` becomes "glibc ncurses libmad".
448
449 Again, using an OpenEmbedded metadata-based kernel recipe file as an
450 example, the following lines will conditionally append to the
451 ``KERNEL_FEATURES`` variable based on the architecture:
452 KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
453 KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
454 KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc
455 cfg/paravirt_kvm.scc"
456
457- *Setting a Variable for a Single Task:* BitBake supports setting a
458 variable just for the duration of a single task. Here is an example:
459 FOO_task-configure = "val 1" FOO_task-compile = "val 2" In the
460 previous example, ``FOO`` has the value "val 1" while the
461 ``do_configure`` task is executed, and the value "val 2" while the
462 ``do_compile`` task is executed.
463
464 Internally, this is implemented by prepending the task (e.g.
465 "task-compile:") to the value of
466 ```OVERRIDES`` <#var-bb-OVERRIDES>`__ for the local datastore of the
467 ``do_compile`` task.
468
469 You can also use this syntax with other combinations (e.g.
470 "``_prepend``") as shown in the following example:
471 EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} "
472
473Key Expansion
474-------------
475
476Key expansion happens when the BitBake datastore is finalized. To better
477understand this, consider the following example: A${B} = "X" B = "2" A2
478= "Y" In this case, after all the parsing is complete, BitBake expands
479``${B}`` into "2". This expansion causes ``A2``, which was set to "Y"
480before the expansion, to become "X".
481
482.. _variable-interaction-worked-examples:
483
484Examples
485--------
486
487Despite the previous explanations that show the different forms of
488variable definitions, it can be hard to work out exactly what happens
489when variable operators, conditional overrides, and unconditional
490overrides are combined. This section presents some common scenarios
491along with explanations for variable interactions that typically confuse
492users.
493
494There is often confusion concerning the order in which overrides and
495various "append" operators take effect. Recall that an append or prepend
496operation using "_append" and "_prepend" does not result in an immediate
497assignment as would "+=", ".=", "=+", or "=.". Consider the following
498example: OVERRIDES = "foo" A = "Z" A_foo_append = "X" For this case,
499``A`` is unconditionally set to "Z" and "X" is unconditionally and
500immediately appended to the variable ``A_foo``. Because overrides have
501not been applied yet, ``A_foo`` is set to "X" due to the append and
502``A`` simply equals "Z".
503
504Applying overrides, however, changes things. Since "foo" is listed in
505``OVERRIDES``, the conditional variable ``A`` is replaced with the "foo"
506version, which is equal to "X". So effectively, ``A_foo`` replaces
507``A``.
508
509This next example changes the order of the override and the append:
510OVERRIDES = "foo" A = "Z" A_append_foo = "X" For this case, before
511overrides are handled, ``A`` is set to "Z" and ``A_append_foo`` is set
512to "X". Once the override for "foo" is applied, however, ``A`` gets
513appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces
514are not appended.
515
516This next example has the order of the appends and overrides reversed
517back as in the first example: OVERRIDES = "foo" A = "Y" A_foo_append =
518"Z" A_foo_append = "X" For this case, before any overrides are resolved,
519``A`` is set to "Y" using an immediate assignment. After this immediate
520assignment, ``A_foo`` is set to "Z", and then further appended with "X"
521leaving the variable set to "ZX". Finally, applying the override for
522"foo" results in the conditional variable ``A`` becoming "ZX" (i.e.
523``A`` is replaced with ``A_foo``).
524
525This final example mixes in some varying operators: A = "1" A_append =
526"2" A_append = "3" A += "4" A .= "5" For this case, the type of append
527operators are affecting the order of assignments as BitBake passes
528through the code multiple times. Initially, ``A`` is set to "1 45"
529because of the three statements that use immediate operators. After
530these assignments are made, BitBake applies the "_append" operations.
531Those operations result in ``A`` becoming "1 4523".
532
533Sharing Functionality
534=====================
535
536BitBake allows for metadata sharing through include files (``.inc``) and
537class files (``.bbclass``). For example, suppose you have a piece of
538common functionality such as a task definition that you want to share
539between more than one recipe. In this case, creating a ``.bbclass`` file
540that contains the common functionality and then using the ``inherit``
541directive in your recipes to inherit the class would be a common way to
542share the task.
543
544This section presents the mechanisms BitBake provides to allow you to
545share functionality between recipes. Specifically, the mechanisms
546include ``include``, ``inherit``, ``INHERIT``, and ``require``
547directives.
548
549Locating Include and Class Files
550--------------------------------
551
552BitBake uses the ```BBPATH`` <#var-bb-BBPATH>`__ variable to locate
553needed include and class files. Additionally, BitBake searches the
554current directory for ``include`` and ``require`` directives.
555
556.. note::
557
558 The
559 BBPATH
560 variable is analogous to the environment variable
561 PATH
562 .
563
564In order for include and class files to be found by BitBake, they need
565to be located in a "classes" subdirectory that can be found in
566``BBPATH``.
567
568``inherit`` Directive
569---------------------
570
571When writing a recipe or class file, you can use the ``inherit``
572directive to inherit the functionality of a class (``.bbclass``).
573BitBake only supports this directive when used within recipe and class
574files (i.e. ``.bb`` and ``.bbclass``).
575
576The ``inherit`` directive is a rudimentary means of specifying
577functionality contained in class files that your recipes require. For
578example, you can easily abstract out the tasks involved in building a
579package that uses Autoconf and Automake and put those tasks into a class
580file and then have your recipe inherit that class file.
581
582As an example, your recipes could use the following directive to inherit
583an ``autotools.bbclass`` file. The class file would contain common
584functionality for using Autotools that could be shared across recipes:
585inherit autotools In this case, BitBake would search for the directory
586``classes/autotools.bbclass`` in ``BBPATH``.
587
588.. note::
589
590 You can override any values and functions of the inherited class
591 within your recipe by doing so after the "inherit" statement.
592
593If you want to use the directive to inherit multiple classes, separate
594them with spaces. The following example shows how to inherit both the
595``buildhistory`` and ``rm_work`` classes: inherit buildhistory rm_work
596
597An advantage with the inherit directive as compared to both the
598`include <#include-directive>`__ and `require <#require-inclusion>`__
599directives is that you can inherit class files conditionally. You can
600accomplish this by using a variable expression after the ``inherit``
601statement. Here is an example: inherit ${VARNAME} If ``VARNAME`` is
602going to be set, it needs to be set before the ``inherit`` statement is
603parsed. One way to achieve a conditional inherit in this case is to use
604overrides: VARIABLE = "" VARIABLE_someoverride = "myclass"
605
606Another method is by using anonymous Python. Here is an example: python
607() { if condition == value: d.setVar('VARIABLE', 'myclass') else:
608d.setVar('VARIABLE', '') }
609
610Alternatively, you could use an in-line Python expression in the
611following form: inherit ${@'classname' if condition else ''} inherit
612${@functionname(params)} In all cases, if the expression evaluates to an
613empty string, the statement does not trigger a syntax error because it
614becomes a no-op.
615
616``include`` Directive
617---------------------
618
619BitBake understands the ``include`` directive. This directive causes
620BitBake to parse whatever file you specify, and to insert that file at
621that location. The directive is much like its equivalent in Make except
622that if the path specified on the include line is a relative path,
623BitBake locates the first file it can find within ``BBPATH``.
624
625The include directive is a more generic method of including
626functionality as compared to the `inherit <#inherit-directive>`__
627directive, which is restricted to class (i.e. ``.bbclass``) files. The
628include directive is applicable for any other kind of shared or
629encapsulated functionality or configuration that does not suit a
630``.bbclass`` file.
631
632As an example, suppose you needed a recipe to include some self-test
633definitions: include test_defs.inc
634
635.. note::
636
637 The
638 include
639 directive does not produce an error when the file cannot be found.
640 Consequently, it is recommended that if the file you are including is
641 expected to exist, you should use
642 require
643 instead of
644 include
645 . Doing so makes sure that an error is produced if the file cannot be
646 found.
647
648.. _require-inclusion:
649
650``require`` Directive
651---------------------
652
653BitBake understands the ``require`` directive. This directive behaves
654just like the ``include`` directive with the exception that BitBake
655raises a parsing error if the file to be included cannot be found. Thus,
656any file you require is inserted into the file that is being parsed at
657the location of the directive.
658
659The require directive, like the include directive previously described,
660is a more generic method of including functionality as compared to the
661`inherit <#inherit-directive>`__ directive, which is restricted to class
662(i.e. ``.bbclass``) files. The require directive is applicable for any
663other kind of shared or encapsulated functionality or configuration that
664does not suit a ``.bbclass`` file.
665
666Similar to how BitBake handles ```include`` <#include-directive>`__, if
667the path specified on the require line is a relative path, BitBake
668locates the first file it can find within ``BBPATH``.
669
670As an example, suppose you have two versions of a recipe (e.g.
671``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some
672identical functionality that could be shared. You could create an
673include file named ``foo.inc`` that contains the common definitions
674needed to build "foo". You need to be sure ``foo.inc`` is located in the
675same directory as your two recipe files as well. Once these conditions
676are set up, you can share the functionality using a ``require``
677directive from within each recipe: require foo.inc
678
679``INHERIT`` Configuration Directive
680-----------------------------------
681
682When creating a configuration file (``.conf``), you can use the
683```INHERIT`` <#var-bb-INHERIT>`__ configuration directive to inherit a
684class. BitBake only supports this directive when used within a
685configuration file.
686
687As an example, suppose you needed to inherit a class file called
688``abc.bbclass`` from a configuration file as follows: INHERIT += "abc"
689This configuration directive causes the named class to be inherited at
690the point of the directive during parsing. As with the ``inherit``
691directive, the ``.bbclass`` file must be located in a "classes"
692subdirectory in one of the directories specified in ``BBPATH``.
693
694.. note::
695
696 Because
697 .conf
698 files are parsed first during BitBake's execution, using
699 INHERIT
700 to inherit a class effectively inherits the class globally (i.e. for
701 all recipes).
702
703If you want to use the directive to inherit multiple classes, you can
704provide them on the same line in the ``local.conf`` file. Use spaces to
705separate the classes. The following example shows how to inherit both
706the ``autotools`` and ``pkgconfig`` classes: INHERIT += "autotools
707pkgconfig"
708
709Functions
710=========
711
712As with most languages, functions are the building blocks that are used
713to build up operations into tasks. BitBake supports these types of
714functions:
715
716- *Shell Functions:* Functions written in shell script and executed
717 either directly as functions, tasks, or both. They can also be called
718 by other shell functions.
719
720- *BitBake-Style Python Functions:* Functions written in Python and
721 executed by BitBake or other Python functions using
722 ``bb.build.exec_func()``.
723
724- *Python Functions:* Functions written in Python and executed by
725 Python.
726
727- *Anonymous Python Functions:* Python functions executed automatically
728 during parsing.
729
730Regardless of the type of function, you can only define them in class
731(``.bbclass``) and recipe (``.bb`` or ``.inc``) files.
732
733Shell Functions
734---------------
735
736Functions written in shell script and executed either directly as
737functions, tasks, or both. They can also be called by other shell
738functions. Here is an example shell function definition: some_function
739() { echo "Hello World" } When you create these types of functions in
740your recipe or class files, you need to follow the shell programming
741rules. The scripts are executed by ``/bin/sh``, which may not be a bash
742shell but might be something such as ``dash``. You should not use
743Bash-specific script (bashisms).
744
745Overrides and override-style operators like ``_append`` and ``_prepend``
746can also be applied to shell functions. Most commonly, this application
747would be used in a ``.bbappend`` file to modify functions in the main
748recipe. It can also be used to modify functions inherited from classes.
749
750As an example, consider the following: do_foo() { bbplain first fn }
751fn_prepend() { bbplain second } fn() { bbplain third } do_foo_append() {
752bbplain fourth } Running ``do_foo`` prints the following: recipename
753do_foo: first recipename do_foo: second recipename do_foo: third
754recipename do_foo: fourth
755
756.. note::
757
758 Overrides and override-style operators can be applied to any shell
759 function, not just
760 tasks
761 .
762
763You can use the ``bitbake -e`` recipename command to view the final
764assembled function after all overrides have been applied.
765
766BitBake-Style Python Functions
767------------------------------
768
769These functions are written in Python and executed by BitBake or other
770Python functions using ``bb.build.exec_func()``.
771
772An example BitBake function is: python some_python_function () {
773d.setVar("TEXT", "Hello World") print d.getVar("TEXT") } Because the
774Python "bb" and "os" modules are already imported, you do not need to
775import these modules. Also in these types of functions, the datastore
776("d") is a global variable and is always automatically available.
777
778.. note::
779
780 Variable expressions (e.g.
781 ${X}
782 ) are no longer expanded within Python functions. This behavior is
783 intentional in order to allow you to freely set variable values to
784 expandable expressions without having them expanded prematurely. If
785 you do wish to expand a variable within a Python function, use
786 d.getVar("X")
787 . Or, for more complicated expressions, use
788 d.expand()
789 .
790
791Similar to shell functions, you can also apply overrides and
792override-style operators to BitBake-style Python functions.
793
794As an example, consider the following: python do_foo_prepend() {
795bb.plain("first") } python do_foo() { bb.plain("second") } python
796do_foo_append() { bb.plain("third") } Running ``do_foo`` prints the
797following: recipename do_foo: first recipename do_foo: second recipename
798do_foo: third You can use the ``bitbake -e`` recipename command to view
799the final assembled function after all overrides have been applied.
800
801Python Functions
802----------------
803
804These functions are written in Python and are executed by other Python
805code. Examples of Python functions are utility functions that you intend
806to call from in-line Python or from within other Python functions. Here
807is an example: def get_depends(d): if d.getVar('SOMECONDITION'): return
808"dependencywithcond" else: return "dependency" SOMECONDITION = "1"
809DEPENDS = "${@get_depends(d)}" This would result in ``DEPENDS``
810containing ``dependencywithcond``.
811
812Here are some things to know about Python functions:
813
814- Python functions can take parameters.
815
816- The BitBake datastore is not automatically available. Consequently,
817 you must pass it in as a parameter to the function.
818
819- The "bb" and "os" Python modules are automatically available. You do
820 not need to import them.
821
822BitBake-Style Python Functions Versus Python Functions
823------------------------------------------------------
824
825Following are some important differences between BitBake-style Python
826functions and regular Python functions defined with "def":
827
828- Only BitBake-style Python functions can be `tasks <#tasks>`__.
829
830- Overrides and override-style operators can only be applied to
831 BitBake-style Python functions.
832
833- Only regular Python functions can take arguments and return values.
834
835- `Variable flags <#variable-flags>`__ such as ``[dirs]``,
836 ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style
837 Python functions, but not on regular Python functions.
838
839- BitBake-style Python functions generate a separate
840 ``${``\ ```T`` <#var-bb-T>`__\ ``}/run.``\ function-name\ ``.``\ pid
841 script that is executed to run the function, and also generate a log
842 file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed
843 as tasks.
844
845 Regular Python functions execute "inline" and do not generate any
846 files in ``${T}``.
847
848- Regular Python functions are called with the usual Python syntax.
849 BitBake-style Python functions are usually tasks and are called
850 directly by BitBake, but can also be called manually from Python code
851 by using the ``bb.build.exec_func()`` function. Here is an example:
852 bb.build.exec_func("my_bitbake_style_function", d)
853
854 .. note::
855
856 bb.build.exec_func()
857 can also be used to run shell functions from Python code. If you
858 want to run a shell function before a Python function within the
859 same task, then you can use a parent helper Python function that
860 starts by running the shell function with
861 bb.build.exec_func()
862 and then runs the Python code.
863
864 To detect errors from functions executed with
865 ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed``
866 exception.
867
868 .. note::
869
870 Functions in metadata (recipes and classes) should not themselves
871 raise
872 bb.build.FuncFailed
873 . Rather,
874 bb.build.FuncFailed
875 should be viewed as a general indicator that the called function
876 failed by raising an exception. For example, an exception raised
877 by
878 bb.fatal()
879 will be caught inside
880 bb.build.exec_func()
881 , and a
882 bb.build.FuncFailed
883 will be raised in response.
884
885Due to their simplicity, you should prefer regular Python functions over
886BitBake-style Python functions unless you need a feature specific to
887BitBake-style Python functions. Regular Python functions in metadata are
888a more recent invention than BitBake-style Python functions, and older
889code tends to use ``bb.build.exec_func()`` more often.
890
891Anonymous Python Functions
892--------------------------
893
894Sometimes it is useful to set variables or perform other operations
895programmatically during parsing. To do this, you can define special
896Python functions, called anonymous Python functions, that run at the end
897of parsing. For example, the following conditionally sets a variable
898based on the value of another variable: python () { if
899d.getVar('SOMEVAR') == 'value': d.setVar('ANOTHERVAR', 'value2') } An
900equivalent way to mark a function as an anonymous function is to give it
901the name "__anonymous", rather than no name.
902
903Anonymous Python functions always run at the end of parsing, regardless
904of where they are defined. If a recipe contains many anonymous
905functions, they run in the same order as they are defined within the
906recipe. As an example, consider the following snippet: python () {
907d.setVar('FOO', 'foo 2') } FOO = "foo 1" python () { d.appendVar('BAR',
908' bar 2') } BAR = "bar 1" The previous example is conceptually
909equivalent to the following snippet: FOO = "foo 1" BAR = "bar 1" FOO =
910"foo 2" BAR += "bar 2" ``FOO`` ends up with the value "foo 2", and
911``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the
912values set for the variables within the anonymous functions become
913available to tasks, which always run after parsing.
914
915Overrides and override-style operators such as "``_append``" are applied
916before anonymous functions run. In the following example, ``FOO`` ends
917up with the value "foo from anonymous": FOO = "foo" FOO_append = " from
918outside" python () { d.setVar("FOO", "foo from anonymous") } For methods
919you can use with anonymous Python functions, see the "`Functions You Can
920Call From Within Python <#functions-you-can-call-from-within-python>`__"
921section. For a different method to run Python code during parsing, see
922the "`Inline Python Variable
923Expansion <#inline-python-variable-expansion>`__" section.
924
925Flexible Inheritance for Class Functions
926----------------------------------------
927
928Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake
929supports exporting a function from a class such that the class function
930appears as the default implementation of the function, but can still be
931called if a recipe inheriting the class needs to define its own version
932of the function.
933
934To understand the benefits of this feature, consider the basic scenario
935where a class defines a task function and your recipe inherits the
936class. In this basic scenario, your recipe inherits the task function as
937defined in the class. If desired, your recipe can add to the start and
938end of the function by using the "_prepend" or "_append" operations
939respectively, or it can redefine the function completely. However, if it
940redefines the function, there is no means for it to call the class
941version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that
942enables the recipe's version of the function to call the original
943version of the function.
944
945To make use of this technique, you need the following things in place:
946
947- The class needs to define the function as follows:
948 classname\ ``_``\ functionname For example, if you have a class file
949 ``bar.bbclass`` and a function named ``do_foo``, the class must
950 define the function as follows: bar_do_foo
951
952- The class needs to contain the ``EXPORT_FUNCTIONS`` statement as
953 follows: EXPORT_FUNCTIONS functionname For example, continuing with
954 the same example, the statement in the ``bar.bbclass`` would be as
955 follows: EXPORT_FUNCTIONS do_foo
956
957- You need to call the function appropriately from within your recipe.
958 Continuing with the same example, if your recipe needs to call the
959 class version of the function, it should call ``bar_do_foo``.
960 Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was
961 used as above, the recipe's function could conditionally call the
962 class version of the function as follows: do_foo() { if [
963 somecondition ] ; then bar_do_foo else # Do something else fi } To
964 call your modified version of the function as defined in your recipe,
965 call it as ``do_foo``.
966
967With these conditions met, your single recipe can freely choose between
968the original function as defined in the class file and the modified
969function in your recipe. If you do not set up these conditions, you are
970limited to using one function or the other.
971
972Tasks
973=====
974
975Tasks are BitBake execution units that make up the steps that BitBake
976can run for a given recipe. Tasks are only supported in recipes and
977classes (i.e. in ``.bb`` files and files included or inherited from
978``.bb`` files). By convention, tasks have names that start with "do_".
979
980Promoting a Function to a Task
981------------------------------
982
983Tasks are either `shell functions <#shell-functions>`__ or
984`BitBake-style Python functions <#bitbake-style-python-functions>`__
985that have been promoted to tasks by using the ``addtask`` command. The
986``addtask`` command can also optionally describe dependencies between
987the task and other tasks. Here is an example that shows how to define a
988task and declare some dependencies: python do_printdate () { import time
989print time.strftime('%Y%m%d', time.gmtime()) } addtask printdate after
990do_fetch before do_build The first argument to ``addtask`` is the name
991of the function to promote to a task. If the name does not start with
992"do_", "do_" is implicitly added, which enforces the convention that all
993task names start with "do_".
994
995In the previous example, the ``do_printdate`` task becomes a dependency
996of the ``do_build`` task, which is the default task (i.e. the task run
997by the ``bitbake`` command unless another task is specified explicitly).
998Additionally, the ``do_printdate`` task becomes dependent upon the
999``do_fetch`` task. Running the ``do_build`` task results in the
1000``do_printdate`` task running first.
1001
1002.. note::
1003
1004 If you try out the previous example, you might see that the
1005 do_printdate
1006 task is only run the first time you build the recipe with the
1007 bitbake
1008 command. This is because BitBake considers the task "up-to-date"
1009 after that initial run. If you want to force the task to always be
1010 rerun for experimentation purposes, you can make BitBake always
1011 consider the task "out-of-date" by using the
1012 [
1013 nostamp
1014 ]
1015 variable flag, as follows:
1016 ::
1017
1018 do_printdate[nostamp] = "1"
1019
1020
1021 You can also explicitly run the task and provide the
1022 -f
1023 option as follows:
1024 ::
1025
1026 $ bitbake recipe -c printdate -f
1027
1028
1029 When manually selecting a task to run with the
1030 bitbake
1031 NBSP
1032 recipe
1033 NBSP
1034 -c
1035 NBSP
1036 task
1037 command, you can omit the "do_" prefix as part of the task name.
1038
1039You might wonder about the practical effects of using ``addtask``
1040without specifying any dependencies as is done in the following example:
1041addtask printdate In this example, assuming dependencies have not been
1042added through some other means, the only way to run the task is by
1043explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You
1044can use the ``do_listtasks`` task to list all tasks defined in a recipe
1045as shown in the following example: $ bitbake recipe -c listtasks For
1046more information on task dependencies, see the
1047"`Dependencies <#dependencies>`__" section.
1048
1049See the "`Variable Flags <#variable-flags>`__" section for information
1050on variable flags you can use with tasks.
1051
1052Deleting a Task
1053---------------
1054
1055As well as being able to add tasks, you can delete them. Simply use the
1056``deltask`` command to delete a task. For example, to delete the example
1057task used in the previous sections, you would use: deltask printdate If
1058you delete a task using the ``deltask`` command and the task has
1059dependencies, the dependencies are not reconnected. For example, suppose
1060you have three tasks named ``do_a``, ``do_b``, and ``do_c``.
1061Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is
1062dependent on ``do_a``. Given this scenario, if you use ``deltask`` to
1063delete ``do_b``, the implicit dependency relationship between ``do_c``
1064and ``do_a`` through ``do_b`` no longer exists, and ``do_c``
1065dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free
1066to run before ``do_a``.
1067
1068If you want dependencies such as these to remain intact, use the
1069``[noexec]`` varflag to disable the task instead of using the
1070``deltask`` command to delete it: do_b[noexec] = "1"
1071
1072Passing Information Into the Build Task Environment
1073---------------------------------------------------
1074
1075When running a task, BitBake tightly controls the shell execution
1076environment of the build tasks to make sure unwanted contamination from
1077the build machine cannot influence the build.
1078
1079.. note::
1080
1081 By default, BitBake cleans the environment to include only those
1082 things exported or listed in its whitelist to ensure that the build
1083 environment is reproducible and consistent. You can prevent this
1084 "cleaning" by setting the
1085 BB_PRESERVE_ENV
1086 variable.
1087
1088Consequently, if you do want something to get passed into the build task
1089environment, you must take these two steps:
1090
10911. Tell BitBake to load what you want from the environment into the
1092 datastore. You can do so through the
1093 ```BB_ENV_WHITELIST`` <#var-bb-BB_ENV_WHITELIST>`__ and
1094 ```BB_ENV_EXTRAWHITE`` <#var-bb-BB_ENV_EXTRAWHITE>`__ variables. For
1095 example, assume you want to prevent the build system from accessing
1096 your ``$HOME/.ccache`` directory. The following command "whitelists"
1097 the environment variable ``CCACHE_DIR`` causing BitBake to allow that
1098 variable into the datastore: export
1099 BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1100
11012. Tell BitBake to export what you have loaded into the datastore to the
1102 task environment of every running task. Loading something from the
1103 environment into the datastore (previous step) only makes it
1104 available in the datastore. To export it to the task environment of
1105 every running task, use a command similar to the following in your
1106 local configuration file ``local.conf`` or your distribution
1107 configuration file: export CCACHE_DIR
1108
1109 .. note::
1110
1111 A side effect of the previous steps is that BitBake records the
1112 variable as a dependency of the build process in things like the
1113 setscene checksums. If doing so results in unnecessary rebuilds of
1114 tasks, you can whitelist the variable so that the setscene code
1115 ignores the dependency when it creates checksums.
1116
1117Sometimes, it is useful to be able to obtain information from the
1118original execution environment. BitBake saves a copy of the original
1119environment into a special variable named
1120```BB_ORIGENV`` <#var-bb-BB_ORIGENV>`__.
1121
1122The ``BB_ORIGENV`` variable returns a datastore object that can be
1123queried using the standard datastore operators such as
1124``getVar(, False)``. The datastore object is useful, for example, to
1125find the original ``DISPLAY`` variable. Here is an example: origenv =
1126d.getVar("BB_ORIGENV", False) bar = origenv.getVar("BAR", False) The
1127previous example returns ``BAR`` from the original execution
1128environment.
1129
1130Variable Flags
1131==============
1132
1133Variable flags (varflags) help control a task's functionality and
1134dependencies. BitBake reads and writes varflags to the datastore using
1135the following command forms: variable = d.getVarFlags("variable")
1136self.d.setVarFlags("FOO", {"func": True})
1137
1138When working with varflags, the same syntax, with the exception of
1139overrides, applies. In other words, you can set, append, and prepend
1140varflags just like variables. See the "`Variable Flag
1141Syntax <#variable-flag-syntax>`__" section for details.
1142
1143BitBake has a defined set of varflags available for recipes and classes.
1144Tasks support a number of these flags which control various
1145functionality of the task:
1146
1147- *``[cleandirs]``:* Empty directories that should be created before
1148 the task runs. Directories that already exist are removed and
1149 recreated to empty them.
1150
1151- *``[depends]``:* Controls inter-task dependencies. See the
1152 ```DEPENDS`` <#var-bb-DEPENDS>`__ variable and the "`Inter-Task
1153 Dependencies <#inter-task-dependencies>`__" section for more
1154 information.
1155
1156- *``[deptask]``:* Controls task build-time dependencies. See the
1157 ```DEPENDS`` <#var-bb-DEPENDS>`__ variable and the "`Build
1158 Dependencies <#build-dependencies>`__" section for more information.
1159
1160- *``[dirs]``:* Directories that should be created before the task
1161 runs. Directories that already exist are left as is. The last
1162 directory listed is used as the current working directory for the
1163 task.
1164
1165- *``[lockfiles]``:* Specifies one or more lockfiles to lock while the
1166 task executes. Only one task may hold a lockfile, and any task that
1167 attempts to lock an already locked file will block until the lock is
1168 released. You can use this variable flag to accomplish mutual
1169 exclusion.
1170
1171- *``[noexec]``:* When set to "1", marks the task as being empty, with
1172 no execution required. You can use the ``[noexec]`` flag to set up
1173 tasks as dependency placeholders, or to disable tasks defined
1174 elsewhere that are not needed in a particular recipe.
1175
1176- *``[nostamp]``:* When set to "1", tells BitBake to not generate a
1177 stamp file for a task, which implies the task should always be
1178 executed.
1179
1180 .. note::
1181
1182 Any task that depends (possibly indirectly) on a
1183 [nostamp]
1184 task will always be executed as well. This can cause unnecessary
1185 rebuilding if you are not careful.
1186
1187- *``[number_threads]``:* Limits tasks to a specific number of
1188 simultaneous threads during execution. This varflag is useful when
1189 your build host has a large number of cores but certain tasks need to
1190 be rate-limited due to various kinds of resource constraints (e.g. to
1191 avoid network throttling). ``number_threads`` works similarly to the
1192 ```BB_NUMBER_THREADS`` <#var-bb-BB_NUMBER_THREADS>`__ variable but is
1193 task-specific.
1194
1195 Set the value globally. For example, the following makes sure the
1196 ``do_fetch`` task uses no more than two simultaneous execution
1197 threads: do_fetch[number_threads] = "2"
1198
1199 .. note::
1200
1201 - Setting the varflag in individual recipes rather than globally
1202 can result in unpredictable behavior.
1203
1204 - Setting the varflag to a value greater than the value used in
1205 the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to
1206 have no effect.
1207
1208- *``[postfuncs]``:* List of functions to call after the completion of
1209 the task.
1210
1211- *``[prefuncs]``:* List of functions to call before the task executes.
1212
1213- *``[rdepends]``:* Controls inter-task runtime dependencies. See the
1214 ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1215 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the
1216 "`Inter-Task Dependencies <#inter-task-dependencies>`__" section for
1217 more information.
1218
1219- *``[rdeptask]``:* Controls task runtime dependencies. See the
1220 ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1221 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the "`Runtime
1222 Dependencies <#runtime-dependencies>`__" section for more
1223 information.
1224
1225- *``[recideptask]``:* When set in conjunction with ``recrdeptask``,
1226 specifies a task that should be inspected for additional
1227 dependencies.
1228
1229- *``[recrdeptask]``:* Controls task recursive runtime dependencies.
1230 See the ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1231 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the
1232 "`Recursive Dependencies <#recursive-dependencies>`__" section for
1233 more information.
1234
1235- *``[stamp-extra-info]``:* Extra stamp information to append to the
1236 task's stamp. As an example, OpenEmbedded uses this flag to allow
1237 machine-specific tasks.
1238
1239- *``[umask]``:* The umask to run the task under.
1240
1241Several varflags are useful for controlling how signatures are
1242calculated for variables. For more information on this process, see the
1243"`Checksums (Signatures) <#checksums>`__" section.
1244
1245- *``[vardeps]``:* Specifies a space-separated list of additional
1246 variables to add to a variable's dependencies for the purposes of
1247 calculating its signature. Adding variables to this list is useful,
1248 for example, when a function refers to a variable in a manner that
1249 does not allow BitBake to automatically determine that the variable
1250 is referred to.
1251
1252- *``[vardepsexclude]``:* Specifies a space-separated list of variables
1253 that should be excluded from a variable's dependencies for the
1254 purposes of calculating its signature.
1255
1256- *``[vardepvalue]``:* If set, instructs BitBake to ignore the actual
1257 value of the variable and instead use the specified value when
1258 calculating the variable's signature.
1259
1260- *``[vardepvalueexclude]``:* Specifies a pipe-separated list of
1261 strings to exclude from the variable's value when calculating the
1262 variable's signature.
1263
1264Events
1265======
1266
1267BitBake allows installation of event handlers within recipe and class
1268files. Events are triggered at certain points during operation, such as
1269the beginning of operation against a given recipe (i.e. ``*.bb``), the
1270start of a given task, a task failure, a task success, and so forth. The
1271intent is to make it easy to do things like email notification on build
1272failures.
1273
1274Following is an example event handler that prints the name of the event
1275and the content of the ``FILE`` variable: addhandler
1276myclass_eventhandler python myclass_eventhandler() { from bb.event
1277import getName print("The name of the Event is %s" % getName(e))
1278print("The file we run for is %s" % d.getVar('FILE')) }
1279myclass_eventhandler[eventmask] = "bb.event.BuildStarted
1280bb.event.BuildCompleted" In the previous example, an eventmask has been
1281set so that the handler only sees the "BuildStarted" and
1282"BuildCompleted" events. This event handler gets called every time an
1283event matching the eventmask is triggered. A global variable "e" is
1284defined, which represents the current event. With the ``getName(e)``
1285method, you can get the name of the triggered event. The global
1286datastore is available as "d". In legacy code, you might see "e.data"
1287used to get the datastore. However, realize that "e.data" is deprecated
1288and you should use "d" going forward.
1289
1290The context of the datastore is appropriate to the event in question.
1291For example, "BuildStarted" and "BuildCompleted" events run before any
1292tasks are executed so would be in the global configuration datastore
1293namespace. No recipe-specific metadata exists in that namespace. The
1294"BuildStarted" and "BuildCompleted" events also run in the main
1295cooker/server process rather than any worker context. Thus, any changes
1296made to the datastore would be seen by other cooker/server events within
1297the current build but not seen outside of that build or in any worker
1298context. Task events run in the actual tasks in question consequently
1299have recipe-specific and task-specific contents. These events run in the
1300worker context and are discarded at the end of task execution.
1301
1302During a standard build, the following common events might occur. The
1303following events are the most common kinds of events that most metadata
1304might have an interest in viewing:
1305
1306- ``bb.event.ConfigParsed()``: Fired when the base configuration; which
1307 consists of ``bitbake.conf``, ``base.bbclass`` and any global
1308 ``INHERIT`` statements; has been parsed. You can see multiple such
1309 events when each of the workers parse the base configuration or if
1310 the server changes configuration and reparses. Any given datastore
1311 only has one such event executed against it, however. If
1312 ```BB_INVALIDCONF`` <#>`__ is set in the datastore by the event
1313 handler, the configuration is reparsed and a new event triggered,
1314 allowing the metadata to update configuration.
1315
1316- ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one
1317 second. You can configure the interval time using the
1318 ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the
1319 ``time.time()`` value when the event is triggered. This event is
1320 useful for activities such as system state monitoring.
1321
1322- ``bb.event.ParseStarted()``: Fired when BitBake is about to start
1323 parsing recipes. This event's "total" attribute represents the number
1324 of recipes BitBake plans to parse.
1325
1326- ``bb.event.ParseProgress()``: Fired as parsing progresses. This
1327 event's "current" attribute is the number of recipes parsed as well
1328 as the "total" attribute.
1329
1330- ``bb.event.ParseCompleted()``: Fired when parsing is complete. This
1331 event's "cached", "parsed", "skipped", "virtuals", "masked", and
1332 "errors" attributes provide statistics for the parsing results.
1333
1334- ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake
1335 fires multiple "BuildStarted" events (one per configuration) when
1336 multiple configuration (multiconfig) is enabled.
1337
1338- ``bb.build.TaskStarted()``: Fired when a task starts. This event's
1339 "taskfile" attribute points to the recipe from which the task
1340 originates. The "taskname" attribute, which is the task's name,
1341 includes the ``do_`` prefix, and the "logfile" attribute point to
1342 where the task's output is stored. Finally, the "time" attribute is
1343 the task's execution start time.
1344
1345- ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task
1346 that does not exist.
1347
1348- ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail
1349 and should not be presented to the user verbosely.
1350
1351- ``bb.build.TaskFailed()``: Fired for normal tasks that fail.
1352
1353- ``bb.build.TaskSucceeded()``: Fired when a task successfully
1354 completes.
1355
1356- ``bb.event.BuildCompleted()``: Fired when a build finishes.
1357
1358- ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker
1359 shuts down. This event is usually only seen by the UIs as a sign they
1360 should also shutdown.
1361
1362This next list of example events occur based on specific requests to the
1363server. These events are often used to communicate larger pieces of
1364information from the BitBake server to other parts of BitBake such as
1365user interfaces:
1366
1367- ``bb.event.TreeDataPreparationStarted()``
1368
1369- ``bb.event.TreeDataPreparationProgress()``
1370
1371- ``bb.event.TreeDataPreparationCompleted()``
1372
1373- ``bb.event.DepTreeGenerated()``
1374
1375- ``bb.event.CoreBaseFilesFound()``
1376
1377- ``bb.event.ConfigFilePathFound()``
1378
1379- ``bb.event.FilesMatchingFound()``
1380
1381- ``bb.event.ConfigFilesFound()``
1382
1383- ``bb.event.TargetsTreeGenerated()``
1384
1385.. _variants-class-extension-mechanism:
1386
1387Variants - Class Extension Mechanism
1388====================================
1389
1390BitBake supports two features that facilitate creating from a single
1391recipe file multiple incarnations of that recipe file where all
1392incarnations are buildable. These features are enabled through the
1393```BBCLASSEXTEND`` <#var-bb-BBCLASSEXTEND>`__ and
1394```BBVERSIONS`` <#var-bb-BBVERSIONS>`__ variables.
1395
1396.. note::
1397
1398 The mechanism for this class extension is extremely specific to the
1399 implementation. Usually, the recipe's
1400 PROVIDES
1401 ,
1402 PN
1403 , and
1404 DEPENDS
1405 variables would need to be modified by the extension class. For
1406 specific examples, see the OE-Core
1407 native
1408 ,
1409 nativesdk
1410 , and
1411 multilib
1412 classes.
1413
1414- *``BBCLASSEXTEND``:* This variable is a space separated list of
1415 classes used to "extend" the recipe for each variant. Here is an
1416 example that results in a second incarnation of the current recipe
1417 being available. This second incarnation will have the "native" class
1418 inherited. BBCLASSEXTEND = "native"
1419
1420- *``BBVERSIONS``:* This variable allows a single recipe to build
1421 multiple versions of a project from a single recipe file. You can
1422 also specify conditional metadata (using the
1423 ```OVERRIDES`` <#var-bb-OVERRIDES>`__ mechanism) for a single
1424 version, or an optionally named range of versions. Here is an
1425 example: BBVERSIONS = "1.0 2.0 git" SRC_URI_git =
1426 "git://someurl/somepath.git" BBVERSIONS = "1.0.[0-6]:1.0.0+ \\
1427 1.0.[7-9]:1.0.7+" SRC_URI_append_1.0.7+ =
1428 "file://some_patch_which_the_new_versions_need.patch;patch=1" The
1429 name of the range defaults to the original version of the recipe. For
1430 example, in OpenEmbedded, the recipe file ``foo_1.0.0+.bb`` creates a
1431 default name range of ``1.0.0+``. This is useful because the range
1432 name is not only placed into overrides, but it is also made available
1433 for the metadata to use in the variable that defines the base recipe
1434 versions for use in ``file://`` search paths
1435 (```FILESPATH`` <#var-bb-FILESPATH>`__).
1436
1437Dependencies
1438============
1439
1440To allow for efficient parallel processing, BitBake handles dependencies
1441at the task level. Dependencies can exist both between tasks within a
1442single recipe and between tasks in different recipes. Following are
1443examples of each:
1444
1445- For tasks within a single recipe, a recipe's ``do_configure`` task
1446 might need to complete before its ``do_compile`` task can run.
1447
1448- For tasks in different recipes, one recipe's ``do_configure`` task
1449 might require another recipe's ``do_populate_sysroot`` task to finish
1450 first such that the libraries and headers provided by the other
1451 recipe are available.
1452
1453This section describes several ways to declare dependencies. Remember,
1454even though dependencies are declared in different ways, they are all
1455simply dependencies between tasks.
1456
1457.. _dependencies-internal-to-the-bb-file:
1458
1459Dependencies Internal to the ``.bb`` File
1460-----------------------------------------
1461
1462BitBake uses the ``addtask`` directive to manage dependencies that are
1463internal to a given recipe file. You can use the ``addtask`` directive
1464to indicate when a task is dependent on other tasks or when other tasks
1465depend on that recipe. Here is an example: addtask printdate after
1466do_fetch before do_build In this example, the ``do_printdate`` task
1467depends on the completion of the ``do_fetch`` task, and the ``do_build``
1468task depends on the completion of the ``do_printdate`` task.
1469
1470.. note::
1471
1472 For a task to run, it must be a direct or indirect dependency of some
1473 other task that is scheduled to run.
1474
1475 For illustration, here are some examples:
1476
1477 - The directive ``addtask mytask before do_configure`` causes
1478 ``do_mytask`` to run before ``do_configure`` runs. Be aware that
1479 ``do_mytask`` still only runs if its `input
1480 checksum <#checksums>`__ has changed since the last time it was
1481 run. Changes to the input checksum of ``do_mytask`` also
1482 indirectly cause ``do_configure`` to run.
1483
1484 - The directive ``addtask mytask after do_configure`` by itself
1485 never causes ``do_mytask`` to run. ``do_mytask`` can still be run
1486 manually as follows: $ bitbake recipe -c mytask Declaring
1487 ``do_mytask`` as a dependency of some other task that is scheduled
1488 to run also causes it to run. Regardless, the task runs after
1489 ``do_configure``.
1490
1491Build Dependencies
1492------------------
1493
1494BitBake uses the ```DEPENDS`` <#var-bb-DEPENDS>`__ variable to manage
1495build time dependencies. The ``[deptask]`` varflag for tasks signifies
1496the task of each item listed in ``DEPENDS`` that must complete before
1497that task can be executed. Here is an example: do_configure[deptask] =
1498"do_populate_sysroot" In this example, the ``do_populate_sysroot`` task
1499of each item in ``DEPENDS`` must complete before ``do_configure`` can
1500execute.
1501
1502Runtime Dependencies
1503--------------------
1504
1505BitBake uses the ```PACKAGES`` <#var-bb-PACKAGES>`__,
1506```RDEPENDS`` <#var-bb-RDEPENDS>`__, and
1507```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variables to manage runtime
1508dependencies.
1509
1510The ``PACKAGES`` variable lists runtime packages. Each of those packages
1511can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The
1512``[rdeptask]`` flag for tasks is used to signify the task of each item
1513runtime dependency which must have completed before that task can be
1514executed. do_package_qa[rdeptask] = "do_packagedata" In the previous
1515example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must
1516have completed before ``do_package_qa`` can execute.
1517Although ``RDEPENDS`` contains entries from the
1518runtime dependency namespace, BitBake knows how to map them back
1519to the build-time dependency namespace, in which the tasks are defined.
1520
1521Recursive Dependencies
1522----------------------
1523
1524BitBake uses the ``[recrdeptask]`` flag to manage recursive task
1525dependencies. BitBake looks through the build-time and runtime
1526dependencies of the current recipe, looks through the task's inter-task
1527dependencies, and then adds dependencies for the listed task. Once
1528BitBake has accomplished this, it recursively works through the
1529dependencies of those tasks. Iterative passes continue until all
1530dependencies are discovered and added.
1531
1532The ``[recrdeptask]`` flag is most commonly used in high-level recipes
1533that need to wait for some task to finish "globally". For example,
1534``image.bbclass`` has the following: do_rootfs[recrdeptask] +=
1535"do_packagedata" This statement says that the ``do_packagedata`` task of
1536the current recipe and all recipes reachable (by way of dependencies)
1537from the image recipe must run before the ``do_rootfs`` task can run.
1538
1539BitBake allows a task to recursively depend on itself by
1540referencing itself in the task list:
1541do_a[recrdeptask] = "do_a do_b"
1542
1543In the same way as before, this means that the ``do_a``
1544and ``do_b`` tasks of the current recipe and all
1545recipes reachable (by way of dependencies) from the recipe
1546must run before the ``do_a`` task can run. In this
1547case BitBake will ignore the current recipe's ``do_a``
1548task circular dependency on itself.
1549
1550Inter-Task Dependencies
1551-----------------------
1552
1553BitBake uses the ``[depends]`` flag in a more generic form to manage
1554inter-task dependencies. This more generic form allows for
1555inter-dependency checks for specific tasks rather than checks for the
1556data in ``DEPENDS``. Here is an example: do_patch[depends] =
1557"quilt-native:do_populate_sysroot" In this example, the
1558``do_populate_sysroot`` task of the target ``quilt-native`` must have
1559completed before the ``do_patch`` task can execute.
1560
1561The ``[rdepends]`` flag works in a similar way but takes targets in the
1562runtime namespace instead of the build-time dependency namespace.
1563
1564Functions You Can Call From Within Python
1565=========================================
1566
1567BitBake provides many functions you can call from within Python
1568functions. This section lists the most commonly used functions, and
1569mentions where to find others.
1570
1571Functions for Accessing Datastore Variables
1572-------------------------------------------
1573
1574It is often necessary to access variables in the BitBake datastore using
1575Python functions. The BitBake datastore has an API that allows you this
1576access. Here is a list of available operations:
1577
1578+-----------------------------------+-----------------------------------+
1579| *Operation* | *Description* |
1580+===================================+===================================+
1581| ``d.getVar("X", expand)`` | Returns the value of variable |
1582| | "X". Using "expand=True" expands |
1583| | the value. Returns "None" if the |
1584| | variable "X" does not exist. |
1585+-----------------------------------+-----------------------------------+
1586| ``d.setVar("X", "value")`` | Sets the variable "X" to "value". |
1587+-----------------------------------+-----------------------------------+
1588| ``d.appendVar("X", "value")`` | Adds "value" to the end of the |
1589| | variable "X". Acts like |
1590| | ``d.setVar("X", "value")`` if the |
1591| | variable "X" does not exist. |
1592+-----------------------------------+-----------------------------------+
1593| ``d.prependVar("X", "value")`` | Adds "value" to the start of the |
1594| | variable "X". Acts like |
1595| | ``d.setVar("X", "value")`` if the |
1596| | variable "X" does not exist. |
1597+-----------------------------------+-----------------------------------+
1598| ``d.delVar("X")`` | Deletes the variable "X" from the |
1599| | datastore. Does nothing if the |
1600| | variable "X" does not exist. |
1601+-----------------------------------+-----------------------------------+
1602| ``d.renameVar("X", "Y")`` | Renames the variable "X" to "Y". |
1603| | Does nothing if the variable "X" |
1604| | does not exist. |
1605+-----------------------------------+-----------------------------------+
1606| `` | Returns the value of variable |
1607| d.getVarFlag("X", flag, expand)`` | "X". Using "expand=True" expands |
1608| | the value. Returns "None" if |
1609| | either the variable "X" or the |
1610| | named flag does not exist. |
1611+-----------------------------------+-----------------------------------+
1612| ``d | Sets the named flag for variable |
1613| .setVarFlag("X", flag, "value")`` | "X" to "value". |
1614+-----------------------------------+-----------------------------------+
1615| ``d.ap | Appends "value" to the named flag |
1616| pendVarFlag("X", flag, "value")`` | on the variable "X". Acts like |
1617| | ``d |
1618| | .setVarFlag("X", flag, "value")`` |
1619| | if the named flag does not exist. |
1620+-----------------------------------+-----------------------------------+
1621| ``d.pre | Prepends "value" to the named |
1622| pendVarFlag("X", flag, "value")`` | flag on the variable "X". Acts |
1623| | like |
1624| | ``d |
1625| | .setVarFlag("X", flag, "value")`` |
1626| | if the named flag does not exist. |
1627+-----------------------------------+-----------------------------------+
1628| ``d.delVarFlag("X", flag)`` | Deletes the named flag on the |
1629| | variable "X" from the datastore. |
1630+-----------------------------------+-----------------------------------+
1631| ``d.setVarFlags("X", flagsdict)`` | Sets the flags specified in the |
1632| | ``flagsdict()`` parameter. |
1633| | ``setVarFlags`` does not clear |
1634| | previous flags. Think of this |
1635| | operation as ``addVarFlags``. |
1636+-----------------------------------+-----------------------------------+
1637| ``d.getVarFlags("X")`` | Returns a ``flagsdict`` of the |
1638| | flags for the variable "X". |
1639| | Returns "None" if the variable |
1640| | "X" does not exist. |
1641+-----------------------------------+-----------------------------------+
1642| ``d.delVarFlags("X")`` | Deletes all the flags for the |
1643| | variable "X". Does nothing if the |
1644| | variable "X" does not exist. |
1645+-----------------------------------+-----------------------------------+
1646| ``d.expand(expression)`` | Expands variable references in |
1647| | the specified string expression. |
1648| | References to variables that do |
1649| | not exist are left as is. For |
1650| | example, ``d.expand("foo ${X}")`` |
1651| | expands to the literal string |
1652| | "foo ${X}" if the variable "X" |
1653| | does not exist. |
1654+-----------------------------------+-----------------------------------+
1655
1656Other Functions
1657---------------
1658
1659You can find many other functions that can be called from Python by
1660looking at the source code of the ``bb`` module, which is in
1661``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes
1662the commonly used functions ``bb.utils.contains()`` and
1663``bb.utils.mkdirhier()``, which come with docstrings.
1664
1665Task Checksums and Setscene
1666===========================
1667
1668BitBake uses checksums (or signatures) along with the setscene to
1669determine if a task needs to be run. This section describes the process.
1670To help understand how BitBake does this, the section assumes an
1671OpenEmbedded metadata-based example.
1672
1673These checksums are stored in ```STAMP`` <#var-bb-STAMP>`__. You can
1674examine the checksums using the following BitBake command: $
1675bitbake-dumpsigs This command returns the signature data in a readable
1676format that allows you to examine the inputs used when the OpenEmbedded
1677build system generates signatures. For example, using
1678``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's
1679“sigdata” for a C application (e.g. ``bash``). Running the command also
1680reveals that the “CC” variable is part of the inputs that are hashed.
1681Any changes to this variable would invalidate the stamp and cause the
1682``do_compile`` task to run.
1683
1684The following list describes related variables:
1685
1686- ```BB_HASHCHECK_FUNCTION`` <#var-bb-BB_HASHCHECK_FUNCTION>`__:
1687 Specifies the name of the function to call during the "setscene" part
1688 of the task's execution in order to validate the list of task hashes.
1689
1690- ```BB_SETSCENE_DEPVALID`` <#var-bb-BB_SETSCENE_DEPVALID>`__:
1691 Specifies a function BitBake calls that determines whether BitBake
1692 requires a setscene dependency to be met.
1693
1694- ```BB_SETSCENE_VERIFY_FUNCTION2`` <#var-bb-BB_SETSCENE_VERIFY_FUNCTION2>`__:
1695 Specifies a function to call that verifies the list of planned task
1696 execution before the main task execution happens.
1697
1698- ```BB_STAMP_POLICY`` <#var-bb-BB_STAMP_POLICY>`__: Defines the mode
1699 for comparing timestamps of stamp files.
1700
1701- ```BB_STAMP_WHITELIST`` <#var-bb-BB_STAMP_WHITELIST>`__: Lists stamp
1702 files that are looked at when the stamp policy is "whitelist".
1703
1704- ```BB_TASKHASH`` <#var-bb-BB_TASKHASH>`__: Within an executing task,
1705 this variable holds the hash of the task as returned by the currently
1706 enabled signature generator.
1707
1708- ```STAMP`` <#var-bb-STAMP>`__: The base path to create stamp files.
1709
1710- ```STAMPCLEAN`` <#var-bb-STAMPCLEAN>`__: Again, the base path to
1711 create stamp files but can use wildcards for matching a range of
1712 files for clean operations.
1713
1714Wildcard Support in Variables
1715=============================
1716
1717Support for wildcard use in variables varies depending on the context in
1718which it is used. For example, some variables and file names allow
1719limited use of wildcards through the "``%``" and "``*``" characters.
1720Other variables or names support Python's
1721```glob`` <https://docs.python.org/3/library/glob.html>`__ syntax,
1722```fnmatch`` <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`__
1723syntax, or
1724```Regular Expression (re)`` <https://docs.python.org/3/library/re.html#re>`__
1725syntax.
1726
1727For variables that have wildcard suport, the documentation describes
1728which form of wildcard, its use, and its limitations.