diff options
Diffstat (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst')
-rw-r--r-- | bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 1969 |
1 files changed, 1969 insertions, 0 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst new file mode 100644 index 0000000000..7ea68ade72 --- /dev/null +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | |||
@@ -0,0 +1,1969 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-2.5 | ||
2 | |||
3 | ==================== | ||
4 | Syntax and Operators | ||
5 | ==================== | ||
6 | |||
7 | | | ||
8 | |||
9 | BitBake files have their own syntax. The syntax has similarities to | ||
10 | several other languages but also has some unique features. This section | ||
11 | describes the available syntax and operators as well as provides | ||
12 | examples. | ||
13 | |||
14 | Basic Syntax | ||
15 | ============ | ||
16 | |||
17 | This section provides some basic syntax examples. | ||
18 | |||
19 | Basic Variable Setting | ||
20 | ---------------------- | ||
21 | |||
22 | The following example sets ``VARIABLE`` to "value". This assignment | ||
23 | occurs immediately as the statement is parsed. It is a "hard" | ||
24 | assignment. :: | ||
25 | |||
26 | VARIABLE = "value" | ||
27 | |||
28 | As expected, if you include leading or | ||
29 | trailing spaces as part of an assignment, the spaces are retained: :: | ||
30 | |||
31 | VARIABLE = " value" | ||
32 | VARIABLE = "value " | ||
33 | |||
34 | Setting ``VARIABLE`` to "" sets | ||
35 | it to an empty string, while setting the variable to " " sets it to a | ||
36 | blank space (i.e. these are not the same values). :: | ||
37 | |||
38 | VARIABLE = "" | ||
39 | VARIABLE = " " | ||
40 | |||
41 | You can use single quotes instead of double quotes when setting a | ||
42 | variable's value. Doing so allows you to use values that contain the | ||
43 | double quote character: :: | ||
44 | |||
45 | VARIABLE = 'I have a " in my value' | ||
46 | |||
47 | .. note:: | ||
48 | |||
49 | Unlike in Bourne shells, single quotes work identically to double | ||
50 | quotes in all other ways. They do not suppress variable expansions. | ||
51 | |||
52 | Modifying Existing Variables | ||
53 | ---------------------------- | ||
54 | |||
55 | Sometimes you need to modify existing variables. Following are some | ||
56 | cases where you might find you want to modify an existing variable: | ||
57 | |||
58 | - Customize a recipe that uses the variable. | ||
59 | |||
60 | - Change a variable's default value used in a ``*.bbclass`` file. | ||
61 | |||
62 | - Change the variable in a ``*.bbappend`` file to override the variable | ||
63 | in the original recipe. | ||
64 | |||
65 | - Change the variable in a configuration file so that the value | ||
66 | overrides an existing configuration. | ||
67 | |||
68 | Changing a variable value can sometimes depend on how the value was | ||
69 | originally assigned and also on the desired intent of the change. In | ||
70 | particular, when you append a value to a variable that has a default | ||
71 | value, the resulting value might not be what you expect. In this case, | ||
72 | the value you provide might replace the value rather than append to the | ||
73 | default value. | ||
74 | |||
75 | If after you have changed a variable's value and something unexplained | ||
76 | occurs, you can use BitBake to check the actual value of the suspect | ||
77 | variable. You can make these checks for both configuration and recipe | ||
78 | level changes: | ||
79 | |||
80 | - For configuration changes, use the following: :: | ||
81 | |||
82 | $ bitbake -e | ||
83 | |||
84 | This | ||
85 | command displays variable values after the configuration files (i.e. | ||
86 | ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth) | ||
87 | have been parsed. | ||
88 | |||
89 | .. note:: | ||
90 | |||
91 | Variables that are exported to the environment are preceded by the | ||
92 | string "export" in the command's output. | ||
93 | |||
94 | - For recipe changes, use the following: :: | ||
95 | |||
96 | $ bitbake recipe -e \| grep VARIABLE=" | ||
97 | |||
98 | This command checks to see if the variable actually makes | ||
99 | it into a specific recipe. | ||
100 | |||
101 | Line Joining | ||
102 | ------------ | ||
103 | |||
104 | Outside of :ref:`functions <bitbake-user-manual/bitbake-user-manual-metadata:functions>`, | ||
105 | BitBake joins any line ending in | ||
106 | a backslash character ("\") with the following line before parsing | ||
107 | statements. The most common use for the "\" character is to split | ||
108 | variable assignments over multiple lines, as in the following example: :: | ||
109 | |||
110 | FOO = "bar \ | ||
111 | baz \ | ||
112 | qaz" | ||
113 | |||
114 | Both the "\" character and the newline | ||
115 | character that follow it are removed when joining lines. Thus, no | ||
116 | newline characters end up in the value of ``FOO``. | ||
117 | |||
118 | Consider this additional example where the two assignments both assign | ||
119 | "barbaz" to ``FOO``: :: | ||
120 | |||
121 | FOO = "barbaz" | ||
122 | FOO = "bar\ | ||
123 | baz" | ||
124 | |||
125 | .. note:: | ||
126 | |||
127 | BitBake does not interpret escape sequences like "\n" in variable | ||
128 | values. For these to have an effect, the value must be passed to some | ||
129 | utility that interprets escape sequences, such as | ||
130 | ``printf`` or ``echo -n``. | ||
131 | |||
132 | Variable Expansion | ||
133 | ------------------ | ||
134 | |||
135 | Variables can reference the contents of other variables using a syntax | ||
136 | that is similar to variable expansion in Bourne shells. The following | ||
137 | assignments result in A containing "aval" and B evaluating to | ||
138 | "preavalpost". :: | ||
139 | |||
140 | A = "aval" | ||
141 | B = "pre${A}post" | ||
142 | |||
143 | .. note:: | ||
144 | |||
145 | Unlike in Bourne shells, the curly braces are mandatory: Only ``${FOO}`` and not | ||
146 | ``$FOO`` is recognized as an expansion of ``FOO``. | ||
147 | |||
148 | The "=" operator does not immediately expand variable references in the | ||
149 | right-hand side. Instead, expansion is deferred until the variable | ||
150 | assigned to is actually used. The result depends on the current values | ||
151 | of the referenced variables. The following example should clarify this | ||
152 | behavior: :: | ||
153 | |||
154 | A = "${B} baz" | ||
155 | B = "${C} bar" | ||
156 | C = "foo" | ||
157 | *At this point, ${A} equals "foo bar baz"* | ||
158 | C = "qux" | ||
159 | *At this point, ${A} equals "qux bar baz"* | ||
160 | B = "norf" | ||
161 | *At this point, ${A} equals "norf baz"\* | ||
162 | |||
163 | Contrast this behavior with the | ||
164 | :ref:`bitbake-user-manual/bitbake-user-manual-metadata:immediate variable | ||
165 | expansion (:=)` operator. | ||
166 | |||
167 | If the variable expansion syntax is used on a variable that does not | ||
168 | exist, the string is kept as is. For example, given the following | ||
169 | assignment, ``BAR`` expands to the literal string "${FOO}" as long as | ||
170 | ``FOO`` does not exist. :: | ||
171 | |||
172 | BAR = "${FOO}" | ||
173 | |||
174 | Setting a default value (?=) | ||
175 | ---------------------------- | ||
176 | |||
177 | You can use the "?=" operator to achieve a "softer" assignment for a | ||
178 | variable. This type of assignment allows you to define a variable if it | ||
179 | is undefined when the statement is parsed, but to leave the value alone | ||
180 | if the variable has a value. Here is an example: :: | ||
181 | |||
182 | A ?= "aval" | ||
183 | |||
184 | If ``A`` is | ||
185 | set at the time this statement is parsed, the variable retains its | ||
186 | value. However, if ``A`` is not set, the variable is set to "aval". | ||
187 | |||
188 | .. note:: | ||
189 | |||
190 | This assignment is immediate. Consequently, if multiple "?=" | ||
191 | assignments to a single variable exist, the first of those ends up | ||
192 | getting used. | ||
193 | |||
194 | Setting a weak default value (??=) | ||
195 | ---------------------------------- | ||
196 | |||
197 | It is possible to use a "weaker" assignment than in the previous section | ||
198 | by using the "??=" operator. This assignment behaves identical to "?=" | ||
199 | except that the assignment is made at the end of the parsing process | ||
200 | rather than immediately. Consequently, when multiple "??=" assignments | ||
201 | exist, the last one is used. Also, any "=" or "?=" assignment will | ||
202 | override the value set with "??=". Here is an example: :: | ||
203 | |||
204 | A ??= "somevalue" | ||
205 | A ??= "someothervalue" | ||
206 | |||
207 | If ``A`` is set before the above statements are | ||
208 | parsed, the variable retains its value. If ``A`` is not set, the | ||
209 | variable is set to "someothervalue". | ||
210 | |||
211 | Again, this assignment is a "lazy" or "weak" assignment because it does | ||
212 | not occur until the end of the parsing process. | ||
213 | |||
214 | Immediate variable expansion (:=) | ||
215 | --------------------------------- | ||
216 | |||
217 | The ":=" operator results in a variable's contents being expanded | ||
218 | immediately, rather than when the variable is actually used: :: | ||
219 | |||
220 | T = "123" | ||
221 | A := "test ${T}" | ||
222 | T = "456" | ||
223 | B := "${T} ${C}" | ||
224 | C = "cval" | ||
225 | C := "${C}append" | ||
226 | |||
227 | In this example, ``A`` contains "test 123", even though the final value | ||
228 | of ``T`` is "456". The variable ``B`` will end up containing "456 | ||
229 | cvalappend". This is because references to undefined variables are | ||
230 | preserved as is during (immediate)expansion. This is in contrast to GNU | ||
231 | Make, where undefined variables expand to nothing. The variable ``C`` | ||
232 | contains "cvalappend" since ``${C}`` immediately expands to "cval". | ||
233 | |||
234 | .. _appending-and-prepending: | ||
235 | |||
236 | Appending (+=) and prepending (=+) With Spaces | ||
237 | ---------------------------------------------- | ||
238 | |||
239 | Appending and prepending values is common and can be accomplished using | ||
240 | the "+=" and "=+" operators. These operators insert a space between the | ||
241 | current value and prepended or appended value. | ||
242 | |||
243 | These operators take immediate effect during parsing. Here are some | ||
244 | examples: :: | ||
245 | |||
246 | B = "bval" | ||
247 | B += "additionaldata" | ||
248 | C = "cval" | ||
249 | C =+ "test" | ||
250 | |||
251 | The variable ``B`` contains "bval additionaldata" and ``C`` contains "test | ||
252 | cval". | ||
253 | |||
254 | .. _appending-and-prepending-without-spaces: | ||
255 | |||
256 | Appending (.=) and Prepending (=.) Without Spaces | ||
257 | ------------------------------------------------- | ||
258 | |||
259 | If you want to append or prepend values without an inserted space, use | ||
260 | the ".=" and "=." operators. | ||
261 | |||
262 | These operators take immediate effect during parsing. Here are some | ||
263 | examples: :: | ||
264 | |||
265 | B = "bval" | ||
266 | B .= "additionaldata" | ||
267 | C = "cval" | ||
268 | C =. "test" | ||
269 | |||
270 | The variable ``B`` contains "bvaladditionaldata" and ``C`` contains | ||
271 | "testcval". | ||
272 | |||
273 | Appending and Prepending (Override Style Syntax) | ||
274 | ------------------------------------------------ | ||
275 | |||
276 | You can also append and prepend a variable's value using an override | ||
277 | style syntax. When you use this syntax, no spaces are inserted. | ||
278 | |||
279 | These operators differ from the ":=", ".=", "=.", "+=", and "=+" | ||
280 | operators in that their effects are applied at variable expansion time | ||
281 | rather than being immediately applied. Here are some examples: :: | ||
282 | |||
283 | B = "bval" | ||
284 | B_append = " additional data" | ||
285 | C = "cval" | ||
286 | C_prepend = "additional data " | ||
287 | D = "dval" | ||
288 | D_append = "additional data" | ||
289 | |||
290 | The variable ``B`` | ||
291 | becomes "bval additional data" and ``C`` becomes "additional data cval". | ||
292 | The variable ``D`` becomes "dvaladditional data". | ||
293 | |||
294 | .. note:: | ||
295 | |||
296 | You must control all spacing when you use the override syntax. | ||
297 | |||
298 | It is also possible to append and prepend to shell functions and | ||
299 | BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`" | ||
300 | sections for examples. | ||
301 | |||
302 | .. _removing-override-style-syntax: | ||
303 | |||
304 | Removal (Override Style Syntax) | ||
305 | ------------------------------- | ||
306 | |||
307 | You can remove values from lists using the removal override style | ||
308 | syntax. Specifying a value for removal causes all occurrences of that | ||
309 | value to be removed from the variable. | ||
310 | |||
311 | When you use this syntax, BitBake expects one or more strings. | ||
312 | Surrounding spaces and spacing are preserved. Here is an example: :: | ||
313 | |||
314 | FOO = "123 456 789 123456 123 456 123 456" | ||
315 | FOO_remove = "123" | ||
316 | FOO_remove = "456" | ||
317 | FOO2 = " abc def ghi abcdef abc def abc def def" | ||
318 | FOO2_remove = "\ | ||
319 | def \ | ||
320 | abc \ | ||
321 | ghi \ | ||
322 | " | ||
323 | |||
324 | The variable ``FOO`` becomes | ||
325 | " 789 123456 " and ``FOO2`` becomes " abcdef ". | ||
326 | |||
327 | Like "_append" and "_prepend", "_remove" is applied at variable | ||
328 | expansion time. | ||
329 | |||
330 | Override Style Operation Advantages | ||
331 | ----------------------------------- | ||
332 | |||
333 | An advantage of the override style operations "_append", "_prepend", and | ||
334 | "_remove" as compared to the "+=" and "=+" operators is that the | ||
335 | override style operators provide guaranteed operations. For example, | ||
336 | consider a class ``foo.bbclass`` that needs to add the value "val" to | ||
337 | the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows: :: | ||
338 | |||
339 | inherit foo | ||
340 | FOO = "initial" | ||
341 | |||
342 | If ``foo.bbclass`` uses the "+=" operator, | ||
343 | as follows, then the final value of ``FOO`` will be "initial", which is | ||
344 | not what is desired: :: | ||
345 | |||
346 | FOO += "val" | ||
347 | |||
348 | If, on the other hand, ``foo.bbclass`` | ||
349 | uses the "_append" operator, then the final value of ``FOO`` will be | ||
350 | "initial val", as intended: :: | ||
351 | |||
352 | FOO_append = " val" | ||
353 | |||
354 | .. note:: | ||
355 | |||
356 | It is never necessary to use "+=" together with "_append". The following | ||
357 | sequence of assignments appends "barbaz" to FOO: :: | ||
358 | |||
359 | FOO_append = "bar" | ||
360 | FOO_append = "baz" | ||
361 | |||
362 | |||
363 | The only effect of changing the second assignment in the previous | ||
364 | example to use "+=" would be to add a space before "baz" in the | ||
365 | appended value (due to how the "+=" operator works). | ||
366 | |||
367 | Another advantage of the override style operations is that you can | ||
368 | combine them with other overrides as described in the | ||
369 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section. | ||
370 | |||
371 | Variable Flag Syntax | ||
372 | -------------------- | ||
373 | |||
374 | Variable flags are BitBake's implementation of variable properties or | ||
375 | attributes. It is a way of tagging extra information onto a variable. | ||
376 | You can find more out about variable flags in general in the | ||
377 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section. | ||
378 | |||
379 | You can define, append, and prepend values to variable flags. All the | ||
380 | standard syntax operations previously mentioned work for variable flags | ||
381 | except for override style syntax (i.e. "_prepend", "_append", and | ||
382 | "_remove"). | ||
383 | |||
384 | Here are some examples showing how to set variable flags: :: | ||
385 | |||
386 | FOO[a] = "abc" | ||
387 | FOO[b] = "123" | ||
388 | FOO[a] += "456" | ||
389 | |||
390 | The variable ``FOO`` has two flags: | ||
391 | ``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123", | ||
392 | respectively. The ``[a]`` flag becomes "abc 456". | ||
393 | |||
394 | No need exists to pre-define variable flags. You can simply start using | ||
395 | them. One extremely common application is to attach some brief | ||
396 | documentation to a BitBake variable as follows: :: | ||
397 | |||
398 | CACHE[doc] = "The directory holding the cache of the metadata." | ||
399 | |||
400 | Inline Python Variable Expansion | ||
401 | -------------------------------- | ||
402 | |||
403 | You can use inline Python variable expansion to set variables. Here is | ||
404 | an example: :: | ||
405 | |||
406 | DATE = "${@time.strftime('%Y%m%d',time.gmtime())}" | ||
407 | |||
408 | This example results in the ``DATE`` variable being set to the current date. | ||
409 | |||
410 | Probably the most common use of this feature is to extract the value of | ||
411 | variables from BitBake's internal data dictionary, ``d``. The following | ||
412 | lines select the values of a package name and its version number, | ||
413 | respectively: :: | ||
414 | |||
415 | PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" | ||
416 | PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}" | ||
417 | |||
418 | .. note:: | ||
419 | |||
420 | Inline Python expressions work just like variable expansions insofar as the | ||
421 | "=" and ":=" operators are concerned. Given the following assignment, foo() | ||
422 | is called each time FOO is expanded: :: | ||
423 | |||
424 | FOO = "${@foo()}" | ||
425 | |||
426 | Contrast this with the following immediate assignment, where foo() is only | ||
427 | called once, while the assignment is parsed: :: | ||
428 | |||
429 | FOO := "${@foo()}" | ||
430 | |||
431 | For a different way to set variables with Python code during parsing, | ||
432 | see the | ||
433 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:anonymous python functions`" section. | ||
434 | |||
435 | Unsetting variables | ||
436 | ------------------- | ||
437 | |||
438 | It is possible to completely remove a variable or a variable flag from | ||
439 | BitBake's internal data dictionary by using the "unset" keyword. Here is | ||
440 | an example: :: | ||
441 | |||
442 | unset DATE | ||
443 | unset do_fetch[noexec] | ||
444 | |||
445 | These two statements remove the ``DATE`` and the ``do_fetch[noexec]`` flag. | ||
446 | |||
447 | Providing Pathnames | ||
448 | ------------------- | ||
449 | |||
450 | When specifying pathnames for use with BitBake, do not use the tilde | ||
451 | ("~") character as a shortcut for your home directory. Doing so might | ||
452 | cause BitBake to not recognize the path since BitBake does not expand | ||
453 | this character in the same way a shell would. | ||
454 | |||
455 | Instead, provide a fuller path as the following example illustrates: :: | ||
456 | |||
457 | BBLAYERS ?= " \ | ||
458 | /home/scott-lenovo/LayerA \ | ||
459 | " | ||
460 | |||
461 | Exporting Variables to the Environment | ||
462 | ====================================== | ||
463 | |||
464 | You can export variables to the environment of running tasks by using | ||
465 | the ``export`` keyword. For example, in the following example, the | ||
466 | ``do_foo`` task prints "value from the environment" when run: :: | ||
467 | |||
468 | export ENV_VARIABLE | ||
469 | ENV_VARIABLE = "value from the environment" | ||
470 | |||
471 | do_foo() { | ||
472 | bbplain "$ENV_VARIABLE" | ||
473 | } | ||
474 | |||
475 | .. note:: | ||
476 | |||
477 | BitBake does not expand ``$ENV_VARIABLE`` in this case because it lacks the | ||
478 | obligatory ``{}`` . Rather, ``$ENV_VARIABLE`` is expanded by the shell. | ||
479 | |||
480 | It does not matter whether ``export ENV_VARIABLE`` appears before or | ||
481 | after assignments to ``ENV_VARIABLE``. | ||
482 | |||
483 | It is also possible to combine ``export`` with setting a value for the | ||
484 | variable. Here is an example: :: | ||
485 | |||
486 | export ENV_VARIABLE = "variable-value" | ||
487 | |||
488 | In the output of ``bitbake -e``, variables that are exported to the | ||
489 | environment are preceded by "export". | ||
490 | |||
491 | Among the variables commonly exported to the environment are ``CC`` and | ||
492 | ``CFLAGS``, which are picked up by many build systems. | ||
493 | |||
494 | Conditional Syntax (Overrides) | ||
495 | ============================== | ||
496 | |||
497 | BitBake uses :term:`OVERRIDES` to control what | ||
498 | variables are overridden after BitBake parses recipes and configuration | ||
499 | files. This section describes how you can use ``OVERRIDES`` as | ||
500 | conditional metadata, talks about key expansion in relationship to | ||
501 | ``OVERRIDES``, and provides some examples to help with understanding. | ||
502 | |||
503 | Conditional Metadata | ||
504 | -------------------- | ||
505 | |||
506 | You can use ``OVERRIDES`` to conditionally select a specific version of | ||
507 | a variable and to conditionally append or prepend the value of a | ||
508 | variable. | ||
509 | |||
510 | .. note:: | ||
511 | |||
512 | Overrides can only use lower-case characters. Additionally, | ||
513 | underscores are not permitted in override names as they are used to | ||
514 | separate overrides from each other and from the variable name. | ||
515 | |||
516 | - *Selecting a Variable:* The ``OVERRIDES`` variable is a | ||
517 | colon-character-separated list that contains items for which you want | ||
518 | to satisfy conditions. Thus, if you have a variable that is | ||
519 | conditional on "arm", and "arm" is in ``OVERRIDES``, then the | ||
520 | "arm"-specific version of the variable is used rather than the | ||
521 | non-conditional version. Here is an example: :: | ||
522 | |||
523 | OVERRIDES = "architecture:os:machine" | ||
524 | TEST = "default" | ||
525 | TEST_os = "osspecific" | ||
526 | TEST_nooverride = "othercondvalue" | ||
527 | |||
528 | In this example, the ``OVERRIDES`` | ||
529 | variable lists three overrides: "architecture", "os", and "machine". | ||
530 | The variable ``TEST`` by itself has a default value of "default". You | ||
531 | select the os-specific version of the ``TEST`` variable by appending | ||
532 | the "os" override to the variable (i.e. ``TEST_os``). | ||
533 | |||
534 | To better understand this, consider a practical example that assumes | ||
535 | an OpenEmbedded metadata-based Linux kernel recipe file. The | ||
536 | following lines from the recipe file first set the kernel branch | ||
537 | variable ``KBRANCH`` to a default value, then conditionally override | ||
538 | that value based on the architecture of the build: :: | ||
539 | |||
540 | KBRANCH = "standard/base" | ||
541 | KBRANCH_qemuarm = "standard/arm-versatile-926ejs" | ||
542 | KBRANCH_qemumips = "standard/mti-malta32" | ||
543 | KBRANCH_qemuppc = "standard/qemuppc" | ||
544 | KBRANCH_qemux86 = "standard/common-pc/base" | ||
545 | KBRANCH_qemux86-64 = "standard/common-pc-64/base" | ||
546 | KBRANCH_qemumips64 = "standard/mti-malta64" | ||
547 | |||
548 | - *Appending and Prepending:* BitBake also supports append and prepend | ||
549 | operations to variable values based on whether a specific item is | ||
550 | listed in ``OVERRIDES``. Here is an example: :: | ||
551 | |||
552 | DEPENDS = "glibc ncurses" | ||
553 | OVERRIDES = "machine:local" | ||
554 | DEPENDS_append_machine = "libmad" | ||
555 | |||
556 | In this example, ``DEPENDS`` becomes "glibc ncurses libmad". | ||
557 | |||
558 | Again, using an OpenEmbedded metadata-based kernel recipe file as an | ||
559 | example, the following lines will conditionally append to the | ||
560 | ``KERNEL_FEATURES`` variable based on the architecture: :: | ||
561 | |||
562 | KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}" | ||
563 | KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc" | ||
564 | KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc" | ||
565 | |||
566 | - *Setting a Variable for a Single Task:* BitBake supports setting a | ||
567 | variable just for the duration of a single task. Here is an example: :: | ||
568 | |||
569 | FOO_task-configure = "val 1" | ||
570 | FOO_task-compile = "val 2" | ||
571 | |||
572 | In the | ||
573 | previous example, ``FOO`` has the value "val 1" while the | ||
574 | ``do_configure`` task is executed, and the value "val 2" while the | ||
575 | ``do_compile`` task is executed. | ||
576 | |||
577 | Internally, this is implemented by prepending the task (e.g. | ||
578 | "task-compile:") to the value of | ||
579 | :term:`OVERRIDES` for the local datastore of the | ||
580 | ``do_compile`` task. | ||
581 | |||
582 | You can also use this syntax with other combinations (e.g. | ||
583 | "``_prepend``") as shown in the following example: :: | ||
584 | |||
585 | EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} " | ||
586 | |||
587 | Key Expansion | ||
588 | ------------- | ||
589 | |||
590 | Key expansion happens when the BitBake datastore is finalized. To better | ||
591 | understand this, consider the following example: :: | ||
592 | |||
593 | A${B} = "X" | ||
594 | B = "2" | ||
595 | A2 = "Y" | ||
596 | |||
597 | In this case, after all the parsing is complete, BitBake expands | ||
598 | ``${B}`` into "2". This expansion causes ``A2``, which was set to "Y" | ||
599 | before the expansion, to become "X". | ||
600 | |||
601 | .. _variable-interaction-worked-examples: | ||
602 | |||
603 | Examples | ||
604 | -------- | ||
605 | |||
606 | Despite the previous explanations that show the different forms of | ||
607 | variable definitions, it can be hard to work out exactly what happens | ||
608 | when variable operators, conditional overrides, and unconditional | ||
609 | overrides are combined. This section presents some common scenarios | ||
610 | along with explanations for variable interactions that typically confuse | ||
611 | users. | ||
612 | |||
613 | There is often confusion concerning the order in which overrides and | ||
614 | various "append" operators take effect. Recall that an append or prepend | ||
615 | operation using "_append" and "_prepend" does not result in an immediate | ||
616 | assignment as would "+=", ".=", "=+", or "=.". Consider the following | ||
617 | example: :: | ||
618 | |||
619 | OVERRIDES = "foo" | ||
620 | A = "Z" | ||
621 | A_foo_append = "X" | ||
622 | |||
623 | For this case, | ||
624 | ``A`` is unconditionally set to "Z" and "X" is unconditionally and | ||
625 | immediately appended to the variable ``A_foo``. Because overrides have | ||
626 | not been applied yet, ``A_foo`` is set to "X" due to the append and | ||
627 | ``A`` simply equals "Z". | ||
628 | |||
629 | Applying overrides, however, changes things. Since "foo" is listed in | ||
630 | ``OVERRIDES``, the conditional variable ``A`` is replaced with the "foo" | ||
631 | version, which is equal to "X". So effectively, ``A_foo`` replaces | ||
632 | ``A``. | ||
633 | |||
634 | This next example changes the order of the override and the append: :: | ||
635 | |||
636 | OVERRIDES = "foo" | ||
637 | A = "Z" | ||
638 | A_append_foo = "X" | ||
639 | |||
640 | For this case, before | ||
641 | overrides are handled, ``A`` is set to "Z" and ``A_append_foo`` is set | ||
642 | to "X". Once the override for "foo" is applied, however, ``A`` gets | ||
643 | appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces | ||
644 | are not appended. | ||
645 | |||
646 | This next example has the order of the appends and overrides reversed | ||
647 | back as in the first example: :: | ||
648 | |||
649 | OVERRIDES = "foo" | ||
650 | A = "Y" | ||
651 | A_foo_append = "Z" | ||
652 | A_foo_append = "X" | ||
653 | |||
654 | For this case, before any overrides are resolved, | ||
655 | ``A`` is set to "Y" using an immediate assignment. After this immediate | ||
656 | assignment, ``A_foo`` is set to "Z", and then further appended with "X" | ||
657 | leaving the variable set to "ZX". Finally, applying the override for | ||
658 | "foo" results in the conditional variable ``A`` becoming "ZX" (i.e. | ||
659 | ``A`` is replaced with ``A_foo``). | ||
660 | |||
661 | This final example mixes in some varying operators: :: | ||
662 | |||
663 | A = "1" | ||
664 | A_append = "2" | ||
665 | A_append = "3" | ||
666 | A += "4" | ||
667 | A .= "5" | ||
668 | |||
669 | For this case, the type of append | ||
670 | operators are affecting the order of assignments as BitBake passes | ||
671 | through the code multiple times. Initially, ``A`` is set to "1 45" | ||
672 | because of the three statements that use immediate operators. After | ||
673 | these assignments are made, BitBake applies the "_append" operations. | ||
674 | Those operations result in ``A`` becoming "1 4523". | ||
675 | |||
676 | Sharing Functionality | ||
677 | ===================== | ||
678 | |||
679 | BitBake allows for metadata sharing through include files (``.inc``) and | ||
680 | class files (``.bbclass``). For example, suppose you have a piece of | ||
681 | common functionality such as a task definition that you want to share | ||
682 | between more than one recipe. In this case, creating a ``.bbclass`` file | ||
683 | that contains the common functionality and then using the ``inherit`` | ||
684 | directive in your recipes to inherit the class would be a common way to | ||
685 | share the task. | ||
686 | |||
687 | This section presents the mechanisms BitBake provides to allow you to | ||
688 | share functionality between recipes. Specifically, the mechanisms | ||
689 | include ``include``, ``inherit``, ``INHERIT``, and ``require`` | ||
690 | directives. | ||
691 | |||
692 | Locating Include and Class Files | ||
693 | -------------------------------- | ||
694 | |||
695 | BitBake uses the :term:`BBPATH` variable to locate | ||
696 | needed include and class files. Additionally, BitBake searches the | ||
697 | current directory for ``include`` and ``require`` directives. | ||
698 | |||
699 | .. note:: | ||
700 | |||
701 | The BBPATH variable is analogous to the environment variable PATH . | ||
702 | |||
703 | In order for include and class files to be found by BitBake, they need | ||
704 | to be located in a "classes" subdirectory that can be found in | ||
705 | ``BBPATH``. | ||
706 | |||
707 | ``inherit`` Directive | ||
708 | --------------------- | ||
709 | |||
710 | When writing a recipe or class file, you can use the ``inherit`` | ||
711 | directive to inherit the functionality of a class (``.bbclass``). | ||
712 | BitBake only supports this directive when used within recipe and class | ||
713 | files (i.e. ``.bb`` and ``.bbclass``). | ||
714 | |||
715 | The ``inherit`` directive is a rudimentary means of specifying | ||
716 | functionality contained in class files that your recipes require. For | ||
717 | example, you can easily abstract out the tasks involved in building a | ||
718 | package that uses Autoconf and Automake and put those tasks into a class | ||
719 | file and then have your recipe inherit that class file. | ||
720 | |||
721 | As an example, your recipes could use the following directive to inherit | ||
722 | an ``autotools.bbclass`` file. The class file would contain common | ||
723 | functionality for using Autotools that could be shared across recipes: :: | ||
724 | |||
725 | inherit autotools | ||
726 | |||
727 | In this case, BitBake would search for the directory | ||
728 | ``classes/autotools.bbclass`` in ``BBPATH``. | ||
729 | |||
730 | .. note:: | ||
731 | |||
732 | You can override any values and functions of the inherited class | ||
733 | within your recipe by doing so after the "inherit" statement. | ||
734 | |||
735 | If you want to use the directive to inherit multiple classes, separate | ||
736 | them with spaces. The following example shows how to inherit both the | ||
737 | ``buildhistory`` and ``rm_work`` classes: :: | ||
738 | |||
739 | inherit buildhistory rm_work | ||
740 | |||
741 | An advantage with the inherit directive as compared to both the | ||
742 | :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>` and :ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>` | ||
743 | directives is that you can inherit class files conditionally. You can | ||
744 | accomplish this by using a variable expression after the ``inherit`` | ||
745 | statement. Here is an example: :: | ||
746 | |||
747 | inherit ${VARNAME} | ||
748 | |||
749 | If ``VARNAME`` is | ||
750 | going to be set, it needs to be set before the ``inherit`` statement is | ||
751 | parsed. One way to achieve a conditional inherit in this case is to use | ||
752 | overrides: :: | ||
753 | |||
754 | VARIABLE = "" | ||
755 | VARIABLE_someoverride = "myclass" | ||
756 | |||
757 | Another method is by using anonymous Python. Here is an example: :: | ||
758 | |||
759 | python () { | ||
760 | if condition == value: | ||
761 | d.setVar('VARIABLE', 'myclass') | ||
762 | else: | ||
763 | d.setVar('VARIABLE', '') | ||
764 | } | ||
765 | |||
766 | Alternatively, you could use an in-line Python expression in the | ||
767 | following form: :: | ||
768 | |||
769 | inherit ${@'classname' if condition else ''} | ||
770 | inherit ${@functionname(params)} | ||
771 | |||
772 | In all cases, if the expression evaluates to an | ||
773 | empty string, the statement does not trigger a syntax error because it | ||
774 | becomes a no-op. | ||
775 | |||
776 | ``include`` Directive | ||
777 | --------------------- | ||
778 | |||
779 | BitBake understands the ``include`` directive. This directive causes | ||
780 | BitBake to parse whatever file you specify, and to insert that file at | ||
781 | that location. The directive is much like its equivalent in Make except | ||
782 | that if the path specified on the include line is a relative path, | ||
783 | BitBake locates the first file it can find within ``BBPATH``. | ||
784 | |||
785 | The include directive is a more generic method of including | ||
786 | functionality as compared to the :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` | ||
787 | directive, which is restricted to class (i.e. ``.bbclass``) files. The | ||
788 | include directive is applicable for any other kind of shared or | ||
789 | encapsulated functionality or configuration that does not suit a | ||
790 | ``.bbclass`` file. | ||
791 | |||
792 | As an example, suppose you needed a recipe to include some self-test | ||
793 | definitions: :: | ||
794 | |||
795 | include test_defs.inc | ||
796 | |||
797 | .. note:: | ||
798 | |||
799 | The include directive does not produce an error when the file cannot be | ||
800 | found. Consequently, it is recommended that if the file you are including is | ||
801 | expected to exist, you should use :ref:`require <require-inclusion>` instead | ||
802 | of include . Doing so makes sure that an error is produced if the file cannot | ||
803 | be found. | ||
804 | |||
805 | .. _require-inclusion: | ||
806 | |||
807 | ``require`` Directive | ||
808 | --------------------- | ||
809 | |||
810 | BitBake understands the ``require`` directive. This directive behaves | ||
811 | just like the ``include`` directive with the exception that BitBake | ||
812 | raises a parsing error if the file to be included cannot be found. Thus, | ||
813 | any file you require is inserted into the file that is being parsed at | ||
814 | the location of the directive. | ||
815 | |||
816 | The require directive, like the include directive previously described, | ||
817 | is a more generic method of including functionality as compared to the | ||
818 | :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` directive, which is restricted to class | ||
819 | (i.e. ``.bbclass``) files. The require directive is applicable for any | ||
820 | other kind of shared or encapsulated functionality or configuration that | ||
821 | does not suit a ``.bbclass`` file. | ||
822 | |||
823 | Similar to how BitBake handles :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>`, if | ||
824 | the path specified on the require line is a relative path, BitBake | ||
825 | locates the first file it can find within ``BBPATH``. | ||
826 | |||
827 | As an example, suppose you have two versions of a recipe (e.g. | ||
828 | ``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some | ||
829 | identical functionality that could be shared. You could create an | ||
830 | include file named ``foo.inc`` that contains the common definitions | ||
831 | needed to build "foo". You need to be sure ``foo.inc`` is located in the | ||
832 | same directory as your two recipe files as well. Once these conditions | ||
833 | are set up, you can share the functionality using a ``require`` | ||
834 | directive from within each recipe: :: | ||
835 | |||
836 | require foo.inc | ||
837 | |||
838 | ``INHERIT`` Configuration Directive | ||
839 | ----------------------------------- | ||
840 | |||
841 | When creating a configuration file (``.conf``), you can use the | ||
842 | :term:`INHERIT` configuration directive to inherit a | ||
843 | class. BitBake only supports this directive when used within a | ||
844 | configuration file. | ||
845 | |||
846 | As an example, suppose you needed to inherit a class file called | ||
847 | ``abc.bbclass`` from a configuration file as follows: :: | ||
848 | |||
849 | INHERIT += "abc" | ||
850 | |||
851 | This configuration directive causes the named class to be inherited at | ||
852 | the point of the directive during parsing. As with the ``inherit`` | ||
853 | directive, the ``.bbclass`` file must be located in a "classes" | ||
854 | subdirectory in one of the directories specified in ``BBPATH``. | ||
855 | |||
856 | .. note:: | ||
857 | |||
858 | Because .conf files are parsed first during BitBake's execution, using | ||
859 | INHERIT to inherit a class effectively inherits the class globally (i.e. for | ||
860 | all recipes). | ||
861 | |||
862 | If you want to use the directive to inherit multiple classes, you can | ||
863 | provide them on the same line in the ``local.conf`` file. Use spaces to | ||
864 | separate the classes. The following example shows how to inherit both | ||
865 | the ``autotools`` and ``pkgconfig`` classes: :: | ||
866 | |||
867 | INHERIT += "autotools pkgconfig" | ||
868 | |||
869 | Functions | ||
870 | ========= | ||
871 | |||
872 | As with most languages, functions are the building blocks that are used | ||
873 | to build up operations into tasks. BitBake supports these types of | ||
874 | functions: | ||
875 | |||
876 | - *Shell Functions:* Functions written in shell script and executed | ||
877 | either directly as functions, tasks, or both. They can also be called | ||
878 | by other shell functions. | ||
879 | |||
880 | - *BitBake-Style Python Functions:* Functions written in Python and | ||
881 | executed by BitBake or other Python functions using | ||
882 | ``bb.build.exec_func()``. | ||
883 | |||
884 | - *Python Functions:* Functions written in Python and executed by | ||
885 | Python. | ||
886 | |||
887 | - *Anonymous Python Functions:* Python functions executed automatically | ||
888 | during parsing. | ||
889 | |||
890 | Regardless of the type of function, you can only define them in class | ||
891 | (``.bbclass``) and recipe (``.bb`` or ``.inc``) files. | ||
892 | |||
893 | Shell Functions | ||
894 | --------------- | ||
895 | |||
896 | Functions written in shell script and executed either directly as | ||
897 | functions, tasks, or both. They can also be called by other shell | ||
898 | functions. Here is an example shell function definition: :: | ||
899 | |||
900 | some_function () { | ||
901 | echo "Hello World" | ||
902 | } | ||
903 | |||
904 | When you create these types of functions in | ||
905 | your recipe or class files, you need to follow the shell programming | ||
906 | rules. The scripts are executed by ``/bin/sh``, which may not be a bash | ||
907 | shell but might be something such as ``dash``. You should not use | ||
908 | Bash-specific script (bashisms). | ||
909 | |||
910 | Overrides and override-style operators like ``_append`` and ``_prepend`` | ||
911 | can also be applied to shell functions. Most commonly, this application | ||
912 | would be used in a ``.bbappend`` file to modify functions in the main | ||
913 | recipe. It can also be used to modify functions inherited from classes. | ||
914 | |||
915 | As an example, consider the following: :: | ||
916 | |||
917 | do_foo() { | ||
918 | bbplain first | ||
919 | fn | ||
920 | } | ||
921 | |||
922 | fn_prepend() { | ||
923 | bbplain second | ||
924 | } | ||
925 | |||
926 | fn() { | ||
927 | bbplain third | ||
928 | } | ||
929 | |||
930 | do_foo_append() { | ||
931 | bbplain fourth | ||
932 | } | ||
933 | |||
934 | Running ``do_foo`` prints the following: :: | ||
935 | |||
936 | recipename do_foo: first | ||
937 | recipename do_foo: second | ||
938 | recipename do_foo: third | ||
939 | recipename do_foo: fourth | ||
940 | |||
941 | .. note:: | ||
942 | |||
943 | Overrides and override-style operators can be applied to any shell | ||
944 | function, not just :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`. | ||
945 | |||
946 | You can use the ``bitbake -e``Â recipename command to view the final | ||
947 | assembled function after all overrides have been applied. | ||
948 | |||
949 | BitBake-Style Python Functions | ||
950 | ------------------------------ | ||
951 | |||
952 | These functions are written in Python and executed by BitBake or other | ||
953 | Python functions using ``bb.build.exec_func()``. | ||
954 | |||
955 | An example BitBake function is: :: | ||
956 | |||
957 | python some_python_function () { | ||
958 | d.setVar("TEXT", "Hello World") | ||
959 | print d.getVar("TEXT") | ||
960 | } | ||
961 | |||
962 | Because the | ||
963 | Python "bb" and "os" modules are already imported, you do not need to | ||
964 | import these modules. Also in these types of functions, the datastore | ||
965 | ("d") is a global variable and is always automatically available. | ||
966 | |||
967 | .. note:: | ||
968 | |||
969 | Variable expressions (e.g. ``${X}`` ) are no longer expanded within Python | ||
970 | functions. This behavior is intentional in order to allow you to freely set | ||
971 | variable values to expandable expressions without having them expanded | ||
972 | prematurely. If you do wish to expand a variable within a Python function, | ||
973 | use ``d.getVar("X")`` . Or, for more complicated expressions, use ``d.expand()``. | ||
974 | |||
975 | Similar to shell functions, you can also apply overrides and | ||
976 | override-style operators to BitBake-style Python functions. | ||
977 | |||
978 | As an example, consider the following: :: | ||
979 | |||
980 | python do_foo_prepend() { | ||
981 | bb.plain("first") | ||
982 | } | ||
983 | |||
984 | python do_foo() { | ||
985 | bb.plain("second") | ||
986 | } | ||
987 | |||
988 | python do_foo_append() { | ||
989 | bb.plain("third") | ||
990 | } | ||
991 | |||
992 | Running ``do_foo`` prints the following: :: | ||
993 | |||
994 | recipename do_foo: first | ||
995 | recipename do_foo: second | ||
996 | recipename do_foo: third | ||
997 | |||
998 | You can use the ``bitbake -e``Â recipename command to view | ||
999 | the final assembled function after all overrides have been applied. | ||
1000 | |||
1001 | Python Functions | ||
1002 | ---------------- | ||
1003 | |||
1004 | These functions are written in Python and are executed by other Python | ||
1005 | code. Examples of Python functions are utility functions that you intend | ||
1006 | to call from in-line Python or from within other Python functions. Here | ||
1007 | is an example: :: | ||
1008 | |||
1009 | def get_depends(d): | ||
1010 | if d.getVar('SOMECONDITION'): | ||
1011 | return "dependencywithcond" | ||
1012 | else: | ||
1013 | return "dependency" | ||
1014 | |||
1015 | SOMECONDITION = "1" | ||
1016 | DEPENDS = "${@get_depends(d)}" | ||
1017 | |||
1018 | This would result in ``DEPENDS`` containing ``dependencywithcond``. | ||
1019 | |||
1020 | Here are some things to know about Python functions: | ||
1021 | |||
1022 | - Python functions can take parameters. | ||
1023 | |||
1024 | - The BitBake datastore is not automatically available. Consequently, | ||
1025 | you must pass it in as a parameter to the function. | ||
1026 | |||
1027 | - The "bb" and "os" Python modules are automatically available. You do | ||
1028 | not need to import them. | ||
1029 | |||
1030 | BitBake-Style Python Functions Versus Python Functions | ||
1031 | ------------------------------------------------------ | ||
1032 | |||
1033 | Following are some important differences between BitBake-style Python | ||
1034 | functions and regular Python functions defined with "def": | ||
1035 | |||
1036 | - Only BitBake-style Python functions can be :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`. | ||
1037 | |||
1038 | - Overrides and override-style operators can only be applied to | ||
1039 | BitBake-style Python functions. | ||
1040 | |||
1041 | - Only regular Python functions can take arguments and return values. | ||
1042 | |||
1043 | - :ref:`Variable flags <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>` such as | ||
1044 | ``[dirs]``, ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style | ||
1045 | Python functions, but not on regular Python functions. | ||
1046 | |||
1047 | - BitBake-style Python functions generate a separate | ||
1048 | ``${``\ :term:`T`\ ``}/run.``\ function-name\ ``.``\ pid | ||
1049 | script that is executed to run the function, and also generate a log | ||
1050 | file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed | ||
1051 | as tasks. | ||
1052 | |||
1053 | Regular Python functions execute "inline" and do not generate any | ||
1054 | files in ``${T}``. | ||
1055 | |||
1056 | - Regular Python functions are called with the usual Python syntax. | ||
1057 | BitBake-style Python functions are usually tasks and are called | ||
1058 | directly by BitBake, but can also be called manually from Python code | ||
1059 | by using the ``bb.build.exec_func()`` function. Here is an example: :: | ||
1060 | |||
1061 | bb.build.exec_func("my_bitbake_style_function", d) | ||
1062 | |||
1063 | .. note:: | ||
1064 | |||
1065 | ``bb.build.exec_func()`` can also be used to run shell functions from Python | ||
1066 | code. If you want to run a shell function before a Python function within | ||
1067 | the same task, then you can use a parent helper Python function that | ||
1068 | starts by running the shell function with ``bb.build.exec_func()`` and then | ||
1069 | runs the Python code. | ||
1070 | |||
1071 | To detect errors from functions executed with | ||
1072 | ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed`` | ||
1073 | exception. | ||
1074 | |||
1075 | .. note:: | ||
1076 | |||
1077 | Functions in metadata (recipes and classes) should not themselves raise | ||
1078 | ``bb.build.FuncFailed``. Rather, ``bb.build.FuncFailed`` should be viewed as a | ||
1079 | general indicator that the called function failed by raising an | ||
1080 | exception. For example, an exception raised by ``bb.fatal()`` will be caught | ||
1081 | inside ``bb.build.exec_func()``, and a ``bb.build.FuncFailed`` will be raised in | ||
1082 | response. | ||
1083 | |||
1084 | Due to their simplicity, you should prefer regular Python functions over | ||
1085 | BitBake-style Python functions unless you need a feature specific to | ||
1086 | BitBake-style Python functions. Regular Python functions in metadata are | ||
1087 | a more recent invention than BitBake-style Python functions, and older | ||
1088 | code tends to use ``bb.build.exec_func()`` more often. | ||
1089 | |||
1090 | Anonymous Python Functions | ||
1091 | -------------------------- | ||
1092 | |||
1093 | Sometimes it is useful to set variables or perform other operations | ||
1094 | programmatically during parsing. To do this, you can define special | ||
1095 | Python functions, called anonymous Python functions, that run at the end | ||
1096 | of parsing. For example, the following conditionally sets a variable | ||
1097 | based on the value of another variable: :: | ||
1098 | |||
1099 | python () { | ||
1100 | if d.getVar('SOMEVAR') == 'value': | ||
1101 | d.setVar('ANOTHERVAR', 'value2') | ||
1102 | } | ||
1103 | |||
1104 | An equivalent way to mark a function as an anonymous function is to give it | ||
1105 | the name "__anonymous", rather than no name. | ||
1106 | |||
1107 | Anonymous Python functions always run at the end of parsing, regardless | ||
1108 | of where they are defined. If a recipe contains many anonymous | ||
1109 | functions, they run in the same order as they are defined within the | ||
1110 | recipe. As an example, consider the following snippet: :: | ||
1111 | |||
1112 | python () { | ||
1113 | d.setVar('FOO', 'foo 2') | ||
1114 | } | ||
1115 | |||
1116 | FOO = "foo 1" | ||
1117 | |||
1118 | python () { | ||
1119 | d.appendVar('BAR',' bar 2') | ||
1120 | } | ||
1121 | |||
1122 | BAR = "bar 1" | ||
1123 | |||
1124 | The previous example is conceptually | ||
1125 | equivalent to the following snippet: :: | ||
1126 | |||
1127 | FOO = "foo 1" | ||
1128 | BAR = "bar 1" | ||
1129 | FOO = "foo 2" | ||
1130 | BAR += "bar 2" | ||
1131 | |||
1132 | ``FOO`` ends up with the value "foo 2", and | ||
1133 | ``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the | ||
1134 | values set for the variables within the anonymous functions become | ||
1135 | available to tasks, which always run after parsing. | ||
1136 | |||
1137 | Overrides and override-style operators such as "``_append``" are applied | ||
1138 | before anonymous functions run. In the following example, ``FOO`` ends | ||
1139 | up with the value "foo from anonymous": :: | ||
1140 | |||
1141 | FOO = "foo" | ||
1142 | FOO_append = " from outside" | ||
1143 | |||
1144 | python () { | ||
1145 | d.setVar("FOO", "foo from anonymous") | ||
1146 | } | ||
1147 | |||
1148 | For methods | ||
1149 | you can use with anonymous Python functions, see the | ||
1150 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`" | ||
1151 | section. For a different method to run Python code during parsing, see | ||
1152 | the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inline python variable expansion`" section. | ||
1153 | |||
1154 | Flexible Inheritance for Class Functions | ||
1155 | ---------------------------------------- | ||
1156 | |||
1157 | Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake | ||
1158 | supports exporting a function from a class such that the class function | ||
1159 | appears as the default implementation of the function, but can still be | ||
1160 | called if a recipe inheriting the class needs to define its own version | ||
1161 | of the function. | ||
1162 | |||
1163 | To understand the benefits of this feature, consider the basic scenario | ||
1164 | where a class defines a task function and your recipe inherits the | ||
1165 | class. In this basic scenario, your recipe inherits the task function as | ||
1166 | defined in the class. If desired, your recipe can add to the start and | ||
1167 | end of the function by using the "_prepend" or "_append" operations | ||
1168 | respectively, or it can redefine the function completely. However, if it | ||
1169 | redefines the function, there is no means for it to call the class | ||
1170 | version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that | ||
1171 | enables the recipe's version of the function to call the original | ||
1172 | version of the function. | ||
1173 | |||
1174 | To make use of this technique, you need the following things in place: | ||
1175 | |||
1176 | - The class needs to define the function as follows: :: | ||
1177 | |||
1178 | classname_functionname | ||
1179 | |||
1180 | For example, if you have a class file | ||
1181 | ``bar.bbclass`` and a function named ``do_foo``, the class must | ||
1182 | define the function as follows: :: | ||
1183 | |||
1184 | bar_do_foo | ||
1185 | |||
1186 | - The class needs to contain the ``EXPORT_FUNCTIONS`` statement as | ||
1187 | follows: :: | ||
1188 | |||
1189 | EXPORT_FUNCTIONS functionname | ||
1190 | |||
1191 | For example, continuing with | ||
1192 | the same example, the statement in the ``bar.bbclass`` would be as | ||
1193 | follows: :: | ||
1194 | |||
1195 | EXPORT_FUNCTIONS do_foo | ||
1196 | |||
1197 | - You need to call the function appropriately from within your recipe. | ||
1198 | Continuing with the same example, if your recipe needs to call the | ||
1199 | class version of the function, it should call ``bar_do_foo``. | ||
1200 | Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was | ||
1201 | used as above, the recipe's function could conditionally call the | ||
1202 | class version of the function as follows: :: | ||
1203 | |||
1204 | do_foo() { | ||
1205 | if [ somecondition ] ; then | ||
1206 | bar_do_foo | ||
1207 | else | ||
1208 | # Do something else | ||
1209 | fi | ||
1210 | } | ||
1211 | |||
1212 | To call your modified version of the function as defined in your recipe, | ||
1213 | call it as ``do_foo``. | ||
1214 | |||
1215 | With these conditions met, your single recipe can freely choose between | ||
1216 | the original function as defined in the class file and the modified | ||
1217 | function in your recipe. If you do not set up these conditions, you are | ||
1218 | limited to using one function or the other. | ||
1219 | |||
1220 | Tasks | ||
1221 | ===== | ||
1222 | |||
1223 | Tasks are BitBake execution units that make up the steps that BitBake | ||
1224 | can run for a given recipe. Tasks are only supported in recipes and | ||
1225 | classes (i.e. in ``.bb`` files and files included or inherited from | ||
1226 | ``.bb`` files). By convention, tasks have names that start with "do\_". | ||
1227 | |||
1228 | Promoting a Function to a Task | ||
1229 | ------------------------------ | ||
1230 | |||
1231 | Tasks are either :ref:`shell functions <bitbake-user-manual/bitbake-user-manual-metadata:shell functions>` or | ||
1232 | :ref:`BitBake-style Python functions <bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions>` | ||
1233 | that have been promoted to tasks by using the ``addtask`` command. The | ||
1234 | ``addtask`` command can also optionally describe dependencies between | ||
1235 | the task and other tasks. Here is an example that shows how to define a | ||
1236 | task and declare some dependencies: :: | ||
1237 | |||
1238 | python do_printdate () { | ||
1239 | import time | ||
1240 | print time.strftime('%Y%m%d', time.gmtime()) | ||
1241 | } | ||
1242 | addtask printdate after do_fetch before do_build | ||
1243 | |||
1244 | The first argument to ``addtask`` is the name | ||
1245 | of the function to promote to a task. If the name does not start with | ||
1246 | "do\_", "do\_" is implicitly added, which enforces the convention that all | ||
1247 | task names start with "do\_". | ||
1248 | |||
1249 | In the previous example, the ``do_printdate`` task becomes a dependency | ||
1250 | of the ``do_build`` task, which is the default task (i.e. the task run | ||
1251 | by the ``bitbake`` command unless another task is specified explicitly). | ||
1252 | Additionally, the ``do_printdate`` task becomes dependent upon the | ||
1253 | ``do_fetch`` task. Running the ``do_build`` task results in the | ||
1254 | ``do_printdate`` task running first. | ||
1255 | |||
1256 | .. note:: | ||
1257 | |||
1258 | If you try out the previous example, you might see that the | ||
1259 | ``do_printdate`` | ||
1260 | task is only run the first time you build the recipe with the | ||
1261 | ``bitbake`` | ||
1262 | command. This is because BitBake considers the task "up-to-date" | ||
1263 | after that initial run. If you want to force the task to always be | ||
1264 | rerun for experimentation purposes, you can make BitBake always | ||
1265 | consider the task "out-of-date" by using the | ||
1266 | :ref:`[nostamp] <bitbake-user-manual/bitbake-user-manual-metadata:Variable Flags>` | ||
1267 | variable flag, as follows: :: | ||
1268 | |||
1269 | do_printdate[nostamp] = "1" | ||
1270 | |||
1271 | You can also explicitly run the task and provide the | ||
1272 | -f option as follows: :: | ||
1273 | |||
1274 | $ bitbake recipe -c printdate -f | ||
1275 | |||
1276 | When manually selecting a task to run with the bitbake ``recipe | ||
1277 | -c task`` command, you can omit the "do\_" prefix as part of the task | ||
1278 | name. | ||
1279 | |||
1280 | You might wonder about the practical effects of using ``addtask`` | ||
1281 | without specifying any dependencies as is done in the following example: :: | ||
1282 | |||
1283 | addtask printdate | ||
1284 | |||
1285 | In this example, assuming dependencies have not been | ||
1286 | added through some other means, the only way to run the task is by | ||
1287 | explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You | ||
1288 | can use the ``do_listtasks`` task to list all tasks defined in a recipe | ||
1289 | as shown in the following example: :: | ||
1290 | |||
1291 | $ bitbake recipe -c listtasks | ||
1292 | |||
1293 | For more information on task dependencies, see the | ||
1294 | ":ref:`bitbake-user-manual/bitbake-user-manual-execution:dependencies`" section. | ||
1295 | |||
1296 | See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section for information | ||
1297 | on variable flags you can use with tasks. | ||
1298 | |||
1299 | Deleting a Task | ||
1300 | --------------- | ||
1301 | |||
1302 | As well as being able to add tasks, you can delete them. Simply use the | ||
1303 | ``deltask`` command to delete a task. For example, to delete the example | ||
1304 | task used in the previous sections, you would use: :: | ||
1305 | |||
1306 | deltask printdate | ||
1307 | |||
1308 | If you delete a task using the ``deltask`` command and the task has | ||
1309 | dependencies, the dependencies are not reconnected. For example, suppose | ||
1310 | you have three tasks named ``do_a``, ``do_b``, and ``do_c``. | ||
1311 | Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is | ||
1312 | dependent on ``do_a``. Given this scenario, if you use ``deltask`` to | ||
1313 | delete ``do_b``, the implicit dependency relationship between ``do_c`` | ||
1314 | and ``do_a`` through ``do_b`` no longer exists, and ``do_c`` | ||
1315 | dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free | ||
1316 | to run before ``do_a``. | ||
1317 | |||
1318 | If you want dependencies such as these to remain intact, use the | ||
1319 | ``[noexec]`` varflag to disable the task instead of using the | ||
1320 | ``deltask`` command to delete it: :: | ||
1321 | |||
1322 | do_b[noexec] = "1" | ||
1323 | |||
1324 | Passing Information Into the Build Task Environment | ||
1325 | --------------------------------------------------- | ||
1326 | |||
1327 | When running a task, BitBake tightly controls the shell execution | ||
1328 | environment of the build tasks to make sure unwanted contamination from | ||
1329 | the build machine cannot influence the build. | ||
1330 | |||
1331 | .. note:: | ||
1332 | |||
1333 | By default, BitBake cleans the environment to include only those | ||
1334 | things exported or listed in its whitelist to ensure that the build | ||
1335 | environment is reproducible and consistent. You can prevent this | ||
1336 | "cleaning" by setting the :term:`BB_PRESERVE_ENV` variable. | ||
1337 | |||
1338 | Consequently, if you do want something to get passed into the build task | ||
1339 | environment, you must take these two steps: | ||
1340 | |||
1341 | #. Tell BitBake to load what you want from the environment into the | ||
1342 | datastore. You can do so through the | ||
1343 | :term:`BB_ENV_WHITELIST` and | ||
1344 | :term:`BB_ENV_EXTRAWHITE` variables. For | ||
1345 | example, assume you want to prevent the build system from accessing | ||
1346 | your ``$HOME/.ccache`` directory. The following command "whitelists" | ||
1347 | the environment variable ``CCACHE_DIR`` causing BitBake to allow that | ||
1348 | variable into the datastore: :: | ||
1349 | |||
1350 | export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR" | ||
1351 | |||
1352 | #. Tell BitBake to export what you have loaded into the datastore to the | ||
1353 | task environment of every running task. Loading something from the | ||
1354 | environment into the datastore (previous step) only makes it | ||
1355 | available in the datastore. To export it to the task environment of | ||
1356 | every running task, use a command similar to the following in your | ||
1357 | local configuration file ``local.conf`` or your distribution | ||
1358 | configuration file: :: | ||
1359 | |||
1360 | export CCACHE_DIR | ||
1361 | |||
1362 | .. note:: | ||
1363 | |||
1364 | A side effect of the previous steps is that BitBake records the | ||
1365 | variable as a dependency of the build process in things like the | ||
1366 | setscene checksums. If doing so results in unnecessary rebuilds of | ||
1367 | tasks, you can whitelist the variable so that the setscene code | ||
1368 | ignores the dependency when it creates checksums. | ||
1369 | |||
1370 | Sometimes, it is useful to be able to obtain information from the | ||
1371 | original execution environment. BitBake saves a copy of the original | ||
1372 | environment into a special variable named :term:`BB_ORIGENV`. | ||
1373 | |||
1374 | The ``BB_ORIGENV`` variable returns a datastore object that can be | ||
1375 | queried using the standard datastore operators such as | ||
1376 | ``getVar(, False)``. The datastore object is useful, for example, to | ||
1377 | find the original ``DISPLAY`` variable. Here is an example: :: | ||
1378 | |||
1379 | origenv = d.getVar("BB_ORIGENV", False) | ||
1380 | bar = origenv.getVar("BAR", False) | ||
1381 | |||
1382 | The previous example returns ``BAR`` from the original execution | ||
1383 | environment. | ||
1384 | |||
1385 | Variable Flags | ||
1386 | ============== | ||
1387 | |||
1388 | Variable flags (varflags) help control a task's functionality and | ||
1389 | dependencies. BitBake reads and writes varflags to the datastore using | ||
1390 | the following command forms: :: | ||
1391 | |||
1392 | variable = d.getVarFlags("variable") | ||
1393 | self.d.setVarFlags("FOO", {"func": True}) | ||
1394 | |||
1395 | When working with varflags, the same syntax, with the exception of | ||
1396 | overrides, applies. In other words, you can set, append, and prepend | ||
1397 | varflags just like variables. See the | ||
1398 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flag syntax`" section for details. | ||
1399 | |||
1400 | BitBake has a defined set of varflags available for recipes and classes. | ||
1401 | Tasks support a number of these flags which control various | ||
1402 | functionality of the task: | ||
1403 | |||
1404 | - ``[cleandirs]``: Empty directories that should be created before | ||
1405 | the task runs. Directories that already exist are removed and | ||
1406 | recreated to empty them. | ||
1407 | |||
1408 | - ``[depends]``: Controls inter-task dependencies. See the | ||
1409 | :term:`DEPENDS` variable and the | ||
1410 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task | ||
1411 | dependencies`" section for more information. | ||
1412 | |||
1413 | - ``[deptask]``: Controls task build-time dependencies. See the | ||
1414 | :term:`DEPENDS` variable and the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:build dependencies`" section for more information. | ||
1415 | |||
1416 | - ``[dirs]``: Directories that should be created before the task | ||
1417 | runs. Directories that already exist are left as is. The last | ||
1418 | directory listed is used as the current working directory for the | ||
1419 | task. | ||
1420 | |||
1421 | - ``[lockfiles]``: Specifies one or more lockfiles to lock while the | ||
1422 | task executes. Only one task may hold a lockfile, and any task that | ||
1423 | attempts to lock an already locked file will block until the lock is | ||
1424 | released. You can use this variable flag to accomplish mutual | ||
1425 | exclusion. | ||
1426 | |||
1427 | - ``[noexec]``: When set to "1", marks the task as being empty, with | ||
1428 | no execution required. You can use the ``[noexec]`` flag to set up | ||
1429 | tasks as dependency placeholders, or to disable tasks defined | ||
1430 | elsewhere that are not needed in a particular recipe. | ||
1431 | |||
1432 | - ``[nostamp]``: When set to "1", tells BitBake to not generate a | ||
1433 | stamp file for a task, which implies the task should always be | ||
1434 | executed. | ||
1435 | |||
1436 | .. caution:: | ||
1437 | |||
1438 | Any task that depends (possibly indirectly) on a ``[nostamp]`` task will | ||
1439 | always be executed as well. This can cause unnecessary rebuilding if you | ||
1440 | are not careful. | ||
1441 | |||
1442 | - ``[number_threads]``: Limits tasks to a specific number of | ||
1443 | simultaneous threads during execution. This varflag is useful when | ||
1444 | your build host has a large number of cores but certain tasks need to | ||
1445 | be rate-limited due to various kinds of resource constraints (e.g. to | ||
1446 | avoid network throttling). ``number_threads`` works similarly to the | ||
1447 | :term:`BB_NUMBER_THREADS` variable but is task-specific. | ||
1448 | |||
1449 | Set the value globally. For example, the following makes sure the | ||
1450 | ``do_fetch`` task uses no more than two simultaneous execution | ||
1451 | threads: do_fetch[number_threads] = "2" | ||
1452 | |||
1453 | .. warning:: | ||
1454 | |||
1455 | - Setting the varflag in individual recipes rather than globally | ||
1456 | can result in unpredictable behavior. | ||
1457 | |||
1458 | - Setting the varflag to a value greater than the value used in | ||
1459 | the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to | ||
1460 | have no effect. | ||
1461 | |||
1462 | - ``[postfuncs]``: List of functions to call after the completion of | ||
1463 | the task. | ||
1464 | |||
1465 | - ``[prefuncs]``: List of functions to call before the task executes. | ||
1466 | |||
1467 | - ``[rdepends]``: Controls inter-task runtime dependencies. See the | ||
1468 | :term:`RDEPENDS` variable, the | ||
1469 | :term:`RRECOMMENDS` variable, and the | ||
1470 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task dependencies`" section for | ||
1471 | more information. | ||
1472 | |||
1473 | - ``[rdeptask]``: Controls task runtime dependencies. See the | ||
1474 | :term:`RDEPENDS` variable, the | ||
1475 | :term:`RRECOMMENDS` variable, and the | ||
1476 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:runtime dependencies`" section for more | ||
1477 | information. | ||
1478 | |||
1479 | - ``[recideptask]``: When set in conjunction with ``recrdeptask``, | ||
1480 | specifies a task that should be inspected for additional | ||
1481 | dependencies. | ||
1482 | |||
1483 | - ``[recrdeptask]``: Controls task recursive runtime dependencies. | ||
1484 | See the :term:`RDEPENDS` variable, the | ||
1485 | :term:`RRECOMMENDS` variable, and the | ||
1486 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:recursive dependencies`" section for | ||
1487 | more information. | ||
1488 | |||
1489 | - ``[stamp-extra-info]``: Extra stamp information to append to the | ||
1490 | task's stamp. As an example, OpenEmbedded uses this flag to allow | ||
1491 | machine-specific tasks. | ||
1492 | |||
1493 | - ``[umask]``: The umask to run the task under. | ||
1494 | |||
1495 | Several varflags are useful for controlling how signatures are | ||
1496 | calculated for variables. For more information on this process, see the | ||
1497 | ":ref:`bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section. | ||
1498 | |||
1499 | - ``[vardeps]``: Specifies a space-separated list of additional | ||
1500 | variables to add to a variable's dependencies for the purposes of | ||
1501 | calculating its signature. Adding variables to this list is useful, | ||
1502 | for example, when a function refers to a variable in a manner that | ||
1503 | does not allow BitBake to automatically determine that the variable | ||
1504 | is referred to. | ||
1505 | |||
1506 | - ``[vardepsexclude]``: Specifies a space-separated list of variables | ||
1507 | that should be excluded from a variable's dependencies for the | ||
1508 | purposes of calculating its signature. | ||
1509 | |||
1510 | - ``[vardepvalue]``: If set, instructs BitBake to ignore the actual | ||
1511 | value of the variable and instead use the specified value when | ||
1512 | calculating the variable's signature. | ||
1513 | |||
1514 | - ``[vardepvalueexclude]``: Specifies a pipe-separated list of | ||
1515 | strings to exclude from the variable's value when calculating the | ||
1516 | variable's signature. | ||
1517 | |||
1518 | Events | ||
1519 | ====== | ||
1520 | |||
1521 | BitBake allows installation of event handlers within recipe and class | ||
1522 | files. Events are triggered at certain points during operation, such as | ||
1523 | the beginning of operation against a given recipe (i.e. ``*.bb``), the | ||
1524 | start of a given task, a task failure, a task success, and so forth. The | ||
1525 | intent is to make it easy to do things like email notification on build | ||
1526 | failures. | ||
1527 | |||
1528 | Following is an example event handler that prints the name of the event | ||
1529 | and the content of the ``FILE`` variable: :: | ||
1530 | |||
1531 | addhandler myclass_eventhandler | ||
1532 | python myclass_eventhandler() { | ||
1533 | from bb.event import getName | ||
1534 | print("The name of the Event is %s" % getName(e)) | ||
1535 | print("The file we run for is %s" % d.getVar('FILE')) | ||
1536 | } | ||
1537 | myclass_eventhandler[eventmask] = "bb.event.BuildStarted | ||
1538 | bb.event.BuildCompleted" | ||
1539 | |||
1540 | In the previous example, an eventmask has been | ||
1541 | set so that the handler only sees the "BuildStarted" and | ||
1542 | "BuildCompleted" events. This event handler gets called every time an | ||
1543 | event matching the eventmask is triggered. A global variable "e" is | ||
1544 | defined, which represents the current event. With the ``getName(e)`` | ||
1545 | method, you can get the name of the triggered event. The global | ||
1546 | datastore is available as "d". In legacy code, you might see "e.data" | ||
1547 | used to get the datastore. However, realize that "e.data" is deprecated | ||
1548 | and you should use "d" going forward. | ||
1549 | |||
1550 | The context of the datastore is appropriate to the event in question. | ||
1551 | For example, "BuildStarted" and "BuildCompleted" events run before any | ||
1552 | tasks are executed so would be in the global configuration datastore | ||
1553 | namespace. No recipe-specific metadata exists in that namespace. The | ||
1554 | "BuildStarted" and "BuildCompleted" events also run in the main | ||
1555 | cooker/server process rather than any worker context. Thus, any changes | ||
1556 | made to the datastore would be seen by other cooker/server events within | ||
1557 | the current build but not seen outside of that build or in any worker | ||
1558 | context. Task events run in the actual tasks in question consequently | ||
1559 | have recipe-specific and task-specific contents. These events run in the | ||
1560 | worker context and are discarded at the end of task execution. | ||
1561 | |||
1562 | During a standard build, the following common events might occur. The | ||
1563 | following events are the most common kinds of events that most metadata | ||
1564 | might have an interest in viewing: | ||
1565 | |||
1566 | - ``bb.event.ConfigParsed()``: Fired when the base configuration; which | ||
1567 | consists of ``bitbake.conf``, ``base.bbclass`` and any global | ||
1568 | ``INHERIT`` statements; has been parsed. You can see multiple such | ||
1569 | events when each of the workers parse the base configuration or if | ||
1570 | the server changes configuration and reparses. Any given datastore | ||
1571 | only has one such event executed against it, however. If | ||
1572 | ```BB_INVALIDCONF`` <#>`__ is set in the datastore by the event | ||
1573 | handler, the configuration is reparsed and a new event triggered, | ||
1574 | allowing the metadata to update configuration. | ||
1575 | |||
1576 | - ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one | ||
1577 | second. You can configure the interval time using the | ||
1578 | ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the | ||
1579 | ``time.time()`` value when the event is triggered. This event is | ||
1580 | useful for activities such as system state monitoring. | ||
1581 | |||
1582 | - ``bb.event.ParseStarted()``: Fired when BitBake is about to start | ||
1583 | parsing recipes. This event's "total" attribute represents the number | ||
1584 | of recipes BitBake plans to parse. | ||
1585 | |||
1586 | - ``bb.event.ParseProgress()``: Fired as parsing progresses. This | ||
1587 | event's "current" attribute is the number of recipes parsed as well | ||
1588 | as the "total" attribute. | ||
1589 | |||
1590 | - ``bb.event.ParseCompleted()``: Fired when parsing is complete. This | ||
1591 | event's "cached", "parsed", "skipped", "virtuals", "masked", and | ||
1592 | "errors" attributes provide statistics for the parsing results. | ||
1593 | |||
1594 | - ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake | ||
1595 | fires multiple "BuildStarted" events (one per configuration) when | ||
1596 | multiple configuration (multiconfig) is enabled. | ||
1597 | |||
1598 | - ``bb.build.TaskStarted()``: Fired when a task starts. This event's | ||
1599 | "taskfile" attribute points to the recipe from which the task | ||
1600 | originates. The "taskname" attribute, which is the task's name, | ||
1601 | includes the ``do_`` prefix, and the "logfile" attribute point to | ||
1602 | where the task's output is stored. Finally, the "time" attribute is | ||
1603 | the task's execution start time. | ||
1604 | |||
1605 | - ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task | ||
1606 | that does not exist. | ||
1607 | |||
1608 | - ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail | ||
1609 | and should not be presented to the user verbosely. | ||
1610 | |||
1611 | - ``bb.build.TaskFailed()``: Fired for normal tasks that fail. | ||
1612 | |||
1613 | - ``bb.build.TaskSucceeded()``: Fired when a task successfully | ||
1614 | completes. | ||
1615 | |||
1616 | - ``bb.event.BuildCompleted()``: Fired when a build finishes. | ||
1617 | |||
1618 | - ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker | ||
1619 | shuts down. This event is usually only seen by the UIs as a sign they | ||
1620 | should also shutdown. | ||
1621 | |||
1622 | This next list of example events occur based on specific requests to the | ||
1623 | server. These events are often used to communicate larger pieces of | ||
1624 | information from the BitBake server to other parts of BitBake such as | ||
1625 | user interfaces: | ||
1626 | |||
1627 | - ``bb.event.TreeDataPreparationStarted()`` | ||
1628 | - ``bb.event.TreeDataPreparationProgress()`` | ||
1629 | - ``bb.event.TreeDataPreparationCompleted()`` | ||
1630 | - ``bb.event.DepTreeGenerated()`` | ||
1631 | - ``bb.event.CoreBaseFilesFound()`` | ||
1632 | - ``bb.event.ConfigFilePathFound()`` | ||
1633 | - ``bb.event.FilesMatchingFound()`` | ||
1634 | - ``bb.event.ConfigFilesFound()`` | ||
1635 | - ``bb.event.TargetsTreeGenerated()`` | ||
1636 | |||
1637 | .. _variants-class-extension-mechanism: | ||
1638 | |||
1639 | Variants - Class Extension Mechanism | ||
1640 | ==================================== | ||
1641 | |||
1642 | BitBake supports two features that facilitate creating from a single | ||
1643 | recipe file multiple incarnations of that recipe file where all | ||
1644 | incarnations are buildable. These features are enabled through the | ||
1645 | :term:`BBCLASSEXTEND` and :term:`BBVERSIONS` variables. | ||
1646 | |||
1647 | .. note:: | ||
1648 | |||
1649 | The mechanism for this class extension is extremely specific to the | ||
1650 | implementation. Usually, the recipe's :term:`PROVIDES` , :term:`PN` , and | ||
1651 | :term:`DEPENDS` variables would need to be modified by the extension | ||
1652 | class. For specific examples, see the OE-Core native , nativesdk , and | ||
1653 | multilib classes. | ||
1654 | |||
1655 | - ``BBCLASSEXTEND``: This variable is a space separated list of | ||
1656 | classes used to "extend" the recipe for each variant. Here is an | ||
1657 | example that results in a second incarnation of the current recipe | ||
1658 | being available. This second incarnation will have the "native" class | ||
1659 | inherited. :: | ||
1660 | |||
1661 | BBCLASSEXTEND = "native" | ||
1662 | |||
1663 | - ``BBVERSIONS``: This variable allows a single recipe to build | ||
1664 | multiple versions of a project from a single recipe file. You can | ||
1665 | also specify conditional metadata (using the | ||
1666 | :term:`OVERRIDES` mechanism) for a single | ||
1667 | version, or an optionally named range of versions. Here is an | ||
1668 | example: :: | ||
1669 | |||
1670 | BBVERSIONS = "1.0 2.0 git" | ||
1671 | SRC_URI_git = "git://someurl/somepath.git" | ||
1672 | |||
1673 | BBVERSIONS = "1.0.[0-6]:1.0.0+ 1.0.[7-9]:1.0.7+" | ||
1674 | SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1" | ||
1675 | |||
1676 | The name of the range defaults to the original version of the recipe. For | ||
1677 | example, in OpenEmbedded, the recipe file ``foo_1.0.0+.bb`` creates a default | ||
1678 | name range of ``1.0.0+``. This is useful because the range name is not only | ||
1679 | placed into overrides, but it is also made available for the metadata to use | ||
1680 | in the variable that defines the base recipe versions for use in ``file://`` | ||
1681 | search paths (:term:`FILESPATH`). | ||
1682 | |||
1683 | Dependencies | ||
1684 | ============ | ||
1685 | |||
1686 | To allow for efficient parallel processing, BitBake handles dependencies | ||
1687 | at the task level. Dependencies can exist both between tasks within a | ||
1688 | single recipe and between tasks in different recipes. Following are | ||
1689 | examples of each: | ||
1690 | |||
1691 | - For tasks within a single recipe, a recipe's ``do_configure`` task | ||
1692 | might need to complete before its ``do_compile`` task can run. | ||
1693 | |||
1694 | - For tasks in different recipes, one recipe's ``do_configure`` task | ||
1695 | might require another recipe's ``do_populate_sysroot`` task to finish | ||
1696 | first such that the libraries and headers provided by the other | ||
1697 | recipe are available. | ||
1698 | |||
1699 | This section describes several ways to declare dependencies. Remember, | ||
1700 | even though dependencies are declared in different ways, they are all | ||
1701 | simply dependencies between tasks. | ||
1702 | |||
1703 | .. _dependencies-internal-to-the-bb-file: | ||
1704 | |||
1705 | Dependencies Internal to the ``.bb`` File | ||
1706 | ----------------------------------------- | ||
1707 | |||
1708 | BitBake uses the ``addtask`` directive to manage dependencies that are | ||
1709 | internal to a given recipe file. You can use the ``addtask`` directive | ||
1710 | to indicate when a task is dependent on other tasks or when other tasks | ||
1711 | depend on that recipe. Here is an example: :: | ||
1712 | |||
1713 | addtask printdate after do_fetch before do_build | ||
1714 | |||
1715 | In this example, the ``do_printdate`` task | ||
1716 | depends on the completion of the ``do_fetch`` task, and the ``do_build`` | ||
1717 | task depends on the completion of the ``do_printdate`` task. | ||
1718 | |||
1719 | .. note:: | ||
1720 | |||
1721 | For a task to run, it must be a direct or indirect dependency of some | ||
1722 | other task that is scheduled to run. | ||
1723 | |||
1724 | For illustration, here are some examples: | ||
1725 | |||
1726 | - The directive ``addtask mytask before do_configure`` causes | ||
1727 | ``do_mytask`` to run before ``do_configure`` runs. Be aware that | ||
1728 | ``do_mytask`` still only runs if its :ref:`input | ||
1729 | checksum <bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)>` has changed since the last time it was | ||
1730 | run. Changes to the input checksum of ``do_mytask`` also | ||
1731 | indirectly cause ``do_configure`` to run. | ||
1732 | |||
1733 | - The directive ``addtask mytask after do_configure`` by itself | ||
1734 | never causes ``do_mytask`` to run. ``do_mytask`` can still be run | ||
1735 | manually as follows: :: | ||
1736 | |||
1737 | $ bitbake recipe -c mytask | ||
1738 | |||
1739 | Declaring ``do_mytask`` as a dependency of some other task that is | ||
1740 | scheduled to run also causes it to run. Regardless, the task runs after | ||
1741 | ``do_configure``. | ||
1742 | |||
1743 | Build Dependencies | ||
1744 | ------------------ | ||
1745 | |||
1746 | BitBake uses the :term:`DEPENDS` variable to manage | ||
1747 | build time dependencies. The ``[deptask]`` varflag for tasks signifies | ||
1748 | the task of each item listed in ``DEPENDS`` that must complete before | ||
1749 | that task can be executed. Here is an example: :: | ||
1750 | |||
1751 | do_configure[deptask] = "do_populate_sysroot" | ||
1752 | |||
1753 | In this example, the ``do_populate_sysroot`` task | ||
1754 | of each item in ``DEPENDS`` must complete before ``do_configure`` can | ||
1755 | execute. | ||
1756 | |||
1757 | Runtime Dependencies | ||
1758 | -------------------- | ||
1759 | |||
1760 | BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS` | ||
1761 | variables to manage runtime dependencies. | ||
1762 | |||
1763 | The ``PACKAGES`` variable lists runtime packages. Each of those packages | ||
1764 | can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The | ||
1765 | ``[rdeptask]`` flag for tasks is used to signify the task of each item | ||
1766 | runtime dependency which must have completed before that task can be | ||
1767 | executed. :: | ||
1768 | |||
1769 | do_package_qa[rdeptask] = "do_packagedata" | ||
1770 | |||
1771 | In the previous | ||
1772 | example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must | ||
1773 | have completed before ``do_package_qa`` can execute. | ||
1774 | Although ``RDEPENDS`` contains entries from the | ||
1775 | runtime dependency namespace, BitBake knows how to map them back | ||
1776 | to the build-time dependency namespace, in which the tasks are defined. | ||
1777 | |||
1778 | Recursive Dependencies | ||
1779 | ---------------------- | ||
1780 | |||
1781 | BitBake uses the ``[recrdeptask]`` flag to manage recursive task | ||
1782 | dependencies. BitBake looks through the build-time and runtime | ||
1783 | dependencies of the current recipe, looks through the task's inter-task | ||
1784 | dependencies, and then adds dependencies for the listed task. Once | ||
1785 | BitBake has accomplished this, it recursively works through the | ||
1786 | dependencies of those tasks. Iterative passes continue until all | ||
1787 | dependencies are discovered and added. | ||
1788 | |||
1789 | The ``[recrdeptask]`` flag is most commonly used in high-level recipes | ||
1790 | that need to wait for some task to finish "globally". For example, | ||
1791 | ``image.bbclass`` has the following: :: | ||
1792 | |||
1793 | do_rootfs[recrdeptask] += "do_packagedata" | ||
1794 | |||
1795 | This statement says that the ``do_packagedata`` task of | ||
1796 | the current recipe and all recipes reachable (by way of dependencies) | ||
1797 | from the image recipe must run before the ``do_rootfs`` task can run. | ||
1798 | |||
1799 | BitBake allows a task to recursively depend on itself by | ||
1800 | referencing itself in the task list: :: | ||
1801 | |||
1802 | do_a[recrdeptask] = "do_a do_b" | ||
1803 | |||
1804 | In the same way as before, this means that the ``do_a`` | ||
1805 | and ``do_b`` tasks of the current recipe and all | ||
1806 | recipes reachable (by way of dependencies) from the recipe | ||
1807 | must run before the ``do_a`` task can run. In this | ||
1808 | case BitBake will ignore the current recipe's ``do_a`` | ||
1809 | task circular dependency on itself. | ||
1810 | |||
1811 | Inter-Task Dependencies | ||
1812 | ----------------------- | ||
1813 | |||
1814 | BitBake uses the ``[depends]`` flag in a more generic form to manage | ||
1815 | inter-task dependencies. This more generic form allows for | ||
1816 | inter-dependency checks for specific tasks rather than checks for the | ||
1817 | data in ``DEPENDS``. Here is an example: :: | ||
1818 | |||
1819 | do_patch[depends] = "quilt-native:do_populate_sysroot" | ||
1820 | |||
1821 | In this example, the ``do_populate_sysroot`` task of the target ``quilt-native`` | ||
1822 | must have completed before the ``do_patch`` task can execute. | ||
1823 | |||
1824 | The ``[rdepends]`` flag works in a similar way but takes targets in the | ||
1825 | runtime namespace instead of the build-time dependency namespace. | ||
1826 | |||
1827 | Functions You Can Call From Within Python | ||
1828 | ========================================= | ||
1829 | |||
1830 | BitBake provides many functions you can call from within Python | ||
1831 | functions. This section lists the most commonly used functions, and | ||
1832 | mentions where to find others. | ||
1833 | |||
1834 | Functions for Accessing Datastore Variables | ||
1835 | ------------------------------------------- | ||
1836 | |||
1837 | It is often necessary to access variables in the BitBake datastore using | ||
1838 | Python functions. The BitBake datastore has an API that allows you this | ||
1839 | access. Here is a list of available operations: | ||
1840 | |||
1841 | .. list-table:: | ||
1842 | :widths: auto | ||
1843 | :header-rows: 1 | ||
1844 | |||
1845 | * - *Operation* | ||
1846 | - *Description* | ||
1847 | * - ``d.getVar("X", expand)`` | ||
1848 | - Returns the value of variable "X". Using "expand=True" expands the | ||
1849 | value. Returns "None" if the variable "X" does not exist. | ||
1850 | * - ``d.setVar("X", "value")`` | ||
1851 | - Sets the variable "X" to "value" | ||
1852 | * - ``d.appendVar("X", "value")`` | ||
1853 | - Adds "value" to the end of the variable "X". Acts like ``d.setVar("X", | ||
1854 | "value")`` if the variable "X" does not exist. | ||
1855 | * - ``d.prependVar("X", "value")`` | ||
1856 | - Adds "value" to the start of the variable "X". Acts like | ||
1857 | ``d.setVar("X","value")`` if the variable "X" does not exist. | ||
1858 | * - ``d.delVar("X")`` | ||
1859 | - Deletes the variable "X" from the datastore. Does nothing if the variable | ||
1860 | "X" does not exist. | ||
1861 | * - ``d.renameVar("X", "Y")`` | ||
1862 | - Renames the variable "X" to "Y". Does nothing if the variable "X" does | ||
1863 | not exist. | ||
1864 | * - ``d.getVarFlag("X", flag, expand)`` | ||
1865 | - Returns the value of variable "X". Using "expand=True" expands the | ||
1866 | value. Returns "None" if either the variable "X" or the named flag does | ||
1867 | not exist. | ||
1868 | * - ``d.setVarFlag("X", flag, "value")`` | ||
1869 | - Sets the named flag for variable "X" to "value". | ||
1870 | * - ``d.appendVarFlag("X", flag, "value")`` | ||
1871 | - Appends "value" to the named flag on the variable "X". Acts like | ||
1872 | ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist. | ||
1873 | * - ``d.prependVarFlag("X", flag, "value")`` | ||
1874 | - Prepends "value" to the named flag on the variable "X". Acts like | ||
1875 | ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist. | ||
1876 | * - ``d.delVarFlag("X", flag)`` | ||
1877 | - Deletes the named flag on the variable "X" from the datastore. | ||
1878 | * - ``d.setVarFlags("X", flagsdict)`` | ||
1879 | - Sets the flags specified in the ``flagsdict()`` | ||
1880 | parameter. ``setVarFlags`` does not clear previous flags. Think of this | ||
1881 | operation as ``addVarFlags``. | ||
1882 | * - ``d.getVarFlags("X")`` | ||
1883 | - Returns a ``flagsdict`` of the flags for the variable "X". Returns "None" | ||
1884 | if the variable "X" does not exist. | ||
1885 | * - ``d.delVarFlags("X")`` | ||
1886 | - Deletes all the flags for the variable "X". Does nothing if the variable | ||
1887 | "X" does not exist. | ||
1888 | * - ``d.expand(expression)`` | ||
1889 | - Expands variable references in the specified string | ||
1890 | expression. References to variables that do not exist are left as is. For | ||
1891 | example, ``d.expand("foo ${X}")`` expands to the literal string "foo | ||
1892 | ${X}" if the variable "X" does not exist. | ||
1893 | |||
1894 | Other Functions | ||
1895 | --------------- | ||
1896 | |||
1897 | You can find many other functions that can be called from Python by | ||
1898 | looking at the source code of the ``bb`` module, which is in | ||
1899 | ``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes | ||
1900 | the commonly used functions ``bb.utils.contains()`` and | ||
1901 | ``bb.utils.mkdirhier()``, which come with docstrings. | ||
1902 | |||
1903 | Task Checksums and Setscene | ||
1904 | =========================== | ||
1905 | |||
1906 | BitBake uses checksums (or signatures) along with the setscene to | ||
1907 | determine if a task needs to be run. This section describes the process. | ||
1908 | To help understand how BitBake does this, the section assumes an | ||
1909 | OpenEmbedded metadata-based example. | ||
1910 | |||
1911 | These checksums are stored in :term:`STAMP`. You can | ||
1912 | examine the checksums using the following BitBake command: :: | ||
1913 | |||
1914 | $ bitbake-dumpsigs | ||
1915 | |||
1916 | This command returns the signature data in a readable | ||
1917 | format that allows you to examine the inputs used when the OpenEmbedded | ||
1918 | build system generates signatures. For example, using | ||
1919 | ``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's | ||
1920 | "sigdata" for a C application (e.g. ``bash``). Running the command also | ||
1921 | reveals that the "CC" variable is part of the inputs that are hashed. | ||
1922 | Any changes to this variable would invalidate the stamp and cause the | ||
1923 | ``do_compile`` task to run. | ||
1924 | |||
1925 | The following list describes related variables: | ||
1926 | |||
1927 | - :term:`BB_HASHCHECK_FUNCTION`: | ||
1928 | Specifies the name of the function to call during the "setscene" part | ||
1929 | of the task's execution in order to validate the list of task hashes. | ||
1930 | |||
1931 | - :term:`BB_SETSCENE_DEPVALID`: | ||
1932 | Specifies a function BitBake calls that determines whether BitBake | ||
1933 | requires a setscene dependency to be met. | ||
1934 | |||
1935 | - :term:`BB_SETSCENE_VERIFY_FUNCTION2`: | ||
1936 | Specifies a function to call that verifies the list of planned task | ||
1937 | execution before the main task execution happens. | ||
1938 | |||
1939 | - :term:`BB_STAMP_POLICY`: Defines the mode | ||
1940 | for comparing timestamps of stamp files. | ||
1941 | |||
1942 | - :term:`BB_STAMP_WHITELIST`: Lists stamp | ||
1943 | files that are looked at when the stamp policy is "whitelist". | ||
1944 | |||
1945 | - :term:`BB_TASKHASH`: Within an executing task, | ||
1946 | this variable holds the hash of the task as returned by the currently | ||
1947 | enabled signature generator. | ||
1948 | |||
1949 | - :term:`STAMP`: The base path to create stamp files. | ||
1950 | |||
1951 | - :term:`STAMPCLEAN`: Again, the base path to | ||
1952 | create stamp files but can use wildcards for matching a range of | ||
1953 | files for clean operations. | ||
1954 | |||
1955 | Wildcard Support in Variables | ||
1956 | ============================= | ||
1957 | |||
1958 | Support for wildcard use in variables varies depending on the context in | ||
1959 | which it is used. For example, some variables and file names allow | ||
1960 | limited use of wildcards through the "``%``" and "``*``" characters. | ||
1961 | Other variables or names support Python's | ||
1962 | `glob <https://docs.python.org/3/library/glob.html>`_ syntax, | ||
1963 | `fnmatch <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`_ | ||
1964 | syntax, or | ||
1965 | `Regular Expression (re) <https://docs.python.org/3/library/re.html>`_ | ||
1966 | syntax. | ||
1967 | |||
1968 | For variables that have wildcard suport, the documentation describes | ||
1969 | which form of wildcard, its use, and its limitations. | ||