summaryrefslogtreecommitdiffstats
path: root/bitbake
diff options
context:
space:
mode:
authorNicolas Dechesne <nicolas.dechesne@linaro.org>2020-07-01 17:00:17 +0200
committerRichard Purdie <richard.purdie@linuxfoundation.org>2020-09-16 18:14:07 +0100
commit080f80d9126fe38a7008a0112bf6cd8ebf1b3491 (patch)
tree2cc5369882ea554a3a8b369abac3a0d0f969d7f0 /bitbake
parent69ed72025ad48ef042bb80cdc912c8486cf635bc (diff)
downloadpoky-080f80d9126fe38a7008a0112bf6cd8ebf1b3491.tar.gz
bitbake: sphinx: initial sphinx support
This commit is autogenerated pandoc to generate an inital set of reST files based on DocBook XML files. A .rst file is generated for each .xml files in all manuals with this command: cd <manual> for i in *.xml; do \ pandoc -f docbook -t rst --shift-heading-level-by=-1 \ $i -o $(basename $i .xml).rst \ done The conversion was done with: pandoc 2.9.2.1-91 (Arch Linux). Also created an initial top level index file for each document, and added all 'books' to the top leve index.rst file. The YP manuals layout is organized as: Book Chapter Section Section Section Sphinx uses section headers to create the document structure. ReStructuredText defines sections headers like that: To break longer text up into sections, you use section headers. These are a single line of text (one or more words) with adornment: an underline alone, or an underline and an overline together, in dashes "-----", equals "======", tildes "~~~~~~" or any of the non-alphanumeric characters = - ` : ' " ~ ^ _ * + # < > that you feel comfortable with. An underline-only adornment is distinct from an overline-and-underline adornment using the same character. The underline/overline must be at least as long as the title text. Be consistent, since all sections marked with the same adornment style are deemed to be at the same level: Let's define the following convention when converting from Docbook: Book => overline === (Title) Chapter => overline *** (1.) Section => ==== (1.1) Section => ---- (1.1.1) Section => ~~~~ (1.1.1.1) Section => ^^^^ (1.1.1.1.1) During the conversion with pandoc, we used --shift-heading-level=-1 to convert most of DocBook headings automatically. However with this setting, the Chapter header was removed, so I added it back manually. Without this setting all headings were off by one, which was more difficult to manually fix. At least with this change, we now have the same TOC with Sphinx and DocBook. (Bitbake rev: 6bf6c8d63787aed7624793c24af3fa603b5ac961) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'bitbake')
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst654
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst629
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst361
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst571
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst1728
-rw-r--r--bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst1235
-rw-r--r--bitbake/doc/index.rst32
7 files changed, 5204 insertions, 6 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst
new file mode 100644
index 0000000000..9488ea9bae
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst
@@ -0,0 +1,654 @@
1=========
2Execution
3=========
4
5The primary purpose for running BitBake is to produce some kind of
6output such as a single installable package, a kernel, a software
7development kit, or even a full, board-specific bootable Linux image,
8complete with bootloader, kernel, and root filesystem. Of course, you
9can execute the ``bitbake`` command with options that cause it to
10execute single tasks, compile single recipe files, capture or clear
11data, or simply return information about the execution environment.
12
13This chapter describes BitBake's execution process from start to finish
14when you use it to create an image. The execution process is launched
15using the following command form: $ bitbake target For information on
16the BitBake command and its options, see "`The BitBake
17Command <#bitbake-user-manual-command>`__" section.
18
19.. note::
20
21 Prior to executing BitBake, you should take advantage of available
22 parallel thread execution on your build host by setting the
23 ```BB_NUMBER_THREADS`` <#var-bb-BB_NUMBER_THREADS>`__ variable in
24 your project's ``local.conf`` configuration file.
25
26 A common method to determine this value for your build host is to run
27 the following: $ grep processor /proc/cpuinfo This command returns
28 the number of processors, which takes into account hyper-threading.
29 Thus, a quad-core build host with hyper-threading most likely shows
30 eight processors, which is the value you would then assign to
31 ``BB_NUMBER_THREADS``.
32
33 A possibly simpler solution is that some Linux distributions (e.g.
34 Debian and Ubuntu) provide the ``ncpus`` command.
35
36Parsing the Base Configuration Metadata
37=======================================
38
39The first thing BitBake does is parse base configuration metadata. Base
40configuration metadata consists of your project's ``bblayers.conf`` file
41to determine what layers BitBake needs to recognize, all necessary
42``layer.conf`` files (one from each layer), and ``bitbake.conf``. The
43data itself is of various types:
44
45- *Recipes:* Details about particular pieces of software.
46
47- *Class Data:* An abstraction of common build information (e.g. how to
48 build a Linux kernel).
49
50- *Configuration Data:* Machine-specific settings, policy decisions,
51 and so forth. Configuration data acts as the glue to bind everything
52 together.
53
54The ``layer.conf`` files are used to construct key variables such as
55```BBPATH`` <#var-bb-BBPATH>`__ and ```BBFILES`` <#var-bb-BBFILES>`__.
56``BBPATH`` is used to search for configuration and class files under the
57``conf`` and ``classes`` directories, respectively. ``BBFILES`` is used
58to locate both recipe and recipe append files (``.bb`` and
59``.bbappend``). If there is no ``bblayers.conf`` file, it is assumed the
60user has set the ``BBPATH`` and ``BBFILES`` directly in the environment.
61
62Next, the ``bitbake.conf`` file is located using the ``BBPATH`` variable
63that was just constructed. The ``bitbake.conf`` file may also include
64other configuration files using the ``include`` or ``require``
65directives.
66
67Prior to parsing configuration files, BitBake looks at certain
68variables, including:
69
70- ```BB_ENV_WHITELIST`` <#var-bb-BB_ENV_WHITELIST>`__
71
72- ```BB_ENV_EXTRAWHITE`` <#var-bb-BB_ENV_EXTRAWHITE>`__
73
74- ```BB_PRESERVE_ENV`` <#var-bb-BB_PRESERVE_ENV>`__
75
76- ```BB_ORIGENV`` <#var-bb-BB_ORIGENV>`__
77
78- ```BITBAKE_UI`` <#var-bb-BITBAKE_UI>`__
79
80The first four variables in this list relate to how BitBake treats shell
81environment variables during task execution. By default, BitBake cleans
82the environment variables and provides tight control over the shell
83execution environment. However, through the use of these first four
84variables, you can apply your control regarding the environment
85variables allowed to be used by BitBake in the shell during execution of
86tasks. See the "`Passing Information Into the Build Task
87Environment <#passing-information-into-the-build-task-environment>`__"
88section and the information about these variables in the variable
89glossary for more information on how they work and on how to use them.
90
91The base configuration metadata is global and therefore affects all
92recipes and tasks that are executed.
93
94BitBake first searches the current working directory for an optional
95``conf/bblayers.conf`` configuration file. This file is expected to
96contain a ```BBLAYERS`` <#var-bb-BBLAYERS>`__ variable that is a
97space-delimited list of 'layer' directories. Recall that if BitBake
98cannot find a ``bblayers.conf`` file, then it is assumed the user has
99set the ``BBPATH`` and ``BBFILES`` variables directly in the
100environment.
101
102For each directory (layer) in this list, a ``conf/layer.conf`` file is
103located and parsed with the ```LAYERDIR`` <#var-bb-LAYERDIR>`__ variable
104being set to the directory where the layer was found. The idea is these
105files automatically set up ```BBPATH`` <#var-bb-BBPATH>`__ and other
106variables correctly for a given build directory.
107
108BitBake then expects to find the ``conf/bitbake.conf`` file somewhere in
109the user-specified ``BBPATH``. That configuration file generally has
110include directives to pull in any other metadata such as files specific
111to the architecture, the machine, the local environment, and so forth.
112
113Only variable definitions and include directives are allowed in BitBake
114``.conf`` files. Some variables directly influence BitBake's behavior.
115These variables might have been set from the environment depending on
116the environment variables previously mentioned or set in the
117configuration files. The "`Variables
118Glossary <#ref-bb-variables-glos>`__" chapter presents a full list of
119variables.
120
121After parsing configuration files, BitBake uses its rudimentary
122inheritance mechanism, which is through class files, to inherit some
123standard classes. BitBake parses a class when the inherit directive
124responsible for getting that class is encountered.
125
126The ``base.bbclass`` file is always included. Other classes that are
127specified in the configuration using the
128```INHERIT`` <#var-bb-INHERIT>`__ variable are also included. BitBake
129searches for class files in a ``classes`` subdirectory under the paths
130in ``BBPATH`` in the same way as configuration files.
131
132A good way to get an idea of the configuration files and the class files
133used in your execution environment is to run the following BitBake
134command: $ bitbake -e > mybb.log Examining the top of the ``mybb.log``
135shows you the many configuration files and class files used in your
136execution environment.
137
138.. note::
139
140 You need to be aware of how BitBake parses curly braces. If a recipe
141 uses a closing curly brace within the function and the character has
142 no leading spaces, BitBake produces a parsing error. If you use a
143 pair of curly braces in a shell function, the closing curly brace
144 must not be located at the start of the line without leading spaces.
145
146 Here is an example that causes BitBake to produce a parsing error:
147 fakeroot create_shar() { cat << "EOF" >
148 ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh usage() { echo "test" ######
149 The following "}" at the start of the line causes a parsing error
150 ###### } EOF } Writing the recipe this way avoids the error: fakeroot
151 create_shar() { cat << "EOF" >
152 ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh usage() { echo "test"
153 ######The following "}" with a leading space at the start of the line
154 avoids the error ###### } EOF }
155
156Locating and Parsing Recipes
157============================
158
159During the configuration phase, BitBake will have set
160```BBFILES`` <#var-bb-BBFILES>`__. BitBake now uses it to construct a
161list of recipes to parse, along with any append files (``.bbappend``) to
162apply. ``BBFILES`` is a space-separated list of available files and
163supports wildcards. An example would be: BBFILES =
164"/path/to/bbfiles/*.bb /path/to/appends/*.bbappend" BitBake parses each
165recipe and append file located with ``BBFILES`` and stores the values of
166various variables into the datastore.
167
168.. note::
169
170 Append files are applied in the order they are encountered in
171 BBFILES
172 .
173
174For each file, a fresh copy of the base configuration is made, then the
175recipe is parsed line by line. Any inherit statements cause BitBake to
176find and then parse class files (``.bbclass``) using
177```BBPATH`` <#var-bb-BBPATH>`__ as the search path. Finally, BitBake
178parses in order any append files found in ``BBFILES``.
179
180One common convention is to use the recipe filename to define pieces of
181metadata. For example, in ``bitbake.conf`` the recipe name and version
182are used to set the variables ```PN`` <#var-bb-PN>`__ and
183```PV`` <#var-bb-PV>`__: PN =
184"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or
185'defaultpkgname'}" PV =
186"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or
187'1.0'}" In this example, a recipe called "something_1.2.3.bb" would set
188``PN`` to "something" and ``PV`` to "1.2.3".
189
190By the time parsing is complete for a recipe, BitBake has a list of
191tasks that the recipe defines and a set of data consisting of keys and
192values as well as dependency information about the tasks.
193
194BitBake does not need all of this information. It only needs a small
195subset of the information to make decisions about the recipe.
196Consequently, BitBake caches the values in which it is interested and
197does not store the rest of the information. Experience has shown it is
198faster to re-parse the metadata than to try and write it out to the disk
199and then reload it.
200
201Where possible, subsequent BitBake commands reuse this cache of recipe
202information. The validity of this cache is determined by first computing
203a checksum of the base configuration data (see
204```BB_HASHCONFIG_WHITELIST`` <#var-bb-BB_HASHCONFIG_WHITELIST>`__) and
205then checking if the checksum matches. If that checksum matches what is
206in the cache and the recipe and class files have not changed, BitBake is
207able to use the cache. BitBake then reloads the cached information about
208the recipe instead of reparsing it from scratch.
209
210Recipe file collections exist to allow the user to have multiple
211repositories of ``.bb`` files that contain the same exact package. For
212example, one could easily use them to make one's own local copy of an
213upstream repository, but with custom modifications that one does not
214want upstream. Here is an example: BBFILES = "/stuff/openembedded/*/*.bb
215/stuff/openembedded.modified/*/*.bb" BBFILE_COLLECTIONS = "upstream
216local" BBFILE_PATTERN_upstream = "^/stuff/openembedded/"
217BBFILE_PATTERN_local = "^/stuff/openembedded.modified/"
218BBFILE_PRIORITY_upstream = "5" BBFILE_PRIORITY_local = "10"
219
220.. note::
221
222 The layers mechanism is now the preferred method of collecting code.
223 While the collections code remains, its main use is to set layer
224 priorities and to deal with overlap (conflicts) between layers.
225
226.. _bb-bitbake-providers:
227
228Providers
229=========
230
231Assuming BitBake has been instructed to execute a target and that all
232the recipe files have been parsed, BitBake starts to figure out how to
233build the target. BitBake looks through the ``PROVIDES`` list for each
234of the recipes. A ``PROVIDES`` list is the list of names by which the
235recipe can be known. Each recipe's ``PROVIDES`` list is created
236implicitly through the recipe's ```PN`` <#var-bb-PN>`__ variable and
237explicitly through the recipe's ```PROVIDES`` <#var-bb-PROVIDES>`__
238variable, which is optional.
239
240When a recipe uses ``PROVIDES``, that recipe's functionality can be
241found under an alternative name or names other than the implicit ``PN``
242name. As an example, suppose a recipe named ``keyboard_1.0.bb``
243contained the following: PROVIDES += "fullkeyboard" The ``PROVIDES``
244list for this recipe becomes "keyboard", which is implicit, and
245"fullkeyboard", which is explicit. Consequently, the functionality found
246in ``keyboard_1.0.bb`` can be found under two different names.
247
248.. _bb-bitbake-preferences:
249
250Preferences
251===========
252
253The ``PROVIDES`` list is only part of the solution for figuring out a
254target's recipes. Because targets might have multiple providers, BitBake
255needs to prioritize providers by determining provider preferences.
256
257A common example in which a target has multiple providers is
258"virtual/kernel", which is on the ``PROVIDES`` list for each kernel
259recipe. Each machine often selects the best kernel provider by using a
260line similar to the following in the machine configuration file:
261PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" The default
262```PREFERRED_PROVIDER`` <#var-bb-PREFERRED_PROVIDER>`__ is the provider
263with the same name as the target. BitBake iterates through each target
264it needs to build and resolves them and their dependencies using this
265process.
266
267Understanding how providers are chosen is made complicated by the fact
268that multiple versions might exist for a given provider. BitBake
269defaults to the highest version of a provider. Version comparisons are
270made using the same method as Debian. You can use the
271```PREFERRED_VERSION`` <#var-bb-PREFERRED_VERSION>`__ variable to
272specify a particular version. You can influence the order by using the
273```DEFAULT_PREFERENCE`` <#var-bb-DEFAULT_PREFERENCE>`__ variable.
274
275By default, files have a preference of "0". Setting
276``DEFAULT_PREFERENCE`` to "-1" makes the recipe unlikely to be used
277unless it is explicitly referenced. Setting ``DEFAULT_PREFERENCE`` to
278"1" makes it likely the recipe is used. ``PREFERRED_VERSION`` overrides
279any ``DEFAULT_PREFERENCE`` setting. ``DEFAULT_PREFERENCE`` is often used
280to mark newer and more experimental recipe versions until they have
281undergone sufficient testing to be considered stable.
282
283When there are multiple “versions” of a given recipe, BitBake defaults
284to selecting the most recent version, unless otherwise specified. If the
285recipe in question has a
286```DEFAULT_PREFERENCE`` <#var-bb-DEFAULT_PREFERENCE>`__ set lower than
287the other recipes (default is 0), then it will not be selected. This
288allows the person or persons maintaining the repository of recipe files
289to specify their preference for the default selected version.
290Additionally, the user can specify their preferred version.
291
292If the first recipe is named ``a_1.1.bb``, then the
293```PN`` <#var-bb-PN>`__ variable will be set to “a”, and the
294```PV`` <#var-bb-PV>`__ variable will be set to 1.1.
295
296Thus, if a recipe named ``a_1.2.bb`` exists, BitBake will choose 1.2 by
297default. However, if you define the following variable in a ``.conf``
298file that BitBake parses, you can change that preference:
299PREFERRED_VERSION_a = "1.1"
300
301.. note::
302
303 It is common for a recipe to provide two versions -- a stable,
304 numbered (and preferred) version, and a version that is automatically
305 checked out from a source code repository that is considered more
306 "bleeding edge" but can be selected only explicitly.
307
308 For example, in the OpenEmbedded codebase, there is a standard,
309 versioned recipe file for BusyBox, ``busybox_1.22.1.bb``, but there
310 is also a Git-based version, ``busybox_git.bb``, which explicitly
311 contains the line DEFAULT_PREFERENCE = "-1" to ensure that the
312 numbered, stable version is always preferred unless the developer
313 selects otherwise.
314
315.. _bb-bitbake-dependencies:
316
317Dependencies
318============
319
320Each target BitBake builds consists of multiple tasks such as ``fetch``,
321``unpack``, ``patch``, ``configure``, and ``compile``. For best
322performance on multi-core systems, BitBake considers each task as an
323independent entity with its own set of dependencies.
324
325Dependencies are defined through several variables. You can find
326information about variables BitBake uses in the `Variables
327Glossary <#ref-bb-variables-glos>`__ near the end of this manual. At a
328basic level, it is sufficient to know that BitBake uses the
329```DEPENDS`` <#var-bb-DEPENDS>`__ and
330```RDEPENDS`` <#var-bb-RDEPENDS>`__ variables when calculating
331dependencies.
332
333For more information on how BitBake handles dependencies, see the
334"`Dependencies <#dependencies>`__" section.
335
336.. _ref-bitbake-tasklist:
337
338The Task List
339=============
340
341Based on the generated list of providers and the dependency information,
342BitBake can now calculate exactly what tasks it needs to run and in what
343order it needs to run them. The "`Executing Tasks <#executing-tasks>`__"
344section has more information on how BitBake chooses which task to
345execute next.
346
347The build now starts with BitBake forking off threads up to the limit
348set in the ```BB_NUMBER_THREADS`` <#var-bb-BB_NUMBER_THREADS>`__
349variable. BitBake continues to fork threads as long as there are tasks
350ready to run, those tasks have all their dependencies met, and the
351thread threshold has not been exceeded.
352
353It is worth noting that you can greatly speed up the build time by
354properly setting the ``BB_NUMBER_THREADS`` variable.
355
356As each task completes, a timestamp is written to the directory
357specified by the ```STAMP`` <#var-bb-STAMP>`__ variable. On subsequent
358runs, BitBake looks in the build directory within ``tmp/stamps`` and
359does not rerun tasks that are already completed unless a timestamp is
360found to be invalid. Currently, invalid timestamps are only considered
361on a per recipe file basis. So, for example, if the configure stamp has
362a timestamp greater than the compile timestamp for a given target, then
363the compile task would rerun. Running the compile task again, however,
364has no effect on other providers that depend on that target.
365
366The exact format of the stamps is partly configurable. In modern
367versions of BitBake, a hash is appended to the stamp so that if the
368configuration changes, the stamp becomes invalid and the task is
369automatically rerun. This hash, or signature used, is governed by the
370signature policy that is configured (see the "`Checksums
371(Signatures) <#checksums>`__" section for information). It is also
372possible to append extra metadata to the stamp using the
373``[stamp-extra-info]`` task flag. For example, OpenEmbedded uses this
374flag to make some tasks machine-specific.
375
376.. note::
377
378 Some tasks are marked as "nostamp" tasks. No timestamp file is
379 created when these tasks are run. Consequently, "nostamp" tasks are
380 always rerun.
381
382For more information on tasks, see the "`Tasks <#tasks>`__" section.
383
384Executing Tasks
385===============
386
387Tasks can be either a shell task or a Python task. For shell tasks,
388BitBake writes a shell script to
389``${``\ ```T`` <#var-bb-T>`__\ ``}/run.do_taskname.pid`` and then
390executes the script. The generated shell script contains all the
391exported variables, and the shell functions with all variables expanded.
392Output from the shell script goes to the file
393``${T}/log.do_taskname.pid``. Looking at the expanded shell functions in
394the run file and the output in the log files is a useful debugging
395technique.
396
397For Python tasks, BitBake executes the task internally and logs
398information to the controlling terminal. Future versions of BitBake will
399write the functions to files similar to the way shell tasks are handled.
400Logging will be handled in a way similar to shell tasks as well.
401
402The order in which BitBake runs the tasks is controlled by its task
403scheduler. It is possible to configure the scheduler and define custom
404implementations for specific use cases. For more information, see these
405variables that control the behavior:
406
407- ```BB_SCHEDULER`` <#var-bb-BB_SCHEDULER>`__
408
409- ```BB_SCHEDULERS`` <#var-bb-BB_SCHEDULERS>`__
410
411It is possible to have functions run before and after a task's main
412function. This is done using the ``[prefuncs]`` and ``[postfuncs]``
413flags of the task that lists the functions to run.
414
415.. _checksums:
416
417Checksums (Signatures)
418======================
419
420A checksum is a unique signature of a task's inputs. The signature of a
421task can be used to determine if a task needs to be run. Because it is a
422change in a task's inputs that triggers running the task, BitBake needs
423to detect all the inputs to a given task. For shell tasks, this turns
424out to be fairly easy because BitBake generates a "run" shell script for
425each task and it is possible to create a checksum that gives you a good
426idea of when the task's data changes.
427
428To complicate the problem, some things should not be included in the
429checksum. First, there is the actual specific build path of a given task
430- the working directory. It does not matter if the working directory
431changes because it should not affect the output for target packages. The
432simplistic approach for excluding the working directory is to set it to
433some fixed value and create the checksum for the "run" script. BitBake
434goes one step better and uses the
435```BB_HASHBASE_WHITELIST`` <#var-bb-BB_HASHBASE_WHITELIST>`__ variable
436to define a list of variables that should never be included when
437generating the signatures.
438
439Another problem results from the "run" scripts containing functions that
440might or might not get called. The incremental build solution contains
441code that figures out dependencies between shell functions. This code is
442used to prune the "run" scripts down to the minimum set, thereby
443alleviating this problem and making the "run" scripts much more readable
444as a bonus.
445
446So far we have solutions for shell scripts. What about Python tasks? The
447same approach applies even though these tasks are more difficult. The
448process needs to figure out what variables a Python function accesses
449and what functions it calls. Again, the incremental build solution
450contains code that first figures out the variable and function
451dependencies, and then creates a checksum for the data used as the input
452to the task.
453
454Like the working directory case, situations exist where dependencies
455should be ignored. For these cases, you can instruct the build process
456to ignore a dependency by using a line like the following:
457PACKAGE_ARCHS[vardepsexclude] = "MACHINE" This example ensures that the
458``PACKAGE_ARCHS`` variable does not depend on the value of ``MACHINE``,
459even if it does reference it.
460
461Equally, there are cases where we need to add dependencies BitBake is
462not able to find. You can accomplish this by using a line like the
463following: PACKAGE_ARCHS[vardeps] = "MACHINE" This example explicitly
464adds the ``MACHINE`` variable as a dependency for ``PACKAGE_ARCHS``.
465
466Consider a case with in-line Python, for example, where BitBake is not
467able to figure out dependencies. When running in debug mode (i.e. using
468``-DDD``), BitBake produces output when it discovers something for which
469it cannot figure out dependencies.
470
471Thus far, this section has limited discussion to the direct inputs into
472a task. Information based on direct inputs is referred to as the
473"basehash" in the code. However, there is still the question of a task's
474indirect inputs - the things that were already built and present in the
475build directory. The checksum (or signature) for a particular task needs
476to add the hashes of all the tasks on which the particular task depends.
477Choosing which dependencies to add is a policy decision. However, the
478effect is to generate a master checksum that combines the basehash and
479the hashes of the task's dependencies.
480
481At the code level, there are a variety of ways both the basehash and the
482dependent task hashes can be influenced. Within the BitBake
483configuration file, we can give BitBake some extra information to help
484it construct the basehash. The following statement effectively results
485in a list of global variable dependency excludes - variables never
486included in any checksum. This example uses variables from OpenEmbedded
487to help illustrate the concept: BB_HASHBASE_WHITELIST ?= "TMPDIR FILE
488PATH PWD BB_TASKHASH BBPATH DL_DIR \\ SSTATE_DIR THISDIR FILESEXTRAPATHS
489FILE_DIRNAME HOME LOGNAME SHELL \\ USER FILESPATH STAGING_DIR_HOST
490STAGING_DIR_TARGET COREBASE PRSERV_HOST \\ PRSERV_DUMPDIR
491PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \\ CCACHE_DIR
492EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX" The
493previous example excludes the work directory, which is part of
494``TMPDIR``.
495
496The rules for deciding which hashes of dependent tasks to include
497through dependency chains are more complex and are generally
498accomplished with a Python function. The code in
499``meta/lib/oe/sstatesig.py`` shows two examples of this and also
500illustrates how you can insert your own policy into the system if so
501desired. This file defines the two basic signature generators
502OpenEmbedded-Core uses: "OEBasic" and "OEBasicHash". By default, there
503is a dummy "noop" signature handler enabled in BitBake. This means that
504behavior is unchanged from previous versions. ``OE-Core`` uses the
505"OEBasicHash" signature handler by default through this setting in the
506``bitbake.conf`` file: BB_SIGNATURE_HANDLER ?= "OEBasicHash" The
507"OEBasicHash" ``BB_SIGNATURE_HANDLER`` is the same as the "OEBasic"
508version but adds the task hash to the stamp files. This results in any
509metadata change that changes the task hash, automatically causing the
510task to be run again. This removes the need to bump
511```PR`` <#var-bb-PR>`__ values, and changes to metadata automatically
512ripple across the build.
513
514It is also worth noting that the end result of these signature
515generators is to make some dependency and hash information available to
516the build. This information includes:
517
518- ``BB_BASEHASH_task-``\ taskname: The base hashes for each task in the
519 recipe.
520
521- ``BB_BASEHASH_``\ filename\ ``:``\ taskname: The base hashes for each
522 dependent task.
523
524- ``BBHASHDEPS_``\ filename\ ``:``\ taskname: The task dependencies for
525 each task.
526
527- ``BB_TASKHASH``: The hash of the currently running task.
528
529It is worth noting that BitBake's "-S" option lets you debug BitBake's
530processing of signatures. The options passed to -S allow different
531debugging modes to be used, either using BitBake's own debug functions
532or possibly those defined in the metadata/signature handler itself. The
533simplest parameter to pass is "none", which causes a set of signature
534information to be written out into ``STAMPS_DIR`` corresponding to the
535targets specified. The other currently available parameter is
536"printdiff", which causes BitBake to try to establish the closest
537signature match it can (e.g. in the sstate cache) and then run
538``bitbake-diffsigs`` over the matches to determine the stamps and delta
539where these two stamp trees diverge.
540
541.. note::
542
543 It is likely that future versions of BitBake will provide other
544 signature handlers triggered through additional "-S" parameters.
545
546You can find more information on checksum metadata in the "`Task
547Checksums and Setscene <#task-checksums-and-setscene>`__" section.
548
549Setscene
550========
551
552The setscene process enables BitBake to handle "pre-built" artifacts.
553The ability to handle and reuse these artifacts allows BitBake the
554luxury of not having to build something from scratch every time.
555Instead, BitBake can use, when possible, existing build artifacts.
556
557BitBake needs to have reliable data indicating whether or not an
558artifact is compatible. Signatures, described in the previous section,
559provide an ideal way of representing whether an artifact is compatible.
560If a signature is the same, an object can be reused.
561
562If an object can be reused, the problem then becomes how to replace a
563given task or set of tasks with the pre-built artifact. BitBake solves
564the problem with the "setscene" process.
565
566When BitBake is asked to build a given target, before building anything,
567it first asks whether cached information is available for any of the
568targets it's building, or any of the intermediate targets. If cached
569information is available, BitBake uses this information instead of
570running the main tasks.
571
572BitBake first calls the function defined by the
573```BB_HASHCHECK_FUNCTION`` <#var-bb-BB_HASHCHECK_FUNCTION>`__ variable
574with a list of tasks and corresponding hashes it wants to build. This
575function is designed to be fast and returns a list of the tasks for
576which it believes in can obtain artifacts.
577
578Next, for each of the tasks that were returned as possibilities, BitBake
579executes a setscene version of the task that the possible artifact
580covers. Setscene versions of a task have the string "_setscene" appended
581to the task name. So, for example, the task with the name ``xxx`` has a
582setscene task named ``xxx_setscene``. The setscene version of the task
583executes and provides the necessary artifacts returning either success
584or failure.
585
586As previously mentioned, an artifact can cover more than one task. For
587example, it is pointless to obtain a compiler if you already have the
588compiled binary. To handle this, BitBake calls the
589```BB_SETSCENE_DEPVALID`` <#var-bb-BB_SETSCENE_DEPVALID>`__ function for
590each successful setscene task to know whether or not it needs to obtain
591the dependencies of that task.
592
593Finally, after all the setscene tasks have executed, BitBake calls the
594function listed in
595```BB_SETSCENE_VERIFY_FUNCTION2`` <#var-bb-BB_SETSCENE_VERIFY_FUNCTION2>`__
596with the list of tasks BitBake thinks has been "covered". The metadata
597can then ensure that this list is correct and can inform BitBake that it
598wants specific tasks to be run regardless of the setscene result.
599
600You can find more information on setscene metadata in the "`Task
601Checksums and Setscene <#task-checksums-and-setscene>`__" section.
602
603Logging
604=======
605
606In addition to the standard command line option to control how verbose
607builds are when execute, bitbake also supports user defined
608configuration of the `Python
609logging <https://docs.python.org/3/library/logging.html>`__ facilities
610through the ```BB_LOGCONFIG`` <#var-bb-BB_LOGCONFIG>`__ variable. This
611variable defines a json or yaml `logging
612configuration <https://docs.python.org/3/library/logging.config.html>`__
613that will be intelligently merged into the default configuration. The
614logging configuration is merged using the following rules:
615
616- The user defined configuration will completely replace the default
617 configuration if top level key ``bitbake_merge`` is set to the value
618 ``False``. In this case, all other rules are ignored.
619
620- The user configuration must have a top level ``version`` which must
621 match the value of the default configuration.
622
623- Any keys defined in the ``handlers``, ``formatters``, or ``filters``,
624 will be merged into the same section in the default configuration,
625 with the user specified keys taking replacing a default one if there
626 is a conflict. In practice, this means that if both the default
627 configuration and user configuration specify a handler named
628 ``myhandler``, the user defined one will replace the default. To
629 prevent the user from inadvertently replacing a default handler,
630 formatter, or filter, all of the default ones are named with a prefix
631 of "``BitBake.``"
632
633- If a logger is defined by the user with the key ``bitbake_merge`` set
634 to ``False``, that logger will be completely replaced by user
635 configuration. In this case, no other rules will apply to that
636 logger.
637
638- All user defined ``filter`` and ``handlers`` properties for a given
639 logger will be merged with corresponding properties from the default
640 logger. For example, if the user configuration adds a filter called
641 ``myFilter`` to the ``BitBake.SigGen``, and the default configuration
642 adds a filter called ``BitBake.defaultFilter``, both filters will be
643 applied to the logger
644
645As an example, consider the following user logging configuration file
646which logs all Hash Equivalence related messages of VERBOSE or higher to
647a file called ``hashequiv.log`` { "version": 1, "handlers": {
648"autobuilderlog": { "class": "logging.FileHandler", "formatter":
649"logfileFormatter", "level": "DEBUG", "filename": "hashequiv.log",
650"mode": "w" } }, "formatters": { "logfileFormatter": { "format":
651"%(name)s: %(levelname)s: %(message)s" } }, "loggers": {
652"BitBake.SigGen.HashEquiv": { "level": "VERBOSE", "handlers":
653["autobuilderlog"] }, "BitBake.RunQueue.HashEquiv": { "level":
654"VERBOSE", "handlers": ["autobuilderlog"] } } }
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
new file mode 100644
index 0000000000..4285a73afe
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -0,0 +1,629 @@
1=====================
2File Download Support
3=====================
4
5BitBake's fetch module is a standalone piece of library code that deals
6with the intricacies of downloading source code and files from remote
7systems. Fetching source code is one of the cornerstones of building
8software. As such, this module forms an important part of BitBake.
9
10The current fetch module is called "fetch2" and refers to the fact that
11it is the second major version of the API. The original version is
12obsolete and has been removed from the codebase. Thus, in all cases,
13"fetch" refers to "fetch2" in this manual.
14
15The Download (Fetch)
16====================
17
18BitBake takes several steps when fetching source code or files. The
19fetcher codebase deals with two distinct processes in order: obtaining
20the files from somewhere (cached or otherwise) and then unpacking those
21files into a specific location and perhaps in a specific way. Getting
22and unpacking the files is often optionally followed by patching.
23Patching, however, is not covered by this module.
24
25The code to execute the first part of this process, a fetch, looks
26something like the following: src_uri = (d.getVar('SRC_URI') or
27"").split() fetcher = bb.fetch2.Fetch(src_uri, d) fetcher.download()
28This code sets up an instance of the fetch class. The instance uses a
29space-separated list of URLs from the ```SRC_URI`` <#var-bb-SRC_URI>`__
30variable and then calls the ``download`` method to download the files.
31
32The instantiation of the fetch class is usually followed by: rootdir =
33l.getVar('WORKDIR') fetcher.unpack(rootdir) This code unpacks the
34downloaded files to the specified by ``WORKDIR``.
35
36.. note::
37
38 For convenience, the naming in these examples matches the variables
39 used by OpenEmbedded. If you want to see the above code in action,
40 examine the OpenEmbedded class file
41 base.bbclass
42 .
43
44The ``SRC_URI`` and ``WORKDIR`` variables are not hardcoded into the
45fetcher, since those fetcher methods can be (and are) called with
46different variable names. In OpenEmbedded for example, the shared state
47(sstate) code uses the fetch module to fetch the sstate files.
48
49When the ``download()`` method is called, BitBake tries to resolve the
50URLs by looking for source files in a specific search order:
51
52- *Pre-mirror Sites:* BitBake first uses pre-mirrors to try and find
53 source files. These locations are defined using the
54 ```PREMIRRORS`` <#var-bb-PREMIRRORS>`__ variable.
55
56- *Source URI:* If pre-mirrors fail, BitBake uses the original URL (e.g
57 from ``SRC_URI``).
58
59- *Mirror Sites:* If fetch failures occur, BitBake next uses mirror
60 locations as defined by the ```MIRRORS`` <#var-bb-MIRRORS>`__
61 variable.
62
63For each URL passed to the fetcher, the fetcher calls the submodule that
64handles that particular URL type. This behavior can be the source of
65some confusion when you are providing URLs for the ``SRC_URI`` variable.
66Consider the following two URLs:
67http://git.yoctoproject.org/git/poky;protocol=git
68git://git.yoctoproject.org/git/poky;protocol=http In the former case,
69the URL is passed to the ``wget`` fetcher, which does not understand
70"git". Therefore, the latter case is the correct form since the Git
71fetcher does know how to use HTTP as a transport.
72
73Here are some examples that show commonly used mirror definitions:
74PREMIRRORS ?= "\\ bzr://.*/.\* http://somemirror.org/sources/ \\n \\
75cvs://.*/.\* http://somemirror.org/sources/ \\n \\ git://.*/.\*
76http://somemirror.org/sources/ \\n \\ hg://.*/.\*
77http://somemirror.org/sources/ \\n \\ osc://.*/.\*
78http://somemirror.org/sources/ \\n \\ p4://.*/.\*
79http://somemirror.org/sources/ \\n \\ svn://.*/.\*
80http://somemirror.org/sources/ \\n" MIRRORS =+ "\\ ftp://.*/.\*
81http://somemirror.org/sources/ \\n \\ http://.*/.\*
82http://somemirror.org/sources/ \\n \\ https://.*/.\*
83http://somemirror.org/sources/ \\n" It is useful to note that BitBake
84supports cross-URLs. It is possible to mirror a Git repository on an
85HTTP server as a tarball. This is what the ``git://`` mapping in the
86previous example does.
87
88Since network accesses are slow, BitBake maintains a cache of files
89downloaded from the network. Any source files that are not local (i.e.
90downloaded from the Internet) are placed into the download directory,
91which is specified by the ```DL_DIR`` <#var-bb-DL_DIR>`__ variable.
92
93File integrity is of key importance for reproducing builds. For
94non-local archive downloads, the fetcher code can verify SHA-256 and MD5
95checksums to ensure the archives have been downloaded correctly. You can
96specify these checksums by using the ``SRC_URI`` variable with the
97appropriate varflags as follows: SRC_URI[md5sum] = "value"
98SRC_URI[sha256sum] = "value" You can also specify the checksums as
99parameters on the ``SRC_URI`` as shown below: SRC_URI =
100"http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
101If multiple URIs exist, you can specify the checksums either directly as
102in the previous example, or you can name the URLs. The following syntax
103shows how you name the URIs: SRC_URI =
104"http://example.com/foobar.tar.bz2;name=foo" SRC_URI[foo.md5sum] =
1054a8e0f237e961fd7785d19d07fdb994d After a file has been downloaded and
106has had its checksum checked, a ".done" stamp is placed in ``DL_DIR``.
107BitBake uses this stamp during subsequent builds to avoid downloading or
108comparing a checksum for the file again.
109
110.. note::
111
112 It is assumed that local storage is safe from data corruption. If
113 this were not the case, there would be bigger issues to worry about.
114
115If ```BB_STRICT_CHECKSUM`` <#var-bb-BB_STRICT_CHECKSUM>`__ is set, any
116download without a checksum triggers an error message. The
117```BB_NO_NETWORK`` <#var-bb-BB_NO_NETWORK>`__ variable can be used to
118make any attempted network access a fatal error, which is useful for
119checking that mirrors are complete as well as other things.
120
121.. _bb-the-unpack:
122
123The Unpack
124==========
125
126The unpack process usually immediately follows the download. For all
127URLs except Git URLs, BitBake uses the common ``unpack`` method.
128
129A number of parameters exist that you can specify within the URL to
130govern the behavior of the unpack stage:
131
132- *unpack:* Controls whether the URL components are unpacked. If set to
133 "1", which is the default, the components are unpacked. If set to
134 "0", the unpack stage leaves the file alone. This parameter is useful
135 when you want an archive to be copied in and not be unpacked.
136
137- *dos:* Applies to ``.zip`` and ``.jar`` files and specifies whether
138 to use DOS line ending conversion on text files.
139
140- *basepath:* Instructs the unpack stage to strip the specified
141 directories from the source path when unpacking.
142
143- *subdir:* Unpacks the specific URL to the specified subdirectory
144 within the root directory.
145
146The unpack call automatically decompresses and extracts files with ".Z",
147".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". ".srpm", ".deb" and
148".bz2" extensions as well as various combinations of tarball extensions.
149
150As mentioned, the Git fetcher has its own unpack method that is
151optimized to work with Git trees. Basically, this method works by
152cloning the tree into the final directory. The process is completed
153using references so that there is only one central copy of the Git
154metadata needed.
155
156.. _bb-fetchers:
157
158Fetchers
159========
160
161As mentioned earlier, the URL prefix determines which fetcher submodule
162BitBake uses. Each submodule can support different URL parameters, which
163are described in the following sections.
164
165.. _local-file-fetcher:
166
167Local file fetcher (``file://``)
168--------------------------------
169
170This submodule handles URLs that begin with ``file://``. The filename
171you specify within the URL can be either an absolute or relative path to
172a file. If the filename is relative, the contents of the
173```FILESPATH`` <#var-bb-FILESPATH>`__ variable is used in the same way
174``PATH`` is used to find executables. If the file cannot be found, it is
175assumed that it is available in ```DL_DIR`` <#var-bb-DL_DIR>`__ by the
176time the ``download()`` method is called.
177
178If you specify a directory, the entire directory is unpacked.
179
180Here are a couple of example URLs, the first relative and the second
181absolute: SRC_URI = "file://relativefile.patch" SRC_URI =
182"file:///Users/ich/very_important_software"
183
184.. _http-ftp-fetcher:
185
186HTTP/FTP wget fetcher (``http://``, ``ftp://``, ``https://``)
187-------------------------------------------------------------
188
189This fetcher obtains files from web and FTP servers. Internally, the
190fetcher uses the wget utility.
191
192The executable and parameters used are specified by the
193``FETCHCMD_wget`` variable, which defaults to sensible values. The
194fetcher supports a parameter "downloadfilename" that allows the name of
195the downloaded file to be specified. Specifying the name of the
196downloaded file is useful for avoiding collisions in
197```DL_DIR`` <#var-bb-DL_DIR>`__ when dealing with multiple files that
198have the same name.
199
200Some example URLs are as follows: SRC_URI =
201"http://oe.handhelds.org/not_there.aac" SRC_URI =
202"ftp://oe.handhelds.org/not_there_as_well.aac" SRC_URI =
203"ftp://you@oe.handhelds.org/home/you/secret.plan"
204
205.. note::
206
207 Because URL parameters are delimited by semi-colons, this can
208 introduce ambiguity when parsing URLs that also contain semi-colons,
209 for example:
210 ::
211
212 SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47"
213
214
215 Such URLs should should be modified by replacing semi-colons with '&'
216 characters:
217 ::
218
219 SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47"
220
221
222 In most cases this should work. Treating semi-colons and '&' in
223 queries identically is recommended by the World Wide Web Consortium
224 (W3C). Note that due to the nature of the URL, you may have to
225 specify the name of the downloaded file as well:
226 ::
227
228 SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47;downloadfilename=myfile.bz2"
229
230
231.. _cvs-fetcher:
232
233CVS fetcher (``(cvs://``)
234-------------------------
235
236This submodule handles checking out files from the CVS version control
237system. You can configure it using a number of different variables:
238
239- *``FETCHCMD_cvs``:* The name of the executable to use when running
240 the ``cvs`` command. This name is usually "cvs".
241
242- *``SRCDATE``:* The date to use when fetching the CVS source code. A
243 special value of "now" causes the checkout to be updated on every
244 build.
245
246- ```CVSDIR`` <#var-bb-CVSDIR>`__\ *:* Specifies where a temporary
247 checkout is saved. The location is often ``DL_DIR/cvs``.
248
249- *``CVS_PROXY_HOST``:* The name to use as a "proxy=" parameter to the
250 ``cvs`` command.
251
252- *``CVS_PROXY_PORT``:* The port number to use as a "proxyport="
253 parameter to the ``cvs`` command.
254
255As well as the standard username and password URL syntax, you can also
256configure the fetcher with various URL parameters:
257
258The supported parameters are as follows:
259
260- *"method":* The protocol over which to communicate with the CVS
261 server. By default, this protocol is "pserver". If "method" is set to
262 "ext", BitBake examines the "rsh" parameter and sets ``CVS_RSH``. You
263 can use "dir" for local directories.
264
265- *"module":* Specifies the module to check out. You must supply this
266 parameter.
267
268- *"tag":* Describes which CVS TAG should be used for the checkout. By
269 default, the TAG is empty.
270
271- *"date":* Specifies a date. If no "date" is specified, the
272 ```SRCDATE`` <#var-bb-SRCDATE>`__ of the configuration is used to
273 checkout a specific date. The special value of "now" causes the
274 checkout to be updated on every build.
275
276- *"localdir":* Used to rename the module. Effectively, you are
277 renaming the output directory to which the module is unpacked. You
278 are forcing the module into a special directory relative to
279 ```CVSDIR`` <#var-bb-CVSDIR>`__.
280
281- *"rsh"* Used in conjunction with the "method" parameter.
282
283- *"scmdata":* Causes the CVS metadata to be maintained in the tarball
284 the fetcher creates when set to "keep". The tarball is expanded into
285 the work directory. By default, the CVS metadata is removed.
286
287- *"fullpath":* Controls whether the resulting checkout is at the
288 module level, which is the default, or is at deeper paths.
289
290- *"norecurse":* Causes the fetcher to only checkout the specified
291 directory with no recurse into any subdirectories.
292
293- *"port":* The port to which the CVS server connects.
294
295Some example URLs are as follows: SRC_URI =
296"cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" SRC_URI =
297"cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
298
299.. _svn-fetcher:
300
301Subversion (SVN) Fetcher (``svn://``)
302-------------------------------------
303
304This fetcher submodule fetches code from the Subversion source control
305system. The executable used is specified by ``FETCHCMD_svn``, which
306defaults to "svn". The fetcher's temporary working directory is set by
307```SVNDIR`` <#var-bb-SVNDIR>`__, which is usually ``DL_DIR/svn``.
308
309The supported parameters are as follows:
310
311- *"module":* The name of the svn module to checkout. You must provide
312 this parameter. You can think of this parameter as the top-level
313 directory of the repository data you want.
314
315- *"path_spec":* A specific directory in which to checkout the
316 specified svn module.
317
318- *"protocol":* The protocol to use, which defaults to "svn". If
319 "protocol" is set to "svn+ssh", the "ssh" parameter is also used.
320
321- *"rev":* The revision of the source code to checkout.
322
323- *"scmdata":* Causes the “.svn” directories to be available during
324 compile-time when set to "keep". By default, these directories are
325 removed.
326
327- *"ssh":* An optional parameter used when "protocol" is set to
328 "svn+ssh". You can use this parameter to specify the ssh program used
329 by svn.
330
331- *"transportuser":* When required, sets the username for the
332 transport. By default, this parameter is empty. The transport
333 username is different than the username used in the main URL, which
334 is passed to the subversion command.
335
336Following are three examples using svn: SRC_URI =
337"svn://myrepos/proj1;module=vip;protocol=http;rev=667" SRC_URI =
338"svn://myrepos/proj1;module=opie;protocol=svn+ssh" SRC_URI =
339"svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${MY_DIR}/proj1"
340
341.. _git-fetcher:
342
343Git Fetcher (``git://``)
344------------------------
345
346This fetcher submodule fetches code from the Git source control system.
347The fetcher works by creating a bare clone of the remote into
348```GITDIR`` <#var-bb-GITDIR>`__, which is usually ``DL_DIR/git2``. This
349bare clone is then cloned into the work directory during the unpack
350stage when a specific tree is checked out. This is done using alternates
351and by reference to minimize the amount of duplicate data on the disk
352and make the unpack process fast. The executable used can be set with
353``FETCHCMD_git``.
354
355This fetcher supports the following parameters:
356
357- *"protocol":* The protocol used to fetch the files. The default is
358 "git" when a hostname is set. If a hostname is not set, the Git
359 protocol is "file". You can also use "http", "https", "ssh" and
360 "rsync".
361
362- *"nocheckout":* Tells the fetcher to not checkout source code when
363 unpacking when set to "1". Set this option for the URL where there is
364 a custom routine to checkout code. The default is "0".
365
366- *"rebaseable":* Indicates that the upstream Git repository can be
367 rebased. You should set this parameter to "1" if revisions can become
368 detached from branches. In this case, the source mirror tarball is
369 done per revision, which has a loss of efficiency. Rebasing the
370 upstream Git repository could cause the current revision to disappear
371 from the upstream repository. This option reminds the fetcher to
372 preserve the local cache carefully for future use. The default value
373 for this parameter is "0".
374
375- *"nobranch":* Tells the fetcher to not check the SHA validation for
376 the branch when set to "1". The default is "0". Set this option for
377 the recipe that refers to the commit that is valid for a tag instead
378 of the branch.
379
380- *"bareclone":* Tells the fetcher to clone a bare clone into the
381 destination directory without checking out a working tree. Only the
382 raw Git metadata is provided. This parameter implies the "nocheckout"
383 parameter as well.
384
385- *"branch":* The branch(es) of the Git tree to clone. If unset, this
386 is assumed to be "master". The number of branch parameters much match
387 the number of name parameters.
388
389- *"rev":* The revision to use for the checkout. The default is
390 "master".
391
392- *"tag":* Specifies a tag to use for the checkout. To correctly
393 resolve tags, BitBake must access the network. For that reason, tags
394 are often not used. As far as Git is concerned, the "tag" parameter
395 behaves effectively the same as the "rev" parameter.
396
397- *"subpath":* Limits the checkout to a specific subpath of the tree.
398 By default, the whole tree is checked out.
399
400- *"destsuffix":* The name of the path in which to place the checkout.
401 By default, the path is ``git/``.
402
403- *"usehead":* Enables local ``git://`` URLs to use the current branch
404 HEAD as the revision for use with ``AUTOREV``. The "usehead"
405 parameter implies no branch and only works when the transfer protocol
406 is ``file://``.
407
408Here are some example URLs: SRC_URI =
409"git://git.oe.handhelds.org/git/vip.git;tag=version-1" SRC_URI =
410"git://git.oe.handhelds.org/git/vip.git;protocol=http"
411
412.. _gitsm-fetcher:
413
414Git Submodule Fetcher (``gitsm://``)
415------------------------------------
416
417This fetcher submodule inherits from the `Git fetcher <#git-fetcher>`__
418and extends that fetcher's behavior by fetching a repository's
419submodules. ```SRC_URI`` <#var-bb-SRC_URI>`__ is passed to the Git
420fetcher as described in the "`Git Fetcher
421(``git://``) <#git-fetcher>`__" section.
422
423.. note::
424
425 You must clean a recipe when switching between '``git://``' and
426 '``gitsm://``' URLs.
427
428 The Git Submodules fetcher is not a complete fetcher implementation.
429 The fetcher has known issues where it does not use the normal source
430 mirroring infrastructure properly. Further, the submodule sources it
431 fetches are not visible to the licensing and source archiving
432 infrastructures.
433
434.. _clearcase-fetcher:
435
436ClearCase Fetcher (``ccrc://``)
437-------------------------------
438
439This fetcher submodule fetches code from a
440`ClearCase <http://en.wikipedia.org/wiki/Rational_ClearCase>`__
441repository.
442
443To use this fetcher, make sure your recipe has proper
444```SRC_URI`` <#var-bb-SRC_URI>`__, ```SRCREV`` <#var-bb-SRCREV>`__, and
445```PV`` <#var-bb-PV>`__ settings. Here is an example: SRC_URI =
446"ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
447SRCREV = "EXAMPLE_CLEARCASE_TAG" PV = "${@d.getVar("SRCREV",
448False).replace("/", "+")}" The fetcher uses the ``rcleartool`` or
449``cleartool`` remote client, depending on which one is available.
450
451Following are options for the ``SRC_URI`` statement:
452
453- *``vob``*: The name, which must include the prepending "/" character,
454 of the ClearCase VOB. This option is required.
455
456- *``module``*: The module, which must include the prepending "/"
457 character, in the selected VOB.
458
459 .. note::
460
461 The
462 module
463 and
464 vob
465 options are combined to create the
466 load
467 rule in the view config spec. As an example, consider the
468 vob
469 and
470 module
471 values from the
472 SRC_URI
473 statement at the start of this section. Combining those values
474 results in the following:
475 ::
476
477 load /example_vob/example_module
478
479
480- *``proto``*: The protocol, which can be either ``http`` or ``https``.
481
482By default, the fetcher creates a configuration specification. If you
483want this specification written to an area other than the default, use
484the ``CCASE_CUSTOM_CONFIG_SPEC`` variable in your recipe to define where
485the specification is written.
486
487.. note::
488
489 the
490 SRCREV
491 loses its functionality if you specify this variable. However,
492 SRCREV
493 is still used to label the archive after a fetch even though it does
494 not define what is fetched.
495
496Here are a couple of other behaviors worth mentioning:
497
498- When using ``cleartool``, the login of ``cleartool`` is handled by
499 the system. The login require no special steps.
500
501- In order to use ``rcleartool`` with authenticated users, an
502 "rcleartool login" is necessary before using the fetcher.
503
504.. _perforce-fetcher:
505
506Perforce Fetcher (``p4://``)
507----------------------------
508
509This fetcher submodule fetches code from the
510`Perforce <https://www.perforce.com/>`__ source control system. The
511executable used is specified by ``FETCHCMD_p4``, which defaults to "p4".
512The fetcher's temporary working directory is set by
513```P4DIR`` <#var-bb-P4DIR>`__, which defaults to "DL_DIR/p4".
514The fetcher does not make use of a perforce client, instead it
515relies on ``p4 files`` to retrieve a list of
516files and ``p4 print`` to transfer the content
517of those files locally.
518
519To use this fetcher, make sure your recipe has proper
520```SRC_URI`` <#var-bb-SRC_URI>`__, ```SRCREV`` <#var-bb-SRCREV>`__, and
521```PV`` <#var-bb-PV>`__ values. The p4 executable is able to use the
522config file defined by your system's ``P4CONFIG`` environment variable
523in order to define the Perforce server URL and port, username, and
524password if you do not wish to keep those values in a recipe itself. If
525you choose not to use ``P4CONFIG``, or to explicitly set variables that
526``P4CONFIG`` can contain, you can specify the ``P4PORT`` value, which is
527the server's URL and port number, and you can specify a username and
528password directly in your recipe within ``SRC_URI``.
529
530Here is an example that relies on ``P4CONFIG`` to specify the server URL
531and port, username, and password, and fetches the Head Revision: SRC_URI
532= "p4://example-depot/main/source/..." SRCREV = "${AUTOREV}" PV =
533"p4-${SRCPV}" S = "${WORKDIR}/p4"
534
535Here is an example that specifies the server URL and port, username, and
536password, and fetches a Revision based on a Label: P4PORT =
537"tcp:p4server.example.net:1666" SRC_URI =
538"p4://user:passwd@example-depot/main/source/..." SRCREV = "release-1.0"
539PV = "p4-${SRCPV}" S = "${WORKDIR}/p4"
540
541.. note::
542
543 You should always set
544 S
545 to
546 "${WORKDIR}/p4"
547 in your recipe.
548
549By default, the fetcher strips the depot location from the local file paths. In
550the above example, the content of ``example-depot/main/source/`` will be placed
551in ``${WORKDIR}/p4``. For situations where preserving parts of the remote depot
552paths locally is desirable, the fetcher supports two parameters:
553
554- **"module":**
555 The top-level depot location or directory to fetch. The value of this
556 parameter can also point to a single file within the depot, in which case
557 the local file path will include the module path.
558- **"remotepath":**
559 When used with the value "``keep``", the fetcher will mirror the full depot
560 paths locally for the specified location, even in combination with the
561 ``module`` parameter.
562
563Here is an example use of the the ``module`` parameter: ::
564
565 SRC_URI = "p4://user:passwd@example-depot/main;module=source/..."
566
567In this case, the content of the top-level directory ``source/`` will be fetched
568to ``${P4DIR}``, including the directory itself. The top-level directory will
569be accesible at ``${P4DIR}/source/``.
570
571Here is an example use of the the ``remotepath`` parameter: ::
572
573 SRC_URI = "p4://user:passwd@example-depot/main;module=source/...;remotepath=keep"
574
575In this case, the content of the top-level directory ``source/`` will be fetched
576to ``${P4DIR}``, but the complete depot paths will be mirrored locally. The
577top-level directory will be accessible at
578``${P4DIR}/example-depot/main/source/``.
579
580.. _repo-fetcher:
581
582Repo Fetcher (``repo://``)
583--------------------------
584
585This fetcher submodule fetches code from ``google-repo`` source control
586system. The fetcher works by initiating and syncing sources of the
587repository into ```REPODIR`` <#var-bb-REPODIR>`__, which is usually
588```DL_DIR`` <#var-bb-DL_DIR>`__\ ``/repo``.
589
590This fetcher supports the following parameters:
591
592- *"protocol":* Protocol to fetch the repository manifest (default:
593 git).
594
595- *"branch":* Branch or tag of repository to get (default: master).
596
597- *"manifest":* Name of the manifest file (default: ``default.xml``).
598
599Here are some example URLs: SRC_URI =
600"repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml"
601SRC_URI =
602"repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml"
603
604Other Fetchers
605--------------
606
607Fetch submodules also exist for the following:
608
609- Bazaar (``bzr://``)
610
611- Mercurial (``hg://``)
612
613- npm (``npm://``)
614
615- OSC (``osc://``)
616
617- Secure FTP (``sftp://``)
618
619- Secure Shell (``ssh://``)
620
621- Trees using Git Annex (``gitannex://``)
622
623No documentation currently exists for these lesser used fetcher
624submodules. However, you might find the code helpful and readable.
625
626Auto Revisions
627==============
628
629We need to document ``AUTOREV`` and ``SRCREV_FORMAT`` here.
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst
new file mode 100644
index 0000000000..174c797ae1
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst
@@ -0,0 +1,361 @@
1===================
2Hello World Example
3===================
4
5BitBake Hello World
6===================
7
8The simplest example commonly used to demonstrate any new programming
9language or tool is the "`Hello
10World <http://en.wikipedia.org/wiki/Hello_world_program>`__" example.
11This appendix demonstrates, in tutorial form, Hello World within the
12context of BitBake. The tutorial describes how to create a new project
13and the applicable metadata files necessary to allow BitBake to build
14it.
15
16Obtaining BitBake
17=================
18
19See the "`Obtaining BitBake <#obtaining-bitbake>`__" section for
20information on how to obtain BitBake. Once you have the source code on
21your machine, the BitBake directory appears as follows: $ ls -al total
22100 drwxrwxr-x. 9 wmat wmat 4096 Jan 31 13:44 . drwxrwxr-x. 3 wmat wmat
234096 Feb 4 10:45 .. -rw-rw-r--. 1 wmat wmat 365 Nov 26 04:55 AUTHORS
24drwxrwxr-x. 2 wmat wmat 4096 Nov 26 04:55 bin drwxrwxr-x. 4 wmat wmat
254096 Jan 31 13:44 build -rw-rw-r--. 1 wmat wmat 16501 Nov 26 04:55
26ChangeLog drwxrwxr-x. 2 wmat wmat 4096 Nov 26 04:55 classes drwxrwxr-x.
272 wmat wmat 4096 Nov 26 04:55 conf drwxrwxr-x. 3 wmat wmat 4096 Nov 26
2804:55 contrib -rw-rw-r--. 1 wmat wmat 17987 Nov 26 04:55 COPYING
29drwxrwxr-x. 3 wmat wmat 4096 Nov 26 04:55 doc -rw-rw-r--. 1 wmat wmat 69
30Nov 26 04:55 .gitignore -rw-rw-r--. 1 wmat wmat 849 Nov 26 04:55 HEADER
31drwxrwxr-x. 5 wmat wmat 4096 Jan 31 13:44 lib -rw-rw-r--. 1 wmat wmat
32195 Nov 26 04:55 MANIFEST.in -rw-rw-r--. 1 wmat wmat 2887 Nov 26 04:55
33TODO
34
35At this point, you should have BitBake cloned to a directory that
36matches the previous listing except for dates and user names.
37
38Setting Up the BitBake Environment
39==================================
40
41First, you need to be sure that you can run BitBake. Set your working
42directory to where your local BitBake files are and run the following
43command: $ ./bin/bitbake --version BitBake Build Tool Core version
441.23.0, bitbake version 1.23.0 The console output tells you what version
45you are running.
46
47The recommended method to run BitBake is from a directory of your
48choice. To be able to run BitBake from any directory, you need to add
49the executable binary to your binary to your shell's environment
50``PATH`` variable. First, look at your current ``PATH`` variable by
51entering the following: $ echo $PATH Next, add the directory location
52for the BitBake binary to the ``PATH``. Here is an example that adds the
53``/home/scott-lenovo/bitbake/bin`` directory to the front of the
54``PATH`` variable: $ export PATH=/home/scott-lenovo/bitbake/bin:$PATH
55You should now be able to enter the ``bitbake`` command from the command
56line while working from any directory.
57
58The Hello World Example
59=======================
60
61The overall goal of this exercise is to build a complete "Hello World"
62example utilizing task and layer concepts. Because this is how modern
63projects such as OpenEmbedded and the Yocto Project utilize BitBake, the
64example provides an excellent starting point for understanding BitBake.
65
66To help you understand how to use BitBake to build targets, the example
67starts with nothing but the ``bitbake`` command, which causes BitBake to
68fail and report problems. The example progresses by adding pieces to the
69build to eventually conclude with a working, minimal "Hello World"
70example.
71
72While every attempt is made to explain what is happening during the
73example, the descriptions cannot cover everything. You can find further
74information throughout this manual. Also, you can actively participate
75in the
76` <http://lists.openembedded.org/mailman/listinfo/bitbake-devel>`__
77discussion mailing list about the BitBake build tool.
78
79.. note::
80
81 This example was inspired by and drew heavily from
82 Mailing List post - The BitBake equivalent of "Hello, World!"
83 .
84
85As stated earlier, the goal of this example is to eventually compile
86"Hello World". However, it is unknown what BitBake needs and what you
87have to provide in order to achieve that goal. Recall that BitBake
88utilizes three types of metadata files: `Configuration
89Files <#configuration-files>`__, `Classes <#classes>`__, and
90`Recipes <#recipes>`__. But where do they go? How does BitBake find
91them? BitBake's error messaging helps you answer these types of
92questions and helps you better understand exactly what is going on.
93
94Following is the complete "Hello World" example.
95
961. *Create a Project Directory:* First, set up a directory for the
97 "Hello World" project. Here is how you can do so in your home
98 directory: $ mkdir ~/hello $ cd ~/hello This is the directory that
99 BitBake will use to do all of its work. You can use this directory
100 to keep all the metafiles needed by BitBake. Having a project
101 directory is a good way to isolate your project.
102
1032. *Run BitBake:* At this point, you have nothing but a project
104 directory. Run the ``bitbake`` command and see what it does: $
105 bitbake The BBPATH variable is not set and bitbake did not find a
106 conf/bblayers.conf file in the expected location. Maybe you
107 accidentally invoked bitbake from the wrong directory? DEBUG:
108 Removed the following variables from the environment:
109 GNOME_DESKTOP_SESSION_ID, XDG_CURRENT_DESKTOP,
110 GNOME_KEYRING_CONTROL, DISPLAY, SSH_AGENT_PID, LANG, no_proxy,
111 XDG_SESSION_PATH, XAUTHORITY, SESSION_MANAGER, SHLVL,
112 MANDATORY_PATH, COMPIZ_CONFIG_PROFILE, WINDOWID, EDITOR,
113 GPG_AGENT_INFO, SSH_AUTH_SOCK, GDMSESSION, GNOME_KEYRING_PID,
114 XDG_SEAT_PATH, XDG_CONFIG_DIRS, LESSOPEN, DBUS_SESSION_BUS_ADDRESS,
115 \_, XDG_SESSION_COOKIE, DESKTOP_SESSION, LESSCLOSE, DEFAULTS_PATH,
116 UBUNTU_MENUPROXY, OLDPWD, XDG_DATA_DIRS, COLORTERM, LS_COLORS The
117 majority of this output is specific to environment variables that
118 are not directly relevant to BitBake. However, the very first
119 message regarding the ``BBPATH`` variable and the
120 ``conf/bblayers.conf`` file is relevant.
121
122 When you run BitBake, it begins looking for metadata files. The
123 ```BBPATH`` <#var-bb-BBPATH>`__ variable is what tells BitBake where
124 to look for those files. ``BBPATH`` is not set and you need to set
125 it. Without ``BBPATH``, BitBake cannot find any configuration files
126 (``.conf``) or recipe files (``.bb``) at all. BitBake also cannot
127 find the ``bitbake.conf`` file.
128
1293. *Setting ``BBPATH``:* For this example, you can set ``BBPATH`` in
130 the same manner that you set ``PATH`` earlier in the appendix. You
131 should realize, though, that it is much more flexible to set the
132 ``BBPATH`` variable up in a configuration file for each project.
133
134 From your shell, enter the following commands to set and export the
135 ``BBPATH`` variable: $ BBPATH="projectdirectory" $ export BBPATH Use
136 your actual project directory in the command. BitBake uses that
137 directory to find the metadata it needs for your project.
138
139 .. note::
140
141 When specifying your project directory, do not use the tilde
142 ("~") character as BitBake does not expand that character as the
143 shell would.
144
1454. *Run BitBake:* Now that you have ``BBPATH`` defined, run the
146 ``bitbake`` command again: $ bitbake ERROR: Traceback (most recent
147 call last): File "/home/scott-lenovo/bitbake/lib/bb/cookerdata.py",
148 line 163, in wrapped return func(fn, \*args) File
149 "/home/scott-lenovo/bitbake/lib/bb/cookerdata.py", line 173, in
150 parse_config_file return bb.parse.handle(fn, data, include) File
151 "/home/scott-lenovo/bitbake/lib/bb/parse/__init__.py", line 99, in
152 handle return h['handle'](fn, data, include) File
153 "/home/scott-lenovo/bitbake/lib/bb/parse/parse_py/ConfHandler.py",
154 line 120, in handle abs_fn = resolve_file(fn, data) File
155 "/home/scott-lenovo/bitbake/lib/bb/parse/__init__.py", line 117, in
156 resolve_file raise IOError("file %s not found in %s" % (fn, bbpath))
157 IOError: file conf/bitbake.conf not found in
158 /home/scott-lenovo/hello ERROR: Unable to parse conf/bitbake.conf:
159 file conf/bitbake.conf not found in /home/scott-lenovo/hello This
160 sample output shows that BitBake could not find the
161 ``conf/bitbake.conf`` file in the project directory. This file is
162 the first thing BitBake must find in order to build a target. And,
163 since the project directory for this example is empty, you need to
164 provide a ``conf/bitbake.conf`` file.
165
1665. *Creating ``conf/bitbake.conf``:* The ``conf/bitbake.conf`` includes
167 a number of configuration variables BitBake uses for metadata and
168 recipe files. For this example, you need to create the file in your
169 project directory and define some key BitBake variables. For more
170 information on the ``bitbake.conf`` file, see
171 ` <http://git.openembedded.org/bitbake/tree/conf/bitbake.conf>`__.
172
173 Use the following commands to create the ``conf`` directory in the
174 project directory: $ mkdir conf From within the ``conf`` directory,
175 use some editor to create the ``bitbake.conf`` so that it contains
176 the following: `PN <#var-bb-PN>`__ =
177 "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0]
178 or 'defaultpkgname'}" TMPDIR = "${`TOPDIR <#var-bb-TOPDIR>`__}/tmp"
179 `CACHE <#var-bb-CACHE>`__ = "${TMPDIR}/cache"
180 `STAMP <#var-bb-STAMP>`__ = "${TMPDIR}/${PN}/stamps"
181 `T <#var-bb-T>`__ = "${TMPDIR}/${PN}/work" `B <#var-bb-B>`__ =
182 "${TMPDIR}/${PN}"
183
184 .. note::
185
186 Without a value for
187 PN
188 , the variables
189 STAMP
190 ,
191 T
192 , and
193 B
194 , prevent more than one recipe from working. You can fix this by
195 either setting
196 PN
197 to have a value similar to what OpenEmbedded and BitBake use in
198 the default
199 bitbake.conf
200 file (see previous example). Or, by manually updating each recipe
201 to set
202 PN
203 . You will also need to include
204 PN
205 as part of the
206 STAMP
207 ,
208 T
209 , and
210 B
211 variable definitions in the
212 local.conf
213 file.
214
215 The ``TMPDIR`` variable establishes a directory that BitBake uses
216 for build output and intermediate files other than the cached
217 information used by the `Setscene <#setscene>`__ process. Here, the
218 ``TMPDIR`` directory is set to ``hello/tmp``.
219
220 .. note::
221
222 You can always safely delete the
223 tmp
224 directory in order to rebuild a BitBake target. The build process
225 creates the directory for you when you run BitBake.
226
227 For information about each of the other variables defined in this
228 example, click on the links to take you to the definitions in the
229 glossary.
230
2316. *Run BitBake:* After making sure that the ``conf/bitbake.conf`` file
232 exists, you can run the ``bitbake`` command again: $ bitbake ERROR:
233 Traceback (most recent call last): File
234 "/home/scott-lenovo/bitbake/lib/bb/cookerdata.py", line 163, in
235 wrapped return func(fn, \*args) File
236 "/home/scott-lenovo/bitbake/lib/bb/cookerdata.py", line 177, in
237 \_inherit bb.parse.BBHandler.inherit(bbclass, "configuration
238 INHERITs", 0, data) File
239 "/home/scott-lenovo/bitbake/lib/bb/parse/parse_py/BBHandler.py",
240 line 92, in inherit include(fn, file, lineno, d, "inherit") File
241 "/home/scott-lenovo/bitbake/lib/bb/parse/parse_py/ConfHandler.py",
242 line 100, in include raise ParseError("Could not %(error_out)s file
243 %(fn)s" % vars(), oldfn, lineno) ParseError: ParseError in
244 configuration INHERITs: Could not inherit file classes/base.bbclass
245 ERROR: Unable to parse base: ParseError in configuration INHERITs:
246 Could not inherit file classes/base.bbclass In the sample output,
247 BitBake could not find the ``classes/base.bbclass`` file. You need
248 to create that file next.
249
2507. *Creating ``classes/base.bbclass``:* BitBake uses class files to
251 provide common code and functionality. The minimally required class
252 for BitBake is the ``classes/base.bbclass`` file. The ``base`` class
253 is implicitly inherited by every recipe. BitBake looks for the class
254 in the ``classes`` directory of the project (i.e ``hello/classes``
255 in this example).
256
257 Create the ``classes`` directory as follows: $ cd $HOME/hello $
258 mkdir classes Move to the ``classes`` directory and then create the
259 ``base.bbclass`` file by inserting this single line: addtask build
260 The minimal task that BitBake runs is the ``do_build`` task. This is
261 all the example needs in order to build the project. Of course, the
262 ``base.bbclass`` can have much more depending on which build
263 environments BitBake is supporting.
264
2658. *Run BitBake:* After making sure that the ``classes/base.bbclass``
266 file exists, you can run the ``bitbake`` command again: $ bitbake
267 Nothing to do. Use 'bitbake world' to build everything, or run
268 'bitbake --help' for usage information. BitBake is finally reporting
269 no errors. However, you can see that it really does not have
270 anything to do. You need to create a recipe that gives BitBake
271 something to do.
272
2739. *Creating a Layer:* While it is not really necessary for such a
274 small example, it is good practice to create a layer in which to
275 keep your code separate from the general metadata used by BitBake.
276 Thus, this example creates and uses a layer called "mylayer".
277
278 .. note::
279
280 You can find additional information on layers in the "
281 Layers
282 " section.
283
284 Minimally, you need a recipe file and a layer configuration file in
285 your layer. The configuration file needs to be in the ``conf``
286 directory inside the layer. Use these commands to set up the layer
287 and the ``conf`` directory: $ cd $HOME $ mkdir mylayer $ cd mylayer
288 $ mkdir conf Move to the ``conf`` directory and create a
289 ``layer.conf`` file that has the following: BBPATH .=
290 ":${`LAYERDIR <#var-bb-LAYERDIR>`__}" `BBFILES <#var-bb-BBFILES>`__
291 += "${LAYERDIR}/*.bb"
292 `BBFILE_COLLECTIONS <#var-bb-BBFILE_COLLECTIONS>`__ += "mylayer"
293 `BBFILE_PATTERN_mylayer <#var-bb-BBFILE_PATTERN>`__ :=
294 "^${LAYERDIR_RE}/" For information on these variables, click the
295 links to go to the definitions in the glossary.
296
297 You need to create the recipe file next. Inside your layer at the
298 top-level, use an editor and create a recipe file named
299 ``printhello.bb`` that has the following:
300 `DESCRIPTION <#var-bb-DESCRIPTION>`__ = "Prints Hello World"
301 `PN <#var-bb-PN>`__ = 'printhello' `PV <#var-bb-PV>`__ = '1' python
302 do_build() { bb.plain("********************"); bb.plain("\* \*");
303 bb.plain("\* Hello, World! \*"); bb.plain("\* \*");
304 bb.plain("********************"); } The recipe file simply provides
305 a description of the recipe, the name, version, and the ``do_build``
306 task, which prints out "Hello World" to the console. For more
307 information on these variables, follow the links to the glossary.
308
30910. *Run BitBake With a Target:* Now that a BitBake target exists, run
310 the command and provide that target: $ cd $HOME/hello $ bitbake
311 printhello ERROR: no recipe files to build, check your BBPATH and
312 BBFILES? Summary: There was 1 ERROR message shown, returning a
313 non-zero exit code. We have created the layer with the recipe and
314 the layer configuration file but it still seems that BitBake cannot
315 find the recipe. BitBake needs a ``conf/bblayers.conf`` that lists
316 the layers for the project. Without this file, BitBake cannot find
317 the recipe.
318
31911. *Creating ``conf/bblayers.conf``:* BitBake uses the
320 ``conf/bblayers.conf`` file to locate layers needed for the project.
321 This file must reside in the ``conf`` directory of the project (i.e.
322 ``hello/conf`` for this example).
323
324 Set your working directory to the ``hello/conf`` directory and then
325 create the ``bblayers.conf`` file so that it contains the following:
326 BBLAYERS ?= " \\ /home/<you>/mylayer \\ " You need to provide your
327 own information for ``you`` in the file.
328
32912. *Run BitBake With a Target:* Now that you have supplied the
330 ``bblayers.conf`` file, run the ``bitbake`` command and provide the
331 target: $ bitbake printhello Parsing recipes: 100%
332 \|##################################################################################\|
333 Time: 00:00:00 Parsing of 1 .bb files complete (0 cached, 1 parsed).
334 1 targets, 0 skipped, 0 masked, 0 errors. NOTE: Resolving any
335 missing task queue dependencies NOTE: Preparing RunQueue NOTE:
336 Executing RunQueue Tasks \*******************\* \* \* \* Hello,
337 World! \* \* \* \*******************\* NOTE: Tasks Summary:
338 Attempted 1 tasks of which 0 didn't need to be rerun and all
339 succeeded. BitBake finds the ``printhello`` recipe and successfully
340 runs the task.
341
342 .. note::
343
344 After the first execution, re-running
345 bitbake printhello
346 again will not result in a BitBake run that prints the same
347 console output. The reason for this is that the first time the
348 printhello.bb
349 recipe's
350 do_build
351 task executes successfully, BitBake writes a stamp file for the
352 task. Thus, the next time you attempt to run the task using that
353 same
354 bitbake
355 command, BitBake notices the stamp and therefore determines that
356 the task does not need to be re-run. If you delete the
357 tmp
358 directory or run
359 bitbake -c clean printhello
360 and then re-run the build, the "Hello, World!" message will be
361 printed again.
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst
new file mode 100644
index 0000000000..3c24e4e2cc
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst
@@ -0,0 +1,571 @@
1========
2Overview
3========
4
5Welcome to the BitBake User Manual. This manual provides information on
6the BitBake tool. The information attempts to be as independent as
7possible regarding systems that use BitBake, such as OpenEmbedded and
8the Yocto Project. In some cases, scenarios or examples within the
9context of a build system are used in the manual to help with
10understanding. For these cases, the manual clearly states the context.
11
12.. _intro:
13
14Introduction
15============
16
17Fundamentally, BitBake is a generic task execution engine that allows
18shell and Python tasks to be run efficiently and in parallel while
19working within complex inter-task dependency constraints. One of
20BitBake's main users, OpenEmbedded, takes this core and builds embedded
21Linux software stacks using a task-oriented approach.
22
23Conceptually, BitBake is similar to GNU Make in some regards but has
24significant differences:
25
26- BitBake executes tasks according to provided metadata that builds up
27 the tasks. Metadata is stored in recipe (``.bb``) and related recipe
28 "append" (``.bbappend``) files, configuration (``.conf``) and
29 underlying include (``.inc``) files, and in class (``.bbclass``)
30 files. The metadata provides BitBake with instructions on what tasks
31 to run and the dependencies between those tasks.
32
33- BitBake includes a fetcher library for obtaining source code from
34 various places such as local files, source control systems, or
35 websites.
36
37- The instructions for each unit to be built (e.g. a piece of software)
38 are known as "recipe" files and contain all the information about the
39 unit (dependencies, source file locations, checksums, description and
40 so on).
41
42- BitBake includes a client/server abstraction and can be used from a
43 command line or used as a service over XML-RPC and has several
44 different user interfaces.
45
46History and Goals
47=================
48
49BitBake was originally a part of the OpenEmbedded project. It was
50inspired by the Portage package management system used by the Gentoo
51Linux distribution. On December 7, 2004, OpenEmbedded project team
52member Chris Larson split the project into two distinct pieces:
53
54- BitBake, a generic task executor
55
56- OpenEmbedded, a metadata set utilized by BitBake
57
58Today, BitBake is the primary basis of the
59`OpenEmbedded <http://www.openembedded.org/>`__ project, which is being
60used to build and maintain Linux distributions such as the `Angstrom
61Distribution <http://www.angstrom-distribution.org/>`__, and which is
62also being used as the build tool for Linux projects such as the `Yocto
63Project <http://www.yoctoproject.org>`__.
64
65Prior to BitBake, no other build tool adequately met the needs of an
66aspiring embedded Linux distribution. All of the build systems used by
67traditional desktop Linux distributions lacked important functionality,
68and none of the ad hoc Buildroot-based systems, prevalent in the
69embedded space, were scalable or maintainable.
70
71Some important original goals for BitBake were:
72
73- Handle cross-compilation.
74
75- Handle inter-package dependencies (build time on target architecture,
76 build time on native architecture, and runtime).
77
78- Support running any number of tasks within a given package,
79 including, but not limited to, fetching upstream sources, unpacking
80 them, patching them, configuring them, and so forth.
81
82- Be Linux distribution agnostic for both build and target systems.
83
84- Be architecture agnostic.
85
86- Support multiple build and target operating systems (e.g. Cygwin, the
87 BSDs, and so forth).
88
89- Be self-contained, rather than tightly integrated into the build
90 machine's root filesystem.
91
92- Handle conditional metadata on the target architecture, operating
93 system, distribution, and machine.
94
95- Be easy to use the tools to supply local metadata and packages
96 against which to operate.
97
98- Be easy to use BitBake to collaborate between multiple projects for
99 their builds.
100
101- Provide an inheritance mechanism to share common metadata between
102 many packages.
103
104Over time it became apparent that some further requirements were
105necessary:
106
107- Handle variants of a base recipe (e.g. native, sdk, and multilib).
108
109- Split metadata into layers and allow layers to enhance or override
110 other layers.
111
112- Allow representation of a given set of input variables to a task as a
113 checksum. Based on that checksum, allow acceleration of builds with
114 prebuilt components.
115
116BitBake satisfies all the original requirements and many more with
117extensions being made to the basic functionality to reflect the
118additional requirements. Flexibility and power have always been the
119priorities. BitBake is highly extensible and supports embedded Python
120code and execution of any arbitrary tasks.
121
122.. _Concepts:
123
124Concepts
125========
126
127BitBake is a program written in the Python language. At the highest
128level, BitBake interprets metadata, decides what tasks are required to
129run, and executes those tasks. Similar to GNU Make, BitBake controls how
130software is built. GNU Make achieves its control through "makefiles",
131while BitBake uses "recipes".
132
133BitBake extends the capabilities of a simple tool like GNU Make by
134allowing for the definition of much more complex tasks, such as
135assembling entire embedded Linux distributions.
136
137The remainder of this section introduces several concepts that should be
138understood in order to better leverage the power of BitBake.
139
140Recipes
141-------
142
143BitBake Recipes, which are denoted by the file extension ``.bb``, are
144the most basic metadata files. These recipe files provide BitBake with
145the following:
146
147- Descriptive information about the package (author, homepage, license,
148 and so on)
149
150- The version of the recipe
151
152- Existing dependencies (both build and runtime dependencies)
153
154- Where the source code resides and how to fetch it
155
156- Whether the source code requires any patches, where to find them, and
157 how to apply them
158
159- How to configure and compile the source code
160
161- How to assemble the generated artifacts into one or more installable
162 packages
163
164- Where on the target machine to install the package or packages
165 created
166
167Within the context of BitBake, or any project utilizing BitBake as its
168build system, files with the ``.bb`` extension are referred to as
169recipes.
170
171.. note::
172
173 The term "package" is also commonly used to describe recipes.
174 However, since the same word is used to describe packaged output from
175 a project, it is best to maintain a single descriptive term -
176 "recipes". Put another way, a single "recipe" file is quite capable
177 of generating a number of related but separately installable
178 "packages". In fact, that ability is fairly common.
179
180Configuration Files
181-------------------
182
183Configuration files, which are denoted by the ``.conf`` extension,
184define various configuration variables that govern the project's build
185process. These files fall into several areas that define machine
186configuration, distribution configuration, possible compiler tuning,
187general common configuration, and user configuration. The main
188configuration file is the sample ``bitbake.conf`` file, which is located
189within the BitBake source tree ``conf`` directory.
190
191Classes
192-------
193
194Class files, which are denoted by the ``.bbclass`` extension, contain
195information that is useful to share between metadata files. The BitBake
196source tree currently comes with one class metadata file called
197``base.bbclass``. You can find this file in the ``classes`` directory.
198The ``base.bbclass`` class files is special since it is always included
199automatically for all recipes and classes. This class contains
200definitions for standard basic tasks such as fetching, unpacking,
201configuring (empty by default), compiling (runs any Makefile present),
202installing (empty by default) and packaging (empty by default). These
203tasks are often overridden or extended by other classes added during the
204project development process.
205
206Layers
207------
208
209Layers allow you to isolate different types of customizations from each
210other. While you might find it tempting to keep everything in one layer
211when working on a single project, the more modular your metadata, the
212easier it is to cope with future changes.
213
214To illustrate how you can use layers to keep things modular, consider
215customizations you might make to support a specific target machine.
216These types of customizations typically reside in a special layer,
217rather than a general layer, called a Board Support Package (BSP) layer.
218Furthermore, the machine customizations should be isolated from recipes
219and metadata that support a new GUI environment, for example. This
220situation gives you a couple of layers: one for the machine
221configurations and one for the GUI environment. It is important to
222understand, however, that the BSP layer can still make machine-specific
223additions to recipes within the GUI environment layer without polluting
224the GUI layer itself with those machine-specific changes. You can
225accomplish this through a recipe that is a BitBake append
226(``.bbappend``) file.
227
228.. _append-bbappend-files:
229
230Append Files
231------------
232
233Append files, which are files that have the ``.bbappend`` file
234extension, extend or override information in an existing recipe file.
235
236BitBake expects every append file to have a corresponding recipe file.
237Furthermore, the append file and corresponding recipe file must use the
238same root filename. The filenames can differ only in the file type
239suffix used (e.g. ``formfactor_0.0.bb`` and
240``formfactor_0.0.bbappend``).
241
242Information in append files extends or overrides the information in the
243underlying, similarly-named recipe files.
244
245When you name an append file, you can use the "``%``" wildcard character
246to allow for matching recipe names. For example, suppose you have an
247append file named as follows: busybox_1.21.%.bbappend That append file
248would match any ``busybox_1.21.``\ x\ ``.bb`` version of the recipe. So,
249the append file would match the following recipe names:
250busybox_1.21.1.bb busybox_1.21.2.bb busybox_1.21.3.bb
251
252.. note::
253
254 The use of the "
255 %
256 " character is limited in that it only works directly in front of the
257 .bbappend
258 portion of the append file's name. You cannot use the wildcard
259 character in any other location of the name.
260
261If the ``busybox`` recipe was updated to ``busybox_1.3.0.bb``, the
262append name would not match. However, if you named the append file
263``busybox_1.%.bbappend``, then you would have a match.
264
265In the most general case, you could name the append file something as
266simple as ``busybox_%.bbappend`` to be entirely version independent.
267
268Obtaining BitBake
269=================
270
271You can obtain BitBake several different ways:
272
273- *Cloning BitBake:* Using Git to clone the BitBake source code
274 repository is the recommended method for obtaining BitBake. Cloning
275 the repository makes it easy to get bug fixes and have access to
276 stable branches and the master branch. Once you have cloned BitBake,
277 you should use the latest stable branch for development since the
278 master branch is for BitBake development and might contain less
279 stable changes.
280
281 You usually need a version of BitBake that matches the metadata you
282 are using. The metadata is generally backwards compatible but not
283 forward compatible.
284
285 Here is an example that clones the BitBake repository: $ git clone
286 git://git.openembedded.org/bitbake This command clones the BitBake
287 Git repository into a directory called ``bitbake``. Alternatively,
288 you can designate a directory after the ``git clone`` command if you
289 want to call the new directory something other than ``bitbake``. Here
290 is an example that names the directory ``bbdev``: $ git clone
291 git://git.openembedded.org/bitbake bbdev
292
293- *Installation using your Distribution Package Management System:*
294 This method is not recommended because the BitBake version that is
295 provided by your distribution, in most cases, is several releases
296 behind a snapshot of the BitBake repository.
297
298- *Taking a snapshot of BitBake:* Downloading a snapshot of BitBake
299 from the source code repository gives you access to a known branch or
300 release of BitBake.
301
302 .. note::
303
304 Cloning the Git repository, as described earlier, is the preferred
305 method for getting BitBake. Cloning the repository makes it easier
306 to update as patches are added to the stable branches.
307
308 The following example downloads a snapshot of BitBake version 1.17.0:
309 $ wget
310 http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz $
311 tar zxpvf bitbake-1.17.0.tar.gz After extraction of the tarball using
312 the tar utility, you have a directory entitled ``bitbake-1.17.0``.
313
314- *Using the BitBake that Comes With Your Build Checkout:* A final
315 possibility for getting a copy of BitBake is that it already comes
316 with your checkout of a larger BitBake-based build system, such as
317 Poky. Rather than manually checking out individual layers and gluing
318 them together yourself, you can check out an entire build system. The
319 checkout will already include a version of BitBake that has been
320 thoroughly tested for compatibility with the other components. For
321 information on how to check out a particular BitBake-based build
322 system, consult that build system's supporting documentation.
323
324.. _bitbake-user-manual-command:
325
326The BitBake Command
327===================
328
329The ``bitbake`` command is the primary interface to the BitBake tool.
330This section presents the BitBake command syntax and provides several
331execution examples.
332
333Usage and syntax
334----------------
335
336Following is the usage and syntax for BitBake: $ bitbake -h Usage:
337bitbake [options] [recipename/target recipe:do_task ...] Executes the
338specified task (default is 'build') for a given set of target recipes
339(.bb files). It is assumed there is a conf/bblayers.conf available in
340cwd or in BBPATH which will provide the layer, BBFILES and other
341configuration information. Options: --version show program's version
342number and exit -h, --help show this help message and exit -b BUILDFILE,
343--buildfile=BUILDFILE Execute tasks from a specific .bb recipe directly.
344WARNING: Does not handle any dependencies from other recipes. -k,
345--continue Continue as much as possible after an error. While the target
346that failed and anything depending on it cannot be built, as much as
347possible will be built before stopping. -f, --force Force the specified
348targets/task to run (invalidating any existing stamp file). -c CMD,
349--cmd=CMD Specify the task to execute. The exact options available
350depend on the metadata. Some examples might be 'compile' or
351'populate_sysroot' or 'listtasks' may give a list of the tasks
352available. -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
353Invalidate the stamp for the specified task such as 'compile' and then
354run the default task for the specified target(s). -r PREFILE,
355--read=PREFILE Read the specified file before bitbake.conf. -R POSTFILE,
356--postread=POSTFILE Read the specified file after bitbake.conf. -v,
357--verbose Enable tracing of shell tasks (with 'set -x'). Also print
358bb.note(...) messages to stdout (in addition to writing them to
359${T}/log.do_<task>). -D, --debug Increase the debug level. You can
360specify this more than once. -D sets the debug level to 1, where only
361bb.debug(1, ...) messages are printed to stdout; -DD sets the debug
362level to 2, where both bb.debug(1, ...) and bb.debug(2, ...) messages
363are printed; etc. Without -D, no debug messages are printed. Note that
364-D only affects output to stdout. All debug messages are written to
365${T}/log.do_taskname, regardless of the debug level. -q, --quiet Output
366less log message data to the terminal. You can specify this more than
367once. -n, --dry-run Don't execute, just go through the motions. -S
368SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER Dump out the
369signature construction information, with no task execution. The
370SIGNATURE_HANDLER parameter is passed to the handler. Two common values
371are none and printdiff but the handler may define more/less. none means
372only dump the signature, printdiff means compare the dumped signature
373with the cached one. -p, --parse-only Quit after parsing the BB recipes.
374-s, --show-versions Show current and preferred versions of all recipes.
375-e, --environment Show the global or per-recipe environment complete
376with information about where variables were set/changed. -g, --graphviz
377Save dependency tree information for the specified targets in the dot
378syntax. -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
379Assume these dependencies don't exist and are already provided
380(equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more
381appealing -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS Show debug
382logging for the specified logging domains -P, --profile Profile the
383command and save reports. -u UI, --ui=UI The user interface to use
384(knotty, ncurses or taskexp - default knotty). --token=XMLRPCTOKEN
385Specify the connection token to be used when connecting to a remote
386server. --revisions-changed Set the exit code depending on whether
387upstream floating revisions have changed or not. --server-only Run
388bitbake without a UI, only starting a server (cooker) process. -B BIND,
389--bind=BIND The name/address for the bitbake xmlrpc server to bind to.
390-T SERVER_TIMEOUT, --idle-timeout=SERVER_TIMEOUT Set timeout to unload
391bitbake server due to inactivity, set to -1 means no unload, default:
392Environment variable BB_SERVER_TIMEOUT. --no-setscene Do not run any
393setscene tasks. sstate will be ignored and everything needed, built.
394--setscene-only Only run setscene tasks, don't run any real tasks.
395--remote-server=REMOTE_SERVER Connect to the specified server. -m,
396--kill-server Terminate any running bitbake server. --observe-only
397Connect to a server as an observing-only client. --status-only Check the
398status of the remote bitbake server. -w WRITEEVENTLOG,
399--write-log=WRITEEVENTLOG Writes the event log of the build to a bitbake
400event json file. Use '' (empty string) to assign the name automatically.
401--runall=RUNALL Run the specified task for any recipe in the taskgraph
402of the specified target (even if it wouldn't otherwise have run).
403--runonly=RUNONLY Run only the specified task within the taskgraph of
404the specified targets (and any task dependencies those tasks may have).
405
406.. _bitbake-examples:
407
408Examples
409--------
410
411This section presents some examples showing how to use BitBake.
412
413.. _example-executing-a-task-against-a-single-recipe:
414
415Executing a Task Against a Single Recipe
416~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
417
418Executing tasks for a single recipe file is relatively simple. You
419specify the file in question, and BitBake parses it and executes the
420specified task. If you do not specify a task, BitBake executes the
421default task, which is "build”. BitBake obeys inter-task dependencies
422when doing so.
423
424The following command runs the build task, which is the default task, on
425the ``foo_1.0.bb`` recipe file: $ bitbake -b foo_1.0.bb The following
426command runs the clean task on the ``foo.bb`` recipe file: $ bitbake -b
427foo.bb -c clean
428
429.. note::
430
431 The "-b" option explicitly does not handle recipe dependencies. Other
432 than for debugging purposes, it is instead recommended that you use
433 the syntax presented in the next section.
434
435Executing Tasks Against a Set of Recipe Files
436~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
437
438There are a number of additional complexities introduced when one wants
439to manage multiple ``.bb`` files. Clearly there needs to be a way to
440tell BitBake what files are available and, of those, which you want to
441execute. There also needs to be a way for each recipe to express its
442dependencies, both for build-time and runtime. There must be a way for
443you to express recipe preferences when multiple recipes provide the same
444functionality, or when there are multiple versions of a recipe.
445
446The ``bitbake`` command, when not using "--buildfile" or "-b" only
447accepts a "PROVIDES". You cannot provide anything else. By default, a
448recipe file generally "PROVIDES" its "packagename" as shown in the
449following example: $ bitbake foo This next example "PROVIDES" the
450package name and also uses the "-c" option to tell BitBake to just
451execute the ``do_clean`` task: $ bitbake -c clean foo
452
453Executing a List of Task and Recipe Combinations
454~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
455
456The BitBake command line supports specifying different tasks for
457individual targets when you specify multiple targets. For example,
458suppose you had two targets (or recipes) ``myfirstrecipe`` and
459``mysecondrecipe`` and you needed BitBake to run ``taskA`` for the first
460recipe and ``taskB`` for the second recipe: $ bitbake
461myfirstrecipe:do_taskA mysecondrecipe:do_taskB
462
463Generating Dependency Graphs
464~~~~~~~~~~~~~~~~~~~~~~~~~~~~
465
466BitBake is able to generate dependency graphs using the ``dot`` syntax.
467You can convert these graphs into images using the ``dot`` tool from
468`Graphviz <http://www.graphviz.org>`__.
469
470When you generate a dependency graph, BitBake writes two files to the
471current working directory:
472
473- *``task-depends.dot``:* Shows dependencies between tasks. These
474 dependencies match BitBake's internal task execution list.
475
476- *``pn-buildlist``:* Shows a simple list of targets that are to be
477 built.
478
479To stop depending on common depends, use the "-I" depend option and
480BitBake omits them from the graph. Leaving this information out can
481produce more readable graphs. This way, you can remove from the graph
482``DEPENDS`` from inherited classes such as ``base.bbclass``.
483
484Here are two examples that create dependency graphs. The second example
485omits depends common in OpenEmbedded from the graph: $ bitbake -g foo $
486bitbake -g -I virtual/kernel -I eglibc foo
487
488Executing a Multiple Configuration Build
489~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
490
491BitBake is able to build multiple images or packages using a single
492command where the different targets require different configurations
493(multiple configuration builds). Each target, in this scenario, is
494referred to as a "multiconfig".
495
496To accomplish a multiple configuration build, you must define each
497target's configuration separately using a parallel configuration file in
498the build directory. The location for these multiconfig configuration
499files is specific. They must reside in the current build directory in a
500sub-directory of ``conf`` named ``multiconfig``. Following is an example
501for two separate targets:
502
503The reason for this required file hierarchy is because the ``BBPATH``
504variable is not constructed until the layers are parsed. Consequently,
505using the configuration file as a pre-configuration file is not possible
506unless it is located in the current working directory.
507
508Minimally, each configuration file must define the machine and the
509temporary directory BitBake uses for the build. Suggested practice
510dictates that you do not overlap the temporary directories used during
511the builds.
512
513Aside from separate configuration files for each target, you must also
514enable BitBake to perform multiple configuration builds. Enabling is
515accomplished by setting the
516```BBMULTICONFIG`` <#var-bb-BBMULTICONFIG>`__ variable in the
517``local.conf`` configuration file. As an example, suppose you had
518configuration files for ``target1`` and ``target2`` defined in the build
519directory. The following statement in the ``local.conf`` file both
520enables BitBake to perform multiple configuration builds and specifies
521the two extra multiconfigs: BBMULTICONFIG = "target1 target2"
522
523Once the target configuration files are in place and BitBake has been
524enabled to perform multiple configuration builds, use the following
525command form to start the builds: $ bitbake [mc:multiconfigname:]target
526[[[mc:multiconfigname:]target] ... ] Here is an example for two extra
527multiconfigs: ``target1`` and ``target2``: $ bitbake mc::target
528mc:target1:target mc:target2:target
529
530.. _bb-enabling-multiple-configuration-build-dependencies:
531
532Enabling Multiple Configuration Build Dependencies
533~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
534
535Sometimes dependencies can exist between targets (multiconfigs) in a
536multiple configuration build. For example, suppose that in order to
537build an image for a particular architecture, the root filesystem of
538another build for a different architecture needs to exist. In other
539words, the image for the first multiconfig depends on the root
540filesystem of the second multiconfig. This dependency is essentially
541that the task in the recipe that builds one multiconfig is dependent on
542the completion of the task in the recipe that builds another
543multiconfig.
544
545To enable dependencies in a multiple configuration build, you must
546declare the dependencies in the recipe using the following statement
547form: task_or_package[mcdepends] =
548"mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
549To better show how to use this statement, consider an example with two
550multiconfigs: ``target1`` and ``target2``: image_task[mcdepends] =
551"mc:target1:target2:image2:rootfs_task" In this example, the
552from_multiconfig is "target1" and the to_multiconfig is "target2". The
553task on which the image whose recipe contains image_task depends on the
554completion of the rootfs_task used to build out image2, which is
555associated with the "target2" multiconfig.
556
557Once you set up this dependency, you can build the "target1" multiconfig
558using a BitBake command as follows: $ bitbake mc:target1:image1 This
559command executes all the tasks needed to create image1 for the "target1"
560multiconfig. Because of the dependency, BitBake also executes through
561the rootfs_task for the "target2" multiconfig build.
562
563Having a recipe depend on the root filesystem of another build might not
564seem that useful. Consider this change to the statement in the image1
565recipe: image_task[mcdepends] = "mc:target1:target2:image2:image_task"
566In this case, BitBake must create image2 for the "target2" build since
567the "target1" build depends on it.
568
569Because "target1" and "target2" are enabled for multiple configuration
570builds and have separate configuration files, BitBake places the
571artifacts for each build in the respective temporary build directories.
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
new file mode 100644
index 0000000000..790065ef64
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -0,0 +1,1728 @@
1====================
2Syntax and Operators
3====================
4
5BitBake files have their own syntax. The syntax has similarities to
6several other languages but also has some unique features. This section
7describes the available syntax and operators as well as provides
8examples.
9
10Basic Syntax
11============
12
13This section provides some basic syntax examples.
14
15Basic Variable Setting
16----------------------
17
18The following example sets ``VARIABLE`` to "value". This assignment
19occurs immediately as the statement is parsed. It is a "hard"
20assignment. VARIABLE = "value" As expected, if you include leading or
21trailing spaces as part of an assignment, the spaces are retained:
22VARIABLE = " value" VARIABLE = "value " Setting ``VARIABLE`` to "" sets
23it to an empty string, while setting the variable to " " sets it to a
24blank space (i.e. these are not the same values). VARIABLE = "" VARIABLE
25= " "
26
27You can use single quotes instead of double quotes when setting a
28variable's value. Doing so allows you to use values that contain the
29double quote character: VARIABLE = 'I have a " in my value'
30
31.. note::
32
33 Unlike in Bourne shells, single quotes work identically to double
34 quotes in all other ways. They do not suppress variable expansions.
35
36Modifying Existing Variables
37----------------------------
38
39Sometimes you need to modify existing variables. Following are some
40cases where you might find you want to modify an existing variable:
41
42- Customize a recipe that uses the variable.
43
44- Change a variable's default value used in a ``*.bbclass`` file.
45
46- Change the variable in a ``*.bbappend`` file to override the variable
47 in the original recipe.
48
49- Change the variable in a configuration file so that the value
50 overrides an existing configuration.
51
52Changing a variable value can sometimes depend on how the value was
53originally assigned and also on the desired intent of the change. In
54particular, when you append a value to a variable that has a default
55value, the resulting value might not be what you expect. In this case,
56the value you provide might replace the value rather than append to the
57default value.
58
59If after you have changed a variable's value and something unexplained
60occurs, you can use BitBake to check the actual value of the suspect
61variable. You can make these checks for both configuration and recipe
62level changes:
63
64- For configuration changes, use the following: $ bitbake -e This
65 command displays variable values after the configuration files (i.e.
66 ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth)
67 have been parsed.
68
69 .. note::
70
71 Variables that are exported to the environment are preceded by the
72 string "export" in the command's output.
73
74- For recipe changes, use the following: $ bitbake recipe -e \| grep
75 VARIABLE=" This command checks to see if the variable actually makes
76 it into a specific recipe.
77
78Line Joining
79------------
80
81Outside of `functions <#functions>`__, BitBake joins any line ending in
82a backslash character ("\") with the following line before parsing
83statements. The most common use for the "\" character is to split
84variable assignments over multiple lines, as in the following example:
85FOO = "bar \\ baz \\ qaz" Both the "\" character and the newline
86character that follow it are removed when joining lines. Thus, no
87newline characters end up in the value of ``FOO``.
88
89Consider this additional example where the two assignments both assign
90"barbaz" to ``FOO``: FOO = "barbaz" FOO = "bar\\ baz"
91
92.. note::
93
94 BitBake does not interpret escape sequences like "\n" in variable
95 values. For these to have an effect, the value must be passed to some
96 utility that interprets escape sequences, such as
97 printf
98 or
99 echo -n
100 .
101
102Variable Expansion
103------------------
104
105Variables can reference the contents of other variables using a syntax
106that is similar to variable expansion in Bourne shells. The following
107assignments result in A containing "aval" and B evaluating to
108"preavalpost". A = "aval" B = "pre${A}post"
109
110.. note::
111
112 Unlike in Bourne shells, the curly braces are mandatory: Only
113 ${FOO}
114 and not
115 $FOO
116 is recognized as an expansion of
117 FOO
118 .
119
120The "=" operator does not immediately expand variable references in the
121right-hand side. Instead, expansion is deferred until the variable
122assigned to is actually used. The result depends on the current values
123of the referenced variables. The following example should clarify this
124behavior: A = "${B} baz" B = "${C} bar" C = "foo" \*At this point, ${A}
125equals "foo bar baz"\* C = "qux" \*At this point, ${A} equals "qux bar
126baz"\* B = "norf" \*At this point, ${A} equals "norf baz"\* Contrast
127this behavior with the `immediate variable
128expansion <#immediate-variable-expansion>`__ operator (i.e. ":=").
129
130If the variable expansion syntax is used on a variable that does not
131exist, the string is kept as is. For example, given the following
132assignment, ``BAR`` expands to the literal string "${FOO}" as long as
133``FOO`` does not exist. BAR = "${FOO}"
134
135Setting a default value (?=)
136----------------------------
137
138You can use the "?=" operator to achieve a "softer" assignment for a
139variable. This type of assignment allows you to define a variable if it
140is undefined when the statement is parsed, but to leave the value alone
141if the variable has a value. Here is an example: A ?= "aval" If ``A`` is
142set at the time this statement is parsed, the variable retains its
143value. However, if ``A`` is not set, the variable is set to "aval".
144
145.. note::
146
147 This assignment is immediate. Consequently, if multiple "?="
148 assignments to a single variable exist, the first of those ends up
149 getting used.
150
151Setting a weak default value (??=)
152----------------------------------
153
154It is possible to use a "weaker" assignment than in the previous section
155by using the "??=" operator. This assignment behaves identical to "?="
156except that the assignment is made at the end of the parsing process
157rather than immediately. Consequently, when multiple "??=" assignments
158exist, the last one is used. Also, any "=" or "?=" assignment will
159override the value set with "??=". Here is an example: A ??= "somevalue"
160A ??= "someothervalue" If ``A`` is set before the above statements are
161parsed, the variable retains its value. If ``A`` is not set, the
162variable is set to "someothervalue".
163
164Again, this assignment is a "lazy" or "weak" assignment because it does
165not occur until the end of the parsing process.
166
167Immediate variable expansion (:=)
168---------------------------------
169
170The ":=" operator results in a variable's contents being expanded
171immediately, rather than when the variable is actually used: T = "123" A
172:= "test ${T}" T = "456" B := "${T} ${C}" C = "cval" C := "${C}append"
173In this example, ``A`` contains "test 123", even though the final value
174of ``T`` is "456". The variable ``B`` will end up containing "456
175cvalappend". This is because references to undefined variables are
176preserved as is during (immediate)expansion. This is in contrast to GNU
177Make, where undefined variables expand to nothing. The variable ``C``
178contains "cvalappend" since ``${C}`` immediately expands to "cval".
179
180.. _appending-and-prepending:
181
182Appending (+=) and prepending (=+) With Spaces
183----------------------------------------------
184
185Appending and prepending values is common and can be accomplished using
186the "+=" and "=+" operators. These operators insert a space between the
187current value and prepended or appended value.
188
189These operators take immediate effect during parsing. Here are some
190examples: B = "bval" B += "additionaldata" C = "cval" C =+ "test" The
191variable ``B`` contains "bval additionaldata" and ``C`` contains "test
192cval".
193
194.. _appending-and-prepending-without-spaces:
195
196Appending (.=) and Prepending (=.) Without Spaces
197-------------------------------------------------
198
199If you want to append or prepend values without an inserted space, use
200the ".=" and "=." operators.
201
202These operators take immediate effect during parsing. Here are some
203examples: B = "bval" B .= "additionaldata" C = "cval" C =. "test" The
204variable ``B`` contains "bvaladditionaldata" and ``C`` contains
205"testcval".
206
207Appending and Prepending (Override Style Syntax)
208------------------------------------------------
209
210You can also append and prepend a variable's value using an override
211style syntax. When you use this syntax, no spaces are inserted.
212
213These operators differ from the ":=", ".=", "=.", "+=", and "=+"
214operators in that their effects are applied at variable expansion time
215rather than being immediately applied. Here are some examples: B =
216"bval" B_append = " additional data" C = "cval" C_prepend = "additional
217data " D = "dval" D_append = "additional data" The variable ``B``
218becomes "bval additional data" and ``C`` becomes "additional data cval".
219The variable ``D`` becomes "dvaladditional data".
220
221.. note::
222
223 You must control all spacing when you use the override syntax.
224
225It is also possible to append and prepend to shell functions and
226BitBake-style Python functions. See the "`Shell
227Functions <#shell-functions>`__" and "`BitBake-Style Python
228Functions <#bitbake-style-python-functions>`__ sections for examples.
229
230.. _removing-override-style-syntax:
231
232Removal (Override Style Syntax)
233-------------------------------
234
235You can remove values from lists using the removal override style
236syntax. Specifying a value for removal causes all occurrences of that
237value to be removed from the variable.
238
239When you use this syntax, BitBake expects one or more strings.
240Surrounding spaces and spacing are preserved. Here is an example: FOO =
241"123 456 789 123456 123 456 123 456" FOO_remove = "123" FOO_remove =
242"456" FOO2 = " abc def ghi abcdef abc def abc def def" FOO2_remove = "
243\\ def \\ abc \\ ghi \\ " The variable ``FOO`` becomes
244"  789 123456    " and ``FOO2`` becomes "     abcdef      ".
245
246Like "_append" and "_prepend", "_remove" is applied at variable
247expansion time.
248
249Override Style Operation Advantages
250-----------------------------------
251
252An advantage of the override style operations "_append", "_prepend", and
253"_remove" as compared to the "+=" and "=+" operators is that the
254override style operators provide guaranteed operations. For example,
255consider a class ``foo.bbclass`` that needs to add the value "val" to
256the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows:
257inherit foo FOO = "initial" If ``foo.bbclass`` uses the "+=" operator,
258as follows, then the final value of ``FOO`` will be "initial", which is
259not what is desired: FOO += "val" If, on the other hand, ``foo.bbclass``
260uses the "_append" operator, then the final value of ``FOO`` will be
261"initial val", as intended: FOO_append = " val"
262
263.. note::
264
265 It is never necessary to use "+=" together with "_append". The
266 following sequence of assignments appends "barbaz" to
267 FOO
268 :
269 ::
270
271 FOO_append = "bar"
272 FOO_append = "baz"
273
274
275 The only effect of changing the second assignment in the previous
276 example to use "+=" would be to add a space before "baz" in the
277 appended value (due to how the "+=" operator works).
278
279Another advantage of the override style operations is that you can
280combine them with other overrides as described in the "`Conditional
281Syntax (Overrides) <#conditional-syntax-overrides>`__" section.
282
283Variable Flag Syntax
284--------------------
285
286Variable flags are BitBake's implementation of variable properties or
287attributes. It is a way of tagging extra information onto a variable.
288You can find more out about variable flags in general in the "`Variable
289Flags <#variable-flags>`__" section.
290
291You can define, append, and prepend values to variable flags. All the
292standard syntax operations previously mentioned work for variable flags
293except for override style syntax (i.e. "_prepend", "_append", and
294"_remove").
295
296Here are some examples showing how to set variable flags: FOO[a] = "abc"
297FOO[b] = "123" FOO[a] += "456" The variable ``FOO`` has two flags:
298``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123",
299respectively. The ``[a]`` flag becomes "abc 456".
300
301No need exists to pre-define variable flags. You can simply start using
302them. One extremely common application is to attach some brief
303documentation to a BitBake variable as follows: CACHE[doc] = "The
304directory holding the cache of the metadata."
305
306Inline Python Variable Expansion
307--------------------------------
308
309You can use inline Python variable expansion to set variables. Here is
310an example: DATE = "${@time.strftime('%Y%m%d',time.gmtime())}" This
311example results in the ``DATE`` variable being set to the current date.
312
313Probably the most common use of this feature is to extract the value of
314variables from BitBake's internal data dictionary, ``d``. The following
315lines select the values of a package name and its version number,
316respectively: PN =
317"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or
318'defaultpkgname'}" PV =
319"${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or
320'1.0'}"
321
322.. note::
323
324 Inline Python expressions work just like variable expansions insofar
325 as the "=" and ":=" operators are concerned. Given the following
326 assignment,
327 foo()
328 is called each time
329 FOO
330 is expanded:
331 ::
332
333 FOO = "${@foo()}"
334
335
336 Contrast this with the following immediate assignment, where
337 foo()
338 is only called once, while the assignment is parsed:
339 ::
340
341 FOO := "${@foo()}"
342
343
344For a different way to set variables with Python code during parsing,
345see the "`Anonymous Python Functions <#anonymous-python-functions>`__"
346section.
347
348Unsetting variables
349-------------------
350
351It is possible to completely remove a variable or a variable flag from
352BitBake's internal data dictionary by using the "unset" keyword. Here is
353an example: unset DATE unset do_fetch[noexec] These two statements
354remove the ``DATE`` and the ``do_fetch[noexec]`` flag.
355
356Providing Pathnames
357-------------------
358
359When specifying pathnames for use with BitBake, do not use the tilde
360("~") character as a shortcut for your home directory. Doing so might
361cause BitBake to not recognize the path since BitBake does not expand
362this character in the same way a shell would.
363
364Instead, provide a fuller path as the following example illustrates:
365BBLAYERS ?= " \\ /home/scott-lenovo/LayerA \\ "
366
367Exporting Variables to the Environment
368======================================
369
370You can export variables to the environment of running tasks by using
371the ``export`` keyword. For example, in the following example, the
372``do_foo`` task prints "value from the environment" when run: export
373ENV_VARIABLE ENV_VARIABLE = "value from the environment" do_foo() {
374bbplain "$ENV_VARIABLE" }
375
376.. note::
377
378 BitBake does not expand
379 $ENV_VARIABLE
380 in this case because it lacks the obligatory
381 {}
382 . Rather,
383 $ENV_VARIABLE
384 is expanded by the shell.
385
386It does not matter whether ``export ENV_VARIABLE`` appears before or
387after assignments to ``ENV_VARIABLE``.
388
389It is also possible to combine ``export`` with setting a value for the
390variable. Here is an example: export ENV_VARIABLE = "variable-value" In
391the output of ``bitbake -e``, variables that are exported to the
392environment are preceded by "export".
393
394Among the variables commonly exported to the environment are ``CC`` and
395``CFLAGS``, which are picked up by many build systems.
396
397Conditional Syntax (Overrides)
398==============================
399
400BitBake uses ```OVERRIDES`` <#var-bb-OVERRIDES>`__ to control what
401variables are overridden after BitBake parses recipes and configuration
402files. This section describes how you can use ``OVERRIDES`` as
403conditional metadata, talks about key expansion in relationship to
404``OVERRIDES``, and provides some examples to help with understanding.
405
406Conditional Metadata
407--------------------
408
409You can use ``OVERRIDES`` to conditionally select a specific version of
410a variable and to conditionally append or prepend the value of a
411variable.
412
413.. note::
414
415 Overrides can only use lower-case characters. Additionally,
416 underscores are not permitted in override names as they are used to
417 separate overrides from each other and from the variable name.
418
419- *Selecting a Variable:* The ``OVERRIDES`` variable is a
420 colon-character-separated list that contains items for which you want
421 to satisfy conditions. Thus, if you have a variable that is
422 conditional on “arm”, and “arm” is in ``OVERRIDES``, then the
423 “arm”-specific version of the variable is used rather than the
424 non-conditional version. Here is an example: OVERRIDES =
425 "architecture:os:machine" TEST = "default" TEST_os = "osspecific"
426 TEST_nooverride = "othercondvalue" In this example, the ``OVERRIDES``
427 variable lists three overrides: "architecture", "os", and "machine".
428 The variable ``TEST`` by itself has a default value of "default". You
429 select the os-specific version of the ``TEST`` variable by appending
430 the "os" override to the variable (i.e.\ ``TEST_os``).
431
432 To better understand this, consider a practical example that assumes
433 an OpenEmbedded metadata-based Linux kernel recipe file. The
434 following lines from the recipe file first set the kernel branch
435 variable ``KBRANCH`` to a default value, then conditionally override
436 that value based on the architecture of the build: KBRANCH =
437 "standard/base" KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
438 KBRANCH_qemumips = "standard/mti-malta32" KBRANCH_qemuppc =
439 "standard/qemuppc" KBRANCH_qemux86 = "standard/common-pc/base"
440 KBRANCH_qemux86-64 = "standard/common-pc-64/base" KBRANCH_qemumips64
441 = "standard/mti-malta64"
442
443- *Appending and Prepending:* BitBake also supports append and prepend
444 operations to variable values based on whether a specific item is
445 listed in ``OVERRIDES``. Here is an example: DEPENDS = "glibc
446 ncurses" OVERRIDES = "machine:local" DEPENDS_append_machine = "
447 libmad" In this example, ``DEPENDS`` becomes "glibc ncurses libmad".
448
449 Again, using an OpenEmbedded metadata-based kernel recipe file as an
450 example, the following lines will conditionally append to the
451 ``KERNEL_FEATURES`` variable based on the architecture:
452 KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
453 KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
454 KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc
455 cfg/paravirt_kvm.scc"
456
457- *Setting a Variable for a Single Task:* BitBake supports setting a
458 variable just for the duration of a single task. Here is an example:
459 FOO_task-configure = "val 1" FOO_task-compile = "val 2" In the
460 previous example, ``FOO`` has the value "val 1" while the
461 ``do_configure`` task is executed, and the value "val 2" while the
462 ``do_compile`` task is executed.
463
464 Internally, this is implemented by prepending the task (e.g.
465 "task-compile:") to the value of
466 ```OVERRIDES`` <#var-bb-OVERRIDES>`__ for the local datastore of the
467 ``do_compile`` task.
468
469 You can also use this syntax with other combinations (e.g.
470 "``_prepend``") as shown in the following example:
471 EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} "
472
473Key Expansion
474-------------
475
476Key expansion happens when the BitBake datastore is finalized. To better
477understand this, consider the following example: A${B} = "X" B = "2" A2
478= "Y" In this case, after all the parsing is complete, BitBake expands
479``${B}`` into "2". This expansion causes ``A2``, which was set to "Y"
480before the expansion, to become "X".
481
482.. _variable-interaction-worked-examples:
483
484Examples
485--------
486
487Despite the previous explanations that show the different forms of
488variable definitions, it can be hard to work out exactly what happens
489when variable operators, conditional overrides, and unconditional
490overrides are combined. This section presents some common scenarios
491along with explanations for variable interactions that typically confuse
492users.
493
494There is often confusion concerning the order in which overrides and
495various "append" operators take effect. Recall that an append or prepend
496operation using "_append" and "_prepend" does not result in an immediate
497assignment as would "+=", ".=", "=+", or "=.". Consider the following
498example: OVERRIDES = "foo" A = "Z" A_foo_append = "X" For this case,
499``A`` is unconditionally set to "Z" and "X" is unconditionally and
500immediately appended to the variable ``A_foo``. Because overrides have
501not been applied yet, ``A_foo`` is set to "X" due to the append and
502``A`` simply equals "Z".
503
504Applying overrides, however, changes things. Since "foo" is listed in
505``OVERRIDES``, the conditional variable ``A`` is replaced with the "foo"
506version, which is equal to "X". So effectively, ``A_foo`` replaces
507``A``.
508
509This next example changes the order of the override and the append:
510OVERRIDES = "foo" A = "Z" A_append_foo = "X" For this case, before
511overrides are handled, ``A`` is set to "Z" and ``A_append_foo`` is set
512to "X". Once the override for "foo" is applied, however, ``A`` gets
513appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces
514are not appended.
515
516This next example has the order of the appends and overrides reversed
517back as in the first example: OVERRIDES = "foo" A = "Y" A_foo_append =
518"Z" A_foo_append = "X" For this case, before any overrides are resolved,
519``A`` is set to "Y" using an immediate assignment. After this immediate
520assignment, ``A_foo`` is set to "Z", and then further appended with "X"
521leaving the variable set to "ZX". Finally, applying the override for
522"foo" results in the conditional variable ``A`` becoming "ZX" (i.e.
523``A`` is replaced with ``A_foo``).
524
525This final example mixes in some varying operators: A = "1" A_append =
526"2" A_append = "3" A += "4" A .= "5" For this case, the type of append
527operators are affecting the order of assignments as BitBake passes
528through the code multiple times. Initially, ``A`` is set to "1 45"
529because of the three statements that use immediate operators. After
530these assignments are made, BitBake applies the "_append" operations.
531Those operations result in ``A`` becoming "1 4523".
532
533Sharing Functionality
534=====================
535
536BitBake allows for metadata sharing through include files (``.inc``) and
537class files (``.bbclass``). For example, suppose you have a piece of
538common functionality such as a task definition that you want to share
539between more than one recipe. In this case, creating a ``.bbclass`` file
540that contains the common functionality and then using the ``inherit``
541directive in your recipes to inherit the class would be a common way to
542share the task.
543
544This section presents the mechanisms BitBake provides to allow you to
545share functionality between recipes. Specifically, the mechanisms
546include ``include``, ``inherit``, ``INHERIT``, and ``require``
547directives.
548
549Locating Include and Class Files
550--------------------------------
551
552BitBake uses the ```BBPATH`` <#var-bb-BBPATH>`__ variable to locate
553needed include and class files. Additionally, BitBake searches the
554current directory for ``include`` and ``require`` directives.
555
556.. note::
557
558 The
559 BBPATH
560 variable is analogous to the environment variable
561 PATH
562 .
563
564In order for include and class files to be found by BitBake, they need
565to be located in a "classes" subdirectory that can be found in
566``BBPATH``.
567
568``inherit`` Directive
569---------------------
570
571When writing a recipe or class file, you can use the ``inherit``
572directive to inherit the functionality of a class (``.bbclass``).
573BitBake only supports this directive when used within recipe and class
574files (i.e. ``.bb`` and ``.bbclass``).
575
576The ``inherit`` directive is a rudimentary means of specifying
577functionality contained in class files that your recipes require. For
578example, you can easily abstract out the tasks involved in building a
579package that uses Autoconf and Automake and put those tasks into a class
580file and then have your recipe inherit that class file.
581
582As an example, your recipes could use the following directive to inherit
583an ``autotools.bbclass`` file. The class file would contain common
584functionality for using Autotools that could be shared across recipes:
585inherit autotools In this case, BitBake would search for the directory
586``classes/autotools.bbclass`` in ``BBPATH``.
587
588.. note::
589
590 You can override any values and functions of the inherited class
591 within your recipe by doing so after the "inherit" statement.
592
593If you want to use the directive to inherit multiple classes, separate
594them with spaces. The following example shows how to inherit both the
595``buildhistory`` and ``rm_work`` classes: inherit buildhistory rm_work
596
597An advantage with the inherit directive as compared to both the
598`include <#include-directive>`__ and `require <#require-inclusion>`__
599directives is that you can inherit class files conditionally. You can
600accomplish this by using a variable expression after the ``inherit``
601statement. Here is an example: inherit ${VARNAME} If ``VARNAME`` is
602going to be set, it needs to be set before the ``inherit`` statement is
603parsed. One way to achieve a conditional inherit in this case is to use
604overrides: VARIABLE = "" VARIABLE_someoverride = "myclass"
605
606Another method is by using anonymous Python. Here is an example: python
607() { if condition == value: d.setVar('VARIABLE', 'myclass') else:
608d.setVar('VARIABLE', '') }
609
610Alternatively, you could use an in-line Python expression in the
611following form: inherit ${@'classname' if condition else ''} inherit
612${@functionname(params)} In all cases, if the expression evaluates to an
613empty string, the statement does not trigger a syntax error because it
614becomes a no-op.
615
616``include`` Directive
617---------------------
618
619BitBake understands the ``include`` directive. This directive causes
620BitBake to parse whatever file you specify, and to insert that file at
621that location. The directive is much like its equivalent in Make except
622that if the path specified on the include line is a relative path,
623BitBake locates the first file it can find within ``BBPATH``.
624
625The include directive is a more generic method of including
626functionality as compared to the `inherit <#inherit-directive>`__
627directive, which is restricted to class (i.e. ``.bbclass``) files. The
628include directive is applicable for any other kind of shared or
629encapsulated functionality or configuration that does not suit a
630``.bbclass`` file.
631
632As an example, suppose you needed a recipe to include some self-test
633definitions: include test_defs.inc
634
635.. note::
636
637 The
638 include
639 directive does not produce an error when the file cannot be found.
640 Consequently, it is recommended that if the file you are including is
641 expected to exist, you should use
642 require
643 instead of
644 include
645 . Doing so makes sure that an error is produced if the file cannot be
646 found.
647
648.. _require-inclusion:
649
650``require`` Directive
651---------------------
652
653BitBake understands the ``require`` directive. This directive behaves
654just like the ``include`` directive with the exception that BitBake
655raises a parsing error if the file to be included cannot be found. Thus,
656any file you require is inserted into the file that is being parsed at
657the location of the directive.
658
659The require directive, like the include directive previously described,
660is a more generic method of including functionality as compared to the
661`inherit <#inherit-directive>`__ directive, which is restricted to class
662(i.e. ``.bbclass``) files. The require directive is applicable for any
663other kind of shared or encapsulated functionality or configuration that
664does not suit a ``.bbclass`` file.
665
666Similar to how BitBake handles ```include`` <#include-directive>`__, if
667the path specified on the require line is a relative path, BitBake
668locates the first file it can find within ``BBPATH``.
669
670As an example, suppose you have two versions of a recipe (e.g.
671``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some
672identical functionality that could be shared. You could create an
673include file named ``foo.inc`` that contains the common definitions
674needed to build "foo". You need to be sure ``foo.inc`` is located in the
675same directory as your two recipe files as well. Once these conditions
676are set up, you can share the functionality using a ``require``
677directive from within each recipe: require foo.inc
678
679``INHERIT`` Configuration Directive
680-----------------------------------
681
682When creating a configuration file (``.conf``), you can use the
683```INHERIT`` <#var-bb-INHERIT>`__ configuration directive to inherit a
684class. BitBake only supports this directive when used within a
685configuration file.
686
687As an example, suppose you needed to inherit a class file called
688``abc.bbclass`` from a configuration file as follows: INHERIT += "abc"
689This configuration directive causes the named class to be inherited at
690the point of the directive during parsing. As with the ``inherit``
691directive, the ``.bbclass`` file must be located in a "classes"
692subdirectory in one of the directories specified in ``BBPATH``.
693
694.. note::
695
696 Because
697 .conf
698 files are parsed first during BitBake's execution, using
699 INHERIT
700 to inherit a class effectively inherits the class globally (i.e. for
701 all recipes).
702
703If you want to use the directive to inherit multiple classes, you can
704provide them on the same line in the ``local.conf`` file. Use spaces to
705separate the classes. The following example shows how to inherit both
706the ``autotools`` and ``pkgconfig`` classes: INHERIT += "autotools
707pkgconfig"
708
709Functions
710=========
711
712As with most languages, functions are the building blocks that are used
713to build up operations into tasks. BitBake supports these types of
714functions:
715
716- *Shell Functions:* Functions written in shell script and executed
717 either directly as functions, tasks, or both. They can also be called
718 by other shell functions.
719
720- *BitBake-Style Python Functions:* Functions written in Python and
721 executed by BitBake or other Python functions using
722 ``bb.build.exec_func()``.
723
724- *Python Functions:* Functions written in Python and executed by
725 Python.
726
727- *Anonymous Python Functions:* Python functions executed automatically
728 during parsing.
729
730Regardless of the type of function, you can only define them in class
731(``.bbclass``) and recipe (``.bb`` or ``.inc``) files.
732
733Shell Functions
734---------------
735
736Functions written in shell script and executed either directly as
737functions, tasks, or both. They can also be called by other shell
738functions. Here is an example shell function definition: some_function
739() { echo "Hello World" } When you create these types of functions in
740your recipe or class files, you need to follow the shell programming
741rules. The scripts are executed by ``/bin/sh``, which may not be a bash
742shell but might be something such as ``dash``. You should not use
743Bash-specific script (bashisms).
744
745Overrides and override-style operators like ``_append`` and ``_prepend``
746can also be applied to shell functions. Most commonly, this application
747would be used in a ``.bbappend`` file to modify functions in the main
748recipe. It can also be used to modify functions inherited from classes.
749
750As an example, consider the following: do_foo() { bbplain first fn }
751fn_prepend() { bbplain second } fn() { bbplain third } do_foo_append() {
752bbplain fourth } Running ``do_foo`` prints the following: recipename
753do_foo: first recipename do_foo: second recipename do_foo: third
754recipename do_foo: fourth
755
756.. note::
757
758 Overrides and override-style operators can be applied to any shell
759 function, not just
760 tasks
761 .
762
763You can use the ``bitbake -e`` recipename command to view the final
764assembled function after all overrides have been applied.
765
766BitBake-Style Python Functions
767------------------------------
768
769These functions are written in Python and executed by BitBake or other
770Python functions using ``bb.build.exec_func()``.
771
772An example BitBake function is: python some_python_function () {
773d.setVar("TEXT", "Hello World") print d.getVar("TEXT") } Because the
774Python "bb" and "os" modules are already imported, you do not need to
775import these modules. Also in these types of functions, the datastore
776("d") is a global variable and is always automatically available.
777
778.. note::
779
780 Variable expressions (e.g.
781 ${X}
782 ) are no longer expanded within Python functions. This behavior is
783 intentional in order to allow you to freely set variable values to
784 expandable expressions without having them expanded prematurely. If
785 you do wish to expand a variable within a Python function, use
786 d.getVar("X")
787 . Or, for more complicated expressions, use
788 d.expand()
789 .
790
791Similar to shell functions, you can also apply overrides and
792override-style operators to BitBake-style Python functions.
793
794As an example, consider the following: python do_foo_prepend() {
795bb.plain("first") } python do_foo() { bb.plain("second") } python
796do_foo_append() { bb.plain("third") } Running ``do_foo`` prints the
797following: recipename do_foo: first recipename do_foo: second recipename
798do_foo: third You can use the ``bitbake -e`` recipename command to view
799the final assembled function after all overrides have been applied.
800
801Python Functions
802----------------
803
804These functions are written in Python and are executed by other Python
805code. Examples of Python functions are utility functions that you intend
806to call from in-line Python or from within other Python functions. Here
807is an example: def get_depends(d): if d.getVar('SOMECONDITION'): return
808"dependencywithcond" else: return "dependency" SOMECONDITION = "1"
809DEPENDS = "${@get_depends(d)}" This would result in ``DEPENDS``
810containing ``dependencywithcond``.
811
812Here are some things to know about Python functions:
813
814- Python functions can take parameters.
815
816- The BitBake datastore is not automatically available. Consequently,
817 you must pass it in as a parameter to the function.
818
819- The "bb" and "os" Python modules are automatically available. You do
820 not need to import them.
821
822BitBake-Style Python Functions Versus Python Functions
823------------------------------------------------------
824
825Following are some important differences between BitBake-style Python
826functions and regular Python functions defined with "def":
827
828- Only BitBake-style Python functions can be `tasks <#tasks>`__.
829
830- Overrides and override-style operators can only be applied to
831 BitBake-style Python functions.
832
833- Only regular Python functions can take arguments and return values.
834
835- `Variable flags <#variable-flags>`__ such as ``[dirs]``,
836 ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style
837 Python functions, but not on regular Python functions.
838
839- BitBake-style Python functions generate a separate
840 ``${``\ ```T`` <#var-bb-T>`__\ ``}/run.``\ function-name\ ``.``\ pid
841 script that is executed to run the function, and also generate a log
842 file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed
843 as tasks.
844
845 Regular Python functions execute "inline" and do not generate any
846 files in ``${T}``.
847
848- Regular Python functions are called with the usual Python syntax.
849 BitBake-style Python functions are usually tasks and are called
850 directly by BitBake, but can also be called manually from Python code
851 by using the ``bb.build.exec_func()`` function. Here is an example:
852 bb.build.exec_func("my_bitbake_style_function", d)
853
854 .. note::
855
856 bb.build.exec_func()
857 can also be used to run shell functions from Python code. If you
858 want to run a shell function before a Python function within the
859 same task, then you can use a parent helper Python function that
860 starts by running the shell function with
861 bb.build.exec_func()
862 and then runs the Python code.
863
864 To detect errors from functions executed with
865 ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed``
866 exception.
867
868 .. note::
869
870 Functions in metadata (recipes and classes) should not themselves
871 raise
872 bb.build.FuncFailed
873 . Rather,
874 bb.build.FuncFailed
875 should be viewed as a general indicator that the called function
876 failed by raising an exception. For example, an exception raised
877 by
878 bb.fatal()
879 will be caught inside
880 bb.build.exec_func()
881 , and a
882 bb.build.FuncFailed
883 will be raised in response.
884
885Due to their simplicity, you should prefer regular Python functions over
886BitBake-style Python functions unless you need a feature specific to
887BitBake-style Python functions. Regular Python functions in metadata are
888a more recent invention than BitBake-style Python functions, and older
889code tends to use ``bb.build.exec_func()`` more often.
890
891Anonymous Python Functions
892--------------------------
893
894Sometimes it is useful to set variables or perform other operations
895programmatically during parsing. To do this, you can define special
896Python functions, called anonymous Python functions, that run at the end
897of parsing. For example, the following conditionally sets a variable
898based on the value of another variable: python () { if
899d.getVar('SOMEVAR') == 'value': d.setVar('ANOTHERVAR', 'value2') } An
900equivalent way to mark a function as an anonymous function is to give it
901the name "__anonymous", rather than no name.
902
903Anonymous Python functions always run at the end of parsing, regardless
904of where they are defined. If a recipe contains many anonymous
905functions, they run in the same order as they are defined within the
906recipe. As an example, consider the following snippet: python () {
907d.setVar('FOO', 'foo 2') } FOO = "foo 1" python () { d.appendVar('BAR',
908' bar 2') } BAR = "bar 1" The previous example is conceptually
909equivalent to the following snippet: FOO = "foo 1" BAR = "bar 1" FOO =
910"foo 2" BAR += "bar 2" ``FOO`` ends up with the value "foo 2", and
911``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the
912values set for the variables within the anonymous functions become
913available to tasks, which always run after parsing.
914
915Overrides and override-style operators such as "``_append``" are applied
916before anonymous functions run. In the following example, ``FOO`` ends
917up with the value "foo from anonymous": FOO = "foo" FOO_append = " from
918outside" python () { d.setVar("FOO", "foo from anonymous") } For methods
919you can use with anonymous Python functions, see the "`Functions You Can
920Call From Within Python <#functions-you-can-call-from-within-python>`__"
921section. For a different method to run Python code during parsing, see
922the "`Inline Python Variable
923Expansion <#inline-python-variable-expansion>`__" section.
924
925Flexible Inheritance for Class Functions
926----------------------------------------
927
928Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake
929supports exporting a function from a class such that the class function
930appears as the default implementation of the function, but can still be
931called if a recipe inheriting the class needs to define its own version
932of the function.
933
934To understand the benefits of this feature, consider the basic scenario
935where a class defines a task function and your recipe inherits the
936class. In this basic scenario, your recipe inherits the task function as
937defined in the class. If desired, your recipe can add to the start and
938end of the function by using the "_prepend" or "_append" operations
939respectively, or it can redefine the function completely. However, if it
940redefines the function, there is no means for it to call the class
941version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that
942enables the recipe's version of the function to call the original
943version of the function.
944
945To make use of this technique, you need the following things in place:
946
947- The class needs to define the function as follows:
948 classname\ ``_``\ functionname For example, if you have a class file
949 ``bar.bbclass`` and a function named ``do_foo``, the class must
950 define the function as follows: bar_do_foo
951
952- The class needs to contain the ``EXPORT_FUNCTIONS`` statement as
953 follows: EXPORT_FUNCTIONS functionname For example, continuing with
954 the same example, the statement in the ``bar.bbclass`` would be as
955 follows: EXPORT_FUNCTIONS do_foo
956
957- You need to call the function appropriately from within your recipe.
958 Continuing with the same example, if your recipe needs to call the
959 class version of the function, it should call ``bar_do_foo``.
960 Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was
961 used as above, the recipe's function could conditionally call the
962 class version of the function as follows: do_foo() { if [
963 somecondition ] ; then bar_do_foo else # Do something else fi } To
964 call your modified version of the function as defined in your recipe,
965 call it as ``do_foo``.
966
967With these conditions met, your single recipe can freely choose between
968the original function as defined in the class file and the modified
969function in your recipe. If you do not set up these conditions, you are
970limited to using one function or the other.
971
972Tasks
973=====
974
975Tasks are BitBake execution units that make up the steps that BitBake
976can run for a given recipe. Tasks are only supported in recipes and
977classes (i.e. in ``.bb`` files and files included or inherited from
978``.bb`` files). By convention, tasks have names that start with "do_".
979
980Promoting a Function to a Task
981------------------------------
982
983Tasks are either `shell functions <#shell-functions>`__ or
984`BitBake-style Python functions <#bitbake-style-python-functions>`__
985that have been promoted to tasks by using the ``addtask`` command. The
986``addtask`` command can also optionally describe dependencies between
987the task and other tasks. Here is an example that shows how to define a
988task and declare some dependencies: python do_printdate () { import time
989print time.strftime('%Y%m%d', time.gmtime()) } addtask printdate after
990do_fetch before do_build The first argument to ``addtask`` is the name
991of the function to promote to a task. If the name does not start with
992"do_", "do_" is implicitly added, which enforces the convention that all
993task names start with "do_".
994
995In the previous example, the ``do_printdate`` task becomes a dependency
996of the ``do_build`` task, which is the default task (i.e. the task run
997by the ``bitbake`` command unless another task is specified explicitly).
998Additionally, the ``do_printdate`` task becomes dependent upon the
999``do_fetch`` task. Running the ``do_build`` task results in the
1000``do_printdate`` task running first.
1001
1002.. note::
1003
1004 If you try out the previous example, you might see that the
1005 do_printdate
1006 task is only run the first time you build the recipe with the
1007 bitbake
1008 command. This is because BitBake considers the task "up-to-date"
1009 after that initial run. If you want to force the task to always be
1010 rerun for experimentation purposes, you can make BitBake always
1011 consider the task "out-of-date" by using the
1012 [
1013 nostamp
1014 ]
1015 variable flag, as follows:
1016 ::
1017
1018 do_printdate[nostamp] = "1"
1019
1020
1021 You can also explicitly run the task and provide the
1022 -f
1023 option as follows:
1024 ::
1025
1026 $ bitbake recipe -c printdate -f
1027
1028
1029 When manually selecting a task to run with the
1030 bitbake
1031 NBSP
1032 recipe
1033 NBSP
1034 -c
1035 NBSP
1036 task
1037 command, you can omit the "do_" prefix as part of the task name.
1038
1039You might wonder about the practical effects of using ``addtask``
1040without specifying any dependencies as is done in the following example:
1041addtask printdate In this example, assuming dependencies have not been
1042added through some other means, the only way to run the task is by
1043explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You
1044can use the ``do_listtasks`` task to list all tasks defined in a recipe
1045as shown in the following example: $ bitbake recipe -c listtasks For
1046more information on task dependencies, see the
1047"`Dependencies <#dependencies>`__" section.
1048
1049See the "`Variable Flags <#variable-flags>`__" section for information
1050on variable flags you can use with tasks.
1051
1052Deleting a Task
1053---------------
1054
1055As well as being able to add tasks, you can delete them. Simply use the
1056``deltask`` command to delete a task. For example, to delete the example
1057task used in the previous sections, you would use: deltask printdate If
1058you delete a task using the ``deltask`` command and the task has
1059dependencies, the dependencies are not reconnected. For example, suppose
1060you have three tasks named ``do_a``, ``do_b``, and ``do_c``.
1061Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is
1062dependent on ``do_a``. Given this scenario, if you use ``deltask`` to
1063delete ``do_b``, the implicit dependency relationship between ``do_c``
1064and ``do_a`` through ``do_b`` no longer exists, and ``do_c``
1065dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free
1066to run before ``do_a``.
1067
1068If you want dependencies such as these to remain intact, use the
1069``[noexec]`` varflag to disable the task instead of using the
1070``deltask`` command to delete it: do_b[noexec] = "1"
1071
1072Passing Information Into the Build Task Environment
1073---------------------------------------------------
1074
1075When running a task, BitBake tightly controls the shell execution
1076environment of the build tasks to make sure unwanted contamination from
1077the build machine cannot influence the build.
1078
1079.. note::
1080
1081 By default, BitBake cleans the environment to include only those
1082 things exported or listed in its whitelist to ensure that the build
1083 environment is reproducible and consistent. You can prevent this
1084 "cleaning" by setting the
1085 BB_PRESERVE_ENV
1086 variable.
1087
1088Consequently, if you do want something to get passed into the build task
1089environment, you must take these two steps:
1090
10911. Tell BitBake to load what you want from the environment into the
1092 datastore. You can do so through the
1093 ```BB_ENV_WHITELIST`` <#var-bb-BB_ENV_WHITELIST>`__ and
1094 ```BB_ENV_EXTRAWHITE`` <#var-bb-BB_ENV_EXTRAWHITE>`__ variables. For
1095 example, assume you want to prevent the build system from accessing
1096 your ``$HOME/.ccache`` directory. The following command "whitelists"
1097 the environment variable ``CCACHE_DIR`` causing BitBake to allow that
1098 variable into the datastore: export
1099 BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1100
11012. Tell BitBake to export what you have loaded into the datastore to the
1102 task environment of every running task. Loading something from the
1103 environment into the datastore (previous step) only makes it
1104 available in the datastore. To export it to the task environment of
1105 every running task, use a command similar to the following in your
1106 local configuration file ``local.conf`` or your distribution
1107 configuration file: export CCACHE_DIR
1108
1109 .. note::
1110
1111 A side effect of the previous steps is that BitBake records the
1112 variable as a dependency of the build process in things like the
1113 setscene checksums. If doing so results in unnecessary rebuilds of
1114 tasks, you can whitelist the variable so that the setscene code
1115 ignores the dependency when it creates checksums.
1116
1117Sometimes, it is useful to be able to obtain information from the
1118original execution environment. BitBake saves a copy of the original
1119environment into a special variable named
1120```BB_ORIGENV`` <#var-bb-BB_ORIGENV>`__.
1121
1122The ``BB_ORIGENV`` variable returns a datastore object that can be
1123queried using the standard datastore operators such as
1124``getVar(, False)``. The datastore object is useful, for example, to
1125find the original ``DISPLAY`` variable. Here is an example: origenv =
1126d.getVar("BB_ORIGENV", False) bar = origenv.getVar("BAR", False) The
1127previous example returns ``BAR`` from the original execution
1128environment.
1129
1130Variable Flags
1131==============
1132
1133Variable flags (varflags) help control a task's functionality and
1134dependencies. BitBake reads and writes varflags to the datastore using
1135the following command forms: variable = d.getVarFlags("variable")
1136self.d.setVarFlags("FOO", {"func": True})
1137
1138When working with varflags, the same syntax, with the exception of
1139overrides, applies. In other words, you can set, append, and prepend
1140varflags just like variables. See the "`Variable Flag
1141Syntax <#variable-flag-syntax>`__" section for details.
1142
1143BitBake has a defined set of varflags available for recipes and classes.
1144Tasks support a number of these flags which control various
1145functionality of the task:
1146
1147- *``[cleandirs]``:* Empty directories that should be created before
1148 the task runs. Directories that already exist are removed and
1149 recreated to empty them.
1150
1151- *``[depends]``:* Controls inter-task dependencies. See the
1152 ```DEPENDS`` <#var-bb-DEPENDS>`__ variable and the "`Inter-Task
1153 Dependencies <#inter-task-dependencies>`__" section for more
1154 information.
1155
1156- *``[deptask]``:* Controls task build-time dependencies. See the
1157 ```DEPENDS`` <#var-bb-DEPENDS>`__ variable and the "`Build
1158 Dependencies <#build-dependencies>`__" section for more information.
1159
1160- *``[dirs]``:* Directories that should be created before the task
1161 runs. Directories that already exist are left as is. The last
1162 directory listed is used as the current working directory for the
1163 task.
1164
1165- *``[lockfiles]``:* Specifies one or more lockfiles to lock while the
1166 task executes. Only one task may hold a lockfile, and any task that
1167 attempts to lock an already locked file will block until the lock is
1168 released. You can use this variable flag to accomplish mutual
1169 exclusion.
1170
1171- *``[noexec]``:* When set to "1", marks the task as being empty, with
1172 no execution required. You can use the ``[noexec]`` flag to set up
1173 tasks as dependency placeholders, or to disable tasks defined
1174 elsewhere that are not needed in a particular recipe.
1175
1176- *``[nostamp]``:* When set to "1", tells BitBake to not generate a
1177 stamp file for a task, which implies the task should always be
1178 executed.
1179
1180 .. note::
1181
1182 Any task that depends (possibly indirectly) on a
1183 [nostamp]
1184 task will always be executed as well. This can cause unnecessary
1185 rebuilding if you are not careful.
1186
1187- *``[number_threads]``:* Limits tasks to a specific number of
1188 simultaneous threads during execution. This varflag is useful when
1189 your build host has a large number of cores but certain tasks need to
1190 be rate-limited due to various kinds of resource constraints (e.g. to
1191 avoid network throttling). ``number_threads`` works similarly to the
1192 ```BB_NUMBER_THREADS`` <#var-bb-BB_NUMBER_THREADS>`__ variable but is
1193 task-specific.
1194
1195 Set the value globally. For example, the following makes sure the
1196 ``do_fetch`` task uses no more than two simultaneous execution
1197 threads: do_fetch[number_threads] = "2"
1198
1199 .. note::
1200
1201 - Setting the varflag in individual recipes rather than globally
1202 can result in unpredictable behavior.
1203
1204 - Setting the varflag to a value greater than the value used in
1205 the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to
1206 have no effect.
1207
1208- *``[postfuncs]``:* List of functions to call after the completion of
1209 the task.
1210
1211- *``[prefuncs]``:* List of functions to call before the task executes.
1212
1213- *``[rdepends]``:* Controls inter-task runtime dependencies. See the
1214 ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1215 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the
1216 "`Inter-Task Dependencies <#inter-task-dependencies>`__" section for
1217 more information.
1218
1219- *``[rdeptask]``:* Controls task runtime dependencies. See the
1220 ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1221 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the "`Runtime
1222 Dependencies <#runtime-dependencies>`__" section for more
1223 information.
1224
1225- *``[recideptask]``:* When set in conjunction with ``recrdeptask``,
1226 specifies a task that should be inspected for additional
1227 dependencies.
1228
1229- *``[recrdeptask]``:* Controls task recursive runtime dependencies.
1230 See the ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable, the
1231 ```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variable, and the
1232 "`Recursive Dependencies <#recursive-dependencies>`__" section for
1233 more information.
1234
1235- *``[stamp-extra-info]``:* Extra stamp information to append to the
1236 task's stamp. As an example, OpenEmbedded uses this flag to allow
1237 machine-specific tasks.
1238
1239- *``[umask]``:* The umask to run the task under.
1240
1241Several varflags are useful for controlling how signatures are
1242calculated for variables. For more information on this process, see the
1243"`Checksums (Signatures) <#checksums>`__" section.
1244
1245- *``[vardeps]``:* Specifies a space-separated list of additional
1246 variables to add to a variable's dependencies for the purposes of
1247 calculating its signature. Adding variables to this list is useful,
1248 for example, when a function refers to a variable in a manner that
1249 does not allow BitBake to automatically determine that the variable
1250 is referred to.
1251
1252- *``[vardepsexclude]``:* Specifies a space-separated list of variables
1253 that should be excluded from a variable's dependencies for the
1254 purposes of calculating its signature.
1255
1256- *``[vardepvalue]``:* If set, instructs BitBake to ignore the actual
1257 value of the variable and instead use the specified value when
1258 calculating the variable's signature.
1259
1260- *``[vardepvalueexclude]``:* Specifies a pipe-separated list of
1261 strings to exclude from the variable's value when calculating the
1262 variable's signature.
1263
1264Events
1265======
1266
1267BitBake allows installation of event handlers within recipe and class
1268files. Events are triggered at certain points during operation, such as
1269the beginning of operation against a given recipe (i.e. ``*.bb``), the
1270start of a given task, a task failure, a task success, and so forth. The
1271intent is to make it easy to do things like email notification on build
1272failures.
1273
1274Following is an example event handler that prints the name of the event
1275and the content of the ``FILE`` variable: addhandler
1276myclass_eventhandler python myclass_eventhandler() { from bb.event
1277import getName print("The name of the Event is %s" % getName(e))
1278print("The file we run for is %s" % d.getVar('FILE')) }
1279myclass_eventhandler[eventmask] = "bb.event.BuildStarted
1280bb.event.BuildCompleted" In the previous example, an eventmask has been
1281set so that the handler only sees the "BuildStarted" and
1282"BuildCompleted" events. This event handler gets called every time an
1283event matching the eventmask is triggered. A global variable "e" is
1284defined, which represents the current event. With the ``getName(e)``
1285method, you can get the name of the triggered event. The global
1286datastore is available as "d". In legacy code, you might see "e.data"
1287used to get the datastore. However, realize that "e.data" is deprecated
1288and you should use "d" going forward.
1289
1290The context of the datastore is appropriate to the event in question.
1291For example, "BuildStarted" and "BuildCompleted" events run before any
1292tasks are executed so would be in the global configuration datastore
1293namespace. No recipe-specific metadata exists in that namespace. The
1294"BuildStarted" and "BuildCompleted" events also run in the main
1295cooker/server process rather than any worker context. Thus, any changes
1296made to the datastore would be seen by other cooker/server events within
1297the current build but not seen outside of that build or in any worker
1298context. Task events run in the actual tasks in question consequently
1299have recipe-specific and task-specific contents. These events run in the
1300worker context and are discarded at the end of task execution.
1301
1302During a standard build, the following common events might occur. The
1303following events are the most common kinds of events that most metadata
1304might have an interest in viewing:
1305
1306- ``bb.event.ConfigParsed()``: Fired when the base configuration; which
1307 consists of ``bitbake.conf``, ``base.bbclass`` and any global
1308 ``INHERIT`` statements; has been parsed. You can see multiple such
1309 events when each of the workers parse the base configuration or if
1310 the server changes configuration and reparses. Any given datastore
1311 only has one such event executed against it, however. If
1312 ```BB_INVALIDCONF`` <#>`__ is set in the datastore by the event
1313 handler, the configuration is reparsed and a new event triggered,
1314 allowing the metadata to update configuration.
1315
1316- ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one
1317 second. You can configure the interval time using the
1318 ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the
1319 ``time.time()`` value when the event is triggered. This event is
1320 useful for activities such as system state monitoring.
1321
1322- ``bb.event.ParseStarted()``: Fired when BitBake is about to start
1323 parsing recipes. This event's "total" attribute represents the number
1324 of recipes BitBake plans to parse.
1325
1326- ``bb.event.ParseProgress()``: Fired as parsing progresses. This
1327 event's "current" attribute is the number of recipes parsed as well
1328 as the "total" attribute.
1329
1330- ``bb.event.ParseCompleted()``: Fired when parsing is complete. This
1331 event's "cached", "parsed", "skipped", "virtuals", "masked", and
1332 "errors" attributes provide statistics for the parsing results.
1333
1334- ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake
1335 fires multiple "BuildStarted" events (one per configuration) when
1336 multiple configuration (multiconfig) is enabled.
1337
1338- ``bb.build.TaskStarted()``: Fired when a task starts. This event's
1339 "taskfile" attribute points to the recipe from which the task
1340 originates. The "taskname" attribute, which is the task's name,
1341 includes the ``do_`` prefix, and the "logfile" attribute point to
1342 where the task's output is stored. Finally, the "time" attribute is
1343 the task's execution start time.
1344
1345- ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task
1346 that does not exist.
1347
1348- ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail
1349 and should not be presented to the user verbosely.
1350
1351- ``bb.build.TaskFailed()``: Fired for normal tasks that fail.
1352
1353- ``bb.build.TaskSucceeded()``: Fired when a task successfully
1354 completes.
1355
1356- ``bb.event.BuildCompleted()``: Fired when a build finishes.
1357
1358- ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker
1359 shuts down. This event is usually only seen by the UIs as a sign they
1360 should also shutdown.
1361
1362This next list of example events occur based on specific requests to the
1363server. These events are often used to communicate larger pieces of
1364information from the BitBake server to other parts of BitBake such as
1365user interfaces:
1366
1367- ``bb.event.TreeDataPreparationStarted()``
1368
1369- ``bb.event.TreeDataPreparationProgress()``
1370
1371- ``bb.event.TreeDataPreparationCompleted()``
1372
1373- ``bb.event.DepTreeGenerated()``
1374
1375- ``bb.event.CoreBaseFilesFound()``
1376
1377- ``bb.event.ConfigFilePathFound()``
1378
1379- ``bb.event.FilesMatchingFound()``
1380
1381- ``bb.event.ConfigFilesFound()``
1382
1383- ``bb.event.TargetsTreeGenerated()``
1384
1385.. _variants-class-extension-mechanism:
1386
1387Variants - Class Extension Mechanism
1388====================================
1389
1390BitBake supports two features that facilitate creating from a single
1391recipe file multiple incarnations of that recipe file where all
1392incarnations are buildable. These features are enabled through the
1393```BBCLASSEXTEND`` <#var-bb-BBCLASSEXTEND>`__ and
1394```BBVERSIONS`` <#var-bb-BBVERSIONS>`__ variables.
1395
1396.. note::
1397
1398 The mechanism for this class extension is extremely specific to the
1399 implementation. Usually, the recipe's
1400 PROVIDES
1401 ,
1402 PN
1403 , and
1404 DEPENDS
1405 variables would need to be modified by the extension class. For
1406 specific examples, see the OE-Core
1407 native
1408 ,
1409 nativesdk
1410 , and
1411 multilib
1412 classes.
1413
1414- *``BBCLASSEXTEND``:* This variable is a space separated list of
1415 classes used to "extend" the recipe for each variant. Here is an
1416 example that results in a second incarnation of the current recipe
1417 being available. This second incarnation will have the "native" class
1418 inherited. BBCLASSEXTEND = "native"
1419
1420- *``BBVERSIONS``:* This variable allows a single recipe to build
1421 multiple versions of a project from a single recipe file. You can
1422 also specify conditional metadata (using the
1423 ```OVERRIDES`` <#var-bb-OVERRIDES>`__ mechanism) for a single
1424 version, or an optionally named range of versions. Here is an
1425 example: BBVERSIONS = "1.0 2.0 git" SRC_URI_git =
1426 "git://someurl/somepath.git" BBVERSIONS = "1.0.[0-6]:1.0.0+ \\
1427 1.0.[7-9]:1.0.7+" SRC_URI_append_1.0.7+ =
1428 "file://some_patch_which_the_new_versions_need.patch;patch=1" The
1429 name of the range defaults to the original version of the recipe. For
1430 example, in OpenEmbedded, the recipe file ``foo_1.0.0+.bb`` creates a
1431 default name range of ``1.0.0+``. This is useful because the range
1432 name is not only placed into overrides, but it is also made available
1433 for the metadata to use in the variable that defines the base recipe
1434 versions for use in ``file://`` search paths
1435 (```FILESPATH`` <#var-bb-FILESPATH>`__).
1436
1437Dependencies
1438============
1439
1440To allow for efficient parallel processing, BitBake handles dependencies
1441at the task level. Dependencies can exist both between tasks within a
1442single recipe and between tasks in different recipes. Following are
1443examples of each:
1444
1445- For tasks within a single recipe, a recipe's ``do_configure`` task
1446 might need to complete before its ``do_compile`` task can run.
1447
1448- For tasks in different recipes, one recipe's ``do_configure`` task
1449 might require another recipe's ``do_populate_sysroot`` task to finish
1450 first such that the libraries and headers provided by the other
1451 recipe are available.
1452
1453This section describes several ways to declare dependencies. Remember,
1454even though dependencies are declared in different ways, they are all
1455simply dependencies between tasks.
1456
1457.. _dependencies-internal-to-the-bb-file:
1458
1459Dependencies Internal to the ``.bb`` File
1460-----------------------------------------
1461
1462BitBake uses the ``addtask`` directive to manage dependencies that are
1463internal to a given recipe file. You can use the ``addtask`` directive
1464to indicate when a task is dependent on other tasks or when other tasks
1465depend on that recipe. Here is an example: addtask printdate after
1466do_fetch before do_build In this example, the ``do_printdate`` task
1467depends on the completion of the ``do_fetch`` task, and the ``do_build``
1468task depends on the completion of the ``do_printdate`` task.
1469
1470.. note::
1471
1472 For a task to run, it must be a direct or indirect dependency of some
1473 other task that is scheduled to run.
1474
1475 For illustration, here are some examples:
1476
1477 - The directive ``addtask mytask before do_configure`` causes
1478 ``do_mytask`` to run before ``do_configure`` runs. Be aware that
1479 ``do_mytask`` still only runs if its `input
1480 checksum <#checksums>`__ has changed since the last time it was
1481 run. Changes to the input checksum of ``do_mytask`` also
1482 indirectly cause ``do_configure`` to run.
1483
1484 - The directive ``addtask mytask after do_configure`` by itself
1485 never causes ``do_mytask`` to run. ``do_mytask`` can still be run
1486 manually as follows: $ bitbake recipe -c mytask Declaring
1487 ``do_mytask`` as a dependency of some other task that is scheduled
1488 to run also causes it to run. Regardless, the task runs after
1489 ``do_configure``.
1490
1491Build Dependencies
1492------------------
1493
1494BitBake uses the ```DEPENDS`` <#var-bb-DEPENDS>`__ variable to manage
1495build time dependencies. The ``[deptask]`` varflag for tasks signifies
1496the task of each item listed in ``DEPENDS`` that must complete before
1497that task can be executed. Here is an example: do_configure[deptask] =
1498"do_populate_sysroot" In this example, the ``do_populate_sysroot`` task
1499of each item in ``DEPENDS`` must complete before ``do_configure`` can
1500execute.
1501
1502Runtime Dependencies
1503--------------------
1504
1505BitBake uses the ```PACKAGES`` <#var-bb-PACKAGES>`__,
1506```RDEPENDS`` <#var-bb-RDEPENDS>`__, and
1507```RRECOMMENDS`` <#var-bb-RRECOMMENDS>`__ variables to manage runtime
1508dependencies.
1509
1510The ``PACKAGES`` variable lists runtime packages. Each of those packages
1511can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The
1512``[rdeptask]`` flag for tasks is used to signify the task of each item
1513runtime dependency which must have completed before that task can be
1514executed. do_package_qa[rdeptask] = "do_packagedata" In the previous
1515example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must
1516have completed before ``do_package_qa`` can execute.
1517Although ``RDEPENDS`` contains entries from the
1518runtime dependency namespace, BitBake knows how to map them back
1519to the build-time dependency namespace, in which the tasks are defined.
1520
1521Recursive Dependencies
1522----------------------
1523
1524BitBake uses the ``[recrdeptask]`` flag to manage recursive task
1525dependencies. BitBake looks through the build-time and runtime
1526dependencies of the current recipe, looks through the task's inter-task
1527dependencies, and then adds dependencies for the listed task. Once
1528BitBake has accomplished this, it recursively works through the
1529dependencies of those tasks. Iterative passes continue until all
1530dependencies are discovered and added.
1531
1532The ``[recrdeptask]`` flag is most commonly used in high-level recipes
1533that need to wait for some task to finish "globally". For example,
1534``image.bbclass`` has the following: do_rootfs[recrdeptask] +=
1535"do_packagedata" This statement says that the ``do_packagedata`` task of
1536the current recipe and all recipes reachable (by way of dependencies)
1537from the image recipe must run before the ``do_rootfs`` task can run.
1538
1539BitBake allows a task to recursively depend on itself by
1540referencing itself in the task list:
1541do_a[recrdeptask] = "do_a do_b"
1542
1543In the same way as before, this means that the ``do_a``
1544and ``do_b`` tasks of the current recipe and all
1545recipes reachable (by way of dependencies) from the recipe
1546must run before the ``do_a`` task can run. In this
1547case BitBake will ignore the current recipe's ``do_a``
1548task circular dependency on itself.
1549
1550Inter-Task Dependencies
1551-----------------------
1552
1553BitBake uses the ``[depends]`` flag in a more generic form to manage
1554inter-task dependencies. This more generic form allows for
1555inter-dependency checks for specific tasks rather than checks for the
1556data in ``DEPENDS``. Here is an example: do_patch[depends] =
1557"quilt-native:do_populate_sysroot" In this example, the
1558``do_populate_sysroot`` task of the target ``quilt-native`` must have
1559completed before the ``do_patch`` task can execute.
1560
1561The ``[rdepends]`` flag works in a similar way but takes targets in the
1562runtime namespace instead of the build-time dependency namespace.
1563
1564Functions You Can Call From Within Python
1565=========================================
1566
1567BitBake provides many functions you can call from within Python
1568functions. This section lists the most commonly used functions, and
1569mentions where to find others.
1570
1571Functions for Accessing Datastore Variables
1572-------------------------------------------
1573
1574It is often necessary to access variables in the BitBake datastore using
1575Python functions. The BitBake datastore has an API that allows you this
1576access. Here is a list of available operations:
1577
1578+-----------------------------------+-----------------------------------+
1579| *Operation* | *Description* |
1580+===================================+===================================+
1581| ``d.getVar("X", expand)`` | Returns the value of variable |
1582| | "X". Using "expand=True" expands |
1583| | the value. Returns "None" if the |
1584| | variable "X" does not exist. |
1585+-----------------------------------+-----------------------------------+
1586| ``d.setVar("X", "value")`` | Sets the variable "X" to "value". |
1587+-----------------------------------+-----------------------------------+
1588| ``d.appendVar("X", "value")`` | Adds "value" to the end of the |
1589| | variable "X". Acts like |
1590| | ``d.setVar("X", "value")`` if the |
1591| | variable "X" does not exist. |
1592+-----------------------------------+-----------------------------------+
1593| ``d.prependVar("X", "value")`` | Adds "value" to the start of the |
1594| | variable "X". Acts like |
1595| | ``d.setVar("X", "value")`` if the |
1596| | variable "X" does not exist. |
1597+-----------------------------------+-----------------------------------+
1598| ``d.delVar("X")`` | Deletes the variable "X" from the |
1599| | datastore. Does nothing if the |
1600| | variable "X" does not exist. |
1601+-----------------------------------+-----------------------------------+
1602| ``d.renameVar("X", "Y")`` | Renames the variable "X" to "Y". |
1603| | Does nothing if the variable "X" |
1604| | does not exist. |
1605+-----------------------------------+-----------------------------------+
1606| `` | Returns the value of variable |
1607| d.getVarFlag("X", flag, expand)`` | "X". Using "expand=True" expands |
1608| | the value. Returns "None" if |
1609| | either the variable "X" or the |
1610| | named flag does not exist. |
1611+-----------------------------------+-----------------------------------+
1612| ``d | Sets the named flag for variable |
1613| .setVarFlag("X", flag, "value")`` | "X" to "value". |
1614+-----------------------------------+-----------------------------------+
1615| ``d.ap | Appends "value" to the named flag |
1616| pendVarFlag("X", flag, "value")`` | on the variable "X". Acts like |
1617| | ``d |
1618| | .setVarFlag("X", flag, "value")`` |
1619| | if the named flag does not exist. |
1620+-----------------------------------+-----------------------------------+
1621| ``d.pre | Prepends "value" to the named |
1622| pendVarFlag("X", flag, "value")`` | flag on the variable "X". Acts |
1623| | like |
1624| | ``d |
1625| | .setVarFlag("X", flag, "value")`` |
1626| | if the named flag does not exist. |
1627+-----------------------------------+-----------------------------------+
1628| ``d.delVarFlag("X", flag)`` | Deletes the named flag on the |
1629| | variable "X" from the datastore. |
1630+-----------------------------------+-----------------------------------+
1631| ``d.setVarFlags("X", flagsdict)`` | Sets the flags specified in the |
1632| | ``flagsdict()`` parameter. |
1633| | ``setVarFlags`` does not clear |
1634| | previous flags. Think of this |
1635| | operation as ``addVarFlags``. |
1636+-----------------------------------+-----------------------------------+
1637| ``d.getVarFlags("X")`` | Returns a ``flagsdict`` of the |
1638| | flags for the variable "X". |
1639| | Returns "None" if the variable |
1640| | "X" does not exist. |
1641+-----------------------------------+-----------------------------------+
1642| ``d.delVarFlags("X")`` | Deletes all the flags for the |
1643| | variable "X". Does nothing if the |
1644| | variable "X" does not exist. |
1645+-----------------------------------+-----------------------------------+
1646| ``d.expand(expression)`` | Expands variable references in |
1647| | the specified string expression. |
1648| | References to variables that do |
1649| | not exist are left as is. For |
1650| | example, ``d.expand("foo ${X}")`` |
1651| | expands to the literal string |
1652| | "foo ${X}" if the variable "X" |
1653| | does not exist. |
1654+-----------------------------------+-----------------------------------+
1655
1656Other Functions
1657---------------
1658
1659You can find many other functions that can be called from Python by
1660looking at the source code of the ``bb`` module, which is in
1661``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes
1662the commonly used functions ``bb.utils.contains()`` and
1663``bb.utils.mkdirhier()``, which come with docstrings.
1664
1665Task Checksums and Setscene
1666===========================
1667
1668BitBake uses checksums (or signatures) along with the setscene to
1669determine if a task needs to be run. This section describes the process.
1670To help understand how BitBake does this, the section assumes an
1671OpenEmbedded metadata-based example.
1672
1673These checksums are stored in ```STAMP`` <#var-bb-STAMP>`__. You can
1674examine the checksums using the following BitBake command: $
1675bitbake-dumpsigs This command returns the signature data in a readable
1676format that allows you to examine the inputs used when the OpenEmbedded
1677build system generates signatures. For example, using
1678``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's
1679“sigdata” for a C application (e.g. ``bash``). Running the command also
1680reveals that the “CC” variable is part of the inputs that are hashed.
1681Any changes to this variable would invalidate the stamp and cause the
1682``do_compile`` task to run.
1683
1684The following list describes related variables:
1685
1686- ```BB_HASHCHECK_FUNCTION`` <#var-bb-BB_HASHCHECK_FUNCTION>`__:
1687 Specifies the name of the function to call during the "setscene" part
1688 of the task's execution in order to validate the list of task hashes.
1689
1690- ```BB_SETSCENE_DEPVALID`` <#var-bb-BB_SETSCENE_DEPVALID>`__:
1691 Specifies a function BitBake calls that determines whether BitBake
1692 requires a setscene dependency to be met.
1693
1694- ```BB_SETSCENE_VERIFY_FUNCTION2`` <#var-bb-BB_SETSCENE_VERIFY_FUNCTION2>`__:
1695 Specifies a function to call that verifies the list of planned task
1696 execution before the main task execution happens.
1697
1698- ```BB_STAMP_POLICY`` <#var-bb-BB_STAMP_POLICY>`__: Defines the mode
1699 for comparing timestamps of stamp files.
1700
1701- ```BB_STAMP_WHITELIST`` <#var-bb-BB_STAMP_WHITELIST>`__: Lists stamp
1702 files that are looked at when the stamp policy is "whitelist".
1703
1704- ```BB_TASKHASH`` <#var-bb-BB_TASKHASH>`__: Within an executing task,
1705 this variable holds the hash of the task as returned by the currently
1706 enabled signature generator.
1707
1708- ```STAMP`` <#var-bb-STAMP>`__: The base path to create stamp files.
1709
1710- ```STAMPCLEAN`` <#var-bb-STAMPCLEAN>`__: Again, the base path to
1711 create stamp files but can use wildcards for matching a range of
1712 files for clean operations.
1713
1714Wildcard Support in Variables
1715=============================
1716
1717Support for wildcard use in variables varies depending on the context in
1718which it is used. For example, some variables and file names allow
1719limited use of wildcards through the "``%``" and "``*``" characters.
1720Other variables or names support Python's
1721```glob`` <https://docs.python.org/3/library/glob.html>`__ syntax,
1722```fnmatch`` <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`__
1723syntax, or
1724```Regular Expression (re)`` <https://docs.python.org/3/library/re.html#re>`__
1725syntax.
1726
1727For variables that have wildcard suport, the documentation describes
1728which form of wildcard, its use, and its limitations.
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
new file mode 100644
index 0000000000..41a77b3bce
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -0,0 +1,1235 @@
1==================
2Variables Glossary
3==================
4
5This chapter lists common variables used by BitBake and gives an
6overview of their function and contents.
7
8.. note::
9
10 Following are some points regarding the variables listed in this
11 glossary:
12
13 - The variables listed in this glossary are specific to BitBake.
14 Consequently, the descriptions are limited to that context.
15
16 - Also, variables exist in other systems that use BitBake (e.g. The
17 Yocto Project and OpenEmbedded) that have names identical to those
18 found in this glossary. For such cases, the variables in those
19 systems extend the functionality of the variable as it is
20 described here in this glossary.
21
22 - Finally, there are variables mentioned in this glossary that do
23 not appear in the BitBake glossary. These other variables are
24 variables used in systems that use BitBake.
25
26`A <#var-bb-ASSUME_PROVIDED>`__ `B <#var-bb-B>`__ `C <#var-bb-CACHE>`__
27`D <#var-bb-DEFAULT_PREFERENCE>`__ `E <#var-bb-EXCLUDE_FROM_WORLD>`__
28`F <#var-bb-FAKEROOT>`__ `G <#var-bb-GITDIR>`__ `H <#var-bb-HGDIR>`__
29`I <#var-bb-INHERIT>`__ `L <#var-bb-LAYERDEPENDS>`__
30`M <#var-bb-MIRRORS>`__ `O <#var-bb-OVERRIDES>`__ `P <#var-bb-P4DIR>`__
31`R <#var-bb-RDEPENDS>`__ `S <#var-bb-SECTION>`__ `T <#var-bb-T>`__
32
33ASSUME_PROVIDED
34 Lists recipe names (```PN`` <#var-bb-PN>`__ values) BitBake does not
35 attempt to build. Instead, BitBake assumes these recipes have already
36 been built.
37
38 In OpenEmbedded-Core, ``ASSUME_PROVIDED`` mostly specifies native
39 tools that should not be built. An example is ``git-native``, which
40 when specified allows for the Git binary from the host to be used
41 rather than building ``git-native``.
42
43B
44 The directory in which BitBake executes functions during a recipe's
45 build process.
46
47BB_ALLOWED_NETWORKS
48 Specifies a space-delimited list of hosts that the fetcher is allowed
49 to use to obtain the required source code. Following are
50 considerations surrounding this variable:
51
52 - This host list is only used if
53 ```BB_NO_NETWORK`` <#var-bb-BB_NO_NETWORK>`__ is either not set or
54 set to "0".
55
56 - Limited support for the "``*``" wildcard character for matching
57 against the beginning of host names exists. For example, the
58 following setting matches ``git.gnu.org``, ``ftp.gnu.org``, and
59 ``foo.git.gnu.org``. BB_ALLOWED_NETWORKS = "*.gnu.org"
60
61 .. note::
62
63 The use of the "``*``" character only works at the beginning of
64 a host name and it must be isolated from the remainder of the
65 host name. You cannot use the wildcard character in any other
66 location of the name or combined with the front part of the
67 name.
68
69 For example, ``*.foo.bar`` is supported, while ``*aa.foo.bar``
70 is not.
71
72 - Mirrors not in the host list are skipped and logged in debug.
73
74 - Attempts to access networks not in the host list cause a failure.
75
76 Using ``BB_ALLOWED_NETWORKS`` in conjunction with
77 ```PREMIRRORS`` <#var-bb-PREMIRRORS>`__ is very useful. Adding the
78 host you want to use to ``PREMIRRORS`` results in the source code
79 being fetched from an allowed location and avoids raising an error
80 when a host that is not allowed is in a
81 ```SRC_URI`` <#var-bb-SRC_URI>`__ statement. This is because the
82 fetcher does not attempt to use the host listed in ``SRC_URI`` after
83 a successful fetch from the ``PREMIRRORS`` occurs.
84
85BB_CONSOLELOG
86 Specifies the path to a log file into which BitBake's user interface
87 writes output during the build.
88
89BB_CURRENTTASK
90 Contains the name of the currently running task. The name does not
91 include the ``do_`` prefix.
92
93BB_DANGLINGAPPENDS_WARNONLY
94 Defines how BitBake handles situations where an append file
95 (``.bbappend``) has no corresponding recipe file (``.bb``). This
96 condition often occurs when layers get out of sync (e.g. ``oe-core``
97 bumps a recipe version and the old recipe no longer exists and the
98 other layer has not been updated to the new version of the recipe
99 yet).
100
101 The default fatal behavior is safest because it is the sane reaction
102 given something is out of sync. It is important to realize when your
103 changes are no longer being applied.
104
105BB_DEFAULT_TASK
106 The default task to use when none is specified (e.g. with the ``-c``
107 command line option). The task name specified should not include the
108 ``do_`` prefix.
109
110BB_DISKMON_DIRS
111 Monitors disk space and available inodes during the build and allows
112 you to control the build based on these parameters.
113
114 Disk space monitoring is disabled by default. When setting this
115 variable, use the following form: BB_DISKMON_DIRS =
116 "<action>,<dir>,<threshold> [...]" where: <action> is: ABORT:
117 Immediately abort the build when a threshold is broken. STOPTASKS:
118 Stop the build after the currently executing tasks have finished when
119 a threshold is broken. WARN: Issue a warning but continue the build
120 when a threshold is broken. Subsequent warnings are issued as defined
121 by the `BB_DISKMON_WARNINTERVAL <#var-bb-BB_DISKMON_WARNINTERVAL>`__
122 variable, which must be defined. <dir> is: Any directory you choose.
123 You can specify one or more directories to monitor by separating the
124 groupings with a space. If two directories are on the same device,
125 only the first directory is monitored. <threshold> is: Either the
126 minimum available disk space, the minimum number of free inodes, or
127 both. You must specify at least one. To omit one or the other, simply
128 omit the value. Specify the threshold using G, M, K for Gbytes,
129 Mbytes, and Kbytes, respectively. If you do not specify G, M, or K,
130 Kbytes is assumed by default. Do not use GB, MB, or KB.
131
132 Here are some examples: BB_DISKMON_DIRS = "ABORT,${TMPDIR},1G,100K
133 WARN,${SSTATE_DIR},1G,100K" BB_DISKMON_DIRS =
134 "STOPTASKS,${TMPDIR},1G" BB_DISKMON_DIRS = "ABORT,${TMPDIR},,100K"
135 The first example works only if you also set the
136 ```BB_DISKMON_WARNINTERVAL`` <#var-bb-BB_DISKMON_WARNINTERVAL>`__
137 variable. This example causes the build system to immediately abort
138 when either the disk space in ``${TMPDIR}`` drops below 1 Gbyte or
139 the available free inodes drops below 100 Kbytes. Because two
140 directories are provided with the variable, the build system also
141 issues a warning when the disk space in the ``${SSTATE_DIR}``
142 directory drops below 1 Gbyte or the number of free inodes drops
143 below 100 Kbytes. Subsequent warnings are issued during intervals as
144 defined by the ``BB_DISKMON_WARNINTERVAL`` variable.
145
146 The second example stops the build after all currently executing
147 tasks complete when the minimum disk space in the ``${TMPDIR}``
148 directory drops below 1 Gbyte. No disk monitoring occurs for the free
149 inodes in this case.
150
151 The final example immediately aborts the build when the number of
152 free inodes in the ``${TMPDIR}`` directory drops below 100 Kbytes. No
153 disk space monitoring for the directory itself occurs in this case.
154
155BB_DISKMON_WARNINTERVAL
156 Defines the disk space and free inode warning intervals.
157
158 If you are going to use the ``BB_DISKMON_WARNINTERVAL`` variable, you
159 must also use the ```BB_DISKMON_DIRS`` <#var-bb-BB_DISKMON_DIRS>`__
160 variable and define its action as "WARN". During the build,
161 subsequent warnings are issued each time disk space or number of free
162 inodes further reduces by the respective interval.
163
164 If you do not provide a ``BB_DISKMON_WARNINTERVAL`` variable and you
165 do use ``BB_DISKMON_DIRS`` with the "WARN" action, the disk
166 monitoring interval defaults to the following:
167 BB_DISKMON_WARNINTERVAL = "50M,5K"
168
169 When specifying the variable in your configuration file, use the
170 following form: BB_DISKMON_WARNINTERVAL =
171 "<disk_space_interval>,<disk_inode_interval>" where:
172 <disk_space_interval> is: An interval of memory expressed in either
173 G, M, or K for Gbytes, Mbytes, or Kbytes, respectively. You cannot
174 use GB, MB, or KB. <disk_inode_interval> is: An interval of free
175 inodes expressed in either G, M, or K for Gbytes, Mbytes, or Kbytes,
176 respectively. You cannot use GB, MB, or KB.
177
178 Here is an example: BB_DISKMON_DIRS = "WARN,${SSTATE_DIR},1G,100K"
179 BB_DISKMON_WARNINTERVAL = "50M,5K" These variables cause BitBake to
180 issue subsequent warnings each time the available disk space further
181 reduces by 50 Mbytes or the number of free inodes further reduces by
182 5 Kbytes in the ``${SSTATE_DIR}`` directory. Subsequent warnings
183 based on the interval occur each time a respective interval is
184 reached beyond the initial warning (i.e. 1 Gbytes and 100 Kbytes).
185
186BB_ENV_WHITELIST
187 Specifies the internal whitelist of variables to allow through from
188 the external environment into BitBake's datastore. If the value of
189 this variable is not specified (which is the default), the following
190 list is used: ```BBPATH`` <#var-bb-BBPATH>`__,
191 ```BB_PRESERVE_ENV`` <#var-bb-BB_PRESERVE_ENV>`__,
192 ```BB_ENV_WHITELIST`` <#var-bb-BB_ENV_WHITELIST>`__, and
193 ```BB_ENV_EXTRAWHITE`` <#var-bb-BB_ENV_EXTRAWHITE>`__.
194
195 .. note::
196
197 You must set this variable in the external environment in order
198 for it to work.
199
200BB_ENV_EXTRAWHITE
201 Specifies an additional set of variables to allow through (whitelist)
202 from the external environment into BitBake's datastore. This list of
203 variables are on top of the internal list set in
204 ```BB_ENV_WHITELIST`` <#var-bb-BB_ENV_WHITELIST>`__.
205
206 .. note::
207
208 You must set this variable in the external environment in order
209 for it to work.
210
211BB_FETCH_PREMIRRORONLY
212 When set to "1", causes BitBake's fetcher module to only search
213 ```PREMIRRORS`` <#var-bb-PREMIRRORS>`__ for files. BitBake will not
214 search the main ```SRC_URI`` <#var-bb-SRC_URI>`__ or
215 ```MIRRORS`` <#var-bb-MIRRORS>`__.
216
217BB_FILENAME
218 Contains the filename of the recipe that owns the currently running
219 task. For example, if the ``do_fetch`` task that resides in the
220 ``my-recipe.bb`` is executing, the ``BB_FILENAME`` variable contains
221 "/foo/path/my-recipe.bb".
222
223BB_GENERATE_MIRROR_TARBALLS
224 Causes tarballs of the Git repositories, including the Git metadata,
225 to be placed in the ```DL_DIR`` <#var-bb-DL_DIR>`__ directory. Anyone
226 wishing to create a source mirror would want to enable this variable.
227
228 For performance reasons, creating and placing tarballs of the Git
229 repositories is not the default action by BitBake.
230 BB_GENERATE_MIRROR_TARBALLS = "1"
231
232BB_HASHCONFIG_WHITELIST
233 Lists variables that are excluded from base configuration checksum,
234 which is used to determine if the cache can be reused.
235
236 One of the ways BitBake determines whether to re-parse the main
237 metadata is through checksums of the variables in the datastore of
238 the base configuration data. There are variables that you typically
239 want to exclude when checking whether or not to re-parse and thus
240 rebuild the cache. As an example, you would usually exclude ``TIME``
241 and ``DATE`` because these variables are always changing. If you did
242 not exclude them, BitBake would never reuse the cache.
243
244BB_HASHBASE_WHITELIST
245 Lists variables that are excluded from checksum and dependency data.
246 Variables that are excluded can therefore change without affecting
247 the checksum mechanism. A common example would be the variable for
248 the path of the build. BitBake's output should not (and usually does
249 not) depend on the directory in which it was built.
250
251BB_HASHCHECK_FUNCTION
252 Specifies the name of the function to call during the "setscene" part
253 of the task's execution in order to validate the list of task hashes.
254 The function returns the list of setscene tasks that should be
255 executed.
256
257 At this point in the execution of the code, the objective is to
258 quickly verify if a given setscene function is likely to work or not.
259 It's easier to check the list of setscene functions in one pass than
260 to call many individual tasks. The returned list need not be
261 completely accurate. A given setscene task can still later fail.
262 However, the more accurate the data returned, the more efficient the
263 build will be.
264
265BB_INVALIDCONF
266 Used in combination with the ``ConfigParsed`` event to trigger
267 re-parsing the base metadata (i.e. all the recipes). The
268 ``ConfigParsed`` event can set the variable to trigger the re-parse.
269 You must be careful to avoid recursive loops with this functionality.
270
271BB_LOGCONFIG
272 Specifies the name of a config file that contains the user logging
273 configuration. See `Logging <#logging>`__ for additional information
274
275BB_LOGFMT
276 Specifies the name of the log files saved into
277 ``${``\ ```T`` <#var-bb-T>`__\ ``}``. By default, the ``BB_LOGFMT``
278 variable is undefined and the log file names get created using the
279 following form: log.{task}.{pid} If you want to force log files to
280 take a specific name, you can set this variable in a configuration
281 file.
282
283BB_NICE_LEVEL
284 Allows BitBake to run at a specific priority (i.e. nice level).
285 System permissions usually mean that BitBake can reduce its priority
286 but not raise it again. See
287 ```BB_TASK_NICE_LEVEL`` <#var-bb-BB_TASK_NICE_LEVEL>`__ for
288 additional information.
289
290BB_NO_NETWORK
291 Disables network access in the BitBake fetcher modules. With this
292 access disabled, any command that attempts to access the network
293 becomes an error.
294
295 Disabling network access is useful for testing source mirrors,
296 running builds when not connected to the Internet, and when operating
297 in certain kinds of firewall environments.
298
299BB_NUMBER_THREADS
300 The maximum number of tasks BitBake should run in parallel at any one
301 time. If your host development system supports multiple cores, a good
302 rule of thumb is to set this variable to twice the number of cores.
303
304BB_NUMBER_PARSE_THREADS
305 Sets the number of threads BitBake uses when parsing. By default, the
306 number of threads is equal to the number of cores on the system.
307
308BB_ORIGENV
309 Contains a copy of the original external environment in which BitBake
310 was run. The copy is taken before any whitelisted variable values are
311 filtered into BitBake's datastore.
312
313 .. note::
314
315 The contents of this variable is a datastore object that can be
316 queried using the normal datastore operations.
317
318BB_PRESERVE_ENV
319 Disables whitelisting and instead allows all variables through from
320 the external environment into BitBake's datastore.
321
322 .. note::
323
324 You must set this variable in the external environment in order
325 for it to work.
326
327BB_RUNFMT
328 Specifies the name of the executable script files (i.e. run files)
329 saved into ``${``\ ```T`` <#var-bb-T>`__\ ``}``. By default, the
330 ``BB_RUNFMT`` variable is undefined and the run file names get
331 created using the following form: run.{task}.{pid} If you want to
332 force run files to take a specific name, you can set this variable in
333 a configuration file.
334
335BB_RUNTASK
336 Contains the name of the currently executing task. The value includes
337 the "do_" prefix. For example, if the currently executing task is
338 ``do_config``, the value is "do_config".
339
340BB_SCHEDULER
341 Selects the name of the scheduler to use for the scheduling of
342 BitBake tasks. Three options exist:
343
344 - *basic* - The basic framework from which everything derives. Using
345 this option causes tasks to be ordered numerically as they are
346 parsed.
347
348 - *speed* - Executes tasks first that have more tasks depending on
349 them. The "speed" option is the default.
350
351 - *completion* - Causes the scheduler to try to complete a given
352 recipe once its build has started.
353
354BB_SCHEDULERS
355 Defines custom schedulers to import. Custom schedulers need to be
356 derived from the ``RunQueueScheduler`` class.
357
358 For information how to select a scheduler, see the
359 ```BB_SCHEDULER`` <#var-bb-BB_SCHEDULER>`__ variable.
360
361BB_SETSCENE_DEPVALID
362 Specifies a function BitBake calls that determines whether BitBake
363 requires a setscene dependency to be met.
364
365 When running a setscene task, BitBake needs to know which
366 dependencies of that setscene task also need to be run. Whether
367 dependencies also need to be run is highly dependent on the metadata.
368 The function specified by this variable returns a "True" or "False"
369 depending on whether the dependency needs to be met.
370
371BB_SETSCENE_VERIFY_FUNCTION2
372 Specifies a function to call that verifies the list of planned task
373 execution before the main task execution happens. The function is
374 called once BitBake has a list of setscene tasks that have run and
375 either succeeded or failed.
376
377 The function allows for a task list check to see if they make sense.
378 Even if BitBake was planning to skip a task, the returned value of
379 the function can force BitBake to run the task, which is necessary
380 under certain metadata defined circumstances.
381
382BB_SIGNATURE_EXCLUDE_FLAGS
383 Lists variable flags (varflags) that can be safely excluded from
384 checksum and dependency data for keys in the datastore. When
385 generating checksum or dependency data for keys in the datastore, the
386 flags set against that key are normally included in the checksum.
387
388 For more information on varflags, see the "`Variable
389 Flags <#variable-flags>`__" section.
390
391BB_SIGNATURE_HANDLER
392 Defines the name of the signature handler BitBake uses. The signature
393 handler defines the way stamp files are created and handled, if and
394 how the signature is incorporated into the stamps, and how the
395 signature itself is generated.
396
397 A new signature handler can be added by injecting a class derived
398 from the ``SignatureGenerator`` class into the global namespace.
399
400BB_SRCREV_POLICY
401 Defines the behavior of the fetcher when it interacts with source
402 control systems and dynamic source revisions. The
403 ``BB_SRCREV_POLICY`` variable is useful when working without a
404 network.
405
406 The variable can be set using one of two policies:
407
408 - *cache* - Retains the value the system obtained previously rather
409 than querying the source control system each time.
410
411 - *clear* - Queries the source controls system every time. With this
412 policy, there is no cache. The "clear" policy is the default.
413
414BB_STAMP_POLICY
415 Defines the mode used for how timestamps of stamp files are compared.
416 You can set the variable to one of the following modes:
417
418 - *perfile* - Timestamp comparisons are only made between timestamps
419 of a specific recipe. This is the default mode.
420
421 - *full* - Timestamp comparisons are made for all dependencies.
422
423 - *whitelist* - Identical to "full" mode except timestamp
424 comparisons are made for recipes listed in the
425 ```BB_STAMP_WHITELIST`` <#var-bb-BB_STAMP_WHITELIST>`__ variable.
426
427 .. note::
428
429 Stamp policies are largely obsolete with the introduction of
430 setscene tasks.
431
432BB_STAMP_WHITELIST
433 Lists files whose stamp file timestamps are compared when the stamp
434 policy mode is set to "whitelist". For information on stamp policies,
435 see the ```BB_STAMP_POLICY`` <#var-bb-BB_STAMP_POLICY>`__ variable.
436
437BB_STRICT_CHECKSUM
438 Sets a more strict checksum mechanism for non-local URLs. Setting
439 this variable to a value causes BitBake to report an error if it
440 encounters a non-local URL that does not have at least one checksum
441 specified.
442
443BB_TASK_IONICE_LEVEL
444 Allows adjustment of a task's Input/Output priority. During
445 Autobuilder testing, random failures can occur for tasks due to I/O
446 starvation. These failures occur during various QEMU runtime
447 timeouts. You can use the ``BB_TASK_IONICE_LEVEL`` variable to adjust
448 the I/O priority of these tasks.
449
450 .. note::
451
452 This variable works similarly to the
453 BB_TASK_NICE_LEVEL
454 variable except with a task's I/O priorities.
455
456 Set the variable as follows: BB_TASK_IONICE_LEVEL = "class.prio" For
457 class, the default value is "2", which is a best effort. You can use
458 "1" for realtime and "3" for idle. If you want to use realtime, you
459 must have superuser privileges.
460
461 For prio, you can use any value from "0", which is the highest
462 priority, to "7", which is the lowest. The default value is "4". You
463 do not need any special privileges to use this range of priority
464 values.
465
466 .. note::
467
468 In order for your I/O priority settings to take effect, you need
469 the Completely Fair Queuing (CFQ) Scheduler selected for the
470 backing block device. To select the scheduler, use the following
471 command form where
472 device
473 is the device (e.g. sda, sdb, and so forth):
474 ::
475
476 $ sudo sh -c “echo cfq > /sys/block/device/queu/scheduler
477
478
479BB_TASK_NICE_LEVEL
480 Allows specific tasks to change their priority (i.e. nice level).
481
482 You can use this variable in combination with task overrides to raise
483 or lower priorities of specific tasks. For example, on the `Yocto
484 Project <http://www.yoctoproject.org>`__ autobuilder, QEMU emulation
485 in images is given a higher priority as compared to build tasks to
486 ensure that images do not suffer timeouts on loaded systems.
487
488BB_TASKHASH
489 Within an executing task, this variable holds the hash of the task as
490 returned by the currently enabled signature generator.
491
492BB_VERBOSE_LOGS
493 Controls how verbose BitBake is during builds. If set, shell scripts
494 echo commands and shell script output appears on standard out
495 (stdout).
496
497BB_WORKERCONTEXT
498 Specifies if the current context is executing a task. BitBake sets
499 this variable to "1" when a task is being executed. The value is not
500 set when the task is in server context during parsing or event
501 handling.
502
503BBCLASSEXTEND
504 Allows you to extend a recipe so that it builds variants of the
505 software. Some examples of these variants for recipes from the
506 OpenEmbedded-Core metadata are "natives" such as ``quilt-native``,
507 which is a copy of Quilt built to run on the build system; "crosses"
508 such as ``gcc-cross``, which is a compiler built to run on the build
509 machine but produces binaries that run on the target ``MACHINE``;
510 "nativesdk", which targets the SDK machine instead of ``MACHINE``;
511 and "mulitlibs" in the form "``multilib:``\ multilib_name".
512
513 To build a different variant of the recipe with a minimal amount of
514 code, it usually is as simple as adding the variable to your recipe.
515 Here are two examples. The "native" variants are from the
516 OpenEmbedded-Core metadata: BBCLASSEXTEND =+ "native nativesdk"
517 BBCLASSEXTEND =+ "multilib:multilib_name"
518
519 .. note::
520
521 Internally, the ``BBCLASSEXTEND`` mechanism generates recipe
522 variants by rewriting variable values and applying overrides such
523 as ``_class-native``. For example, to generate a native version of
524 a recipe, a ```DEPENDS`` <#var-bb-DEPENDS>`__ on "foo" is
525 rewritten to a ``DEPENDS`` on "foo-native".
526
527 Even when using ``BBCLASSEXTEND``, the recipe is only parsed once.
528 Parsing once adds some limitations. For example, it is not
529 possible to include a different file depending on the variant,
530 since ``include`` statements are processed when the recipe is
531 parsed.
532
533BBDEBUG
534 Sets the BitBake debug output level to a specific value as
535 incremented by the ``-D`` command line option.
536
537 .. note::
538
539 You must set this variable in the external environment in order
540 for it to work.
541
542BBFILE_COLLECTIONS
543 Lists the names of configured layers. These names are used to find
544 the other ``BBFILE_*`` variables. Typically, each layer appends its
545 name to this variable in its ``conf/layer.conf`` file.
546
547BBFILE_PATTERN
548 Variable that expands to match files from
549 ```BBFILES`` <#var-bb-BBFILES>`__ in a particular layer. This
550 variable is used in the ``conf/layer.conf`` file and must be suffixed
551 with the name of the specific layer (e.g.
552 ``BBFILE_PATTERN_emenlow``).
553
554BBFILE_PRIORITY
555 Assigns the priority for recipe files in each layer.
556
557 This variable is useful in situations where the same recipe appears
558 in more than one layer. Setting this variable allows you to
559 prioritize a layer against other layers that contain the same recipe
560 - effectively letting you control the precedence for the multiple
561 layers. The precedence established through this variable stands
562 regardless of a recipe's version (```PV`` <#var-bb-PV>`__ variable).
563 For example, a layer that has a recipe with a higher ``PV`` value but
564 for which the ``BBFILE_PRIORITY`` is set to have a lower precedence
565 still has a lower precedence.
566
567 A larger value for the ``BBFILE_PRIORITY`` variable results in a
568 higher precedence. For example, the value 6 has a higher precedence
569 than the value 5. If not specified, the ``BBFILE_PRIORITY`` variable
570 is set based on layer dependencies (see the ``LAYERDEPENDS`` variable
571 for more information. The default priority, if unspecified for a
572 layer with no dependencies, is the lowest defined priority + 1 (or 1
573 if no priorities are defined).
574
575 .. tip::
576
577 You can use the command
578 bitbake-layers show-layers
579 to list all configured layers along with their priorities.
580
581BBFILES
582 A space-separated list of recipe files BitBake uses to build
583 software.
584
585 When specifying recipe files, you can pattern match using Python's
586 ```glob`` <https://docs.python.org/3/library/glob.html>`__ syntax.
587 For details on the syntax, see the documentation by following the
588 previous link.
589
590BBINCLUDED
591 Contains a space-separated list of all of all files that BitBake's
592 parser included during parsing of the current file.
593
594BBINCLUDELOGS
595 If set to a value, enables printing the task log when reporting a
596 failed task.
597
598BBINCLUDELOGS_LINES
599 If ```BBINCLUDELOGS`` <#var-bb-BBINCLUDELOGS>`__ is set, specifies
600 the maximum number of lines from the task log file to print when
601 reporting a failed task. If you do not set ``BBINCLUDELOGS_LINES``,
602 the entire log is printed.
603
604BBLAYERS
605 Lists the layers to enable during the build. This variable is defined
606 in the ``bblayers.conf`` configuration file in the build directory.
607 Here is an example: BBLAYERS = " \\ /home/scottrif/poky/meta \\
608 /home/scottrif/poky/meta-yocto \\ /home/scottrif/poky/meta-yocto-bsp
609 \\ /home/scottrif/poky/meta-mykernel \\ " This example enables four
610 layers, one of which is a custom, user-defined layer named
611 ``meta-mykernel``.
612
613BBLAYERS_FETCH_DIR
614 Sets the base location where layers are stored. This setting is used
615 in conjunction with ``bitbake-layers layerindex-fetch`` and tells
616 ``bitbake-layers`` where to place the fetched layers.
617
618BBMASK
619 Prevents BitBake from processing recipes and recipe append files.
620
621 You can use the ``BBMASK`` variable to "hide" these ``.bb`` and
622 ``.bbappend`` files. BitBake ignores any recipe or recipe append
623 files that match any of the expressions. It is as if BitBake does not
624 see them at all. Consequently, matching files are not parsed or
625 otherwise used by BitBake.
626
627 The values you provide are passed to Python's regular expression
628 compiler. Consequently, the syntax follows Python's Regular
629 Expression (re) syntax. The expressions are compared against the full
630 paths to the files. For complete syntax information, see Python's
631 documentation at ` <http://docs.python.org/3/library/re.html#re>`__.
632
633 The following example uses a complete regular expression to tell
634 BitBake to ignore all recipe and recipe append files in the
635 ``meta-ti/recipes-misc/`` directory: BBMASK = "meta-ti/recipes-misc/"
636 If you want to mask out multiple directories or recipes, you can
637 specify multiple regular expression fragments. This next example
638 masks out multiple directories and individual recipes: BBMASK +=
639 "/meta-ti/recipes-misc/ meta-ti/recipes-ti/packagegroup/" BBMASK +=
640 "/meta-oe/recipes-support/" BBMASK += "/meta-foo/.*/openldap" BBMASK
641 += "opencv.*\.bbappend" BBMASK += "lzma"
642
643 .. note::
644
645 When specifying a directory name, use the trailing slash character
646 to ensure you match just that directory name.
647
648BBMULTICONFIG
649 Enables BitBake to perform multiple configuration builds and lists
650 each separate configuration (multiconfig). You can use this variable
651 to cause BitBake to build multiple targets where each target has a
652 separate configuration. Define ``BBMULTICONFIG`` in your
653 ``conf/local.conf`` configuration file.
654
655 As an example, the following line specifies three multiconfigs, each
656 having a separate configuration file: BBMULTIFONFIG = "configA
657 configB configC" Each configuration file you use must reside in the
658 build directory within a directory named ``conf/multiconfig`` (e.g.
659 build_directory\ ``/conf/multiconfig/configA.conf``).
660
661 For information on how to use ``BBMULTICONFIG`` in an environment
662 that supports building targets with multiple configurations, see the
663 "`Executing a Multiple Configuration
664 Build <#executing-a-multiple-configuration-build>`__" section.
665
666BBPATH
667 Used by BitBake to locate class (``.bbclass``) and configuration
668 (``.conf``) files. This variable is analogous to the ``PATH``
669 variable.
670
671 If you run BitBake from a directory outside of the build directory,
672 you must be sure to set ``BBPATH`` to point to the build directory.
673 Set the variable as you would any environment variable and then run
674 BitBake: $ BBPATH="build_directory" $ export BBPATH $ bitbake target
675
676BBSERVER
677 Points to the server that runs memory-resident BitBake. The variable
678 is only used when you employ memory-resident BitBake.
679
680BBTARGETS
681 Allows you to use a configuration file to add to the list of
682 command-line target recipes you want to build.
683
684BBVERSIONS
685 Allows a single recipe to build multiple versions of a project from a
686 single recipe file. You also able to specify conditional metadata
687 using the ```OVERRIDES`` <#var-bb-OVERRIDES>`__ mechanism for a
688 single version or for an optionally named range of versions.
689
690 For more information on ``BBVERSIONS``, see the "`Variants - Class
691 Extension Mechanism <#variants-class-extension-mechanism>`__"
692 section.
693
694BITBAKE_UI
695 Used to specify the UI module to use when running BitBake. Using this
696 variable is equivalent to using the ``-u`` command-line option.
697
698 .. note::
699
700 You must set this variable in the external environment in order
701 for it to work.
702
703BUILDNAME
704 A name assigned to the build. The name defaults to a datetime stamp
705 of when the build was started but can be defined by the metadata.
706
707BZRDIR
708 The directory in which files checked out of a Bazaar system are
709 stored.
710
711CACHE
712 Specifies the directory BitBake uses to store a cache of the metadata
713 so it does not need to be parsed every time BitBake is started.
714
715CVSDIR
716 The directory in which files checked out under the CVS system are
717 stored.
718
719DEFAULT_PREFERENCE
720 Specifies a weak bias for recipe selection priority.
721
722 The most common usage of this is variable is to set it to "-1" within
723 a recipe for a development version of a piece of software. Using the
724 variable in this way causes the stable version of the recipe to build
725 by default in the absence of ``PREFERRED_VERSION`` being used to
726 build the development version.
727
728 .. note::
729
730 The bias provided by
731 DEFAULT_PREFERENCE
732 is weak and is overridden by
733 BBFILE_PRIORITY
734 if that variable is different between two layers that contain
735 different versions of the same recipe.
736
737DEPENDS
738 Lists a recipe's build-time dependencies (i.e. other recipe files).
739
740 Consider this simple example for two recipes named "a" and "b" that
741 produce similarly named packages. In this example, the ``DEPENDS``
742 statement appears in the "a" recipe: DEPENDS = "b" Here, the
743 dependency is such that the ``do_configure`` task for recipe "a"
744 depends on the ``do_populate_sysroot`` task of recipe "b". This means
745 anything that recipe "b" puts into sysroot is available when recipe
746 "a" is configuring itself.
747
748 For information on runtime dependencies, see the
749 ```RDEPENDS`` <#var-bb-RDEPENDS>`__ variable.
750
751DESCRIPTION
752 A long description for the recipe.
753
754DL_DIR
755 The central download directory used by the build process to store
756 downloads. By default, ``DL_DIR`` gets files suitable for mirroring
757 for everything except Git repositories. If you want tarballs of Git
758 repositories, use the
759 ```BB_GENERATE_MIRROR_TARBALLS`` <#var-bb-BB_GENERATE_MIRROR_TARBALLS>`__
760 variable.
761
762EXCLUDE_FROM_WORLD
763 Directs BitBake to exclude a recipe from world builds (i.e.
764 ``bitbake world``). During world builds, BitBake locates, parses and
765 builds all recipes found in every layer exposed in the
766 ``bblayers.conf`` configuration file.
767
768 To exclude a recipe from a world build using this variable, set the
769 variable to "1" in the recipe.
770
771 .. note::
772
773 Recipes added to
774 EXCLUDE_FROM_WORLD
775 may still be built during a world build in order to satisfy
776 dependencies of other recipes. Adding a recipe to
777 EXCLUDE_FROM_WORLD
778 only ensures that the recipe is not explicitly added to the list
779 of build targets in a world build.
780
781FAKEROOT
782 Contains the command to use when running a shell script in a fakeroot
783 environment. The ``FAKEROOT`` variable is obsolete and has been
784 replaced by the other ``FAKEROOT*`` variables. See these entries in
785 the glossary for more information.
786
787FAKEROOTBASEENV
788 Lists environment variables to set when executing the command defined
789 by ```FAKEROOTCMD`` <#var-bb-FAKEROOTCMD>`__ that starts the
790 bitbake-worker process in the fakeroot environment.
791
792FAKEROOTCMD
793 Contains the command that starts the bitbake-worker process in the
794 fakeroot environment.
795
796FAKEROOTDIRS
797 Lists directories to create before running a task in the fakeroot
798 environment.
799
800FAKEROOTENV
801 Lists environment variables to set when running a task in the
802 fakeroot environment. For additional information on environment
803 variables and the fakeroot environment, see the
804 ```FAKEROOTBASEENV`` <#var-bb-FAKEROOTBASEENV>`__ variable.
805
806FAKEROOTNOENV
807 Lists environment variables to set when running a task that is not in
808 the fakeroot environment. For additional information on environment
809 variables and the fakeroot environment, see the
810 ```FAKEROOTENV`` <#var-bb-FAKEROOTENV>`__ variable.
811
812FETCHCMD
813 Defines the command the BitBake fetcher module executes when running
814 fetch operations. You need to use an override suffix when you use the
815 variable (e.g. ``FETCHCMD_git`` or ``FETCHCMD_svn``).
816
817FILE
818 Points at the current file. BitBake sets this variable during the
819 parsing process to identify the file being parsed. BitBake also sets
820 this variable when a recipe is being executed to identify the recipe
821 file.
822
823FILESPATH
824 Specifies directories BitBake uses when searching for patches and
825 files. The "local" fetcher module uses these directories when
826 handling ``file://`` URLs. The variable behaves like a shell ``PATH``
827 environment variable. The value is a colon-separated list of
828 directories that are searched left-to-right in order.
829
830GITDIR
831 The directory in which a local copy of a Git repository is stored
832 when it is cloned.
833
834HGDIR
835 The directory in which files checked out of a Mercurial system are
836 stored.
837
838HOMEPAGE
839 Website where more information about the software the recipe is
840 building can be found.
841
842INHERIT
843 Causes the named class or classes to be inherited globally. Anonymous
844 functions in the class or classes are not executed for the base
845 configuration and in each individual recipe. The OpenEmbedded build
846 system ignores changes to ``INHERIT`` in individual recipes.
847
848 For more information on ``INHERIT``, see the "```INHERIT``
849 Configuration Directive <#inherit-configuration-directive>`__"
850 section.
851
852LAYERDEPENDS
853 Lists the layers, separated by spaces, upon which this recipe
854 depends. Optionally, you can specify a specific layer version for a
855 dependency by adding it to the end of the layer name with a colon,
856 (e.g. "anotherlayer:3" to be compared against
857 ```LAYERVERSION`` <#var-bb-LAYERVERSION>`__\ ``_anotherlayer`` in
858 this case). BitBake produces an error if any dependency is missing or
859 the version numbers do not match exactly (if specified).
860
861 You use this variable in the ``conf/layer.conf`` file. You must also
862 use the specific layer name as a suffix to the variable (e.g.
863 ``LAYERDEPENDS_mylayer``).
864
865LAYERDIR
866 When used inside the ``layer.conf`` configuration file, this variable
867 provides the path of the current layer. This variable is not
868 available outside of ``layer.conf`` and references are expanded
869 immediately when parsing of the file completes.
870
871LAYERDIR_RE
872 When used inside the ``layer.conf`` configuration file, this variable
873 provides the path of the current layer, escaped for use in a regular
874 expression (```BBFILE_PATTERN`` <#var-bb-BBFILE_PATTERN>`__). This
875 variable is not available outside of ``layer.conf`` and references
876 are expanded immediately when parsing of the file completes.
877
878LAYERVERSION
879 Optionally specifies the version of a layer as a single number. You
880 can use this variable within
881 ```LAYERDEPENDS`` <#var-bb-LAYERDEPENDS>`__ for another layer in
882 order to depend on a specific version of the layer.
883
884 You use this variable in the ``conf/layer.conf`` file. You must also
885 use the specific layer name as a suffix to the variable (e.g.
886 ``LAYERDEPENDS_mylayer``).
887
888LICENSE
889 The list of source licenses for the recipe.
890
891MIRRORS
892 Specifies additional paths from which BitBake gets source code. When
893 the build system searches for source code, it first tries the local
894 download directory. If that location fails, the build system tries
895 locations defined by ```PREMIRRORS`` <#var-bb-PREMIRRORS>`__, the
896 upstream source, and then locations specified by ``MIRRORS`` in that
897 order.
898
899MULTI_PROVIDER_WHITELIST
900 Allows you to suppress BitBake warnings caused when building two
901 separate recipes that provide the same output.
902
903 BitBake normally issues a warning when building two different recipes
904 where each provides the same output. This scenario is usually
905 something the user does not want. However, cases do exist where it
906 makes sense, particularly in the ``virtual/*`` namespace. You can use
907 this variable to suppress BitBake's warnings.
908
909 To use the variable, list provider names (e.g. recipe names,
910 ``virtual/kernel``, and so forth).
911
912OVERRIDES
913 BitBake uses ``OVERRIDES`` to control what variables are overridden
914 after BitBake parses recipes and configuration files.
915
916 Following is a simple example that uses an overrides list based on
917 machine architectures: OVERRIDES = "arm:x86:mips:powerpc" You can
918 find information on how to use ``OVERRIDES`` in the "`Conditional
919 Syntax (Overrides) <#conditional-syntax-overrides>`__" section.
920
921P4DIR
922 The directory in which a local copy of a Perforce depot is stored
923 when it is fetched.
924
925PACKAGES
926 The list of packages the recipe creates.
927
928PACKAGES_DYNAMIC
929 A promise that your recipe satisfies runtime dependencies for
930 optional modules that are found in other recipes.
931 ``PACKAGES_DYNAMIC`` does not actually satisfy the dependencies, it
932 only states that they should be satisfied. For example, if a hard,
933 runtime dependency (```RDEPENDS`` <#var-bb-RDEPENDS>`__) of another
934 package is satisfied during the build through the
935 ``PACKAGES_DYNAMIC`` variable, but a package with the module name is
936 never actually produced, then the other package will be broken.
937
938PE
939 The epoch of the recipe. By default, this variable is unset. The
940 variable is used to make upgrades possible when the versioning scheme
941 changes in some backwards incompatible way.
942
943PERSISTENT_DIR
944 Specifies the directory BitBake uses to store data that should be
945 preserved between builds. In particular, the data stored is the data
946 that uses BitBake's persistent data API and the data used by the PR
947 Server and PR Service.
948
949PF
950 Specifies the recipe or package name and includes all version and
951 revision numbers (i.e. ``eglibc-2.13-r20+svnr15508/`` and
952 ``bash-4.2-r1/``).
953
954PN
955 The recipe name.
956
957PR
958 The revision of the recipe.
959
960PREFERRED_PROVIDER
961 Determines which recipe should be given preference when multiple
962 recipes provide the same item. You should always suffix the variable
963 with the name of the provided item, and you should set it to the
964 ```PN`` <#var-bb-PN>`__ of the recipe to which you want to give
965 precedence. Some examples: PREFERRED_PROVIDER_virtual/kernel ?=
966 "linux-yocto" PREFERRED_PROVIDER_virtual/xserver = "xserver-xf86"
967 PREFERRED_PROVIDER_virtual/libgl ?= "mesa"
968
969PREFERRED_PROVIDERS
970 Determines which recipe should be given preference for cases where
971 multiple recipes provide the same item. Functionally,
972 ``PREFERRED_PROVIDERS`` is identical to
973 ```PREFERRED_PROVIDER`` <#var-bb-PREFERRED_PROVIDER>`__. However, the
974 ``PREFERRED_PROVIDERS`` variable lets you define preferences for
975 multiple situations using the following form: PREFERRED_PROVIDERS =
976 "xxx:yyy aaa:bbb ..." This form is a convenient replacement for the
977 following: PREFERRED_PROVIDER_xxx = "yyy" PREFERRED_PROVIDER_aaa =
978 "bbb"
979
980PREFERRED_VERSION
981 If there are multiple versions of recipes available, this variable
982 determines which recipe should be given preference. You must always
983 suffix the variable with the ```PN`` <#var-bb-PN>`__ you want to
984 select, and you should set ```PV`` <#var-bb-PV>`__ accordingly for
985 precedence.
986
987 The ``PREFERRED_VERSION`` variable supports limited wildcard use
988 through the "``%``" character. You can use the character to match any
989 number of characters, which can be useful when specifying versions
990 that contain long revision numbers that potentially change. Here are
991 two examples: PREFERRED_VERSION_python = "2.7.3"
992 PREFERRED_VERSION_linux-yocto = "4.12%"
993
994 .. note::
995
996 The use of the "
997 %
998 " character is limited in that it only works at the end of the
999 string. You cannot use the wildcard character in any other
1000 location of the string.
1001
1002PREMIRRORS
1003 Specifies additional paths from which BitBake gets source code. When
1004 the build system searches for source code, it first tries the local
1005 download directory. If that location fails, the build system tries
1006 locations defined by ``PREMIRRORS``, the upstream source, and then
1007 locations specified by ```MIRRORS`` <#var-bb-MIRRORS>`__ in that
1008 order.
1009
1010 Typically, you would add a specific server for the build system to
1011 attempt before any others by adding something like the following to
1012 your configuration: PREMIRRORS_prepend = "\\ git://.*/.\*
1013 http://www.yoctoproject.org/sources/ \\n \\ ftp://.*/.\*
1014 http://www.yoctoproject.org/sources/ \\n \\ http://.*/.\*
1015 http://www.yoctoproject.org/sources/ \\n \\ https://.*/.\*
1016 http://www.yoctoproject.org/sources/ \\n" These changes cause the
1017 build system to intercept Git, FTP, HTTP, and HTTPS requests and
1018 direct them to the ``http://`` sources mirror. You can use
1019 ``file://`` URLs to point to local directories or network shares as
1020 well.
1021
1022PROVIDES
1023 A list of aliases by which a particular recipe can be known. By
1024 default, a recipe's own ``PN`` is implicitly already in its
1025 ``PROVIDES`` list. If a recipe uses ``PROVIDES``, the additional
1026 aliases are synonyms for the recipe and can be useful satisfying
1027 dependencies of other recipes during the build as specified by
1028 ``DEPENDS``.
1029
1030 Consider the following example ``PROVIDES`` statement from a recipe
1031 file ``libav_0.8.11.bb``: PROVIDES += "libpostproc" The ``PROVIDES``
1032 statement results in the "libav" recipe also being known as
1033 "libpostproc".
1034
1035 In addition to providing recipes under alternate names, the
1036 ``PROVIDES`` mechanism is also used to implement virtual targets. A
1037 virtual target is a name that corresponds to some particular
1038 functionality (e.g. a Linux kernel). Recipes that provide the
1039 functionality in question list the virtual target in ``PROVIDES``.
1040 Recipes that depend on the functionality in question can include the
1041 virtual target in ```DEPENDS`` <#var-bb-DEPENDS>`__ to leave the
1042 choice of provider open.
1043
1044 Conventionally, virtual targets have names on the form
1045 "virtual/function" (e.g. "virtual/kernel"). The slash is simply part
1046 of the name and has no syntactical significance.
1047
1048PRSERV_HOST
1049 The network based ```PR`` <#var-bb-PR>`__ service host and port.
1050
1051 Following is an example of how the ``PRSERV_HOST`` variable is set:
1052 PRSERV_HOST = "localhost:0" You must set the variable if you want to
1053 automatically start a local PR service. You can set ``PRSERV_HOST``
1054 to other values to use a remote PR service.
1055
1056PV
1057 The version of the recipe.
1058
1059RDEPENDS
1060 Lists a package's runtime dependencies (i.e. other packages) that
1061 must be installed in order for the built package to run correctly. If
1062 a package in this list cannot be found during the build, you will get
1063 a build error.
1064
1065 Because the ``RDEPENDS`` variable applies to packages being built,
1066 you should always use the variable in a form with an attached package
1067 name. For example, suppose you are building a development package
1068 that depends on the ``perl`` package. In this case, you would use the
1069 following ``RDEPENDS`` statement: RDEPENDS_${PN}-dev += "perl" In the
1070 example, the development package depends on the ``perl`` package.
1071 Thus, the ``RDEPENDS`` variable has the ``${PN}-dev`` package name as
1072 part of the variable.
1073
1074 BitBake supports specifying versioned dependencies. Although the
1075 syntax varies depending on the packaging format, BitBake hides these
1076 differences from you. Here is the general syntax to specify versions
1077 with the ``RDEPENDS`` variable: RDEPENDS_${PN} = "package (operator
1078 version)" For ``operator``, you can specify the following: = < > <=
1079 >= For example, the following sets up a dependency on version 1.2 or
1080 greater of the package ``foo``: RDEPENDS_${PN} = "foo (>= 1.2)"
1081
1082 For information on build-time dependencies, see the
1083 ```DEPENDS`` <#var-bb-DEPENDS>`__ variable.
1084
1085REPODIR
1086 The directory in which a local copy of a ``google-repo`` directory is
1087 stored when it is synced.
1088
1089RPROVIDES
1090 A list of package name aliases that a package also provides. These
1091 aliases are useful for satisfying runtime dependencies of other
1092 packages both during the build and on the target (as specified by
1093 ``RDEPENDS``).
1094
1095 As with all package-controlling variables, you must always use the
1096 variable in conjunction with a package name override. Here is an
1097 example: RPROVIDES_${PN} = "widget-abi-2"
1098
1099RRECOMMENDS
1100 A list of packages that extends the usability of a package being
1101 built. The package being built does not depend on this list of
1102 packages in order to successfully build, but needs them for the
1103 extended usability. To specify runtime dependencies for packages, see
1104 the ``RDEPENDS`` variable.
1105
1106 BitBake supports specifying versioned recommends. Although the syntax
1107 varies depending on the packaging format, BitBake hides these
1108 differences from you. Here is the general syntax to specify versions
1109 with the ``RRECOMMENDS`` variable: RRECOMMENDS_${PN} = "package
1110 (operator version)" For ``operator``, you can specify the following:
1111 = < > <= >= For example, the following sets up a recommend on version
1112 1.2 or greater of the package ``foo``: RRECOMMENDS_${PN} = "foo (>=
1113 1.2)"
1114
1115SECTION
1116 The section in which packages should be categorized.
1117
1118SRC_URI
1119 The list of source files - local or remote. This variable tells
1120 BitBake which bits to pull for the build and how to pull them. For
1121 example, if the recipe or append file needs to fetch a single tarball
1122 from the Internet, the recipe or append file uses a ``SRC_URI`` entry
1123 that specifies that tarball. On the other hand, if the recipe or
1124 append file needs to fetch a tarball and include a custom file, the
1125 recipe or append file needs an ``SRC_URI`` variable that specifies
1126 all those sources.
1127
1128 The following list explains the available URI protocols:
1129
1130 - *``file://`` -* Fetches files, which are usually files shipped
1131 with the metadata, from the local machine. The path is relative to
1132 the ```FILESPATH`` <#var-bb-FILESPATH>`__ variable.
1133
1134 - *``bzr://`` -* Fetches files from a Bazaar revision control
1135 repository.
1136
1137 - *``git://`` -* Fetches files from a Git revision control
1138 repository.
1139
1140 - *``osc://`` -* Fetches files from an OSC (OpenSUSE Build service)
1141 revision control repository.
1142
1143 - *``repo://`` -* Fetches files from a repo (Git) repository.
1144
1145 - *``http://`` -* Fetches files from the Internet using HTTP.
1146
1147 - *``https://`` -* Fetches files from the Internet using HTTPS.
1148
1149 - *``ftp://`` -* Fetches files from the Internet using FTP.
1150
1151 - *``cvs://`` -* Fetches files from a CVS revision control
1152 repository.
1153
1154 - *``hg://`` -* Fetches files from a Mercurial (``hg``) revision
1155 control repository.
1156
1157 - *``p4://`` -* Fetches files from a Perforce (``p4``) revision
1158 control repository.
1159
1160 - *``ssh://`` -* Fetches files from a secure shell.
1161
1162 - *``svn://`` -* Fetches files from a Subversion (``svn``) revision
1163 control repository.
1164
1165 Here are some additional options worth mentioning:
1166
1167 - *``unpack`` -* Controls whether or not to unpack the file if it is
1168 an archive. The default action is to unpack the file.
1169
1170 - *``subdir`` -* Places the file (or extracts its contents) into the
1171 specified subdirectory. This option is useful for unusual tarballs
1172 or other archives that do not have their files already in a
1173 subdirectory within the archive.
1174
1175 - *``name`` -* Specifies a name to be used for association with
1176 ``SRC_URI`` checksums when you have more than one file specified
1177 in ``SRC_URI``.
1178
1179 - *``downloadfilename`` -* Specifies the filename used when storing
1180 the downloaded file.
1181
1182SRCDATE
1183 The date of the source code used to build the package. This variable
1184 applies only if the source was fetched from a Source Code Manager
1185 (SCM).
1186
1187SRCREV
1188 The revision of the source code used to build the package. This
1189 variable applies only when using Subversion, Git, Mercurial and
1190 Bazaar. If you want to build a fixed revision and you want to avoid
1191 performing a query on the remote repository every time BitBake parses
1192 your recipe, you should specify a ``SRCREV`` that is a full revision
1193 identifier and not just a tag.
1194
1195SRCREV_FORMAT
1196 Helps construct valid ```SRCREV`` <#var-bb-SRCREV>`__ values when
1197 multiple source controlled URLs are used in
1198 ```SRC_URI`` <#var-bb-SRC_URI>`__.
1199
1200 The system needs help constructing these values under these
1201 circumstances. Each component in the ``SRC_URI`` is assigned a name
1202 and these are referenced in the ``SRCREV_FORMAT`` variable. Consider
1203 an example with URLs named "machine" and "meta". In this case,
1204 ``SRCREV_FORMAT`` could look like "machine_meta" and those names
1205 would have the SCM versions substituted into each position. Only one
1206 ``AUTOINC`` placeholder is added and if needed. And, this placeholder
1207 is placed at the start of the returned string.
1208
1209STAMP
1210 Specifies the base path used to create recipe stamp files. The path
1211 to an actual stamp file is constructed by evaluating this string and
1212 then appending additional information.
1213
1214STAMPCLEAN
1215 Specifies the base path used to create recipe stamp files. Unlike the
1216 ```STAMP`` <#var-bb-STAMP>`__ variable, ``STAMPCLEAN`` can contain
1217 wildcards to match the range of files a clean operation should
1218 remove. BitBake uses a clean operation to remove any other stamps it
1219 should be removing when creating a new stamp.
1220
1221SUMMARY
1222 A short summary for the recipe, which is 72 characters or less.
1223
1224SVNDIR
1225 The directory in which files checked out of a Subversion system are
1226 stored.
1227
1228T
1229 Points to a directory were BitBake places temporary files, which
1230 consist mostly of task logs and scripts, when building a particular
1231 recipe.
1232
1233TOPDIR
1234 Points to the build directory. BitBake automatically sets this
1235 variable.
diff --git a/bitbake/doc/index.rst b/bitbake/doc/index.rst
index 1cc9a4d849..3ec7031f57 100644
--- a/bitbake/doc/index.rst
+++ b/bitbake/doc/index.rst
@@ -1,11 +1,31 @@
1.. The Yocto Project documentation master file, created by 1===================
2 sphinx-quickstart on Mon Apr 13 09:38:33 2020. 2BitBake User Manual
3 You can adapt this file completely to your liking, but it should at least 3===================
4 contain the root `toctree` directive.
5 4
6Welcome to Bitbake documentation! 5.. include:: <xhtml1-lat1.txt>
7================================= 6
7| BitBake Community
8| Copyright |copy| |copyright|
9| <bitbake-devel@lists.openembedded.org>
10
11This work is licensed under the Creative Commons Attribution License. To view a
12copy of this license, visit http://creativecommons.org/licenses/by/2.5/ or send
13a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View,
14California 94041, USA.
15
16.. toctree::
17 :caption: Table of Contents
18 :numbered:
19
20 bitbake-user-manual/bitbake-user-manual-intro
21 bitbake-user-manual/bitbake-user-manual-execution
22 bitbake-user-manual/bitbake-user-manual-metadata
23 bitbake-user-manual/bitbake-user-manual-fetching
24 bitbake-user-manual/bitbake-user-manual-ref-variables
25 bitbake-user-manual/bitbake-user-manual-hello
8 26
9.. toctree:: 27.. toctree::
10 :maxdepth: 1 28 :maxdepth: 1
29 :hidden:
11 30
31 genindex