diff options
Diffstat (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst')
-rw-r--r-- | bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst | 182 |
1 files changed, 105 insertions, 77 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst index d74e768f69..d58fbb32ea 100644 --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst | |||
@@ -16,7 +16,7 @@ data, or simply return information about the execution environment. | |||
16 | 16 | ||
17 | This chapter describes BitBake's execution process from start to finish | 17 | This chapter describes BitBake's execution process from start to finish |
18 | when you use it to create an image. The execution process is launched | 18 | when you use it to create an image. The execution process is launched |
19 | using the following command form: :: | 19 | using the following command form:: |
20 | 20 | ||
21 | $ bitbake target | 21 | $ bitbake target |
22 | 22 | ||
@@ -32,7 +32,7 @@ the BitBake command and its options, see ":ref:`The BitBake Command | |||
32 | your project's ``local.conf`` configuration file. | 32 | your project's ``local.conf`` configuration file. |
33 | 33 | ||
34 | A common method to determine this value for your build host is to run | 34 | A common method to determine this value for your build host is to run |
35 | the following: :: | 35 | the following:: |
36 | 36 | ||
37 | $ grep processor /proc/cpuinfo | 37 | $ grep processor /proc/cpuinfo |
38 | 38 | ||
@@ -40,7 +40,7 @@ the BitBake command and its options, see ":ref:`The BitBake Command | |||
40 | the number of processors, which takes into account hyper-threading. | 40 | the number of processors, which takes into account hyper-threading. |
41 | Thus, a quad-core build host with hyper-threading most likely shows | 41 | Thus, a quad-core build host with hyper-threading most likely shows |
42 | eight processors, which is the value you would then assign to | 42 | eight processors, which is the value you would then assign to |
43 | ``BB_NUMBER_THREADS``. | 43 | :term:`BB_NUMBER_THREADS`. |
44 | 44 | ||
45 | A possibly simpler solution is that some Linux distributions (e.g. | 45 | A possibly simpler solution is that some Linux distributions (e.g. |
46 | Debian and Ubuntu) provide the ``ncpus`` command. | 46 | Debian and Ubuntu) provide the ``ncpus`` command. |
@@ -65,13 +65,13 @@ data itself is of various types: | |||
65 | 65 | ||
66 | The ``layer.conf`` files are used to construct key variables such as | 66 | The ``layer.conf`` files are used to construct key variables such as |
67 | :term:`BBPATH` and :term:`BBFILES`. | 67 | :term:`BBPATH` and :term:`BBFILES`. |
68 | ``BBPATH`` is used to search for configuration and class files under the | 68 | :term:`BBPATH` is used to search for configuration and class files under the |
69 | ``conf`` and ``classes`` directories, respectively. ``BBFILES`` is used | 69 | ``conf`` and ``classes`` directories, respectively. :term:`BBFILES` is used |
70 | to locate both recipe and recipe append files (``.bb`` and | 70 | to locate both recipe and recipe append files (``.bb`` and |
71 | ``.bbappend``). If there is no ``bblayers.conf`` file, it is assumed the | 71 | ``.bbappend``). If there is no ``bblayers.conf`` file, it is assumed the |
72 | user has set the ``BBPATH`` and ``BBFILES`` directly in the environment. | 72 | user has set the :term:`BBPATH` and :term:`BBFILES` directly in the environment. |
73 | 73 | ||
74 | Next, the ``bitbake.conf`` file is located using the ``BBPATH`` variable | 74 | Next, the ``bitbake.conf`` file is located using the :term:`BBPATH` variable |
75 | that was just constructed. The ``bitbake.conf`` file may also include | 75 | that was just constructed. The ``bitbake.conf`` file may also include |
76 | other configuration files using the ``include`` or ``require`` | 76 | other configuration files using the ``include`` or ``require`` |
77 | directives. | 77 | directives. |
@@ -79,8 +79,8 @@ directives. | |||
79 | Prior to parsing configuration files, BitBake looks at certain | 79 | Prior to parsing configuration files, BitBake looks at certain |
80 | variables, including: | 80 | variables, including: |
81 | 81 | ||
82 | - :term:`BB_ENV_WHITELIST` | 82 | - :term:`BB_ENV_PASSTHROUGH` |
83 | - :term:`BB_ENV_EXTRAWHITE` | 83 | - :term:`BB_ENV_PASSTHROUGH_ADDITIONS` |
84 | - :term:`BB_PRESERVE_ENV` | 84 | - :term:`BB_PRESERVE_ENV` |
85 | - :term:`BB_ORIGENV` | 85 | - :term:`BB_ORIGENV` |
86 | - :term:`BITBAKE_UI` | 86 | - :term:`BITBAKE_UI` |
@@ -104,7 +104,7 @@ BitBake first searches the current working directory for an optional | |||
104 | contain a :term:`BBLAYERS` variable that is a | 104 | contain a :term:`BBLAYERS` variable that is a |
105 | space-delimited list of 'layer' directories. Recall that if BitBake | 105 | space-delimited list of 'layer' directories. Recall that if BitBake |
106 | cannot find a ``bblayers.conf`` file, then it is assumed the user has | 106 | cannot find a ``bblayers.conf`` file, then it is assumed the user has |
107 | set the ``BBPATH`` and ``BBFILES`` variables directly in the | 107 | set the :term:`BBPATH` and :term:`BBFILES` variables directly in the |
108 | environment. | 108 | environment. |
109 | 109 | ||
110 | For each directory (layer) in this list, a ``conf/layer.conf`` file is | 110 | For each directory (layer) in this list, a ``conf/layer.conf`` file is |
@@ -114,7 +114,7 @@ files automatically set up :term:`BBPATH` and other | |||
114 | variables correctly for a given build directory. | 114 | variables correctly for a given build directory. |
115 | 115 | ||
116 | BitBake then expects to find the ``conf/bitbake.conf`` file somewhere in | 116 | BitBake then expects to find the ``conf/bitbake.conf`` file somewhere in |
117 | the user-specified ``BBPATH``. That configuration file generally has | 117 | the user-specified :term:`BBPATH`. That configuration file generally has |
118 | include directives to pull in any other metadata such as files specific | 118 | include directives to pull in any other metadata such as files specific |
119 | to the architecture, the machine, the local environment, and so forth. | 119 | to the architecture, the machine, the local environment, and so forth. |
120 | 120 | ||
@@ -135,11 +135,11 @@ The ``base.bbclass`` file is always included. Other classes that are | |||
135 | specified in the configuration using the | 135 | specified in the configuration using the |
136 | :term:`INHERIT` variable are also included. BitBake | 136 | :term:`INHERIT` variable are also included. BitBake |
137 | searches for class files in a ``classes`` subdirectory under the paths | 137 | searches for class files in a ``classes`` subdirectory under the paths |
138 | in ``BBPATH`` in the same way as configuration files. | 138 | in :term:`BBPATH` in the same way as configuration files. |
139 | 139 | ||
140 | A good way to get an idea of the configuration files and the class files | 140 | A good way to get an idea of the configuration files and the class files |
141 | used in your execution environment is to run the following BitBake | 141 | used in your execution environment is to run the following BitBake |
142 | command: :: | 142 | command:: |
143 | 143 | ||
144 | $ bitbake -e > mybb.log | 144 | $ bitbake -e > mybb.log |
145 | 145 | ||
@@ -155,7 +155,7 @@ execution environment. | |||
155 | pair of curly braces in a shell function, the closing curly brace | 155 | pair of curly braces in a shell function, the closing curly brace |
156 | must not be located at the start of the line without leading spaces. | 156 | must not be located at the start of the line without leading spaces. |
157 | 157 | ||
158 | Here is an example that causes BitBake to produce a parsing error: :: | 158 | Here is an example that causes BitBake to produce a parsing error:: |
159 | 159 | ||
160 | fakeroot create_shar() { | 160 | fakeroot create_shar() { |
161 | cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh | 161 | cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh |
@@ -184,13 +184,13 @@ Locating and Parsing Recipes | |||
184 | During the configuration phase, BitBake will have set | 184 | During the configuration phase, BitBake will have set |
185 | :term:`BBFILES`. BitBake now uses it to construct a | 185 | :term:`BBFILES`. BitBake now uses it to construct a |
186 | list of recipes to parse, along with any append files (``.bbappend``) to | 186 | list of recipes to parse, along with any append files (``.bbappend``) to |
187 | apply. ``BBFILES`` is a space-separated list of available files and | 187 | apply. :term:`BBFILES` is a space-separated list of available files and |
188 | supports wildcards. An example would be: :: | 188 | supports wildcards. An example would be:: |
189 | 189 | ||
190 | BBFILES = "/path/to/bbfiles/*.bb /path/to/appends/*.bbappend" | 190 | BBFILES = "/path/to/bbfiles/*.bb /path/to/appends/*.bbappend" |
191 | 191 | ||
192 | BitBake parses each | 192 | BitBake parses each |
193 | recipe and append file located with ``BBFILES`` and stores the values of | 193 | recipe and append file located with :term:`BBFILES` and stores the values of |
194 | various variables into the datastore. | 194 | various variables into the datastore. |
195 | 195 | ||
196 | .. note:: | 196 | .. note:: |
@@ -201,18 +201,18 @@ For each file, a fresh copy of the base configuration is made, then the | |||
201 | recipe is parsed line by line. Any inherit statements cause BitBake to | 201 | recipe is parsed line by line. Any inherit statements cause BitBake to |
202 | find and then parse class files (``.bbclass``) using | 202 | find and then parse class files (``.bbclass``) using |
203 | :term:`BBPATH` as the search path. Finally, BitBake | 203 | :term:`BBPATH` as the search path. Finally, BitBake |
204 | parses in order any append files found in ``BBFILES``. | 204 | parses in order any append files found in :term:`BBFILES`. |
205 | 205 | ||
206 | One common convention is to use the recipe filename to define pieces of | 206 | One common convention is to use the recipe filename to define pieces of |
207 | metadata. For example, in ``bitbake.conf`` the recipe name and version | 207 | metadata. For example, in ``bitbake.conf`` the recipe name and version |
208 | are used to set the variables :term:`PN` and | 208 | are used to set the variables :term:`PN` and |
209 | :term:`PV`: :: | 209 | :term:`PV`:: |
210 | 210 | ||
211 | PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" | 211 | PN = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" |
212 | PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}" | 212 | PV = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}" |
213 | 213 | ||
214 | In this example, a recipe called "something_1.2.3.bb" would set | 214 | In this example, a recipe called "something_1.2.3.bb" would set |
215 | ``PN`` to "something" and ``PV`` to "1.2.3". | 215 | :term:`PN` to "something" and :term:`PV` to "1.2.3". |
216 | 216 | ||
217 | By the time parsing is complete for a recipe, BitBake has a list of | 217 | By the time parsing is complete for a recipe, BitBake has a list of |
218 | tasks that the recipe defines and a set of data consisting of keys and | 218 | tasks that the recipe defines and a set of data consisting of keys and |
@@ -228,7 +228,7 @@ and then reload it. | |||
228 | Where possible, subsequent BitBake commands reuse this cache of recipe | 228 | Where possible, subsequent BitBake commands reuse this cache of recipe |
229 | information. The validity of this cache is determined by first computing | 229 | information. The validity of this cache is determined by first computing |
230 | a checksum of the base configuration data (see | 230 | a checksum of the base configuration data (see |
231 | :term:`BB_HASHCONFIG_WHITELIST`) and | 231 | :term:`BB_HASHCONFIG_IGNORE_VARS`) and |
232 | then checking if the checksum matches. If that checksum matches what is | 232 | then checking if the checksum matches. If that checksum matches what is |
233 | in the cache and the recipe and class files have not changed, BitBake is | 233 | in the cache and the recipe and class files have not changed, BitBake is |
234 | able to use the cache. BitBake then reloads the cached information about | 234 | able to use the cache. BitBake then reloads the cached information about |
@@ -238,13 +238,14 @@ Recipe file collections exist to allow the user to have multiple | |||
238 | repositories of ``.bb`` files that contain the same exact package. For | 238 | repositories of ``.bb`` files that contain the same exact package. For |
239 | example, one could easily use them to make one's own local copy of an | 239 | example, one could easily use them to make one's own local copy of an |
240 | upstream repository, but with custom modifications that one does not | 240 | upstream repository, but with custom modifications that one does not |
241 | want upstream. Here is an example: :: | 241 | want upstream. Here is an example:: |
242 | 242 | ||
243 | BBFILES = "/stuff/openembedded/*/*.bb /stuff/openembedded.modified/*/*.bb" | 243 | BBFILES = "/stuff/openembedded/*/*.bb /stuff/openembedded.modified/*/*.bb" |
244 | BBFILE_COLLECTIONS = "upstream local" | 244 | BBFILE_COLLECTIONS = "upstream local" |
245 | BBFILE_PATTERN_upstream = "^/stuff/openembedded/" | 245 | BBFILE_PATTERN_upstream = "^/stuff/openembedded/" |
246 | BBFILE_PATTERN_local = "^/stuff/openembedded.modified/" | 246 | BBFILE_PATTERN_local = "^/stuff/openembedded.modified/" |
247 | BBFILE_PRIORITY_upstream = "5" BBFILE_PRIORITY_local = "10" | 247 | BBFILE_PRIORITY_upstream = "5" |
248 | BBFILE_PRIORITY_local = "10" | ||
248 | 249 | ||
249 | .. note:: | 250 | .. note:: |
250 | 251 | ||
@@ -259,21 +260,21 @@ Providers | |||
259 | 260 | ||
260 | Assuming BitBake has been instructed to execute a target and that all | 261 | Assuming BitBake has been instructed to execute a target and that all |
261 | the recipe files have been parsed, BitBake starts to figure out how to | 262 | the recipe files have been parsed, BitBake starts to figure out how to |
262 | build the target. BitBake looks through the ``PROVIDES`` list for each | 263 | build the target. BitBake looks through the :term:`PROVIDES` list for each |
263 | of the recipes. A ``PROVIDES`` list is the list of names by which the | 264 | of the recipes. A :term:`PROVIDES` list is the list of names by which the |
264 | recipe can be known. Each recipe's ``PROVIDES`` list is created | 265 | recipe can be known. Each recipe's :term:`PROVIDES` list is created |
265 | implicitly through the recipe's :term:`PN` variable and | 266 | implicitly through the recipe's :term:`PN` variable and |
266 | explicitly through the recipe's :term:`PROVIDES` | 267 | explicitly through the recipe's :term:`PROVIDES` |
267 | variable, which is optional. | 268 | variable, which is optional. |
268 | 269 | ||
269 | When a recipe uses ``PROVIDES``, that recipe's functionality can be | 270 | When a recipe uses :term:`PROVIDES`, that recipe's functionality can be |
270 | found under an alternative name or names other than the implicit ``PN`` | 271 | found under an alternative name or names other than the implicit :term:`PN` |
271 | name. As an example, suppose a recipe named ``keyboard_1.0.bb`` | 272 | name. As an example, suppose a recipe named ``keyboard_1.0.bb`` |
272 | contained the following: :: | 273 | contained the following:: |
273 | 274 | ||
274 | PROVIDES += "fullkeyboard" | 275 | PROVIDES += "fullkeyboard" |
275 | 276 | ||
276 | The ``PROVIDES`` | 277 | The :term:`PROVIDES` |
277 | list for this recipe becomes "keyboard", which is implicit, and | 278 | list for this recipe becomes "keyboard", which is implicit, and |
278 | "fullkeyboard", which is explicit. Consequently, the functionality found | 279 | "fullkeyboard", which is explicit. Consequently, the functionality found |
279 | in ``keyboard_1.0.bb`` can be found under two different names. | 280 | in ``keyboard_1.0.bb`` can be found under two different names. |
@@ -283,14 +284,14 @@ in ``keyboard_1.0.bb`` can be found under two different names. | |||
283 | Preferences | 284 | Preferences |
284 | =========== | 285 | =========== |
285 | 286 | ||
286 | The ``PROVIDES`` list is only part of the solution for figuring out a | 287 | The :term:`PROVIDES` list is only part of the solution for figuring out a |
287 | target's recipes. Because targets might have multiple providers, BitBake | 288 | target's recipes. Because targets might have multiple providers, BitBake |
288 | needs to prioritize providers by determining provider preferences. | 289 | needs to prioritize providers by determining provider preferences. |
289 | 290 | ||
290 | A common example in which a target has multiple providers is | 291 | A common example in which a target has multiple providers is |
291 | "virtual/kernel", which is on the ``PROVIDES`` list for each kernel | 292 | "virtual/kernel", which is on the :term:`PROVIDES` list for each kernel |
292 | recipe. Each machine often selects the best kernel provider by using a | 293 | recipe. Each machine often selects the best kernel provider by using a |
293 | line similar to the following in the machine configuration file: :: | 294 | line similar to the following in the machine configuration file:: |
294 | 295 | ||
295 | PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" | 296 | PREFERRED_PROVIDER_virtual/kernel = "linux-yocto" |
296 | 297 | ||
@@ -308,10 +309,10 @@ specify a particular version. You can influence the order by using the | |||
308 | :term:`DEFAULT_PREFERENCE` variable. | 309 | :term:`DEFAULT_PREFERENCE` variable. |
309 | 310 | ||
310 | By default, files have a preference of "0". Setting | 311 | By default, files have a preference of "0". Setting |
311 | ``DEFAULT_PREFERENCE`` to "-1" makes the recipe unlikely to be used | 312 | :term:`DEFAULT_PREFERENCE` to "-1" makes the recipe unlikely to be used |
312 | unless it is explicitly referenced. Setting ``DEFAULT_PREFERENCE`` to | 313 | unless it is explicitly referenced. Setting :term:`DEFAULT_PREFERENCE` to |
313 | "1" makes it likely the recipe is used. ``PREFERRED_VERSION`` overrides | 314 | "1" makes it likely the recipe is used. :term:`PREFERRED_VERSION` overrides |
314 | any ``DEFAULT_PREFERENCE`` setting. ``DEFAULT_PREFERENCE`` is often used | 315 | any :term:`DEFAULT_PREFERENCE` setting. :term:`DEFAULT_PREFERENCE` is often used |
315 | to mark newer and more experimental recipe versions until they have | 316 | to mark newer and more experimental recipe versions until they have |
316 | undergone sufficient testing to be considered stable. | 317 | undergone sufficient testing to be considered stable. |
317 | 318 | ||
@@ -330,7 +331,7 @@ If the first recipe is named ``a_1.1.bb``, then the | |||
330 | 331 | ||
331 | Thus, if a recipe named ``a_1.2.bb`` exists, BitBake will choose 1.2 by | 332 | Thus, if a recipe named ``a_1.2.bb`` exists, BitBake will choose 1.2 by |
332 | default. However, if you define the following variable in a ``.conf`` | 333 | default. However, if you define the following variable in a ``.conf`` |
333 | file that BitBake parses, you can change that preference: :: | 334 | file that BitBake parses, you can change that preference:: |
334 | 335 | ||
335 | PREFERRED_VERSION_a = "1.1" | 336 | PREFERRED_VERSION_a = "1.1" |
336 | 337 | ||
@@ -393,7 +394,7 @@ ready to run, those tasks have all their dependencies met, and the | |||
393 | thread threshold has not been exceeded. | 394 | thread threshold has not been exceeded. |
394 | 395 | ||
395 | It is worth noting that you can greatly speed up the build time by | 396 | It is worth noting that you can greatly speed up the build time by |
396 | properly setting the ``BB_NUMBER_THREADS`` variable. | 397 | properly setting the :term:`BB_NUMBER_THREADS` variable. |
397 | 398 | ||
398 | As each task completes, a timestamp is written to the directory | 399 | As each task completes, a timestamp is written to the directory |
399 | specified by the :term:`STAMP` variable. On subsequent | 400 | specified by the :term:`STAMP` variable. On subsequent |
@@ -434,7 +435,7 @@ BitBake writes a shell script to | |||
434 | executes the script. The generated shell script contains all the | 435 | executes the script. The generated shell script contains all the |
435 | exported variables, and the shell functions with all variables expanded. | 436 | exported variables, and the shell functions with all variables expanded. |
436 | Output from the shell script goes to the file | 437 | Output from the shell script goes to the file |
437 | ``${T}/log.do_taskname.pid``. Looking at the expanded shell functions in | 438 | ``${``\ :term:`T`\ ``}/log.do_taskname.pid``. Looking at the expanded shell functions in |
438 | the run file and the output in the log files is a useful debugging | 439 | the run file and the output in the log files is a useful debugging |
439 | technique. | 440 | technique. |
440 | 441 | ||
@@ -476,7 +477,7 @@ changes because it should not affect the output for target packages. The | |||
476 | simplistic approach for excluding the working directory is to set it to | 477 | simplistic approach for excluding the working directory is to set it to |
477 | some fixed value and create the checksum for the "run" script. BitBake | 478 | some fixed value and create the checksum for the "run" script. BitBake |
478 | goes one step better and uses the | 479 | goes one step better and uses the |
479 | :term:`BB_HASHBASE_WHITELIST` variable | 480 | :term:`BB_BASEHASH_IGNORE_VARS` variable |
480 | to define a list of variables that should never be included when | 481 | to define a list of variables that should never be included when |
481 | generating the signatures. | 482 | generating the signatures. |
482 | 483 | ||
@@ -497,7 +498,7 @@ to the task. | |||
497 | 498 | ||
498 | Like the working directory case, situations exist where dependencies | 499 | Like the working directory case, situations exist where dependencies |
499 | should be ignored. For these cases, you can instruct the build process | 500 | should be ignored. For these cases, you can instruct the build process |
500 | to ignore a dependency by using a line like the following: :: | 501 | to ignore a dependency by using a line like the following:: |
501 | 502 | ||
502 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" | 503 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" |
503 | 504 | ||
@@ -507,7 +508,7 @@ even if it does reference it. | |||
507 | 508 | ||
508 | Equally, there are cases where we need to add dependencies BitBake is | 509 | Equally, there are cases where we need to add dependencies BitBake is |
509 | not able to find. You can accomplish this by using a line like the | 510 | not able to find. You can accomplish this by using a line like the |
510 | following: :: | 511 | following:: |
511 | 512 | ||
512 | PACKAGE_ARCHS[vardeps] = "MACHINE" | 513 | PACKAGE_ARCHS[vardeps] = "MACHINE" |
513 | 514 | ||
@@ -522,7 +523,7 @@ it cannot figure out dependencies. | |||
522 | Thus far, this section has limited discussion to the direct inputs into | 523 | Thus far, this section has limited discussion to the direct inputs into |
523 | a task. Information based on direct inputs is referred to as the | 524 | a task. Information based on direct inputs is referred to as the |
524 | "basehash" in the code. However, there is still the question of a task's | 525 | "basehash" in the code. However, there is still the question of a task's |
525 | indirect inputs - the things that were already built and present in the | 526 | indirect inputs --- the things that were already built and present in the |
526 | build directory. The checksum (or signature) for a particular task needs | 527 | build directory. The checksum (or signature) for a particular task needs |
527 | to add the hashes of all the tasks on which the particular task depends. | 528 | to add the hashes of all the tasks on which the particular task depends. |
528 | Choosing which dependencies to add is a policy decision. However, the | 529 | Choosing which dependencies to add is a policy decision. However, the |
@@ -533,11 +534,11 @@ At the code level, there are a variety of ways both the basehash and the | |||
533 | dependent task hashes can be influenced. Within the BitBake | 534 | dependent task hashes can be influenced. Within the BitBake |
534 | configuration file, we can give BitBake some extra information to help | 535 | configuration file, we can give BitBake some extra information to help |
535 | it construct the basehash. The following statement effectively results | 536 | it construct the basehash. The following statement effectively results |
536 | in a list of global variable dependency excludes - variables never | 537 | in a list of global variable dependency excludes --- variables never |
537 | included in any checksum. This example uses variables from OpenEmbedded | 538 | included in any checksum. This example uses variables from OpenEmbedded |
538 | to help illustrate the concept: :: | 539 | to help illustrate the concept:: |
539 | 540 | ||
540 | BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \ | 541 | BB_BASEHASH_IGNORE_VARS ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \ |
541 | SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL \ | 542 | SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL \ |
542 | USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \ | 543 | USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \ |
543 | PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \ | 544 | PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \ |
@@ -551,23 +552,22 @@ through dependency chains are more complex and are generally | |||
551 | accomplished with a Python function. The code in | 552 | accomplished with a Python function. The code in |
552 | ``meta/lib/oe/sstatesig.py`` shows two examples of this and also | 553 | ``meta/lib/oe/sstatesig.py`` shows two examples of this and also |
553 | illustrates how you can insert your own policy into the system if so | 554 | illustrates how you can insert your own policy into the system if so |
554 | desired. This file defines the two basic signature generators | 555 | desired. This file defines the basic signature generator |
555 | OpenEmbedded-Core uses: "OEBasic" and "OEBasicHash". By default, there | 556 | OpenEmbedded-Core uses: "OEBasicHash". By default, there |
556 | is a dummy "noop" signature handler enabled in BitBake. This means that | 557 | is a dummy "noop" signature handler enabled in BitBake. This means that |
557 | behavior is unchanged from previous versions. ``OE-Core`` uses the | 558 | behavior is unchanged from previous versions. ``OE-Core`` uses the |
558 | "OEBasicHash" signature handler by default through this setting in the | 559 | "OEBasicHash" signature handler by default through this setting in the |
559 | ``bitbake.conf`` file: :: | 560 | ``bitbake.conf`` file:: |
560 | 561 | ||
561 | BB_SIGNATURE_HANDLER ?= "OEBasicHash" | 562 | BB_SIGNATURE_HANDLER ?= "OEBasicHash" |
562 | 563 | ||
563 | The "OEBasicHash" ``BB_SIGNATURE_HANDLER`` is the same as the "OEBasic" | 564 | The main feature of the "OEBasicHash" :term:`BB_SIGNATURE_HANDLER` is that |
564 | version but adds the task hash to the stamp files. This results in any | 565 | it adds the task hash to the stamp files. Thanks to this, any metadata |
565 | metadata change that changes the task hash, automatically causing the | 566 | change will change the task hash, automatically causing the task to be run |
566 | task to be run again. This removes the need to bump | 567 | again. This removes the need to bump :term:`PR` values, and changes to |
567 | :term:`PR` values, and changes to metadata automatically | 568 | metadata automatically ripple across the build. |
568 | ripple across the build. | ||
569 | 569 | ||
570 | It is also worth noting that the end result of these signature | 570 | It is also worth noting that the end result of signature |
571 | generators is to make some dependency and hash information available to | 571 | generators is to make some dependency and hash information available to |
572 | the build. This information includes: | 572 | the build. This information includes: |
573 | 573 | ||
@@ -577,10 +577,7 @@ the build. This information includes: | |||
577 | - ``BB_BASEHASH_``\ *filename:taskname*: The base hashes for each | 577 | - ``BB_BASEHASH_``\ *filename:taskname*: The base hashes for each |
578 | dependent task. | 578 | dependent task. |
579 | 579 | ||
580 | - ``BBHASHDEPS_``\ *filename:taskname*: The task dependencies for | 580 | - :term:`BB_TASKHASH`: The hash of the currently running task. |
581 | each task. | ||
582 | |||
583 | - ``BB_TASKHASH``: The hash of the currently running task. | ||
584 | 581 | ||
585 | It is worth noting that BitBake's "-S" option lets you debug BitBake's | 582 | It is worth noting that BitBake's "-S" option lets you debug BitBake's |
586 | processing of signatures. The options passed to -S allow different | 583 | processing of signatures. The options passed to -S allow different |
@@ -589,10 +586,11 @@ or possibly those defined in the metadata/signature handler itself. The | |||
589 | simplest parameter to pass is "none", which causes a set of signature | 586 | simplest parameter to pass is "none", which causes a set of signature |
590 | information to be written out into ``STAMPS_DIR`` corresponding to the | 587 | information to be written out into ``STAMPS_DIR`` corresponding to the |
591 | targets specified. The other currently available parameter is | 588 | targets specified. The other currently available parameter is |
592 | "printdiff", which causes BitBake to try to establish the closest | 589 | "printdiff", which causes BitBake to try to establish the most recent |
593 | signature match it can (e.g. in the sstate cache) and then run | 590 | signature match it can (e.g. in the sstate cache) and then run |
594 | ``bitbake-diffsigs`` over the matches to determine the stamps and delta | 591 | compare the matched signatures to determine the stamps and delta |
595 | where these two stamp trees diverge. | 592 | where these two stamp trees diverge. This can be used to determine why |
593 | tasks need to be re-run in situations where that is not expected. | ||
596 | 594 | ||
597 | .. note:: | 595 | .. note:: |
598 | 596 | ||
@@ -647,13 +645,6 @@ compiled binary. To handle this, BitBake calls the | |||
647 | each successful setscene task to know whether or not it needs to obtain | 645 | each successful setscene task to know whether or not it needs to obtain |
648 | the dependencies of that task. | 646 | the dependencies of that task. |
649 | 647 | ||
650 | Finally, after all the setscene tasks have executed, BitBake calls the | ||
651 | function listed in | ||
652 | :term:`BB_SETSCENE_VERIFY_FUNCTION2` | ||
653 | with the list of tasks BitBake thinks has been "covered". The metadata | ||
654 | can then ensure that this list is correct and can inform BitBake that it | ||
655 | wants specific tasks to be run regardless of the setscene result. | ||
656 | |||
657 | You can find more information on setscene metadata in the | 648 | You can find more information on setscene metadata in the |
658 | :ref:`bitbake-user-manual/bitbake-user-manual-metadata:task checksums and setscene` | 649 | :ref:`bitbake-user-manual/bitbake-user-manual-metadata:task checksums and setscene` |
659 | section. | 650 | section. |
@@ -666,7 +657,7 @@ builds are when execute, bitbake also supports user defined | |||
666 | configuration of the `Python | 657 | configuration of the `Python |
667 | logging <https://docs.python.org/3/library/logging.html>`__ facilities | 658 | logging <https://docs.python.org/3/library/logging.html>`__ facilities |
668 | through the :term:`BB_LOGCONFIG` variable. This | 659 | through the :term:`BB_LOGCONFIG` variable. This |
669 | variable defines a json or yaml `logging | 660 | variable defines a JSON or YAML `logging |
670 | configuration <https://docs.python.org/3/library/logging.config.html>`__ | 661 | configuration <https://docs.python.org/3/library/logging.config.html>`__ |
671 | that will be intelligently merged into the default configuration. The | 662 | that will be intelligently merged into the default configuration. The |
672 | logging configuration is merged using the following rules: | 663 | logging configuration is merged using the following rules: |
@@ -700,9 +691,9 @@ logging configuration is merged using the following rules: | |||
700 | adds a filter called ``BitBake.defaultFilter``, both filters will be | 691 | adds a filter called ``BitBake.defaultFilter``, both filters will be |
701 | applied to the logger | 692 | applied to the logger |
702 | 693 | ||
703 | As an example, consider the following user logging configuration file | 694 | As a first example, you can create a ``hashequiv.json`` user logging |
704 | which logs all Hash Equivalence related messages of VERBOSE or higher to | 695 | configuration file to log all Hash Equivalence related messages of ``VERBOSE`` |
705 | a file called ``hashequiv.log`` :: | 696 | or higher priority to a file called ``hashequiv.log``:: |
706 | 697 | ||
707 | { | 698 | { |
708 | "version": 1, | 699 | "version": 1, |
@@ -731,3 +722,40 @@ a file called ``hashequiv.log`` :: | |||
731 | } | 722 | } |
732 | } | 723 | } |
733 | } | 724 | } |
725 | |||
726 | Then set the :term:`BB_LOGCONFIG` variable in ``conf/local.conf``:: | ||
727 | |||
728 | BB_LOGCONFIG = "hashequiv.json" | ||
729 | |||
730 | Another example is this ``warn.json`` file to log all ``WARNING`` and | ||
731 | higher priority messages to a ``warn.log`` file:: | ||
732 | |||
733 | { | ||
734 | "version": 1, | ||
735 | "formatters": { | ||
736 | "warnlogFormatter": { | ||
737 | "()": "bb.msg.BBLogFormatter", | ||
738 | "format": "%(levelname)s: %(message)s" | ||
739 | } | ||
740 | }, | ||
741 | |||
742 | "handlers": { | ||
743 | "warnlog": { | ||
744 | "class": "logging.FileHandler", | ||
745 | "formatter": "warnlogFormatter", | ||
746 | "level": "WARNING", | ||
747 | "filename": "warn.log" | ||
748 | } | ||
749 | }, | ||
750 | |||
751 | "loggers": { | ||
752 | "BitBake": { | ||
753 | "handlers": ["warnlog"] | ||
754 | } | ||
755 | }, | ||
756 | |||
757 | "@disable_existing_loggers": false | ||
758 | } | ||
759 | |||
760 | Note that BitBake's helper classes for structured logging are implemented in | ||
761 | ``lib/bb/msg.py``. | ||