diff options
Diffstat (limited to 'documentation/dev-manual/debugging.rst')
| -rw-r--r-- | documentation/dev-manual/debugging.rst | 1271 |
1 files changed, 0 insertions, 1271 deletions
diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst deleted file mode 100644 index 6c45ccf652..0000000000 --- a/documentation/dev-manual/debugging.rst +++ /dev/null | |||
| @@ -1,1271 +0,0 @@ | |||
| 1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
| 2 | |||
| 3 | Debugging Tools and Techniques | ||
| 4 | ****************************** | ||
| 5 | |||
| 6 | The exact method for debugging build failures depends on the nature of | ||
| 7 | the problem and on the system's area from which the bug originates. | ||
| 8 | Standard debugging practices such as comparison against the last known | ||
| 9 | working version with examination of the changes and the re-application | ||
| 10 | of steps to identify the one causing the problem are valid for the Yocto | ||
| 11 | Project just as they are for any other system. Even though it is | ||
| 12 | impossible to detail every possible potential failure, this section | ||
| 13 | provides some general tips to aid in debugging given a variety of | ||
| 14 | situations. | ||
| 15 | |||
| 16 | .. note:: | ||
| 17 | |||
| 18 | A useful feature for debugging is the error reporting tool. | ||
| 19 | Configuring the Yocto Project to use this tool causes the | ||
| 20 | OpenEmbedded build system to produce error reporting commands as part | ||
| 21 | of the console output. You can enter the commands after the build | ||
| 22 | completes to log error information into a common database, that can | ||
| 23 | help you figure out what might be going wrong. For information on how | ||
| 24 | to enable and use this feature, see the | ||
| 25 | ":ref:`dev-manual/error-reporting-tool:using the error reporting tool`" | ||
| 26 | section. | ||
| 27 | |||
| 28 | The following list shows the debugging topics in the remainder of this | ||
| 29 | section: | ||
| 30 | |||
| 31 | - ":ref:`dev-manual/debugging:viewing logs from failed tasks`" describes | ||
| 32 | how to find and view logs from tasks that failed during the build | ||
| 33 | process. | ||
| 34 | |||
| 35 | - ":ref:`dev-manual/debugging:viewing variable values`" describes how to | ||
| 36 | use the BitBake ``-e`` option to examine variable values after a | ||
| 37 | recipe has been parsed. | ||
| 38 | |||
| 39 | - ":ref:`dev-manual/debugging:viewing package information with ``oe-pkgdata-util```" | ||
| 40 | describes how to use the ``oe-pkgdata-util`` utility to query | ||
| 41 | :term:`PKGDATA_DIR` and | ||
| 42 | display package-related information for built packages. | ||
| 43 | |||
| 44 | - ":ref:`dev-manual/debugging:viewing dependencies between recipes and tasks`" | ||
| 45 | describes how to use the BitBake ``-g`` option to display recipe | ||
| 46 | dependency information used during the build. | ||
| 47 | |||
| 48 | - ":ref:`dev-manual/debugging:viewing task variable dependencies`" describes | ||
| 49 | how to use the ``bitbake-dumpsig`` command in conjunction with key | ||
| 50 | subdirectories in the :term:`Build Directory` to determine variable | ||
| 51 | dependencies. | ||
| 52 | |||
| 53 | - ":ref:`dev-manual/debugging:running specific tasks`" describes | ||
| 54 | how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``) | ||
| 55 | to run specific tasks in the build chain. It can be useful to run | ||
| 56 | tasks "out-of-order" when trying isolate build issues. | ||
| 57 | |||
| 58 | - ":ref:`dev-manual/debugging:general BitBake problems`" describes how | ||
| 59 | to use BitBake's ``-D`` debug output option to reveal more about what | ||
| 60 | BitBake is doing during the build. | ||
| 61 | |||
| 62 | - ":ref:`dev-manual/debugging:building with no dependencies`" | ||
| 63 | describes how to use the BitBake ``-b`` option to build a recipe | ||
| 64 | while ignoring dependencies. | ||
| 65 | |||
| 66 | - ":ref:`dev-manual/debugging:recipe logging mechanisms`" | ||
| 67 | describes how to use the many recipe logging functions to produce | ||
| 68 | debugging output and report errors and warnings. | ||
| 69 | |||
| 70 | - ":ref:`dev-manual/debugging:debugging parallel make races`" | ||
| 71 | describes how to debug situations where the build consists of several | ||
| 72 | parts that are run simultaneously and when the output or result of | ||
| 73 | one part is not ready for use with a different part of the build that | ||
| 74 | depends on that output. | ||
| 75 | |||
| 76 | - ":ref:`dev-manual/debugging:debugging with the gnu project debugger (gdb) remotely`" | ||
| 77 | describes how to use GDB to allow you to examine running programs, which can | ||
| 78 | help you fix problems. | ||
| 79 | |||
| 80 | - ":ref:`dev-manual/debugging:debugging with the gnu project debugger (gdb) on the target`" | ||
| 81 | describes how to use GDB directly on target hardware for debugging. | ||
| 82 | |||
| 83 | - ":ref:`dev-manual/debugging:other debugging tips`" describes | ||
| 84 | miscellaneous debugging tips that can be useful. | ||
| 85 | |||
| 86 | Viewing Logs from Failed Tasks | ||
| 87 | ============================== | ||
| 88 | |||
| 89 | You can find the log for a task in the file | ||
| 90 | ``${``\ :term:`WORKDIR`\ ``}/temp/log.do_``\ `taskname`. | ||
| 91 | For example, the log for the | ||
| 92 | :ref:`ref-tasks-compile` task of the | ||
| 93 | QEMU minimal image for the x86 machine (``qemux86``) might be in | ||
| 94 | ``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``. | ||
| 95 | To see the commands :term:`BitBake` ran | ||
| 96 | to generate a log, look at the corresponding ``run.do_``\ `taskname` file | ||
| 97 | in the same directory. | ||
| 98 | |||
| 99 | ``log.do_``\ `taskname` and ``run.do_``\ `taskname` are actually symbolic | ||
| 100 | links to ``log.do_``\ `taskname`\ ``.``\ `pid` and | ||
| 101 | ``log.run_``\ `taskname`\ ``.``\ `pid`, where `pid` is the PID the task had | ||
| 102 | when it ran. The symlinks always point to the files corresponding to the | ||
| 103 | most recent run. | ||
| 104 | |||
| 105 | Viewing Variable Values | ||
| 106 | ======================= | ||
| 107 | |||
| 108 | Sometimes you need to know the value of a variable as a result of | ||
| 109 | BitBake's parsing step. This could be because some unexpected behavior | ||
| 110 | occurred in your project. Perhaps an attempt to :ref:`modify a variable | ||
| 111 | <bitbake-user-manual/bitbake-user-manual-metadata:modifying existing | ||
| 112 | variables>` did not work out as expected. | ||
| 113 | |||
| 114 | BitBake's ``-e`` option is used to display variable values after | ||
| 115 | parsing. The following command displays the variable values after the | ||
| 116 | configuration files (i.e. ``local.conf``, ``bblayers.conf``, | ||
| 117 | ``bitbake.conf`` and so forth) have been parsed:: | ||
| 118 | |||
| 119 | $ bitbake -e | ||
| 120 | |||
| 121 | The following command displays variable values after a specific recipe has | ||
| 122 | been parsed. The variables include those from the configuration as well:: | ||
| 123 | |||
| 124 | $ bitbake -e recipename | ||
| 125 | |||
| 126 | .. note:: | ||
| 127 | |||
| 128 | Each recipe has its own private set of variables (datastore). | ||
| 129 | Internally, after parsing the configuration, a copy of the resulting | ||
| 130 | datastore is made prior to parsing each recipe. This copying implies | ||
| 131 | that variables set in one recipe will not be visible to other | ||
| 132 | recipes. | ||
| 133 | |||
| 134 | Likewise, each task within a recipe gets a private datastore based on | ||
| 135 | the recipe datastore, which means that variables set within one task | ||
| 136 | will not be visible to other tasks. | ||
| 137 | |||
| 138 | In the output of ``bitbake -e``, each variable is preceded by a | ||
| 139 | description of how the variable got its value, including temporary | ||
| 140 | values that were later overridden. This description also includes | ||
| 141 | variable flags (varflags) set on the variable. The output can be very | ||
| 142 | helpful during debugging. | ||
| 143 | |||
| 144 | Variables that are exported to the environment are preceded by | ||
| 145 | ``export`` in the output of ``bitbake -e``. See the following example:: | ||
| 146 | |||
| 147 | export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" | ||
| 148 | |||
| 149 | In addition to variable values, the output of the ``bitbake -e`` and | ||
| 150 | ``bitbake -e`` recipe commands includes the following information: | ||
| 151 | |||
| 152 | - The output starts with a tree listing all configuration files and | ||
| 153 | classes included globally, recursively listing the files they include | ||
| 154 | or inherit in turn. Much of the behavior of the OpenEmbedded build | ||
| 155 | system (including the behavior of the :ref:`ref-manual/tasks:normal recipe build tasks`) is | ||
| 156 | implemented in the :ref:`ref-classes-base` class and the | ||
| 157 | classes it inherits, rather than being built into BitBake itself. | ||
| 158 | |||
| 159 | - After the variable values, all functions appear in the output. For | ||
| 160 | shell functions, variables referenced within the function body are | ||
| 161 | expanded. If a function has been modified using overrides or using | ||
| 162 | override-style operators like ``:append`` and ``:prepend``, then the | ||
| 163 | final assembled function body appears in the output. | ||
| 164 | |||
| 165 | Viewing Package Information with ``oe-pkgdata-util`` | ||
| 166 | ==================================================== | ||
| 167 | |||
| 168 | You can use the ``oe-pkgdata-util`` command-line utility to query | ||
| 169 | :term:`PKGDATA_DIR` and display | ||
| 170 | various package-related information. When you use the utility, you must | ||
| 171 | use it to view information on packages that have already been built. | ||
| 172 | |||
| 173 | Here are a few of the available ``oe-pkgdata-util`` subcommands. | ||
| 174 | |||
| 175 | .. note:: | ||
| 176 | |||
| 177 | You can use the standard \* and ? globbing wildcards as part of | ||
| 178 | package names and paths. | ||
| 179 | |||
| 180 | - ``oe-pkgdata-util list-pkgs [pattern]``: Lists all packages | ||
| 181 | that have been built, optionally limiting the match to packages that | ||
| 182 | match pattern. | ||
| 183 | |||
| 184 | - ``oe-pkgdata-util list-pkg-files package ...``: Lists the | ||
| 185 | files and directories contained in the given packages. | ||
| 186 | |||
| 187 | .. note:: | ||
| 188 | |||
| 189 | A different way to view the contents of a package is to look at | ||
| 190 | the | ||
| 191 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
| 192 | directory of the recipe that generates the package. This directory | ||
| 193 | is created by the | ||
| 194 | :ref:`ref-tasks-package` task | ||
| 195 | and has one subdirectory for each package the recipe generates, | ||
| 196 | which contains the files stored in that package. | ||
| 197 | |||
| 198 | If you want to inspect the ``${WORKDIR}/packages-split`` | ||
| 199 | directory, make sure that :ref:`ref-classes-rm-work` is not | ||
| 200 | enabled when you build the recipe. | ||
| 201 | |||
| 202 | - ``oe-pkgdata-util find-path path ...``: Lists the names of | ||
| 203 | the packages that contain the given paths. For example, the following | ||
| 204 | tells us that ``/usr/share/man/man1/make.1`` is contained in the | ||
| 205 | ``make-doc`` package:: | ||
| 206 | |||
| 207 | $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 | ||
| 208 | make-doc: /usr/share/man/man1/make.1 | ||
| 209 | |||
| 210 | - ``oe-pkgdata-util lookup-recipe package ...``: Lists the name | ||
| 211 | of the recipes that produce the given packages. | ||
| 212 | |||
| 213 | For more information on the ``oe-pkgdata-util`` command, use the help | ||
| 214 | facility:: | ||
| 215 | |||
| 216 | $ oe-pkgdata-util --help | ||
| 217 | $ oe-pkgdata-util subcommand --help | ||
| 218 | |||
| 219 | Viewing Dependencies Between Recipes and Tasks | ||
| 220 | ============================================== | ||
| 221 | |||
| 222 | Sometimes it can be hard to see why BitBake wants to build other recipes | ||
| 223 | before the one you have specified. Dependency information can help you | ||
| 224 | understand why a recipe is built. | ||
| 225 | |||
| 226 | To generate dependency information for a recipe, run the following | ||
| 227 | command:: | ||
| 228 | |||
| 229 | $ bitbake -g recipename | ||
| 230 | |||
| 231 | This command writes the following files in the current directory: | ||
| 232 | |||
| 233 | - ``pn-buildlist``: A list of recipes/targets involved in building | ||
| 234 | `recipename`. "Involved" here means that at least one task from the | ||
| 235 | recipe needs to run when building `recipename` from scratch. Targets | ||
| 236 | that are in | ||
| 237 | :term:`ASSUME_PROVIDED` | ||
| 238 | are not listed. | ||
| 239 | |||
| 240 | - ``task-depends.dot``: A graph showing dependencies between tasks. | ||
| 241 | |||
| 242 | The graphs are in :wikipedia:`DOT <DOT_%28graph_description_language%29>` | ||
| 243 | format and can be converted to images (e.g. using the ``dot`` tool from | ||
| 244 | `Graphviz <https://www.graphviz.org/>`__). | ||
| 245 | |||
| 246 | .. note:: | ||
| 247 | |||
| 248 | - DOT files use a plain text format. The graphs generated using the | ||
| 249 | ``bitbake -g`` command are often so large as to be difficult to | ||
| 250 | read without special pruning (e.g. with BitBake's ``-I`` option) | ||
| 251 | and processing. Despite the form and size of the graphs, the | ||
| 252 | corresponding ``.dot`` files can still be possible to read and | ||
| 253 | provide useful information. | ||
| 254 | |||
| 255 | As an example, the ``task-depends.dot`` file contains lines such | ||
| 256 | as the following:: | ||
| 257 | |||
| 258 | "libxslt.do_configure" -> "libxml2.do_populate_sysroot" | ||
| 259 | |||
| 260 | The above example line reveals that the | ||
| 261 | :ref:`ref-tasks-configure` | ||
| 262 | task in ``libxslt`` depends on the | ||
| 263 | :ref:`ref-tasks-populate_sysroot` | ||
| 264 | task in ``libxml2``, which is a normal | ||
| 265 | :term:`DEPENDS` dependency | ||
| 266 | between the two recipes. | ||
| 267 | |||
| 268 | - For an example of how ``.dot`` files can be processed, see the | ||
| 269 | ``scripts/contrib/graph-tool`` Python script, which finds and | ||
| 270 | displays paths between graph nodes. | ||
| 271 | |||
| 272 | You can use a different method to view dependency information by using | ||
| 273 | either:: | ||
| 274 | |||
| 275 | $ bitbake -g -u taskexp recipename | ||
| 276 | |||
| 277 | or:: | ||
| 278 | |||
| 279 | $ bitbake -g -u taskexp_ncurses recipename | ||
| 280 | |||
| 281 | The ``-u taskdep`` option GUI window from which you can view build-time and | ||
| 282 | runtime dependencies for the recipes involved in building recipename. The | ||
| 283 | ``-u taskexp_ncurses`` option uses ncurses instead of GTK to render the UI. | ||
| 284 | |||
| 285 | Viewing Task Variable Dependencies | ||
| 286 | ================================== | ||
| 287 | |||
| 288 | As mentioned in the | ||
| 289 | ":ref:`bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" | ||
| 290 | section of the BitBake User Manual, BitBake tries to automatically determine | ||
| 291 | what variables a task depends on so that it can rerun the task if any values of | ||
| 292 | the variables change. This determination is usually reliable. However, if you | ||
| 293 | do things like construct variable names at runtime, then you might have to | ||
| 294 | manually declare dependencies on those variables using ``vardeps`` as described | ||
| 295 | in the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" | ||
| 296 | section of the BitBake User Manual. | ||
| 297 | |||
| 298 | If you are unsure whether a variable dependency is being picked up | ||
| 299 | automatically for a given task, you can list the variable dependencies | ||
| 300 | BitBake has determined by doing the following: | ||
| 301 | |||
| 302 | #. Build the recipe containing the task:: | ||
| 303 | |||
| 304 | $ bitbake recipename | ||
| 305 | |||
| 306 | #. Inside the :term:`STAMPS_DIR` | ||
| 307 | directory, find the signature data (``sigdata``) file that | ||
| 308 | corresponds to the task. The ``sigdata`` files contain a pickled | ||
| 309 | Python database of all the metadata that went into creating the input | ||
| 310 | checksum for the task. As an example, for the | ||
| 311 | :ref:`ref-tasks-fetch` task of the | ||
| 312 | ``db`` recipe, the ``sigdata`` file might be found in the following | ||
| 313 | location:: | ||
| 314 | |||
| 315 | ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
| 316 | |||
| 317 | For tasks that are accelerated through the shared state | ||
| 318 | (:ref:`sstate <overview-manual/concepts:shared state cache>`) cache, an | ||
| 319 | additional ``siginfo`` file is written into | ||
| 320 | :term:`SSTATE_DIR` along with | ||
| 321 | the cached task output. The ``siginfo`` files contain exactly the | ||
| 322 | same information as ``sigdata`` files. | ||
| 323 | |||
| 324 | #. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here | ||
| 325 | is an example:: | ||
| 326 | |||
| 327 | $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
| 328 | |||
| 329 | In the output of the above command, you will find a line like the | ||
| 330 | following, which lists all the (inferred) variable dependencies for | ||
| 331 | the task. This list also includes indirect dependencies from | ||
| 332 | variables depending on other variables, recursively:: | ||
| 333 | |||
| 334 | Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[sha256sum]', 'base_do_fetch'] | ||
| 335 | |||
| 336 | .. note:: | ||
| 337 | |||
| 338 | Functions (e.g. ``base_do_fetch``) also count as variable dependencies. | ||
| 339 | These functions in turn depend on the variables they reference. | ||
| 340 | |||
| 341 | The output of ``bitbake-dumpsig`` also includes the value each | ||
| 342 | variable had, a list of dependencies for each variable, and | ||
| 343 | :term:`BB_BASEHASH_IGNORE_VARS` | ||
| 344 | information. | ||
| 345 | |||
| 346 | Debugging signature construction and unexpected task executions | ||
| 347 | =============================================================== | ||
| 348 | |||
| 349 | There is a ``bitbake-diffsigs`` command for comparing two | ||
| 350 | ``siginfo`` or ``sigdata`` files. This command can be helpful when | ||
| 351 | trying to figure out what changed between two versions of a task. If you | ||
| 352 | call ``bitbake-diffsigs`` with just one file, the command behaves like | ||
| 353 | ``bitbake-dumpsig``. | ||
| 354 | |||
| 355 | You can also use BitBake to dump out the signature construction | ||
| 356 | information without executing tasks by using either of the following | ||
| 357 | BitBake command-line options:: | ||
| 358 | |||
| 359 | ‐‐dump-signatures=SIGNATURE_HANDLER | ||
| 360 | -S SIGNATURE_HANDLER | ||
| 361 | |||
| 362 | |||
| 363 | .. note:: | ||
| 364 | |||
| 365 | Two common values for `SIGNATURE_HANDLER` are "none" and "printdiff", which | ||
| 366 | dump only the signature or compare the dumped signature with the most recent one, | ||
| 367 | respectively. "printdiff" will try to establish the most recent | ||
| 368 | signature match (e.g. in the sstate cache) and then | ||
| 369 | compare the matched signatures to determine the stamps and delta | ||
| 370 | where these two stamp trees diverge. This can be used to determine why | ||
| 371 | tasks need to be re-run in situations where that is not expected. | ||
| 372 | |||
| 373 | Using BitBake with either of these options causes BitBake to dump out | ||
| 374 | ``sigdata`` files in the ``stamps`` directory for every task it would | ||
| 375 | have executed instead of building the specified target package. | ||
| 376 | |||
| 377 | Viewing Metadata Used to Create the Input Signature of a Shared State Task | ||
| 378 | ========================================================================== | ||
| 379 | |||
| 380 | Seeing what metadata went into creating the input signature of a shared | ||
| 381 | state (sstate) task can be a useful debugging aid. This information is | ||
| 382 | available in signature information (``siginfo``) files in | ||
| 383 | :term:`SSTATE_DIR`. For | ||
| 384 | information on how to view and interpret information in ``siginfo`` | ||
| 385 | files, see the | ||
| 386 | ":ref:`dev-manual/debugging:viewing task variable dependencies`" section. | ||
| 387 | |||
| 388 | For conceptual information on shared state, see the | ||
| 389 | ":ref:`overview-manual/concepts:shared state`" | ||
| 390 | section in the Yocto Project Overview and Concepts Manual. | ||
| 391 | |||
| 392 | Invalidating Shared State to Force a Task to Run | ||
| 393 | ================================================ | ||
| 394 | |||
| 395 | The OpenEmbedded build system uses | ||
| 396 | :ref:`checksums <overview-manual/concepts:checksums (signatures)>` and | ||
| 397 | :ref:`overview-manual/concepts:shared state` cache to avoid unnecessarily | ||
| 398 | rebuilding tasks. Collectively, this scheme is known as "shared state | ||
| 399 | code". | ||
| 400 | |||
| 401 | As with all schemes, this one has some drawbacks. It is possible that | ||
| 402 | you could make implicit changes to your code that the checksum | ||
| 403 | calculations do not take into account. These implicit changes affect a | ||
| 404 | task's output but do not trigger the shared state code into rebuilding a | ||
| 405 | recipe. Consider an example during which a tool changes its output. | ||
| 406 | Assume that the output of ``rpmdeps`` changes. The result of the change | ||
| 407 | should be that all the ``package`` and ``package_write_rpm`` shared | ||
| 408 | state cache items become invalid. However, because the change to the | ||
| 409 | output is external to the code and therefore implicit, the associated | ||
| 410 | shared state cache items do not become invalidated. In this case, the | ||
| 411 | build process uses the cached items rather than running the task again. | ||
| 412 | Obviously, these types of implicit changes can cause problems. | ||
| 413 | |||
| 414 | To avoid these problems during the build, you need to understand the | ||
| 415 | effects of any changes you make. Realize that changes you make directly | ||
| 416 | to a function are automatically factored into the checksum calculation. | ||
| 417 | Thus, these explicit changes invalidate the associated area of shared | ||
| 418 | state cache. However, you need to be aware of any implicit changes that | ||
| 419 | are not obvious changes to the code and could affect the output of a | ||
| 420 | given task. | ||
| 421 | |||
| 422 | When you identify an implicit change, you can easily take steps to | ||
| 423 | invalidate the cache and force the tasks to run. The steps you can take | ||
| 424 | are as simple as changing a function's comments in the source code. For | ||
| 425 | example, to invalidate package shared state files, change the comment | ||
| 426 | statements of | ||
| 427 | :ref:`ref-tasks-package` or the | ||
| 428 | comments of one of the functions it calls. Even though the change is | ||
| 429 | purely cosmetic, it causes the checksum to be recalculated and forces | ||
| 430 | the build system to run the task again. | ||
| 431 | |||
| 432 | .. note:: | ||
| 433 | |||
| 434 | For an example of a commit that makes a cosmetic change to invalidate | ||
| 435 | shared state, see this | ||
| 436 | :yocto_git:`commit </poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54>`. | ||
| 437 | |||
| 438 | Running Specific Tasks | ||
| 439 | ====================== | ||
| 440 | |||
| 441 | Any given recipe consists of a set of tasks. The standard BitBake | ||
| 442 | behavior in most cases is: :ref:`ref-tasks-fetch`, :ref:`ref-tasks-unpack`, :ref:`ref-tasks-patch`, | ||
| 443 | :ref:`ref-tasks-configure`, :ref:`ref-tasks-compile`, :ref:`ref-tasks-install`, :ref:`ref-tasks-package`, | ||
| 444 | :ref:`do_package_write_* <ref-tasks-package_write_deb>`, and :ref:`ref-tasks-build`. The default task is | ||
| 445 | :ref:`ref-tasks-build` and any tasks on which it depends build first. Some tasks, | ||
| 446 | such as :ref:`ref-tasks-devshell`, are not part of the default build chain. If you | ||
| 447 | wish to run a task that is not part of the default build chain, you can | ||
| 448 | use the ``-c`` option in BitBake. Here is an example:: | ||
| 449 | |||
| 450 | $ bitbake matchbox-desktop -c devshell | ||
| 451 | |||
| 452 | The ``-c`` option respects task dependencies, which means that all other | ||
| 453 | tasks (including tasks from other recipes) that the specified task | ||
| 454 | depends on will be run before the task. Even when you manually specify a | ||
| 455 | task to run with ``-c``, BitBake will only run the task if it considers | ||
| 456 | it "out of date". See the | ||
| 457 | ":ref:`overview-manual/concepts:stamp files and the rerunning of tasks`" | ||
| 458 | section in the Yocto Project Overview and Concepts Manual for how | ||
| 459 | BitBake determines whether a task is "out of date". | ||
| 460 | |||
| 461 | If you want to force an up-to-date task to be rerun (e.g. because you | ||
| 462 | made manual modifications to the recipe's | ||
| 463 | :term:`WORKDIR` that you want to try | ||
| 464 | out), then you can use the ``-f`` option. | ||
| 465 | |||
| 466 | .. note:: | ||
| 467 | |||
| 468 | The reason ``-f`` is never required when running the | ||
| 469 | :ref:`ref-tasks-devshell` task is because the | ||
| 470 | [\ :ref:`nostamp <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ] | ||
| 471 | variable flag is already set for the task. | ||
| 472 | |||
| 473 | The following example shows one way you can use the ``-f`` option:: | ||
| 474 | |||
| 475 | $ bitbake matchbox-desktop | ||
| 476 | . | ||
| 477 | . | ||
| 478 | make some changes to the source code in the work directory | ||
| 479 | . | ||
| 480 | . | ||
| 481 | $ bitbake matchbox-desktop -c compile -f | ||
| 482 | $ bitbake matchbox-desktop | ||
| 483 | |||
| 484 | This sequence first builds and then recompiles ``matchbox-desktop``. The | ||
| 485 | last command reruns all tasks (basically the packaging tasks) after the | ||
| 486 | compile. BitBake recognizes that the :ref:`ref-tasks-compile` task was rerun and | ||
| 487 | therefore understands that the other tasks also need to be run again. | ||
| 488 | |||
| 489 | Another, shorter way to rerun a task and all | ||
| 490 | :ref:`ref-manual/tasks:normal recipe build tasks` | ||
| 491 | that depend on it is to use the ``-C`` option. | ||
| 492 | |||
| 493 | .. note:: | ||
| 494 | |||
| 495 | This option is upper-cased and is separate from the ``-c`` | ||
| 496 | option, which is lower-cased. | ||
| 497 | |||
| 498 | Using this option invalidates the given task and then runs the | ||
| 499 | :ref:`ref-tasks-build` task, which is | ||
| 500 | the default task if no task is given, and the tasks on which it depends. | ||
| 501 | You could replace the final two commands in the previous example with | ||
| 502 | the following single command:: | ||
| 503 | |||
| 504 | $ bitbake matchbox-desktop -C compile | ||
| 505 | |||
| 506 | Internally, the ``-f`` and ``-C`` options work by tainting (modifying) | ||
| 507 | the input checksum of the specified task. This tainting indirectly | ||
| 508 | causes the task and its dependent tasks to be rerun through the normal | ||
| 509 | task dependency mechanisms. | ||
| 510 | |||
| 511 | .. note:: | ||
| 512 | |||
| 513 | BitBake explicitly keeps track of which tasks have been tainted in | ||
| 514 | this fashion, and will print warnings such as the following for | ||
| 515 | builds involving such tasks: | ||
| 516 | |||
| 517 | .. code-block:: none | ||
| 518 | |||
| 519 | WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run | ||
| 520 | |||
| 521 | |||
| 522 | The purpose of the warning is to let you know that the work directory | ||
| 523 | and build output might not be in the clean state they would be in for | ||
| 524 | a "normal" build, depending on what actions you took. To get rid of | ||
| 525 | such warnings, you can remove the work directory and rebuild the | ||
| 526 | recipe, as follows:: | ||
| 527 | |||
| 528 | $ bitbake matchbox-desktop -c clean | ||
| 529 | $ bitbake matchbox-desktop | ||
| 530 | |||
| 531 | |||
| 532 | You can view a list of tasks in a given package by running the | ||
| 533 | :ref:`ref-tasks-listtasks` task as follows:: | ||
| 534 | |||
| 535 | $ bitbake matchbox-desktop -c listtasks | ||
| 536 | |||
| 537 | The results appear as output to the console and are also in | ||
| 538 | the file ``${WORKDIR}/temp/log.do_listtasks``. | ||
| 539 | |||
| 540 | General BitBake Problems | ||
| 541 | ======================== | ||
| 542 | |||
| 543 | You can see debug output from BitBake by using the ``-D`` option. The | ||
| 544 | debug output gives more information about what BitBake is doing and the | ||
| 545 | reason behind it. Each ``-D`` option you use increases the logging | ||
| 546 | level. The most common usage is ``-DDD``. | ||
| 547 | |||
| 548 | The output from ``bitbake -DDD -v targetname`` can reveal why BitBake | ||
| 549 | chose a certain version of a package or why BitBake picked a certain | ||
| 550 | provider. This command could also help you in a situation where you | ||
| 551 | think BitBake did something unexpected. | ||
| 552 | |||
| 553 | Building with No Dependencies | ||
| 554 | ============================= | ||
| 555 | |||
| 556 | To build a specific recipe (``.bb`` file), you can use the following | ||
| 557 | command form:: | ||
| 558 | |||
| 559 | $ bitbake -b somepath/somerecipe.bb | ||
| 560 | |||
| 561 | This command form does | ||
| 562 | not check for dependencies. Consequently, you should use it only when | ||
| 563 | you know existing dependencies have been met. | ||
| 564 | |||
| 565 | .. note:: | ||
| 566 | |||
| 567 | You can also specify fragments of the filename. In this case, BitBake | ||
| 568 | checks for a unique match. | ||
| 569 | |||
| 570 | Recipe Logging Mechanisms | ||
| 571 | ========================= | ||
| 572 | |||
| 573 | The Yocto Project provides several logging functions for producing | ||
| 574 | debugging output and reporting errors and warnings. For Python | ||
| 575 | functions, the following logging functions are available. All of these functions | ||
| 576 | log to ``${T}/log.do_``\ `task`, and can also log to standard output | ||
| 577 | (stdout) with the right settings: | ||
| 578 | |||
| 579 | - ``bb.plain(msg)``: Writes msg as is to the log while also | ||
| 580 | logging to stdout. | ||
| 581 | |||
| 582 | - ``bb.note(msg)``: Writes "NOTE: msg" to the log. Also logs to | ||
| 583 | stdout if BitBake is called with "-v". | ||
| 584 | |||
| 585 | - ``bb.debug(level, msg)``: Writes "DEBUG: msg" to the log. Also logs to | ||
| 586 | stdout if the log level is greater than or equal to level. See the | ||
| 587 | ":ref:`bitbake-user-manual/bitbake-user-manual-intro:usage and syntax`" | ||
| 588 | option in the BitBake User Manual for more information. | ||
| 589 | |||
| 590 | - ``bb.warn(msg)``: Writes "WARNING: msg" to the log while also | ||
| 591 | logging to stdout. | ||
| 592 | |||
| 593 | - ``bb.error(msg)``: Writes "ERROR: msg" to the log while also | ||
| 594 | logging to standard out (stdout). | ||
| 595 | |||
| 596 | .. note:: | ||
| 597 | |||
| 598 | Calling this function does not cause the task to fail. | ||
| 599 | |||
| 600 | - ``bb.fatal(msg)``: This logging function is similar to | ||
| 601 | ``bb.error(msg)`` but also causes the calling task to fail. | ||
| 602 | |||
| 603 | .. note:: | ||
| 604 | |||
| 605 | ``bb.fatal()`` raises an exception, which means you do not need to put a | ||
| 606 | "return" statement after the function. | ||
| 607 | |||
| 608 | The same logging functions are also available in shell functions, under | ||
| 609 | the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``, | ||
| 610 | and ``bbfatal``. The :ref:`ref-classes-logging` class | ||
| 611 | implements these functions. See that class in the ``meta/classes`` | ||
| 612 | folder of the :term:`Source Directory` for information. | ||
| 613 | |||
| 614 | Logging With Python | ||
| 615 | ------------------- | ||
| 616 | |||
| 617 | When creating recipes using Python and inserting code that handles build | ||
| 618 | logs, keep in mind the goal is to have informative logs while keeping | ||
| 619 | the console as "silent" as possible. Also, if you want status messages | ||
| 620 | in the log, use the "debug" loglevel. | ||
| 621 | |||
| 622 | Here is an example written in Python. The code handles logging for | ||
| 623 | a function that determines the number of tasks needed to be run. See the | ||
| 624 | ":ref:`ref-tasks-listtasks`" | ||
| 625 | section for additional information:: | ||
| 626 | |||
| 627 | python do_listtasks() { | ||
| 628 | bb.debug(2, "Starting to figure out the task list") | ||
| 629 | if noteworthy_condition: | ||
| 630 | bb.note("There are 47 tasks to run") | ||
| 631 | bb.debug(2, "Got to point xyz") | ||
| 632 | if warning_trigger: | ||
| 633 | bb.warn("Detected warning_trigger, this might be a problem later.") | ||
| 634 | if recoverable_error: | ||
| 635 | bb.error("Hit recoverable_error, you really need to fix this!") | ||
| 636 | if fatal_error: | ||
| 637 | bb.fatal("fatal_error detected, unable to print the task list") | ||
| 638 | bb.plain("The tasks present are abc") | ||
| 639 | bb.debug(2, "Finished figuring out the tasklist") | ||
| 640 | } | ||
| 641 | |||
| 642 | Logging With Bash | ||
| 643 | ----------------- | ||
| 644 | |||
| 645 | When creating recipes using Bash and inserting code that handles build | ||
| 646 | logs, you have the same goals --- informative with minimal console output. | ||
| 647 | The syntax you use for recipes written in Bash is similar to that of | ||
| 648 | recipes written in Python described in the previous section. | ||
| 649 | |||
| 650 | Here is an example written in Bash. The code logs the progress of | ||
| 651 | the ``do_my_function`` function:: | ||
| 652 | |||
| 653 | do_my_function() { | ||
| 654 | bbdebug 2 "Running do_my_function" | ||
| 655 | if [ exceptional_condition ]; then | ||
| 656 | bbnote "Hit exceptional_condition" | ||
| 657 | fi | ||
| 658 | bbdebug 2 "Got to point xyz" | ||
| 659 | if [ warning_trigger ]; then | ||
| 660 | bbwarn "Detected warning_trigger, this might cause a problem later." | ||
| 661 | fi | ||
| 662 | if [ recoverable_error ]; then | ||
| 663 | bberror "Hit recoverable_error, correcting" | ||
| 664 | fi | ||
| 665 | if [ fatal_error ]; then | ||
| 666 | bbfatal "fatal_error detected" | ||
| 667 | fi | ||
| 668 | bbdebug 2 "Completed do_my_function" | ||
| 669 | } | ||
| 670 | |||
| 671 | |||
| 672 | Debugging Parallel Make Races | ||
| 673 | ============================= | ||
| 674 | |||
| 675 | A parallel ``make`` race occurs when the build consists of several parts | ||
| 676 | that are run simultaneously and a situation occurs when the output or | ||
| 677 | result of one part is not ready for use with a different part of the | ||
| 678 | build that depends on that output. Parallel make races are annoying and | ||
| 679 | can sometimes be difficult to reproduce and fix. However, there are some simple | ||
| 680 | tips and tricks that can help you debug and fix them. This section | ||
| 681 | presents a real-world example of an error encountered on the Yocto | ||
| 682 | Project autobuilder and the process used to fix it. | ||
| 683 | |||
| 684 | .. note:: | ||
| 685 | |||
| 686 | If you cannot properly fix a ``make`` race condition, you can work around it | ||
| 687 | by clearing either the :term:`PARALLEL_MAKE` or :term:`PARALLEL_MAKEINST` | ||
| 688 | variables. | ||
| 689 | |||
| 690 | The Failure | ||
| 691 | ----------- | ||
| 692 | |||
| 693 | For this example, assume that you are building an image that depends on | ||
| 694 | the "neard" package. And, during the build, BitBake runs into problems | ||
| 695 | and creates the following output. | ||
| 696 | |||
| 697 | .. note:: | ||
| 698 | |||
| 699 | This example log file has longer lines artificially broken to make | ||
| 700 | the listing easier to read. | ||
| 701 | |||
| 702 | If you examine the output or the log file, you see the failure during | ||
| 703 | ``make``: | ||
| 704 | |||
| 705 | .. code-block:: none | ||
| 706 | |||
| 707 | | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] | ||
| 708 | | DEBUG: Executing shell function do_compile | ||
| 709 | | NOTE: make -j 16 | ||
| 710 | | make --no-print-directory all-am | ||
| 711 | | /bin/mkdir -p include/near | ||
| 712 | | /bin/mkdir -p include/near | ||
| 713 | | /bin/mkdir -p include/near | ||
| 714 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 715 | 0.14-r0/neard-0.14/include/types.h include/near/types.h | ||
| 716 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 717 | 0.14-r0/neard-0.14/include/log.h include/near/log.h | ||
| 718 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 719 | 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h | ||
| 720 | | /bin/mkdir -p include/near | ||
| 721 | | /bin/mkdir -p include/near | ||
| 722 | | /bin/mkdir -p include/near | ||
| 723 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 724 | 0.14-r0/neard-0.14/include/tag.h include/near/tag.h | ||
| 725 | | /bin/mkdir -p include/near | ||
| 726 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 727 | 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h | ||
| 728 | | /bin/mkdir -p include/near | ||
| 729 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 730 | 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h | ||
| 731 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 732 | 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h | ||
| 733 | | /bin/mkdir -p include/near | ||
| 734 | | /bin/mkdir -p include/near | ||
| 735 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 736 | 0.14-r0/neard-0.14/include/setting.h include/near/setting.h | ||
| 737 | | /bin/mkdir -p include/near | ||
| 738 | | /bin/mkdir -p include/near | ||
| 739 | | /bin/mkdir -p include/near | ||
| 740 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 741 | 0.14-r0/neard-0.14/include/device.h include/near/device.h | ||
| 742 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 743 | 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h | ||
| 744 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 745 | 0.14-r0/neard-0.14/include/snep.h include/near/snep.h | ||
| 746 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 747 | 0.14-r0/neard-0.14/include/version.h include/near/version.h | ||
| 748 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
| 749 | 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h | ||
| 750 | | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h | ||
| 751 | | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/nightly-x86/ | ||
| 752 | build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ | ||
| 753 | yocto-autobuilder/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 | ||
| 754 | -I/home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ | ||
| 755 | lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/nightly-x86/build/build/ | ||
| 756 | tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/ | ||
| 757 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ | ||
| 758 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 | ||
| 759 | -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" | ||
| 760 | -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c | ||
| 761 | -o tools/snep-send.o tools/snep-send.c | ||
| 762 | | In file included from tools/snep-send.c:16:0: | ||
| 763 | | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
| 764 | | #include <near/dbus.h> | ||
| 765 | | ^ | ||
| 766 | | compilation terminated. | ||
| 767 | | make[1]: *** [tools/snep-send.o] Error 1 | ||
| 768 | | make[1]: *** Waiting for unfinished jobs.... | ||
| 769 | | make: *** [all] Error 2 | ||
| 770 | | ERROR: oe_runmake failed | ||
| 771 | |||
| 772 | Reproducing the Error | ||
| 773 | --------------------- | ||
| 774 | |||
| 775 | Because race conditions are intermittent, they do not manifest | ||
| 776 | themselves every time you do the build. In fact, most times the build | ||
| 777 | will complete without problems even though the potential race condition | ||
| 778 | exists. Thus, once the error surfaces, you need a way to reproduce it. | ||
| 779 | |||
| 780 | In this example, compiling the "neard" package is causing the problem. | ||
| 781 | So the first thing to do is build "neard" locally. Before you start the | ||
| 782 | build, set the | ||
| 783 | :term:`PARALLEL_MAKE` variable | ||
| 784 | in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a | ||
| 785 | high value for :term:`PARALLEL_MAKE` increases the chances of the race | ||
| 786 | condition showing up:: | ||
| 787 | |||
| 788 | $ bitbake neard | ||
| 789 | |||
| 790 | Once the local build for "neard" completes, start a ``devshell`` build:: | ||
| 791 | |||
| 792 | $ bitbake neard -c devshell | ||
| 793 | |||
| 794 | For information on how to use a ``devshell``, see the | ||
| 795 | ":ref:`dev-manual/development-shell:using a development shell`" section. | ||
| 796 | |||
| 797 | In the ``devshell``, do the following:: | ||
| 798 | |||
| 799 | $ make clean | ||
| 800 | $ make tools/snep-send.o | ||
| 801 | |||
| 802 | The ``devshell`` commands cause the failure to clearly | ||
| 803 | be visible. In this case, there is a missing dependency for the ``neard`` | ||
| 804 | Makefile target. Here is some abbreviated, sample output with the | ||
| 805 | missing dependency clearly visible at the end:: | ||
| 806 | |||
| 807 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... | ||
| 808 | . | ||
| 809 | . | ||
| 810 | . | ||
| 811 | tools/snep-send.c | ||
| 812 | In file included from tools/snep-send.c:16:0: | ||
| 813 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
| 814 | #include <near/dbus.h> | ||
| 815 | ^ | ||
| 816 | compilation terminated. | ||
| 817 | make: *** [tools/snep-send.o] Error 1 | ||
| 818 | $ | ||
| 819 | |||
| 820 | |||
| 821 | Creating a Patch for the Fix | ||
| 822 | ---------------------------- | ||
| 823 | |||
| 824 | Because there is a missing dependency for the Makefile target, you need | ||
| 825 | to patch the ``Makefile.am`` file, which is generated from | ||
| 826 | ``Makefile.in``. You can use Quilt to create the patch:: | ||
| 827 | |||
| 828 | $ quilt new parallelmake.patch | ||
| 829 | Patch patches/parallelmake.patch is now on top | ||
| 830 | $ quilt add Makefile.am | ||
| 831 | File Makefile.am added to patch patches/parallelmake.patch | ||
| 832 | |||
| 833 | For more information on using Quilt, see the | ||
| 834 | ":ref:`dev-manual/quilt:using quilt in your workflow`" section. | ||
| 835 | |||
| 836 | At this point you need to make the edits to ``Makefile.am`` to add the | ||
| 837 | missing dependency. For our example, you have to add the following line | ||
| 838 | to the file:: | ||
| 839 | |||
| 840 | tools/snep-send.$(OBJEXT): include/near/dbus.h | ||
| 841 | |||
| 842 | Once you have edited the file, use the ``refresh`` command to create the | ||
| 843 | patch:: | ||
| 844 | |||
| 845 | $ quilt refresh | ||
| 846 | Refreshed patch patches/parallelmake.patch | ||
| 847 | |||
| 848 | Once the patch file is created, you need to add it back to the originating | ||
| 849 | recipe folder. Here is an example assuming a top-level | ||
| 850 | :term:`Source Directory` named ``poky``:: | ||
| 851 | |||
| 852 | $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard | ||
| 853 | |||
| 854 | The final thing you need to do to implement the fix in the build is to | ||
| 855 | update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the | ||
| 856 | :term:`SRC_URI` statement includes | ||
| 857 | the patch file. The recipe file is in the folder above the patch. Here | ||
| 858 | is what the edited :term:`SRC_URI` statement would look like:: | ||
| 859 | |||
| 860 | SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ | ||
| 861 | file://neard.in \ | ||
| 862 | file://neard.service.in \ | ||
| 863 | file://parallelmake.patch \ | ||
| 864 | " | ||
| 865 | |||
| 866 | With the patch complete and moved to the correct folder and the | ||
| 867 | :term:`SRC_URI` statement updated, you can exit the ``devshell``:: | ||
| 868 | |||
| 869 | $ exit | ||
| 870 | |||
| 871 | Testing the Build | ||
| 872 | ----------------- | ||
| 873 | |||
| 874 | With everything in place, you can get back to trying the build again | ||
| 875 | locally:: | ||
| 876 | |||
| 877 | $ bitbake neard | ||
| 878 | |||
| 879 | This build should succeed. | ||
| 880 | |||
| 881 | Now you can open up a ``devshell`` again and repeat the clean and make | ||
| 882 | operations as follows:: | ||
| 883 | |||
| 884 | $ bitbake neard -c devshell | ||
| 885 | $ make clean | ||
| 886 | $ make tools/snep-send.o | ||
| 887 | |||
| 888 | The build should work without issue. | ||
| 889 | |||
| 890 | As with all solved problems, if they originated upstream, you need to | ||
| 891 | submit the fix for the recipe in OE-Core and upstream so that the | ||
| 892 | problem is taken care of at its source. See the | ||
| 893 | ":doc:`/contributor-guide/submit-changes`" section for more information. | ||
| 894 | |||
| 895 | Debugging With the GNU Project Debugger (GDB) Remotely | ||
| 896 | ====================================================== | ||
| 897 | |||
| 898 | GDB allows you to examine running programs, which in turn helps you to | ||
| 899 | understand and fix problems. It also allows you to perform post-mortem | ||
| 900 | style analysis of program crashes. GDB is available as a package within | ||
| 901 | the Yocto Project and is installed in SDK images by default. See the | ||
| 902 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
| 903 | Project Reference Manual for a description of these images. You can find | ||
| 904 | information on GDB at https://sourceware.org/gdb/. | ||
| 905 | |||
| 906 | .. note:: | ||
| 907 | |||
| 908 | For best results, install debug (``-dbg``) packages for the applications you | ||
| 909 | are going to debug. Doing so makes extra debug symbols available that give | ||
| 910 | you more meaningful output. | ||
| 911 | |||
| 912 | Sometimes, due to memory or disk space constraints, it is not possible | ||
| 913 | to use GDB directly on the remote target to debug applications. These | ||
| 914 | constraints arise because GDB needs to load the debugging information | ||
| 915 | and the binaries of the process being debugged. Additionally, GDB needs | ||
| 916 | to perform many computations to locate information such as function | ||
| 917 | names, variable names and values, stack traces and so forth --- even | ||
| 918 | before starting the debugging process. These extra computations place | ||
| 919 | more load on the target system and can alter the characteristics of the | ||
| 920 | program being debugged. | ||
| 921 | |||
| 922 | To help get past the previously mentioned constraints, there are two | ||
| 923 | methods you can use: running a debuginfod server and using gdbserver. | ||
| 924 | |||
| 925 | Using the debuginfod server method | ||
| 926 | ---------------------------------- | ||
| 927 | |||
| 928 | ``debuginfod`` from ``elfutils`` is a way to distribute ``debuginfo`` files. | ||
| 929 | Running a ``debuginfod`` server makes debug symbols readily available, | ||
| 930 | which means you don't need to download debugging information | ||
| 931 | and the binaries of the process being debugged. You can just fetch | ||
| 932 | debug symbols from the server. | ||
| 933 | |||
| 934 | To run a ``debuginfod`` server, you need to do the following: | ||
| 935 | |||
| 936 | - Ensure that ``debuginfod`` is present in :term:`DISTRO_FEATURES` | ||
| 937 | (it already is in ``OpenEmbedded-core`` defaults and ``poky`` reference distribution). | ||
| 938 | If not, set in your distro config file or in ``local.conf``:: | ||
| 939 | |||
| 940 | DISTRO_FEATURES:append = " debuginfod" | ||
| 941 | |||
| 942 | This distro feature enables the server and client library in ``elfutils``, | ||
| 943 | and enables ``debuginfod`` support in clients (at the moment, ``gdb`` and ``binutils``). | ||
| 944 | |||
| 945 | - Run the following commands to launch the ``debuginfod`` server on the host:: | ||
| 946 | |||
| 947 | $ oe-debuginfod | ||
| 948 | |||
| 949 | - To use ``debuginfod`` on the target, you need to know the ip:port where | ||
| 950 | ``debuginfod`` is listening on the host (port defaults to 8002), and export | ||
| 951 | that into the shell environment, for example in ``qemu``:: | ||
| 952 | |||
| 953 | root@qemux86-64:~# export DEBUGINFOD_URLS="http://192.168.7.1:8002/" | ||
| 954 | |||
| 955 | - Then debug info fetching should simply work when running the target ``gdb``, | ||
| 956 | ``readelf`` or ``objdump``, for example:: | ||
| 957 | |||
| 958 | root@qemux86-64:~# gdb /bin/cat | ||
| 959 | ... | ||
| 960 | Reading symbols from /bin/cat... | ||
| 961 | Downloading separate debug info for /bin/cat... | ||
| 962 | Reading symbols from /home/root/.cache/debuginfod_client/923dc4780cfbc545850c616bffa884b6b5eaf322/debuginfo... | ||
| 963 | |||
| 964 | - It's also possible to use ``debuginfod-find`` to just query the server:: | ||
| 965 | |||
| 966 | root@qemux86-64:~# debuginfod-find debuginfo /bin/ls | ||
| 967 | /home/root/.cache/debuginfod_client/356edc585f7f82d46f94fcb87a86a3fe2d2e60bd/debuginfo | ||
| 968 | |||
| 969 | |||
| 970 | Using the gdbserver method | ||
| 971 | -------------------------- | ||
| 972 | |||
| 973 | gdbserver, which runs on the remote target and does not load any | ||
| 974 | debugging information from the debugged process. Instead, a GDB instance | ||
| 975 | processes the debugging information that is run on a remote computer - | ||
| 976 | the host GDB. The host GDB then sends control commands to gdbserver to | ||
| 977 | make it stop or start the debugged program, as well as read or write | ||
| 978 | memory regions of that debugged program. All the debugging information | ||
| 979 | loaded and processed as well as all the heavy debugging is done by the | ||
| 980 | host GDB. Offloading these processes gives the gdbserver running on the | ||
| 981 | target a chance to remain small and fast. | ||
| 982 | |||
| 983 | Because the host GDB is responsible for loading the debugging | ||
| 984 | information and for doing the necessary processing to make actual | ||
| 985 | debugging happen, you have to make sure the host can access the | ||
| 986 | unstripped binaries complete with their debugging information and also | ||
| 987 | be sure the target is compiled with no optimizations. The host GDB must | ||
| 988 | also have local access to all the libraries used by the debugged | ||
| 989 | program. Because gdbserver does not need any local debugging | ||
| 990 | information, the binaries on the remote target can remain stripped. | ||
| 991 | However, the binaries must also be compiled without optimization so they | ||
| 992 | match the host's binaries. | ||
| 993 | |||
| 994 | To remain consistent with GDB documentation and terminology, the binary | ||
| 995 | being debugged on the remote target machine is referred to as the | ||
| 996 | "inferior" binary. For documentation on GDB see the `GDB | ||
| 997 | site <https://sourceware.org/gdb/documentation/>`__. | ||
| 998 | |||
| 999 | The following steps show you how to debug using the GNU project | ||
| 1000 | debugger. | ||
| 1001 | |||
| 1002 | #. *Configure your build system to construct the companion debug | ||
| 1003 | filesystem:* | ||
| 1004 | |||
| 1005 | In your ``local.conf`` file, set the following:: | ||
| 1006 | |||
| 1007 | IMAGE_GEN_DEBUGFS = "1" | ||
| 1008 | IMAGE_FSTYPES_DEBUGFS = "tar.bz2" | ||
| 1009 | |||
| 1010 | These options cause the | ||
| 1011 | OpenEmbedded build system to generate a special companion filesystem | ||
| 1012 | fragment, which contains the matching source and debug symbols to | ||
| 1013 | your deployable filesystem. The build system does this by looking at | ||
| 1014 | what is in the deployed filesystem, and pulling the corresponding | ||
| 1015 | ``-dbg`` packages. | ||
| 1016 | |||
| 1017 | The companion debug filesystem is not a complete filesystem, but only | ||
| 1018 | contains the debug fragments. This filesystem must be combined with | ||
| 1019 | the full filesystem for debugging. Subsequent steps in this procedure | ||
| 1020 | show how to combine the partial filesystem with the full filesystem. | ||
| 1021 | |||
| 1022 | #. *Configure the system to include gdbserver in the target filesystem:* | ||
| 1023 | |||
| 1024 | Make the following addition in your ``local.conf`` file:: | ||
| 1025 | |||
| 1026 | EXTRA_IMAGE_FEATURES:append = " tools-debug" | ||
| 1027 | |||
| 1028 | The change makes | ||
| 1029 | sure the ``gdbserver`` package is included. | ||
| 1030 | |||
| 1031 | #. *Build the environment:* | ||
| 1032 | |||
| 1033 | Use the following command to construct the image and the companion | ||
| 1034 | Debug Filesystem:: | ||
| 1035 | |||
| 1036 | $ bitbake image | ||
| 1037 | |||
| 1038 | Build the cross GDB component and | ||
| 1039 | make it available for debugging. Build the SDK that matches the | ||
| 1040 | image. Building the SDK is best for a production build that can be | ||
| 1041 | used later for debugging, especially during long term maintenance:: | ||
| 1042 | |||
| 1043 | $ bitbake -c populate_sdk image | ||
| 1044 | |||
| 1045 | Alternatively, you can build the minimal toolchain components that | ||
| 1046 | match the target. Doing so creates a smaller than typical SDK and | ||
| 1047 | only contains a minimal set of components with which to build simple | ||
| 1048 | test applications, as well as run the debugger:: | ||
| 1049 | |||
| 1050 | $ bitbake meta-toolchain | ||
| 1051 | |||
| 1052 | A final method is to build Gdb itself within the build system:: | ||
| 1053 | |||
| 1054 | $ bitbake gdb-cross-<architecture> | ||
| 1055 | |||
| 1056 | Doing so produces a temporary copy of | ||
| 1057 | ``cross-gdb`` you can use for debugging during development. While | ||
| 1058 | this is the quickest approach, the two previous methods in this step | ||
| 1059 | are better when considering long-term maintenance strategies. | ||
| 1060 | |||
| 1061 | .. note:: | ||
| 1062 | |||
| 1063 | If you run ``bitbake gdb-cross``, the OpenEmbedded build system suggests | ||
| 1064 | the actual image (e.g. ``gdb-cross-i586``). The suggestion is usually the | ||
| 1065 | actual name you want to use. | ||
| 1066 | |||
| 1067 | #. *Set up the* ``debugfs``\ *:* | ||
| 1068 | |||
| 1069 | Run the following commands to set up the ``debugfs``:: | ||
| 1070 | |||
| 1071 | $ mkdir debugfs | ||
| 1072 | $ cd debugfs | ||
| 1073 | $ tar xvfj build-dir/tmp/deploy/images/machine/image.rootfs.tar.bz2 | ||
| 1074 | $ tar xvfj build-dir/tmp/deploy/images/machine/image-dbg.rootfs.tar.bz2 | ||
| 1075 | |||
| 1076 | #. *Set up GDB:* | ||
| 1077 | |||
| 1078 | Install the SDK (if you built one) and then source the correct | ||
| 1079 | environment file. Sourcing the environment file puts the SDK in your | ||
| 1080 | ``PATH`` environment variable and sets ``$GDB`` to the SDK's debugger. | ||
| 1081 | |||
| 1082 | If you are using the build system, Gdb is located in | ||
| 1083 | `build-dir`\ ``/tmp/sysroots/``\ `host`\ ``/usr/bin/``\ `architecture`\ ``/``\ `architecture`\ ``-gdb`` | ||
| 1084 | |||
| 1085 | #. *Boot the target:* | ||
| 1086 | |||
| 1087 | For information on how to run QEMU, see the `QEMU | ||
| 1088 | Documentation <https://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__. | ||
| 1089 | |||
| 1090 | .. note:: | ||
| 1091 | |||
| 1092 | Be sure to verify that your host can access the target via TCP. | ||
| 1093 | |||
| 1094 | #. *Debug a program:* | ||
| 1095 | |||
| 1096 | Debugging a program involves running gdbserver on the target and then | ||
| 1097 | running Gdb on the host. The example in this step debugs ``gzip``: | ||
| 1098 | |||
| 1099 | .. code-block:: shell | ||
| 1100 | |||
| 1101 | root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help | ||
| 1102 | |||
| 1103 | For | ||
| 1104 | additional gdbserver options, see the `GDB Server | ||
| 1105 | Documentation <https://www.gnu.org/software/gdb/documentation/>`__. | ||
| 1106 | |||
| 1107 | After running gdbserver on the target, you need to run Gdb on the | ||
| 1108 | host and configure it and connect to the target. Use these commands:: | ||
| 1109 | |||
| 1110 | $ cd directory-holding-the-debugfs-directory | ||
| 1111 | $ arch-gdb | ||
| 1112 | (gdb) set sysroot debugfs | ||
| 1113 | (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug | ||
| 1114 | (gdb) target remote IP-of-target:1234 | ||
| 1115 | |||
| 1116 | At this | ||
| 1117 | point, everything should automatically load (i.e. matching binaries, | ||
| 1118 | symbols and headers). | ||
| 1119 | |||
| 1120 | .. note:: | ||
| 1121 | |||
| 1122 | The Gdb ``set`` commands in the previous example can be placed into the | ||
| 1123 | users ``~/.gdbinit`` file. Upon starting, Gdb automatically runs whatever | ||
| 1124 | commands are in that file. | ||
| 1125 | |||
| 1126 | #. *Deploying without a full image rebuild:* | ||
| 1127 | |||
| 1128 | In many cases, during development you want a quick method to deploy a | ||
| 1129 | new binary to the target and debug it, without waiting for a full | ||
| 1130 | image build. | ||
| 1131 | |||
| 1132 | One approach to solving this situation is to just build the component | ||
| 1133 | you want to debug. Once you have built the component, copy the | ||
| 1134 | executable directly to both the target and the host ``debugfs``. | ||
| 1135 | |||
| 1136 | If the binary is processed through the debug splitting in | ||
| 1137 | OpenEmbedded, you should also copy the debug items (i.e. ``.debug`` | ||
| 1138 | contents and corresponding ``/usr/src/debug`` files) from the work | ||
| 1139 | directory. Here is an example:: | ||
| 1140 | |||
| 1141 | $ bitbake bash | ||
| 1142 | $ bitbake -c devshell bash | ||
| 1143 | $ cd .. | ||
| 1144 | $ scp packages-split/bash/bin/bash target:/bin/bash | ||
| 1145 | $ cp -a packages-split/bash-dbg/\* path/debugfs | ||
| 1146 | |||
| 1147 | Debugging with the GNU Project Debugger (GDB) on the Target | ||
| 1148 | =========================================================== | ||
| 1149 | |||
| 1150 | The previous section addressed using GDB remotely for debugging | ||
| 1151 | purposes, which is the most usual case due to the inherent hardware | ||
| 1152 | limitations on many embedded devices. However, debugging in the target | ||
| 1153 | hardware itself is also possible with more powerful devices. This | ||
| 1154 | section describes what you need to do in order to support using GDB to | ||
| 1155 | debug on the target hardware. | ||
| 1156 | |||
| 1157 | To support this kind of debugging, you need do the following: | ||
| 1158 | |||
| 1159 | - Ensure that GDB is on the target. You can do this by making | ||
| 1160 | the following addition to your ``local.conf`` file:: | ||
| 1161 | |||
| 1162 | EXTRA_IMAGE_FEATURES:append = " tools-debug" | ||
| 1163 | |||
| 1164 | - Ensure that debug symbols are present. You can do so by adding the | ||
| 1165 | corresponding ``-dbg`` package to :term:`IMAGE_INSTALL`:: | ||
| 1166 | |||
| 1167 | IMAGE_INSTALL:append = " packagename-dbg" | ||
| 1168 | |||
| 1169 | Alternatively, you can add the following to ``local.conf`` to include | ||
| 1170 | all the debug symbols:: | ||
| 1171 | |||
| 1172 | EXTRA_IMAGE_FEATURES:append = " dbg-pkgs" | ||
| 1173 | |||
| 1174 | .. note:: | ||
| 1175 | |||
| 1176 | To improve the debug information accuracy, you can reduce the level | ||
| 1177 | of optimization used by the compiler. For example, when adding the | ||
| 1178 | following line to your ``local.conf`` file, you will reduce optimization | ||
| 1179 | from :term:`FULL_OPTIMIZATION` of "-O2" to :term:`DEBUG_OPTIMIZATION` | ||
| 1180 | of "-O -fno-omit-frame-pointer":: | ||
| 1181 | |||
| 1182 | DEBUG_BUILD = "1" | ||
| 1183 | |||
| 1184 | Consider that this will reduce the application's performance and is | ||
| 1185 | recommended only for debugging purposes. | ||
| 1186 | |||
| 1187 | Enabling Minidebuginfo | ||
| 1188 | ====================== | ||
| 1189 | |||
| 1190 | Enabling the :term:`DISTRO_FEATURES` minidebuginfo adds a compressed ELF section ``.gnu_debugdata`` | ||
| 1191 | to all binary files, containing only function names, and thus increasing the size of the | ||
| 1192 | binaries only by 5 to 10%. For comparison, full debug symbols can be 10 times as big as | ||
| 1193 | a stripped binary, and it is thus not always possible to deploy full debug symbols. | ||
| 1194 | Minidebuginfo data allows, on the one side, to retrieve a call-stack using | ||
| 1195 | GDB (command backtrace) without deploying full debug symbols to the target. It also | ||
| 1196 | allows to retrieve a symbolicated call-stack when using ``systemd-coredump`` to manage | ||
| 1197 | coredumps (commands ``coredumpctl list`` and ``coredumpctl info``). | ||
| 1198 | |||
| 1199 | This feature was created by Fedora, see https://fedoraproject.org/wiki/Features/MiniDebugInfo for | ||
| 1200 | more details. | ||
| 1201 | |||
| 1202 | Other Debugging Tips | ||
| 1203 | ==================== | ||
| 1204 | |||
| 1205 | Here are some other tips that you might find useful: | ||
| 1206 | |||
| 1207 | - When adding new packages, it is worth watching for undesirable items | ||
| 1208 | making their way into compiler command lines. For example, you do not | ||
| 1209 | want references to local system files like ``/usr/lib/`` or | ||
| 1210 | ``/usr/include/``. | ||
| 1211 | |||
| 1212 | - If you want to remove the ``psplash`` boot splashscreen, add | ||
| 1213 | ``psplash=false`` to the kernel command line. Doing so prevents | ||
| 1214 | ``psplash`` from loading and thus allows you to see the console. It | ||
| 1215 | is also possible to switch out of the splashscreen by switching the | ||
| 1216 | virtual console (e.g. Fn+Left or Fn+Right on a Zaurus). | ||
| 1217 | |||
| 1218 | - Removing :term:`TMPDIR` (usually ``tmp/``, within the | ||
| 1219 | :term:`Build Directory`) can often fix temporary build issues. Removing | ||
| 1220 | :term:`TMPDIR` is usually a relatively cheap operation, because task output | ||
| 1221 | will be cached in :term:`SSTATE_DIR` (usually ``sstate-cache/``, which is | ||
| 1222 | also in the :term:`Build Directory`). | ||
| 1223 | |||
| 1224 | .. note:: | ||
| 1225 | |||
| 1226 | Removing :term:`TMPDIR` might be a workaround rather than a fix. | ||
| 1227 | Consequently, trying to determine the underlying cause of an issue before | ||
| 1228 | removing the directory is a good idea. | ||
| 1229 | |||
| 1230 | - Understanding how a feature is used in practice within existing | ||
| 1231 | recipes can be very helpful. It is recommended that you configure | ||
| 1232 | some method that allows you to quickly search through files. | ||
| 1233 | |||
| 1234 | Using GNU Grep, you can use the following shell function to | ||
| 1235 | recursively search through common recipe-related files, skipping | ||
| 1236 | binary files, ``.git`` directories, and the :term:`Build Directory` | ||
| 1237 | (assuming its name starts with "build"):: | ||
| 1238 | |||
| 1239 | g() { | ||
| 1240 | grep -Ir \ | ||
| 1241 | --exclude-dir=.git \ | ||
| 1242 | --exclude-dir='build*' \ | ||
| 1243 | --include='*.bb*' \ | ||
| 1244 | --include='*.inc*' \ | ||
| 1245 | --include='*.conf*' \ | ||
| 1246 | --include='*.py*' \ | ||
| 1247 | "$@" | ||
| 1248 | } | ||
| 1249 | |||
| 1250 | Here are some usage examples:: | ||
| 1251 | |||
| 1252 | $ g FOO # Search recursively for "FOO" | ||
| 1253 | $ g -i foo # Search recursively for "foo", ignoring case | ||
| 1254 | $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" | ||
| 1255 | |||
| 1256 | If figuring | ||
| 1257 | out how some feature works requires a lot of searching, it might | ||
| 1258 | indicate that the documentation should be extended or improved. In | ||
| 1259 | such cases, consider filing a documentation bug using the Yocto | ||
| 1260 | Project implementation of | ||
| 1261 | :yocto_bugs:`Bugzilla <>`. For information on | ||
| 1262 | how to submit a bug against the Yocto Project, see the Yocto Project | ||
| 1263 | Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` | ||
| 1264 | and the ":doc:`/contributor-guide/report-defect`" section. | ||
| 1265 | |||
| 1266 | .. note:: | ||
| 1267 | |||
| 1268 | The manuals might not be the right place to document variables | ||
| 1269 | that are purely internal and have a limited scope (e.g. internal | ||
| 1270 | variables used to implement a single ``.bbclass`` file). | ||
| 1271 | |||
