diff options
Diffstat (limited to 'documentation/dev-manual/debugging.rst')
-rw-r--r-- | documentation/dev-manual/debugging.rst | 1253 |
1 files changed, 1253 insertions, 0 deletions
diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst new file mode 100644 index 0000000000..ef296de7ac --- /dev/null +++ b/documentation/dev-manual/debugging.rst | |||
@@ -0,0 +1,1253 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Debugging Tools and Techniques | ||
4 | ****************************** | ||
5 | |||
6 | The exact method for debugging build failures depends on the nature of | ||
7 | the problem and on the system's area from which the bug originates. | ||
8 | Standard debugging practices such as comparison against the last known | ||
9 | working version with examination of the changes and the re-application | ||
10 | of steps to identify the one causing the problem are valid for the Yocto | ||
11 | Project just as they are for any other system. Even though it is | ||
12 | impossible to detail every possible potential failure, this section | ||
13 | provides some general tips to aid in debugging given a variety of | ||
14 | situations. | ||
15 | |||
16 | .. note:: | ||
17 | |||
18 | A useful feature for debugging is the error reporting tool. | ||
19 | Configuring the Yocto Project to use this tool causes the | ||
20 | OpenEmbedded build system to produce error reporting commands as part | ||
21 | of the console output. You can enter the commands after the build | ||
22 | completes to log error information into a common database, that can | ||
23 | help you figure out what might be going wrong. For information on how | ||
24 | to enable and use this feature, see the | ||
25 | ":ref:`dev-manual/error-reporting-tool:using the error reporting tool`" | ||
26 | section. | ||
27 | |||
28 | The following list shows the debugging topics in the remainder of this | ||
29 | section: | ||
30 | |||
31 | - ":ref:`dev-manual/debugging:viewing logs from failed tasks`" describes | ||
32 | how to find and view logs from tasks that failed during the build | ||
33 | process. | ||
34 | |||
35 | - ":ref:`dev-manual/debugging:viewing variable values`" describes how to | ||
36 | use the BitBake ``-e`` option to examine variable values after a | ||
37 | recipe has been parsed. | ||
38 | |||
39 | - ":ref:`dev-manual/debugging:viewing package information with \`\`oe-pkgdata-util\`\``" | ||
40 | describes how to use the ``oe-pkgdata-util`` utility to query | ||
41 | :term:`PKGDATA_DIR` and | ||
42 | display package-related information for built packages. | ||
43 | |||
44 | - ":ref:`dev-manual/debugging:viewing dependencies between recipes and tasks`" | ||
45 | describes how to use the BitBake ``-g`` option to display recipe | ||
46 | dependency information used during the build. | ||
47 | |||
48 | - ":ref:`dev-manual/debugging:viewing task variable dependencies`" describes | ||
49 | how to use the ``bitbake-dumpsig`` command in conjunction with key | ||
50 | subdirectories in the :term:`Build Directory` to determine variable | ||
51 | dependencies. | ||
52 | |||
53 | - ":ref:`dev-manual/debugging:running specific tasks`" describes | ||
54 | how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``) | ||
55 | to run specific tasks in the build chain. It can be useful to run | ||
56 | tasks "out-of-order" when trying isolate build issues. | ||
57 | |||
58 | - ":ref:`dev-manual/debugging:general BitBake problems`" describes how | ||
59 | to use BitBake's ``-D`` debug output option to reveal more about what | ||
60 | BitBake is doing during the build. | ||
61 | |||
62 | - ":ref:`dev-manual/debugging:building with no dependencies`" | ||
63 | describes how to use the BitBake ``-b`` option to build a recipe | ||
64 | while ignoring dependencies. | ||
65 | |||
66 | - ":ref:`dev-manual/debugging:recipe logging mechanisms`" | ||
67 | describes how to use the many recipe logging functions to produce | ||
68 | debugging output and report errors and warnings. | ||
69 | |||
70 | - ":ref:`dev-manual/debugging:debugging parallel make races`" | ||
71 | describes how to debug situations where the build consists of several | ||
72 | parts that are run simultaneously and when the output or result of | ||
73 | one part is not ready for use with a different part of the build that | ||
74 | depends on that output. | ||
75 | |||
76 | - ":ref:`dev-manual/debugging:debugging with the gnu project debugger (gdb) remotely`" | ||
77 | describes how to use GDB to allow you to examine running programs, which can | ||
78 | help you fix problems. | ||
79 | |||
80 | - ":ref:`dev-manual/debugging:debugging with the gnu project debugger (gdb) on the target`" | ||
81 | describes how to use GDB directly on target hardware for debugging. | ||
82 | |||
83 | - ":ref:`dev-manual/debugging:other debugging tips`" describes | ||
84 | miscellaneous debugging tips that can be useful. | ||
85 | |||
86 | Viewing Logs from Failed Tasks | ||
87 | ============================== | ||
88 | |||
89 | You can find the log for a task in the file | ||
90 | ``${``\ :term:`WORKDIR`\ ``}/temp/log.do_``\ `taskname`. | ||
91 | For example, the log for the | ||
92 | :ref:`ref-tasks-compile` task of the | ||
93 | QEMU minimal image for the x86 machine (``qemux86``) might be in | ||
94 | ``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``. | ||
95 | To see the commands :term:`BitBake` ran | ||
96 | to generate a log, look at the corresponding ``run.do_``\ `taskname` file | ||
97 | in the same directory. | ||
98 | |||
99 | ``log.do_``\ `taskname` and ``run.do_``\ `taskname` are actually symbolic | ||
100 | links to ``log.do_``\ `taskname`\ ``.``\ `pid` and | ||
101 | ``log.run_``\ `taskname`\ ``.``\ `pid`, where `pid` is the PID the task had | ||
102 | when it ran. The symlinks always point to the files corresponding to the | ||
103 | most recent run. | ||
104 | |||
105 | Viewing Variable Values | ||
106 | ======================= | ||
107 | |||
108 | Sometimes you need to know the value of a variable as a result of | ||
109 | BitBake's parsing step. This could be because some unexpected behavior | ||
110 | occurred in your project. Perhaps an attempt to :ref:`modify a variable | ||
111 | <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:modifying existing | ||
112 | variables>` did not work out as expected. | ||
113 | |||
114 | BitBake's ``-e`` option is used to display variable values after | ||
115 | parsing. The following command displays the variable values after the | ||
116 | configuration files (i.e. ``local.conf``, ``bblayers.conf``, | ||
117 | ``bitbake.conf`` and so forth) have been parsed:: | ||
118 | |||
119 | $ bitbake -e | ||
120 | |||
121 | The following command displays variable values after a specific recipe has | ||
122 | been parsed. The variables include those from the configuration as well:: | ||
123 | |||
124 | $ bitbake -e recipename | ||
125 | |||
126 | .. note:: | ||
127 | |||
128 | Each recipe has its own private set of variables (datastore). | ||
129 | Internally, after parsing the configuration, a copy of the resulting | ||
130 | datastore is made prior to parsing each recipe. This copying implies | ||
131 | that variables set in one recipe will not be visible to other | ||
132 | recipes. | ||
133 | |||
134 | Likewise, each task within a recipe gets a private datastore based on | ||
135 | the recipe datastore, which means that variables set within one task | ||
136 | will not be visible to other tasks. | ||
137 | |||
138 | In the output of ``bitbake -e``, each variable is preceded by a | ||
139 | description of how the variable got its value, including temporary | ||
140 | values that were later overridden. This description also includes | ||
141 | variable flags (varflags) set on the variable. The output can be very | ||
142 | helpful during debugging. | ||
143 | |||
144 | Variables that are exported to the environment are preceded by | ||
145 | ``export`` in the output of ``bitbake -e``. See the following example:: | ||
146 | |||
147 | export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" | ||
148 | |||
149 | In addition to variable values, the output of the ``bitbake -e`` and | ||
150 | ``bitbake -e`` recipe commands includes the following information: | ||
151 | |||
152 | - The output starts with a tree listing all configuration files and | ||
153 | classes included globally, recursively listing the files they include | ||
154 | or inherit in turn. Much of the behavior of the OpenEmbedded build | ||
155 | system (including the behavior of the :ref:`ref-manual/tasks:normal recipe build tasks`) is | ||
156 | implemented in the :ref:`base <ref-classes-base>` class and the | ||
157 | classes it inherits, rather than being built into BitBake itself. | ||
158 | |||
159 | - After the variable values, all functions appear in the output. For | ||
160 | shell functions, variables referenced within the function body are | ||
161 | expanded. If a function has been modified using overrides or using | ||
162 | override-style operators like ``:append`` and ``:prepend``, then the | ||
163 | final assembled function body appears in the output. | ||
164 | |||
165 | Viewing Package Information with ``oe-pkgdata-util`` | ||
166 | ==================================================== | ||
167 | |||
168 | You can use the ``oe-pkgdata-util`` command-line utility to query | ||
169 | :term:`PKGDATA_DIR` and display | ||
170 | various package-related information. When you use the utility, you must | ||
171 | use it to view information on packages that have already been built. | ||
172 | |||
173 | Following are a few of the available ``oe-pkgdata-util`` subcommands. | ||
174 | |||
175 | .. note:: | ||
176 | |||
177 | You can use the standard \* and ? globbing wildcards as part of | ||
178 | package names and paths. | ||
179 | |||
180 | - ``oe-pkgdata-util list-pkgs [pattern]``: Lists all packages | ||
181 | that have been built, optionally limiting the match to packages that | ||
182 | match pattern. | ||
183 | |||
184 | - ``oe-pkgdata-util list-pkg-files package ...``: Lists the | ||
185 | files and directories contained in the given packages. | ||
186 | |||
187 | .. note:: | ||
188 | |||
189 | A different way to view the contents of a package is to look at | ||
190 | the | ||
191 | ``${``\ :term:`WORKDIR`\ ``}/packages-split`` | ||
192 | directory of the recipe that generates the package. This directory | ||
193 | is created by the | ||
194 | :ref:`ref-tasks-package` task | ||
195 | and has one subdirectory for each package the recipe generates, | ||
196 | which contains the files stored in that package. | ||
197 | |||
198 | If you want to inspect the ``${WORKDIR}/packages-split`` | ||
199 | directory, make sure that | ||
200 | :ref:`rm_work <ref-classes-rm-work>` is not | ||
201 | enabled when you build the recipe. | ||
202 | |||
203 | - ``oe-pkgdata-util find-path path ...``: Lists the names of | ||
204 | the packages that contain the given paths. For example, the following | ||
205 | tells us that ``/usr/share/man/man1/make.1`` is contained in the | ||
206 | ``make-doc`` package:: | ||
207 | |||
208 | $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 | ||
209 | make-doc: /usr/share/man/man1/make.1 | ||
210 | |||
211 | - ``oe-pkgdata-util lookup-recipe package ...``: Lists the name | ||
212 | of the recipes that produce the given packages. | ||
213 | |||
214 | For more information on the ``oe-pkgdata-util`` command, use the help | ||
215 | facility:: | ||
216 | |||
217 | $ oe-pkgdata-util --help | ||
218 | $ oe-pkgdata-util subcommand --help | ||
219 | |||
220 | Viewing Dependencies Between Recipes and Tasks | ||
221 | ============================================== | ||
222 | |||
223 | Sometimes it can be hard to see why BitBake wants to build other recipes | ||
224 | before the one you have specified. Dependency information can help you | ||
225 | understand why a recipe is built. | ||
226 | |||
227 | To generate dependency information for a recipe, run the following | ||
228 | command:: | ||
229 | |||
230 | $ bitbake -g recipename | ||
231 | |||
232 | This command writes the following files in the current directory: | ||
233 | |||
234 | - ``pn-buildlist``: A list of recipes/targets involved in building | ||
235 | `recipename`. "Involved" here means that at least one task from the | ||
236 | recipe needs to run when building `recipename` from scratch. Targets | ||
237 | that are in | ||
238 | :term:`ASSUME_PROVIDED` | ||
239 | are not listed. | ||
240 | |||
241 | - ``task-depends.dot``: A graph showing dependencies between tasks. | ||
242 | |||
243 | The graphs are in :wikipedia:`DOT <DOT_%28graph_description_language%29>` | ||
244 | format and can be converted to images (e.g. using the ``dot`` tool from | ||
245 | `Graphviz <https://www.graphviz.org/>`__). | ||
246 | |||
247 | .. note:: | ||
248 | |||
249 | - DOT files use a plain text format. The graphs generated using the | ||
250 | ``bitbake -g`` command are often so large as to be difficult to | ||
251 | read without special pruning (e.g. with BitBake's ``-I`` option) | ||
252 | and processing. Despite the form and size of the graphs, the | ||
253 | corresponding ``.dot`` files can still be possible to read and | ||
254 | provide useful information. | ||
255 | |||
256 | As an example, the ``task-depends.dot`` file contains lines such | ||
257 | as the following:: | ||
258 | |||
259 | "libxslt.do_configure" -> "libxml2.do_populate_sysroot" | ||
260 | |||
261 | The above example line reveals that the | ||
262 | :ref:`ref-tasks-configure` | ||
263 | task in ``libxslt`` depends on the | ||
264 | :ref:`ref-tasks-populate_sysroot` | ||
265 | task in ``libxml2``, which is a normal | ||
266 | :term:`DEPENDS` dependency | ||
267 | between the two recipes. | ||
268 | |||
269 | - For an example of how ``.dot`` files can be processed, see the | ||
270 | ``scripts/contrib/graph-tool`` Python script, which finds and | ||
271 | displays paths between graph nodes. | ||
272 | |||
273 | You can use a different method to view dependency information by using | ||
274 | the following command:: | ||
275 | |||
276 | $ bitbake -g -u taskexp recipename | ||
277 | |||
278 | This command | ||
279 | displays a GUI window from which you can view build-time and runtime | ||
280 | dependencies for the recipes involved in building recipename. | ||
281 | |||
282 | Viewing Task Variable Dependencies | ||
283 | ================================== | ||
284 | |||
285 | As mentioned in the | ||
286 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section of the BitBake | ||
287 | User Manual, BitBake tries to automatically determine what variables a | ||
288 | task depends on so that it can rerun the task if any values of the | ||
289 | variables change. This determination is usually reliable. However, if | ||
290 | you do things like construct variable names at runtime, then you might | ||
291 | have to manually declare dependencies on those variables using | ||
292 | ``vardeps`` as described in the | ||
293 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section of the BitBake | ||
294 | User Manual. | ||
295 | |||
296 | If you are unsure whether a variable dependency is being picked up | ||
297 | automatically for a given task, you can list the variable dependencies | ||
298 | BitBake has determined by doing the following: | ||
299 | |||
300 | 1. Build the recipe containing the task:: | ||
301 | |||
302 | $ bitbake recipename | ||
303 | |||
304 | 2. Inside the :term:`STAMPS_DIR` | ||
305 | directory, find the signature data (``sigdata``) file that | ||
306 | corresponds to the task. The ``sigdata`` files contain a pickled | ||
307 | Python database of all the metadata that went into creating the input | ||
308 | checksum for the task. As an example, for the | ||
309 | :ref:`ref-tasks-fetch` task of the | ||
310 | ``db`` recipe, the ``sigdata`` file might be found in the following | ||
311 | location:: | ||
312 | |||
313 | ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
314 | |||
315 | For tasks that are accelerated through the shared state | ||
316 | (:ref:`sstate <overview-manual/concepts:shared state cache>`) cache, an | ||
317 | additional ``siginfo`` file is written into | ||
318 | :term:`SSTATE_DIR` along with | ||
319 | the cached task output. The ``siginfo`` files contain exactly the | ||
320 | same information as ``sigdata`` files. | ||
321 | |||
322 | 3. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here | ||
323 | is an example:: | ||
324 | |||
325 | $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 | ||
326 | |||
327 | In the output of the above command, you will find a line like the | ||
328 | following, which lists all the (inferred) variable dependencies for | ||
329 | the task. This list also includes indirect dependencies from | ||
330 | variables depending on other variables, recursively. | ||
331 | :: | ||
332 | |||
333 | Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] | ||
334 | |||
335 | .. note:: | ||
336 | |||
337 | Functions (e.g. ``base_do_fetch``) also count as variable dependencies. | ||
338 | These functions in turn depend on the variables they reference. | ||
339 | |||
340 | The output of ``bitbake-dumpsig`` also includes the value each | ||
341 | variable had, a list of dependencies for each variable, and | ||
342 | :term:`BB_BASEHASH_IGNORE_VARS` | ||
343 | information. | ||
344 | |||
345 | There is also a ``bitbake-diffsigs`` command for comparing two | ||
346 | ``siginfo`` or ``sigdata`` files. This command can be helpful when | ||
347 | trying to figure out what changed between two versions of a task. If you | ||
348 | call ``bitbake-diffsigs`` with just one file, the command behaves like | ||
349 | ``bitbake-dumpsig``. | ||
350 | |||
351 | You can also use BitBake to dump out the signature construction | ||
352 | information without executing tasks by using either of the following | ||
353 | BitBake command-line options:: | ||
354 | |||
355 | ‐‐dump-signatures=SIGNATURE_HANDLER | ||
356 | -S SIGNATURE_HANDLER | ||
357 | |||
358 | |||
359 | .. note:: | ||
360 | |||
361 | Two common values for `SIGNATURE_HANDLER` are "none" and "printdiff", which | ||
362 | dump only the signature or compare the dumped signature with the cached one, | ||
363 | respectively. | ||
364 | |||
365 | Using BitBake with either of these options causes BitBake to dump out | ||
366 | ``sigdata`` files in the ``stamps`` directory for every task it would | ||
367 | have executed instead of building the specified target package. | ||
368 | |||
369 | Viewing Metadata Used to Create the Input Signature of a Shared State Task | ||
370 | ========================================================================== | ||
371 | |||
372 | Seeing what metadata went into creating the input signature of a shared | ||
373 | state (sstate) task can be a useful debugging aid. This information is | ||
374 | available in signature information (``siginfo``) files in | ||
375 | :term:`SSTATE_DIR`. For | ||
376 | information on how to view and interpret information in ``siginfo`` | ||
377 | files, see the | ||
378 | ":ref:`dev-manual/debugging:viewing task variable dependencies`" section. | ||
379 | |||
380 | For conceptual information on shared state, see the | ||
381 | ":ref:`overview-manual/concepts:shared state`" | ||
382 | section in the Yocto Project Overview and Concepts Manual. | ||
383 | |||
384 | Invalidating Shared State to Force a Task to Run | ||
385 | ================================================ | ||
386 | |||
387 | The OpenEmbedded build system uses | ||
388 | :ref:`checksums <overview-manual/concepts:checksums (signatures)>` and | ||
389 | :ref:`overview-manual/concepts:shared state` cache to avoid unnecessarily | ||
390 | rebuilding tasks. Collectively, this scheme is known as "shared state | ||
391 | code". | ||
392 | |||
393 | As with all schemes, this one has some drawbacks. It is possible that | ||
394 | you could make implicit changes to your code that the checksum | ||
395 | calculations do not take into account. These implicit changes affect a | ||
396 | task's output but do not trigger the shared state code into rebuilding a | ||
397 | recipe. Consider an example during which a tool changes its output. | ||
398 | Assume that the output of ``rpmdeps`` changes. The result of the change | ||
399 | should be that all the ``package`` and ``package_write_rpm`` shared | ||
400 | state cache items become invalid. However, because the change to the | ||
401 | output is external to the code and therefore implicit, the associated | ||
402 | shared state cache items do not become invalidated. In this case, the | ||
403 | build process uses the cached items rather than running the task again. | ||
404 | Obviously, these types of implicit changes can cause problems. | ||
405 | |||
406 | To avoid these problems during the build, you need to understand the | ||
407 | effects of any changes you make. Realize that changes you make directly | ||
408 | to a function are automatically factored into the checksum calculation. | ||
409 | Thus, these explicit changes invalidate the associated area of shared | ||
410 | state cache. However, you need to be aware of any implicit changes that | ||
411 | are not obvious changes to the code and could affect the output of a | ||
412 | given task. | ||
413 | |||
414 | When you identify an implicit change, you can easily take steps to | ||
415 | invalidate the cache and force the tasks to run. The steps you can take | ||
416 | are as simple as changing a function's comments in the source code. For | ||
417 | example, to invalidate package shared state files, change the comment | ||
418 | statements of | ||
419 | :ref:`ref-tasks-package` or the | ||
420 | comments of one of the functions it calls. Even though the change is | ||
421 | purely cosmetic, it causes the checksum to be recalculated and forces | ||
422 | the build system to run the task again. | ||
423 | |||
424 | .. note:: | ||
425 | |||
426 | For an example of a commit that makes a cosmetic change to invalidate | ||
427 | shared state, see this | ||
428 | :yocto_git:`commit </poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54>`. | ||
429 | |||
430 | Running Specific Tasks | ||
431 | ====================== | ||
432 | |||
433 | Any given recipe consists of a set of tasks. The standard BitBake | ||
434 | behavior in most cases is: :ref:`ref-tasks-fetch`, :ref:`ref-tasks-unpack`, :ref:`ref-tasks-patch`, | ||
435 | :ref:`ref-tasks-configure`, :ref:`ref-tasks-compile`, :ref:`ref-tasks-install`, :ref:`ref-tasks-package`, | ||
436 | :ref:`do_package_write_* <ref-tasks-package_write_deb>`, and :ref:`ref-tasks-build`. The default task is | ||
437 | :ref:`ref-tasks-build` and any tasks on which it depends build first. Some tasks, | ||
438 | such as :ref:`ref-tasks-devshell`, are not part of the default build chain. If you | ||
439 | wish to run a task that is not part of the default build chain, you can | ||
440 | use the ``-c`` option in BitBake. Here is an example:: | ||
441 | |||
442 | $ bitbake matchbox-desktop -c devshell | ||
443 | |||
444 | The ``-c`` option respects task dependencies, which means that all other | ||
445 | tasks (including tasks from other recipes) that the specified task | ||
446 | depends on will be run before the task. Even when you manually specify a | ||
447 | task to run with ``-c``, BitBake will only run the task if it considers | ||
448 | it "out of date". See the | ||
449 | ":ref:`overview-manual/concepts:stamp files and the rerunning of tasks`" | ||
450 | section in the Yocto Project Overview and Concepts Manual for how | ||
451 | BitBake determines whether a task is "out of date". | ||
452 | |||
453 | If you want to force an up-to-date task to be rerun (e.g. because you | ||
454 | made manual modifications to the recipe's | ||
455 | :term:`WORKDIR` that you want to try | ||
456 | out), then you can use the ``-f`` option. | ||
457 | |||
458 | .. note:: | ||
459 | |||
460 | The reason ``-f`` is never required when running the | ||
461 | :ref:`ref-tasks-devshell` task is because the | ||
462 | [\ :ref:`nostamp <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ] | ||
463 | variable flag is already set for the task. | ||
464 | |||
465 | The following example shows one way you can use the ``-f`` option:: | ||
466 | |||
467 | $ bitbake matchbox-desktop | ||
468 | . | ||
469 | . | ||
470 | make some changes to the source code in the work directory | ||
471 | . | ||
472 | . | ||
473 | $ bitbake matchbox-desktop -c compile -f | ||
474 | $ bitbake matchbox-desktop | ||
475 | |||
476 | This sequence first builds and then recompiles ``matchbox-desktop``. The | ||
477 | last command reruns all tasks (basically the packaging tasks) after the | ||
478 | compile. BitBake recognizes that the :ref:`ref-tasks-compile` task was rerun and | ||
479 | therefore understands that the other tasks also need to be run again. | ||
480 | |||
481 | Another, shorter way to rerun a task and all | ||
482 | :ref:`ref-manual/tasks:normal recipe build tasks` | ||
483 | that depend on it is to use the ``-C`` option. | ||
484 | |||
485 | .. note:: | ||
486 | |||
487 | This option is upper-cased and is separate from the ``-c`` | ||
488 | option, which is lower-cased. | ||
489 | |||
490 | Using this option invalidates the given task and then runs the | ||
491 | :ref:`ref-tasks-build` task, which is | ||
492 | the default task if no task is given, and the tasks on which it depends. | ||
493 | You could replace the final two commands in the previous example with | ||
494 | the following single command:: | ||
495 | |||
496 | $ bitbake matchbox-desktop -C compile | ||
497 | |||
498 | Internally, the ``-f`` and ``-C`` options work by tainting (modifying) | ||
499 | the input checksum of the specified task. This tainting indirectly | ||
500 | causes the task and its dependent tasks to be rerun through the normal | ||
501 | task dependency mechanisms. | ||
502 | |||
503 | .. note:: | ||
504 | |||
505 | BitBake explicitly keeps track of which tasks have been tainted in | ||
506 | this fashion, and will print warnings such as the following for | ||
507 | builds involving such tasks: | ||
508 | |||
509 | .. code-block:: none | ||
510 | |||
511 | WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run | ||
512 | |||
513 | |||
514 | The purpose of the warning is to let you know that the work directory | ||
515 | and build output might not be in the clean state they would be in for | ||
516 | a "normal" build, depending on what actions you took. To get rid of | ||
517 | such warnings, you can remove the work directory and rebuild the | ||
518 | recipe, as follows:: | ||
519 | |||
520 | $ bitbake matchbox-desktop -c clean | ||
521 | $ bitbake matchbox-desktop | ||
522 | |||
523 | |||
524 | You can view a list of tasks in a given package by running the | ||
525 | :ref:`ref-tasks-listtasks` task as follows:: | ||
526 | |||
527 | $ bitbake matchbox-desktop -c listtasks | ||
528 | |||
529 | The results appear as output to the console and are also in | ||
530 | the file ``${WORKDIR}/temp/log.do_listtasks``. | ||
531 | |||
532 | General BitBake Problems | ||
533 | ======================== | ||
534 | |||
535 | You can see debug output from BitBake by using the ``-D`` option. The | ||
536 | debug output gives more information about what BitBake is doing and the | ||
537 | reason behind it. Each ``-D`` option you use increases the logging | ||
538 | level. The most common usage is ``-DDD``. | ||
539 | |||
540 | The output from ``bitbake -DDD -v targetname`` can reveal why BitBake | ||
541 | chose a certain version of a package or why BitBake picked a certain | ||
542 | provider. This command could also help you in a situation where you | ||
543 | think BitBake did something unexpected. | ||
544 | |||
545 | Building with No Dependencies | ||
546 | ============================= | ||
547 | |||
548 | To build a specific recipe (``.bb`` file), you can use the following | ||
549 | command form:: | ||
550 | |||
551 | $ bitbake -b somepath/somerecipe.bb | ||
552 | |||
553 | This command form does | ||
554 | not check for dependencies. Consequently, you should use it only when | ||
555 | you know existing dependencies have been met. | ||
556 | |||
557 | .. note:: | ||
558 | |||
559 | You can also specify fragments of the filename. In this case, BitBake | ||
560 | checks for a unique match. | ||
561 | |||
562 | Recipe Logging Mechanisms | ||
563 | ========================= | ||
564 | |||
565 | The Yocto Project provides several logging functions for producing | ||
566 | debugging output and reporting errors and warnings. For Python | ||
567 | functions, the following logging functions are available. All of these functions | ||
568 | log to ``${T}/log.do_``\ `task`, and can also log to standard output | ||
569 | (stdout) with the right settings: | ||
570 | |||
571 | - ``bb.plain(msg)``: Writes msg as is to the log while also | ||
572 | logging to stdout. | ||
573 | |||
574 | - ``bb.note(msg)``: Writes "NOTE: msg" to the log. Also logs to | ||
575 | stdout if BitBake is called with "-v". | ||
576 | |||
577 | - ``bb.debug(level, msg)``: Writes "DEBUG: msg" to the | ||
578 | log. Also logs to stdout if the log level is greater than or equal to | ||
579 | level. See the ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax`" option | ||
580 | in the BitBake User Manual for more information. | ||
581 | |||
582 | - ``bb.warn(msg)``: Writes "WARNING: msg" to the log while also | ||
583 | logging to stdout. | ||
584 | |||
585 | - ``bb.error(msg)``: Writes "ERROR: msg" to the log while also | ||
586 | logging to standard out (stdout). | ||
587 | |||
588 | .. note:: | ||
589 | |||
590 | Calling this function does not cause the task to fail. | ||
591 | |||
592 | - ``bb.fatal(msg)``: This logging function is similar to | ||
593 | ``bb.error(msg)`` but also causes the calling task to fail. | ||
594 | |||
595 | .. note:: | ||
596 | |||
597 | ``bb.fatal()`` raises an exception, which means you do not need to put a | ||
598 | "return" statement after the function. | ||
599 | |||
600 | The same logging functions are also available in shell functions, under | ||
601 | the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``, | ||
602 | and ``bbfatal``. The | ||
603 | :ref:`logging <ref-classes-logging>` class | ||
604 | implements these functions. See that class in the ``meta/classes`` | ||
605 | folder of the :term:`Source Directory` for information. | ||
606 | |||
607 | Logging With Python | ||
608 | ------------------- | ||
609 | |||
610 | When creating recipes using Python and inserting code that handles build | ||
611 | logs, keep in mind the goal is to have informative logs while keeping | ||
612 | the console as "silent" as possible. Also, if you want status messages | ||
613 | in the log, use the "debug" loglevel. | ||
614 | |||
615 | Following is an example written in Python. The code handles logging for | ||
616 | a function that determines the number of tasks needed to be run. See the | ||
617 | ":ref:`ref-tasks-listtasks`" | ||
618 | section for additional information:: | ||
619 | |||
620 | python do_listtasks() { | ||
621 | bb.debug(2, "Starting to figure out the task list") | ||
622 | if noteworthy_condition: | ||
623 | bb.note("There are 47 tasks to run") | ||
624 | bb.debug(2, "Got to point xyz") | ||
625 | if warning_trigger: | ||
626 | bb.warn("Detected warning_trigger, this might be a problem later.") | ||
627 | if recoverable_error: | ||
628 | bb.error("Hit recoverable_error, you really need to fix this!") | ||
629 | if fatal_error: | ||
630 | bb.fatal("fatal_error detected, unable to print the task list") | ||
631 | bb.plain("The tasks present are abc") | ||
632 | bb.debug(2, "Finished figuring out the tasklist") | ||
633 | } | ||
634 | |||
635 | Logging With Bash | ||
636 | ----------------- | ||
637 | |||
638 | When creating recipes using Bash and inserting code that handles build | ||
639 | logs, you have the same goals --- informative with minimal console output. | ||
640 | The syntax you use for recipes written in Bash is similar to that of | ||
641 | recipes written in Python described in the previous section. | ||
642 | |||
643 | Following is an example written in Bash. The code logs the progress of | ||
644 | the ``do_my_function`` function. | ||
645 | :: | ||
646 | |||
647 | do_my_function() { | ||
648 | bbdebug 2 "Running do_my_function" | ||
649 | if [ exceptional_condition ]; then | ||
650 | bbnote "Hit exceptional_condition" | ||
651 | fi | ||
652 | bbdebug 2 "Got to point xyz" | ||
653 | if [ warning_trigger ]; then | ||
654 | bbwarn "Detected warning_trigger, this might cause a problem later." | ||
655 | fi | ||
656 | if [ recoverable_error ]; then | ||
657 | bberror "Hit recoverable_error, correcting" | ||
658 | fi | ||
659 | if [ fatal_error ]; then | ||
660 | bbfatal "fatal_error detected" | ||
661 | fi | ||
662 | bbdebug 2 "Completed do_my_function" | ||
663 | } | ||
664 | |||
665 | |||
666 | Debugging Parallel Make Races | ||
667 | ============================= | ||
668 | |||
669 | A parallel ``make`` race occurs when the build consists of several parts | ||
670 | that are run simultaneously and a situation occurs when the output or | ||
671 | result of one part is not ready for use with a different part of the | ||
672 | build that depends on that output. Parallel make races are annoying and | ||
673 | can sometimes be difficult to reproduce and fix. However, there are some simple | ||
674 | tips and tricks that can help you debug and fix them. This section | ||
675 | presents a real-world example of an error encountered on the Yocto | ||
676 | Project autobuilder and the process used to fix it. | ||
677 | |||
678 | .. note:: | ||
679 | |||
680 | If you cannot properly fix a ``make`` race condition, you can work around it | ||
681 | by clearing either the :term:`PARALLEL_MAKE` or :term:`PARALLEL_MAKEINST` | ||
682 | variables. | ||
683 | |||
684 | The Failure | ||
685 | ----------- | ||
686 | |||
687 | For this example, assume that you are building an image that depends on | ||
688 | the "neard" package. And, during the build, BitBake runs into problems | ||
689 | and creates the following output. | ||
690 | |||
691 | .. note:: | ||
692 | |||
693 | This example log file has longer lines artificially broken to make | ||
694 | the listing easier to read. | ||
695 | |||
696 | If you examine the output or the log file, you see the failure during | ||
697 | ``make``: | ||
698 | |||
699 | .. code-block:: none | ||
700 | |||
701 | | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] | ||
702 | | DEBUG: Executing shell function do_compile | ||
703 | | NOTE: make -j 16 | ||
704 | | make --no-print-directory all-am | ||
705 | | /bin/mkdir -p include/near | ||
706 | | /bin/mkdir -p include/near | ||
707 | | /bin/mkdir -p include/near | ||
708 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
709 | 0.14-r0/neard-0.14/include/types.h include/near/types.h | ||
710 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
711 | 0.14-r0/neard-0.14/include/log.h include/near/log.h | ||
712 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
713 | 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h | ||
714 | | /bin/mkdir -p include/near | ||
715 | | /bin/mkdir -p include/near | ||
716 | | /bin/mkdir -p include/near | ||
717 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
718 | 0.14-r0/neard-0.14/include/tag.h include/near/tag.h | ||
719 | | /bin/mkdir -p include/near | ||
720 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
721 | 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h | ||
722 | | /bin/mkdir -p include/near | ||
723 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
724 | 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h | ||
725 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
726 | 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h | ||
727 | | /bin/mkdir -p include/near | ||
728 | | /bin/mkdir -p include/near | ||
729 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
730 | 0.14-r0/neard-0.14/include/setting.h include/near/setting.h | ||
731 | | /bin/mkdir -p include/near | ||
732 | | /bin/mkdir -p include/near | ||
733 | | /bin/mkdir -p include/near | ||
734 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
735 | 0.14-r0/neard-0.14/include/device.h include/near/device.h | ||
736 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
737 | 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h | ||
738 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
739 | 0.14-r0/neard-0.14/include/snep.h include/near/snep.h | ||
740 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
741 | 0.14-r0/neard-0.14/include/version.h include/near/version.h | ||
742 | | ln -s /home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ | ||
743 | 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h | ||
744 | | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h | ||
745 | | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/nightly-x86/ | ||
746 | build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ | ||
747 | yocto-autobuilder/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 | ||
748 | -I/home/pokybuild/yocto-autobuilder/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ | ||
749 | lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/nightly-x86/build/build/ | ||
750 | tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/ | ||
751 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ | ||
752 | nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 | ||
753 | -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" | ||
754 | -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c | ||
755 | -o tools/snep-send.o tools/snep-send.c | ||
756 | | In file included from tools/snep-send.c:16:0: | ||
757 | | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
758 | | #include <near/dbus.h> | ||
759 | | ^ | ||
760 | | compilation terminated. | ||
761 | | make[1]: *** [tools/snep-send.o] Error 1 | ||
762 | | make[1]: *** Waiting for unfinished jobs.... | ||
763 | | make: *** [all] Error 2 | ||
764 | | ERROR: oe_runmake failed | ||
765 | |||
766 | Reproducing the Error | ||
767 | --------------------- | ||
768 | |||
769 | Because race conditions are intermittent, they do not manifest | ||
770 | themselves every time you do the build. In fact, most times the build | ||
771 | will complete without problems even though the potential race condition | ||
772 | exists. Thus, once the error surfaces, you need a way to reproduce it. | ||
773 | |||
774 | In this example, compiling the "neard" package is causing the problem. | ||
775 | So the first thing to do is build "neard" locally. Before you start the | ||
776 | build, set the | ||
777 | :term:`PARALLEL_MAKE` variable | ||
778 | in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a | ||
779 | high value for :term:`PARALLEL_MAKE` increases the chances of the race | ||
780 | condition showing up:: | ||
781 | |||
782 | $ bitbake neard | ||
783 | |||
784 | Once the local build for "neard" completes, start a ``devshell`` build:: | ||
785 | |||
786 | $ bitbake neard -c devshell | ||
787 | |||
788 | For information on how to use a ``devshell``, see the | ||
789 | ":ref:`dev-manual/development-shell:using a development shell`" section. | ||
790 | |||
791 | In the ``devshell``, do the following:: | ||
792 | |||
793 | $ make clean | ||
794 | $ make tools/snep-send.o | ||
795 | |||
796 | The ``devshell`` commands cause the failure to clearly | ||
797 | be visible. In this case, there is a missing dependency for the ``neard`` | ||
798 | Makefile target. Here is some abbreviated, sample output with the | ||
799 | missing dependency clearly visible at the end:: | ||
800 | |||
801 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... | ||
802 | . | ||
803 | . | ||
804 | . | ||
805 | tools/snep-send.c | ||
806 | In file included from tools/snep-send.c:16:0: | ||
807 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory | ||
808 | #include <near/dbus.h> | ||
809 | ^ | ||
810 | compilation terminated. | ||
811 | make: *** [tools/snep-send.o] Error 1 | ||
812 | $ | ||
813 | |||
814 | |||
815 | Creating a Patch for the Fix | ||
816 | ---------------------------- | ||
817 | |||
818 | Because there is a missing dependency for the Makefile target, you need | ||
819 | to patch the ``Makefile.am`` file, which is generated from | ||
820 | ``Makefile.in``. You can use Quilt to create the patch:: | ||
821 | |||
822 | $ quilt new parallelmake.patch | ||
823 | Patch patches/parallelmake.patch is now on top | ||
824 | $ quilt add Makefile.am | ||
825 | File Makefile.am added to patch patches/parallelmake.patch | ||
826 | |||
827 | For more information on using Quilt, see the | ||
828 | ":ref:`dev-manual/quilt:using quilt in your workflow`" section. | ||
829 | |||
830 | At this point you need to make the edits to ``Makefile.am`` to add the | ||
831 | missing dependency. For our example, you have to add the following line | ||
832 | to the file:: | ||
833 | |||
834 | tools/snep-send.$(OBJEXT): include/near/dbus.h | ||
835 | |||
836 | Once you have edited the file, use the ``refresh`` command to create the | ||
837 | patch:: | ||
838 | |||
839 | $ quilt refresh | ||
840 | Refreshed patch patches/parallelmake.patch | ||
841 | |||
842 | Once the patch file is created, you need to add it back to the originating | ||
843 | recipe folder. Here is an example assuming a top-level | ||
844 | :term:`Source Directory` named ``poky``:: | ||
845 | |||
846 | $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard | ||
847 | |||
848 | The final thing you need to do to implement the fix in the build is to | ||
849 | update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the | ||
850 | :term:`SRC_URI` statement includes | ||
851 | the patch file. The recipe file is in the folder above the patch. Here | ||
852 | is what the edited :term:`SRC_URI` statement would look like:: | ||
853 | |||
854 | SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ | ||
855 | file://neard.in \ | ||
856 | file://neard.service.in \ | ||
857 | file://parallelmake.patch \ | ||
858 | " | ||
859 | |||
860 | With the patch complete and moved to the correct folder and the | ||
861 | :term:`SRC_URI` statement updated, you can exit the ``devshell``:: | ||
862 | |||
863 | $ exit | ||
864 | |||
865 | Testing the Build | ||
866 | ----------------- | ||
867 | |||
868 | With everything in place, you can get back to trying the build again | ||
869 | locally:: | ||
870 | |||
871 | $ bitbake neard | ||
872 | |||
873 | This build should succeed. | ||
874 | |||
875 | Now you can open up a ``devshell`` again and repeat the clean and make | ||
876 | operations as follows:: | ||
877 | |||
878 | $ bitbake neard -c devshell | ||
879 | $ make clean | ||
880 | $ make tools/snep-send.o | ||
881 | |||
882 | The build should work without issue. | ||
883 | |||
884 | As with all solved problems, if they originated upstream, you need to | ||
885 | submit the fix for the recipe in OE-Core and upstream so that the | ||
886 | problem is taken care of at its source. See the | ||
887 | ":ref:`dev-manual/changes:submitting a change to the yocto project`" | ||
888 | section for more information. | ||
889 | |||
890 | Debugging With the GNU Project Debugger (GDB) Remotely | ||
891 | ====================================================== | ||
892 | |||
893 | GDB allows you to examine running programs, which in turn helps you to | ||
894 | understand and fix problems. It also allows you to perform post-mortem | ||
895 | style analysis of program crashes. GDB is available as a package within | ||
896 | the Yocto Project and is installed in SDK images by default. See the | ||
897 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
898 | Project Reference Manual for a description of these images. You can find | ||
899 | information on GDB at https://sourceware.org/gdb/. | ||
900 | |||
901 | .. note:: | ||
902 | |||
903 | For best results, install debug (``-dbg``) packages for the applications you | ||
904 | are going to debug. Doing so makes extra debug symbols available that give | ||
905 | you more meaningful output. | ||
906 | |||
907 | Sometimes, due to memory or disk space constraints, it is not possible | ||
908 | to use GDB directly on the remote target to debug applications. These | ||
909 | constraints arise because GDB needs to load the debugging information | ||
910 | and the binaries of the process being debugged. Additionally, GDB needs | ||
911 | to perform many computations to locate information such as function | ||
912 | names, variable names and values, stack traces and so forth --- even | ||
913 | before starting the debugging process. These extra computations place | ||
914 | more load on the target system and can alter the characteristics of the | ||
915 | program being debugged. | ||
916 | |||
917 | To help get past the previously mentioned constraints, there are two | ||
918 | methods you can use: running a debuginfod server and using gdbserver. | ||
919 | |||
920 | Using the debuginfod server method | ||
921 | ---------------------------------- | ||
922 | |||
923 | ``debuginfod`` from ``elfutils`` is a way to distribute ``debuginfo`` files. | ||
924 | Running a ``debuginfod`` server makes debug symbols readily available, | ||
925 | which means you don't need to download debugging information | ||
926 | and the binaries of the process being debugged. You can just fetch | ||
927 | debug symbols from the server. | ||
928 | |||
929 | To run a ``debuginfod`` server, you need to do the following: | ||
930 | |||
931 | - Ensure that ``debuginfod`` is present in :term:`DISTRO_FEATURES` | ||
932 | (it already is in ``OpenEmbedded-core`` defaults and ``poky`` reference distribution). | ||
933 | If not, set in your distro config file or in ``local.conf``:: | ||
934 | |||
935 | DISTRO_FEATURES:append = " debuginfod" | ||
936 | |||
937 | This distro feature enables the server and client library in ``elfutils``, | ||
938 | and enables ``debuginfod`` support in clients (at the moment, ``gdb`` and ``binutils``). | ||
939 | |||
940 | - Run the following commands to launch the ``debuginfod`` server on the host:: | ||
941 | |||
942 | $ oe-debuginfod | ||
943 | |||
944 | - To use ``debuginfod`` on the target, you need to know the ip:port where | ||
945 | ``debuginfod`` is listening on the host (port defaults to 8002), and export | ||
946 | that into the shell environment, for example in ``qemu``:: | ||
947 | |||
948 | root@qemux86-64:~# export DEBUGINFOD_URLS="http://192.168.7.1:8002/" | ||
949 | |||
950 | - Then debug info fetching should simply work when running the target ``gdb``, | ||
951 | ``readelf`` or ``objdump``, for example:: | ||
952 | |||
953 | root@qemux86-64:~# gdb /bin/cat | ||
954 | ... | ||
955 | Reading symbols from /bin/cat... | ||
956 | Downloading separate debug info for /bin/cat... | ||
957 | Reading symbols from /home/root/.cache/debuginfod_client/923dc4780cfbc545850c616bffa884b6b5eaf322/debuginfo... | ||
958 | |||
959 | - It's also possible to use ``debuginfod-find`` to just query the server:: | ||
960 | |||
961 | root@qemux86-64:~# debuginfod-find debuginfo /bin/ls | ||
962 | /home/root/.cache/debuginfod_client/356edc585f7f82d46f94fcb87a86a3fe2d2e60bd/debuginfo | ||
963 | |||
964 | |||
965 | Using the gdbserver method | ||
966 | -------------------------- | ||
967 | |||
968 | gdbserver, which runs on the remote target and does not load any | ||
969 | debugging information from the debugged process. Instead, a GDB instance | ||
970 | processes the debugging information that is run on a remote computer - | ||
971 | the host GDB. The host GDB then sends control commands to gdbserver to | ||
972 | make it stop or start the debugged program, as well as read or write | ||
973 | memory regions of that debugged program. All the debugging information | ||
974 | loaded and processed as well as all the heavy debugging is done by the | ||
975 | host GDB. Offloading these processes gives the gdbserver running on the | ||
976 | target a chance to remain small and fast. | ||
977 | |||
978 | Because the host GDB is responsible for loading the debugging | ||
979 | information and for doing the necessary processing to make actual | ||
980 | debugging happen, you have to make sure the host can access the | ||
981 | unstripped binaries complete with their debugging information and also | ||
982 | be sure the target is compiled with no optimizations. The host GDB must | ||
983 | also have local access to all the libraries used by the debugged | ||
984 | program. Because gdbserver does not need any local debugging | ||
985 | information, the binaries on the remote target can remain stripped. | ||
986 | However, the binaries must also be compiled without optimization so they | ||
987 | match the host's binaries. | ||
988 | |||
989 | To remain consistent with GDB documentation and terminology, the binary | ||
990 | being debugged on the remote target machine is referred to as the | ||
991 | "inferior" binary. For documentation on GDB see the `GDB | ||
992 | site <https://sourceware.org/gdb/documentation/>`__. | ||
993 | |||
994 | The following steps show you how to debug using the GNU project | ||
995 | debugger. | ||
996 | |||
997 | 1. *Configure your build system to construct the companion debug | ||
998 | filesystem:* | ||
999 | |||
1000 | In your ``local.conf`` file, set the following:: | ||
1001 | |||
1002 | IMAGE_GEN_DEBUGFS = "1" | ||
1003 | IMAGE_FSTYPES_DEBUGFS = "tar.bz2" | ||
1004 | |||
1005 | These options cause the | ||
1006 | OpenEmbedded build system to generate a special companion filesystem | ||
1007 | fragment, which contains the matching source and debug symbols to | ||
1008 | your deployable filesystem. The build system does this by looking at | ||
1009 | what is in the deployed filesystem, and pulling the corresponding | ||
1010 | ``-dbg`` packages. | ||
1011 | |||
1012 | The companion debug filesystem is not a complete filesystem, but only | ||
1013 | contains the debug fragments. This filesystem must be combined with | ||
1014 | the full filesystem for debugging. Subsequent steps in this procedure | ||
1015 | show how to combine the partial filesystem with the full filesystem. | ||
1016 | |||
1017 | 2. *Configure the system to include gdbserver in the target filesystem:* | ||
1018 | |||
1019 | Make the following addition in your ``local.conf`` file:: | ||
1020 | |||
1021 | EXTRA_IMAGE_FEATURES:append = " tools-debug" | ||
1022 | |||
1023 | The change makes | ||
1024 | sure the ``gdbserver`` package is included. | ||
1025 | |||
1026 | 3. *Build the environment:* | ||
1027 | |||
1028 | Use the following command to construct the image and the companion | ||
1029 | Debug Filesystem:: | ||
1030 | |||
1031 | $ bitbake image | ||
1032 | |||
1033 | Build the cross GDB component and | ||
1034 | make it available for debugging. Build the SDK that matches the | ||
1035 | image. Building the SDK is best for a production build that can be | ||
1036 | used later for debugging, especially during long term maintenance:: | ||
1037 | |||
1038 | $ bitbake -c populate_sdk image | ||
1039 | |||
1040 | Alternatively, you can build the minimal toolchain components that | ||
1041 | match the target. Doing so creates a smaller than typical SDK and | ||
1042 | only contains a minimal set of components with which to build simple | ||
1043 | test applications, as well as run the debugger:: | ||
1044 | |||
1045 | $ bitbake meta-toolchain | ||
1046 | |||
1047 | A final method is to build Gdb itself within the build system:: | ||
1048 | |||
1049 | $ bitbake gdb-cross-<architecture> | ||
1050 | |||
1051 | Doing so produces a temporary copy of | ||
1052 | ``cross-gdb`` you can use for debugging during development. While | ||
1053 | this is the quickest approach, the two previous methods in this step | ||
1054 | are better when considering long-term maintenance strategies. | ||
1055 | |||
1056 | .. note:: | ||
1057 | |||
1058 | If you run ``bitbake gdb-cross``, the OpenEmbedded build system suggests | ||
1059 | the actual image (e.g. ``gdb-cross-i586``). The suggestion is usually the | ||
1060 | actual name you want to use. | ||
1061 | |||
1062 | 4. *Set up the* ``debugfs``\ *:* | ||
1063 | |||
1064 | Run the following commands to set up the ``debugfs``:: | ||
1065 | |||
1066 | $ mkdir debugfs | ||
1067 | $ cd debugfs | ||
1068 | $ tar xvfj build-dir/tmp/deploy/images/machine/image.rootfs.tar.bz2 | ||
1069 | $ tar xvfj build-dir/tmp/deploy/images/machine/image-dbg.rootfs.tar.bz2 | ||
1070 | |||
1071 | 5. *Set up GDB:* | ||
1072 | |||
1073 | Install the SDK (if you built one) and then source the correct | ||
1074 | environment file. Sourcing the environment file puts the SDK in your | ||
1075 | ``PATH`` environment variable and sets ``$GDB`` to the SDK's debugger. | ||
1076 | |||
1077 | If you are using the build system, Gdb is located in | ||
1078 | `build-dir`\ ``/tmp/sysroots/``\ `host`\ ``/usr/bin/``\ `architecture`\ ``/``\ `architecture`\ ``-gdb`` | ||
1079 | |||
1080 | 6. *Boot the target:* | ||
1081 | |||
1082 | For information on how to run QEMU, see the `QEMU | ||
1083 | Documentation <https://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__. | ||
1084 | |||
1085 | .. note:: | ||
1086 | |||
1087 | Be sure to verify that your host can access the target via TCP. | ||
1088 | |||
1089 | 7. *Debug a program:* | ||
1090 | |||
1091 | Debugging a program involves running gdbserver on the target and then | ||
1092 | running Gdb on the host. The example in this step debugs ``gzip``: | ||
1093 | |||
1094 | .. code-block:: shell | ||
1095 | |||
1096 | root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help | ||
1097 | |||
1098 | For | ||
1099 | additional gdbserver options, see the `GDB Server | ||
1100 | Documentation <https://www.gnu.org/software/gdb/documentation/>`__. | ||
1101 | |||
1102 | After running gdbserver on the target, you need to run Gdb on the | ||
1103 | host and configure it and connect to the target. Use these commands:: | ||
1104 | |||
1105 | $ cd directory-holding-the-debugfs-directory | ||
1106 | $ arch-gdb | ||
1107 | (gdb) set sysroot debugfs | ||
1108 | (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug | ||
1109 | (gdb) target remote IP-of-target:1234 | ||
1110 | |||
1111 | At this | ||
1112 | point, everything should automatically load (i.e. matching binaries, | ||
1113 | symbols and headers). | ||
1114 | |||
1115 | .. note:: | ||
1116 | |||
1117 | The Gdb ``set`` commands in the previous example can be placed into the | ||
1118 | users ``~/.gdbinit`` file. Upon starting, Gdb automatically runs whatever | ||
1119 | commands are in that file. | ||
1120 | |||
1121 | 8. *Deploying without a full image rebuild:* | ||
1122 | |||
1123 | In many cases, during development you want a quick method to deploy a | ||
1124 | new binary to the target and debug it, without waiting for a full | ||
1125 | image build. | ||
1126 | |||
1127 | One approach to solving this situation is to just build the component | ||
1128 | you want to debug. Once you have built the component, copy the | ||
1129 | executable directly to both the target and the host ``debugfs``. | ||
1130 | |||
1131 | If the binary is processed through the debug splitting in | ||
1132 | OpenEmbedded, you should also copy the debug items (i.e. ``.debug`` | ||
1133 | contents and corresponding ``/usr/src/debug`` files) from the work | ||
1134 | directory. Here is an example:: | ||
1135 | |||
1136 | $ bitbake bash | ||
1137 | $ bitbake -c devshell bash | ||
1138 | $ cd .. | ||
1139 | $ scp packages-split/bash/bin/bash target:/bin/bash | ||
1140 | $ cp -a packages-split/bash-dbg/\* path/debugfs | ||
1141 | |||
1142 | Debugging with the GNU Project Debugger (GDB) on the Target | ||
1143 | =========================================================== | ||
1144 | |||
1145 | The previous section addressed using GDB remotely for debugging | ||
1146 | purposes, which is the most usual case due to the inherent hardware | ||
1147 | limitations on many embedded devices. However, debugging in the target | ||
1148 | hardware itself is also possible with more powerful devices. This | ||
1149 | section describes what you need to do in order to support using GDB to | ||
1150 | debug on the target hardware. | ||
1151 | |||
1152 | To support this kind of debugging, you need do the following: | ||
1153 | |||
1154 | - Ensure that GDB is on the target. You can do this by making | ||
1155 | the following addition to your ``local.conf`` file:: | ||
1156 | |||
1157 | EXTRA_IMAGE_FEATURES:append = " tools-debug" | ||
1158 | |||
1159 | - Ensure that debug symbols are present. You can do so by adding the | ||
1160 | corresponding ``-dbg`` package to :term:`IMAGE_INSTALL`:: | ||
1161 | |||
1162 | IMAGE_INSTALL:append = " packagename-dbg" | ||
1163 | |||
1164 | Alternatively, you can add the following to ``local.conf`` to include | ||
1165 | all the debug symbols:: | ||
1166 | |||
1167 | EXTRA_IMAGE_FEATURES:append = " dbg-pkgs" | ||
1168 | |||
1169 | .. note:: | ||
1170 | |||
1171 | To improve the debug information accuracy, you can reduce the level | ||
1172 | of optimization used by the compiler. For example, when adding the | ||
1173 | following line to your ``local.conf`` file, you will reduce optimization | ||
1174 | from :term:`FULL_OPTIMIZATION` of "-O2" to :term:`DEBUG_OPTIMIZATION` | ||
1175 | of "-O -fno-omit-frame-pointer":: | ||
1176 | |||
1177 | DEBUG_BUILD = "1" | ||
1178 | |||
1179 | Consider that this will reduce the application's performance and is | ||
1180 | recommended only for debugging purposes. | ||
1181 | |||
1182 | Other Debugging Tips | ||
1183 | ==================== | ||
1184 | |||
1185 | Here are some other tips that you might find useful: | ||
1186 | |||
1187 | - When adding new packages, it is worth watching for undesirable items | ||
1188 | making their way into compiler command lines. For example, you do not | ||
1189 | want references to local system files like ``/usr/lib/`` or | ||
1190 | ``/usr/include/``. | ||
1191 | |||
1192 | - If you want to remove the ``psplash`` boot splashscreen, add | ||
1193 | ``psplash=false`` to the kernel command line. Doing so prevents | ||
1194 | ``psplash`` from loading and thus allows you to see the console. It | ||
1195 | is also possible to switch out of the splashscreen by switching the | ||
1196 | virtual console (e.g. Fn+Left or Fn+Right on a Zaurus). | ||
1197 | |||
1198 | - Removing :term:`TMPDIR` (usually ``tmp/``, within the | ||
1199 | :term:`Build Directory`) can often fix temporary build issues. Removing | ||
1200 | :term:`TMPDIR` is usually a relatively cheap operation, because task output | ||
1201 | will be cached in :term:`SSTATE_DIR` (usually ``sstate-cache/``, which is | ||
1202 | also in the :term:`Build Directory`). | ||
1203 | |||
1204 | .. note:: | ||
1205 | |||
1206 | Removing :term:`TMPDIR` might be a workaround rather than a fix. | ||
1207 | Consequently, trying to determine the underlying cause of an issue before | ||
1208 | removing the directory is a good idea. | ||
1209 | |||
1210 | - Understanding how a feature is used in practice within existing | ||
1211 | recipes can be very helpful. It is recommended that you configure | ||
1212 | some method that allows you to quickly search through files. | ||
1213 | |||
1214 | Using GNU Grep, you can use the following shell function to | ||
1215 | recursively search through common recipe-related files, skipping | ||
1216 | binary files, ``.git`` directories, and the :term:`Build Directory` | ||
1217 | (assuming its name starts with "build"):: | ||
1218 | |||
1219 | g() { | ||
1220 | grep -Ir \ | ||
1221 | --exclude-dir=.git \ | ||
1222 | --exclude-dir='build*' \ | ||
1223 | --include='*.bb*' \ | ||
1224 | --include='*.inc*' \ | ||
1225 | --include='*.conf*' \ | ||
1226 | --include='*.py*' \ | ||
1227 | "$@" | ||
1228 | } | ||
1229 | |||
1230 | Following are some usage examples:: | ||
1231 | |||
1232 | $ g FOO # Search recursively for "FOO" | ||
1233 | $ g -i foo # Search recursively for "foo", ignoring case | ||
1234 | $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" | ||
1235 | |||
1236 | If figuring | ||
1237 | out how some feature works requires a lot of searching, it might | ||
1238 | indicate that the documentation should be extended or improved. In | ||
1239 | such cases, consider filing a documentation bug using the Yocto | ||
1240 | Project implementation of | ||
1241 | :yocto_bugs:`Bugzilla <>`. For information on | ||
1242 | how to submit a bug against the Yocto Project, see the Yocto Project | ||
1243 | Bugzilla :yocto_wiki:`wiki page </Bugzilla_Configuration_and_Bug_Tracking>` | ||
1244 | and the | ||
1245 | ":ref:`dev-manual/changes:submitting a defect against the yocto project`" | ||
1246 | section. | ||
1247 | |||
1248 | .. note:: | ||
1249 | |||
1250 | The manuals might not be the right place to document variables | ||
1251 | that are purely internal and have a limited scope (e.g. internal | ||
1252 | variables used to implement a single ``.bbclass`` file). | ||
1253 | |||