diff options
author | Antonin Godard <antonin.godard@bootlin.com> | 2025-01-27 10:58:22 +0100 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2025-02-07 17:52:48 +0000 |
commit | cfe5da1f75e7b9e43c89cd5f93976f05bb12edfe (patch) | |
tree | 0c1cfbf8048a9fb31ac21d597a365624fc2b14e5 /documentation | |
parent | 21521c621895019e8fc46b4118f9cbb6abf8b5bb (diff) | |
download | poky-cfe5da1f75e7b9e43c89cd5f93976f05bb12edfe.tar.gz |
Move devtool doc from extensible to dev manual
[YOCTO #13417]
devtool is not directly tied to the Extensible SDK anymore, and
instructions in there can be split and added to the development manual.
This patch is a simple bulk move, except for the following:
- Intro added to the devtool.rst document (mostly copy-pasted from the
devtool quick reference in the ref-manual).
- Figures renamed to remove "sdk-" from filenames.
- References to sdk-manual/extensible replaced by dev-manual/devtool.
Reported-By: Paul Eggleton <bluelightning@bluelightning.org>
(From yocto-docs rev: 044d3185b858fce1febcfe3a6834b883f9a598fa)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
13 files changed, 1338 insertions, 1327 deletions
diff --git a/documentation/dev-manual/devtool.rst b/documentation/dev-manual/devtool.rst new file mode 100644 index 0000000000..2e3de7a096 --- /dev/null +++ b/documentation/dev-manual/devtool.rst | |||
@@ -0,0 +1,1327 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Using the ``devtool`` command-line tool | ||
4 | *************************************** | ||
5 | |||
6 | The ``devtool`` command-line tool provides a number of features that | ||
7 | help you build, test, and package software. This command is available | ||
8 | alongside the ``bitbake`` command. Additionally, the ``devtool`` command | ||
9 | is a key part of the :term:`Extensible Software Development Kit (eSDK)`. | ||
10 | |||
11 | Use ``devtool add`` to Add an Application | ||
12 | ========================================= | ||
13 | |||
14 | The ``devtool add`` command generates a new recipe based on existing | ||
15 | source code. This command takes advantage of the | ||
16 | :ref:`devtool-the-workspace-layer-structure` | ||
17 | layer that many ``devtool`` commands use. The command is flexible enough | ||
18 | to allow you to extract source code into both the workspace or a | ||
19 | separate local Git repository and to use existing code that does not | ||
20 | need to be extracted. | ||
21 | |||
22 | Depending on your particular scenario, the arguments and options you use | ||
23 | with ``devtool add`` form different combinations. The following diagram | ||
24 | shows common development flows you would use with the ``devtool add`` | ||
25 | command: | ||
26 | |||
27 | .. image:: figures/devtool-add-flow.png | ||
28 | :width: 100% | ||
29 | |||
30 | #. *Generating the New Recipe*: The top part of the flow shows three | ||
31 | scenarios by which you could use ``devtool add`` to generate a recipe | ||
32 | based on existing source code. | ||
33 | |||
34 | In a shared development environment, it is typical for other | ||
35 | developers to be responsible for various areas of source code. As a | ||
36 | developer, you are probably interested in using that source code as | ||
37 | part of your development within the Yocto Project. All you need is | ||
38 | access to the code, a recipe, and a controlled area in which to do | ||
39 | your work. | ||
40 | |||
41 | Within the diagram, three possible scenarios feed into the | ||
42 | ``devtool add`` workflow: | ||
43 | |||
44 | - *Left*: The left scenario in the figure represents a common | ||
45 | situation where the source code does not exist locally and needs | ||
46 | to be extracted. In this situation, the source code is extracted | ||
47 | to the default workspace --- you do not want the files in some | ||
48 | specific location outside of the workspace. Thus, everything you | ||
49 | need will be located in the workspace:: | ||
50 | |||
51 | $ devtool add recipe fetchuri | ||
52 | |||
53 | With this command, ``devtool`` extracts the upstream | ||
54 | source files into a local Git repository within the ``sources`` | ||
55 | folder. The command then creates a recipe named recipe and a | ||
56 | corresponding append file in the workspace. If you do not provide | ||
57 | recipe, the command makes an attempt to determine the recipe name. | ||
58 | |||
59 | - *Middle*: The middle scenario in the figure also represents a | ||
60 | situation where the source code does not exist locally. In this | ||
61 | case, the code is again upstream and needs to be extracted to some | ||
62 | local area --- this time outside of the default workspace. | ||
63 | |||
64 | .. note:: | ||
65 | |||
66 | If required, ``devtool`` always creates a Git repository locally | ||
67 | during the extraction. | ||
68 | |||
69 | Furthermore, the first positional argument ``srctree`` in this case | ||
70 | identifies where the ``devtool add`` command will locate the | ||
71 | extracted code outside of the workspace. You need to specify an | ||
72 | empty directory:: | ||
73 | |||
74 | $ devtool add recipe srctree fetchuri | ||
75 | |||
76 | In summary, the source code is pulled from fetchuri and extracted into the | ||
77 | location defined by ``srctree`` as a local Git repository. | ||
78 | |||
79 | Within workspace, ``devtool`` creates a recipe named recipe along | ||
80 | with an associated append file. | ||
81 | |||
82 | - *Right*: The right scenario in the figure represents a situation | ||
83 | where the ``srctree`` has been previously prepared outside of the | ||
84 | ``devtool`` workspace. | ||
85 | |||
86 | The following command provides a new recipe name and identifies | ||
87 | the existing source tree location:: | ||
88 | |||
89 | $ devtool add recipe srctree | ||
90 | |||
91 | The command examines the source code and creates a recipe named | ||
92 | recipe for the code and places the recipe into the workspace. | ||
93 | |||
94 | Because the extracted source code already exists, ``devtool`` does | ||
95 | not try to relocate the source code into the workspace --- only the | ||
96 | new recipe is placed in the workspace. | ||
97 | |||
98 | Aside from a recipe folder, the command also creates an associated | ||
99 | append folder and places an initial ``*.bbappend`` file within. | ||
100 | |||
101 | #. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the | ||
102 | editor as defined by the ``$EDITOR`` environment variable and modify | ||
103 | the file:: | ||
104 | |||
105 | $ devtool edit-recipe recipe | ||
106 | |||
107 | From within the editor, you can make modifications to the recipe that | ||
108 | take effect when you build it later. | ||
109 | |||
110 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
111 | depends on what you are going to do with the new code. | ||
112 | |||
113 | If you need to eventually move the build output to the target | ||
114 | hardware, use the following ``devtool`` command:: | ||
115 | |||
116 | $ devtool build recipe | ||
117 | |||
118 | On the other hand, if you want an image to contain the recipe's | ||
119 | packages from the workspace for immediate deployment onto a device | ||
120 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
121 | command:: | ||
122 | |||
123 | $ devtool build-image image | ||
124 | |||
125 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
126 | to build out your recipe, you probably want to see if the resulting | ||
127 | build output works as expected on the target hardware. | ||
128 | |||
129 | .. note:: | ||
130 | |||
131 | This step assumes you have a previously built image that is | ||
132 | already either running in QEMU or is running on actual hardware. | ||
133 | Also, it is assumed that for deployment of the image to the | ||
134 | target, SSH is installed in the image and, if the image is running | ||
135 | on real hardware, you have network access to and from your | ||
136 | development machine. | ||
137 | |||
138 | You can deploy your build output to that target hardware by using the | ||
139 | ``devtool deploy-target`` command:: | ||
140 | |||
141 | $ devtool deploy-target recipe target | ||
142 | |||
143 | The target is a live target machine running as an SSH server. | ||
144 | |||
145 | You can, of course, also deploy the image you build to actual | ||
146 | hardware by using the ``devtool build-image`` command. However, | ||
147 | ``devtool`` does not provide a specific command that allows you to | ||
148 | deploy the image to actual hardware. | ||
149 | |||
150 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
151 | creates any patches corresponding to commits in the local Git | ||
152 | repository, moves the new recipe to a more permanent layer, and then | ||
153 | resets the recipe so that the recipe is built normally rather than | ||
154 | from the workspace:: | ||
155 | |||
156 | $ devtool finish recipe layer | ||
157 | |||
158 | .. note:: | ||
159 | |||
160 | Any changes you want to turn into patches must be committed to the | ||
161 | Git repository in the source tree. | ||
162 | |||
163 | As mentioned, the ``devtool finish`` command moves the final recipe | ||
164 | to its permanent layer. | ||
165 | |||
166 | As a final process of the ``devtool finish`` command, the state of | ||
167 | the standard layers and the upstream source is restored so that you | ||
168 | can build the recipe from those areas rather than the workspace. | ||
169 | |||
170 | .. note:: | ||
171 | |||
172 | You can use the ``devtool reset`` command to put things back should you | ||
173 | decide you do not want to proceed with your work. If you do use this | ||
174 | command, realize that the source tree is preserved. | ||
175 | |||
176 | Use ``devtool modify`` to Modify the Source of an Existing Component | ||
177 | ==================================================================== | ||
178 | |||
179 | The ``devtool modify`` command prepares the way to work on existing code | ||
180 | that already has a local recipe in place that is used to build the | ||
181 | software. The command is flexible enough to allow you to extract code | ||
182 | from an upstream source, specify the existing recipe, and keep track of | ||
183 | and gather any patch files from other developers that are associated | ||
184 | with the code. | ||
185 | |||
186 | Depending on your particular scenario, the arguments and options you use | ||
187 | with ``devtool modify`` form different combinations. The following | ||
188 | diagram shows common development flows for the ``devtool modify`` | ||
189 | command: | ||
190 | |||
191 | .. image:: figures/devtool-modify-flow.png | ||
192 | :width: 100% | ||
193 | |||
194 | #. *Preparing to Modify the Code*: The top part of the flow shows three | ||
195 | scenarios by which you could use ``devtool modify`` to prepare to | ||
196 | work on source files. Each scenario assumes the following: | ||
197 | |||
198 | - The recipe exists locally in a layer external to the ``devtool`` | ||
199 | workspace. | ||
200 | |||
201 | - The source files exist either upstream in an un-extracted state or | ||
202 | locally in a previously extracted state. | ||
203 | |||
204 | The typical situation is where another developer has created a layer | ||
205 | for use with the Yocto Project and their recipe already resides in | ||
206 | that layer. Furthermore, their source code is readily available | ||
207 | either upstream or locally. | ||
208 | |||
209 | - *Left*: The left scenario in the figure represents a common | ||
210 | situation where the source code does not exist locally and it | ||
211 | needs to be extracted from an upstream source. In this situation, | ||
212 | the source is extracted into the default ``devtool`` workspace | ||
213 | location. The recipe, in this scenario, is in its own layer | ||
214 | outside the workspace (i.e. ``meta-``\ layername). | ||
215 | |||
216 | The following command identifies the recipe and, by default, | ||
217 | extracts the source files:: | ||
218 | |||
219 | $ devtool modify recipe | ||
220 | |||
221 | Once ``devtool`` locates the recipe, ``devtool`` uses the recipe's | ||
222 | :term:`SRC_URI` statements to locate the source code and any local | ||
223 | patch files from other developers. | ||
224 | |||
225 | With this scenario, there is no ``srctree`` argument. Consequently, the | ||
226 | default behavior of the ``devtool modify`` command is to extract | ||
227 | the source files pointed to by the :term:`SRC_URI` statements into a | ||
228 | local Git structure. Furthermore, the location for the extracted | ||
229 | source is the default area within the ``devtool`` workspace. The | ||
230 | result is that the command sets up both the source code and an | ||
231 | append file within the workspace while the recipe remains in its | ||
232 | original location. | ||
233 | |||
234 | Additionally, if you have any non-patch local files (i.e. files | ||
235 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
236 | excluding ``*.patch/`` or ``*.diff``), these files are copied to | ||
237 | an ``oe-local-files`` folder under the newly created source tree. | ||
238 | Copying the files here gives you a convenient area from which you | ||
239 | can modify the files. Any changes or additions you make to those | ||
240 | files are incorporated into the build the next time you build the | ||
241 | software just as are other changes you might have made to the | ||
242 | source. | ||
243 | |||
244 | - *Middle*: The middle scenario in the figure represents a situation | ||
245 | where the source code also does not exist locally. In this case, | ||
246 | the code is again upstream and needs to be extracted to some local | ||
247 | area as a Git repository. The recipe, in this scenario, is again | ||
248 | local and in its own layer outside the workspace. | ||
249 | |||
250 | The following command tells ``devtool`` the recipe with which to | ||
251 | work and, in this case, identifies a local area for the extracted | ||
252 | source files that exists outside of the default ``devtool`` | ||
253 | workspace:: | ||
254 | |||
255 | $ devtool modify recipe srctree | ||
256 | |||
257 | .. note:: | ||
258 | |||
259 | You cannot provide a URL for ``srctree`` using the ``devtool`` command. | ||
260 | |||
261 | As with all extractions, the command uses the recipe's :term:`SRC_URI` | ||
262 | statements to locate the source files and any associated patch | ||
263 | files. Non-patch files are copied to an ``oe-local-files`` folder | ||
264 | under the newly created source tree. | ||
265 | |||
266 | Once the files are located, the command by default extracts them | ||
267 | into ``srctree``. | ||
268 | |||
269 | Within workspace, ``devtool`` creates an append file for the | ||
270 | recipe. The recipe remains in its original location but the source | ||
271 | files are extracted to the location you provide with ``srctree``. | ||
272 | |||
273 | - *Right*: The right scenario in the figure represents a situation | ||
274 | where the source tree (``srctree``) already exists locally as a | ||
275 | previously extracted Git structure outside of the ``devtool`` | ||
276 | workspace. In this example, the recipe also exists elsewhere | ||
277 | locally in its own layer. | ||
278 | |||
279 | The following command tells ``devtool`` the recipe with which to | ||
280 | work, uses the "-n" option to indicate source does not need to be | ||
281 | extracted, and uses ``srctree`` to point to the previously extracted | ||
282 | source files:: | ||
283 | |||
284 | $ devtool modify -n recipe srctree | ||
285 | |||
286 | If an ``oe-local-files`` subdirectory happens to exist and it | ||
287 | contains non-patch files, the files are used. However, if the | ||
288 | subdirectory does not exist and you run the ``devtool finish`` | ||
289 | command, any non-patch files that might exist next to the recipe | ||
290 | are removed because it appears to ``devtool`` that you have | ||
291 | deleted those files. | ||
292 | |||
293 | Once the ``devtool modify`` command finishes, it creates only an | ||
294 | append file for the recipe in the ``devtool`` workspace. The | ||
295 | recipe and the source code remain in their original locations. | ||
296 | |||
297 | #. *Edit the Source*: Once you have used the ``devtool modify`` command, | ||
298 | you are free to make changes to the source files. You can use any | ||
299 | editor you like to make and save your source code modifications. | ||
300 | |||
301 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
302 | depends on what you are going to do with the new code. | ||
303 | |||
304 | If you need to eventually move the build output to the target | ||
305 | hardware, use the following ``devtool`` command:: | ||
306 | |||
307 | $ devtool build recipe | ||
308 | |||
309 | On the other hand, if you want an image to contain the recipe's | ||
310 | packages from the workspace for immediate deployment onto a device | ||
311 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
312 | command:: | ||
313 | |||
314 | $ devtool build-image image | ||
315 | |||
316 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
317 | to build out your recipe, you probably want to see if the resulting | ||
318 | build output works as expected on target hardware. | ||
319 | |||
320 | .. note:: | ||
321 | |||
322 | This step assumes you have a previously built image that is | ||
323 | already either running in QEMU or running on actual hardware. | ||
324 | Also, it is assumed that for deployment of the image to the | ||
325 | target, SSH is installed in the image and if the image is running | ||
326 | on real hardware that you have network access to and from your | ||
327 | development machine. | ||
328 | |||
329 | You can deploy your build output to that target hardware by using the | ||
330 | ``devtool deploy-target`` command:: | ||
331 | |||
332 | $ devtool deploy-target recipe target | ||
333 | |||
334 | The target is a live target machine running as an SSH server. | ||
335 | |||
336 | You can, of course, use other methods to deploy the image you built | ||
337 | using the ``devtool build-image`` command to actual hardware. | ||
338 | ``devtool`` does not provide a specific command to deploy the image | ||
339 | to actual hardware. | ||
340 | |||
341 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
342 | creates any patches corresponding to commits in the local Git | ||
343 | repository, updates the recipe to point to them (or creates a | ||
344 | ``.bbappend`` file to do so, depending on the specified destination | ||
345 | layer), and then resets the recipe so that the recipe is built | ||
346 | normally rather than from the workspace:: | ||
347 | |||
348 | $ devtool finish recipe layer | ||
349 | |||
350 | .. note:: | ||
351 | |||
352 | Any changes you want to turn into patches must be staged and | ||
353 | committed within the local Git repository before you use the | ||
354 | ``devtool finish`` command. | ||
355 | |||
356 | Because there is no need to move the recipe, ``devtool finish`` | ||
357 | either updates the original recipe in the original layer or the | ||
358 | command creates a ``.bbappend`` file in a different layer as provided | ||
359 | by layer. Any work you did in the ``oe-local-files`` directory is | ||
360 | preserved in the original files next to the recipe during the | ||
361 | ``devtool finish`` command. | ||
362 | |||
363 | As a final process of the ``devtool finish`` command, the state of | ||
364 | the standard layers and the upstream source is restored so that you | ||
365 | can build the recipe from those areas rather than from the workspace. | ||
366 | |||
367 | .. note:: | ||
368 | |||
369 | You can use the ``devtool reset`` command to put things back should you | ||
370 | decide you do not want to proceed with your work. If you do use this | ||
371 | command, realize that the source tree is preserved. | ||
372 | |||
373 | ``devtool ide-sdk`` configures IDEs and bootstraps SDKs | ||
374 | ======================================================= | ||
375 | |||
376 | The ``devtool ide-sdk`` command can provide an IDE configuration for IDEs when | ||
377 | working on the source code of one or more recipes. | ||
378 | Depending on the programming language, and the build system used by the recipe, | ||
379 | the tools required for cross-development and remote debugging are different. | ||
380 | For example: | ||
381 | |||
382 | - A C/C++ project usually uses CMake or Meson. | ||
383 | |||
384 | - A Python project uses setuptools or one of its successors. | ||
385 | |||
386 | - A Rust project uses Cargo. | ||
387 | |||
388 | Also, the IDE plugins needed for the integration of a build system with the | ||
389 | IDE and the corresponding settings are usually specific to these build-systems. | ||
390 | To hide all these details from the user, ``devtool ide-sdk`` does two things: | ||
391 | |||
392 | - It generates any kind of SDK needed for cross-development and remote | ||
393 | debugging of the specified recipes. | ||
394 | |||
395 | - It generates the configuration for the IDE (and the IDE plugins) for using | ||
396 | the cross-toolchain and remote debugging tools provided by the SDK directly | ||
397 | from the IDE. | ||
398 | |||
399 | For supported build systems the configurations generated by ``devtool ide-sdk`` | ||
400 | combine the advantages of the ``devtool modify`` based workflow | ||
401 | (see :ref:`using_devtool`) with the advantages of the simple Environment Setup | ||
402 | script based workflow (see :ref:`running_the_ext_sdk_env`) provided by Yocto's | ||
403 | SDK or eSDK: | ||
404 | |||
405 | - The source code of the recipe is in the workspace created by | ||
406 | ``devtool modify`` or ``devtool add``. | ||
407 | Using ``devtool build``, ``devtool build-image``, | ||
408 | ``devtool deploy-target`` or ``bitbake`` is possible. | ||
409 | Also ``devtool ide-sdk`` can be used to update the SDK and the IDE | ||
410 | configuration at any time. | ||
411 | |||
412 | - ``devtool ide-sdk`` aims to support multiple programming languages and | ||
413 | multiple IDEs natively. "Natively" means that the IDE is configured to call | ||
414 | the build tool (e.g. ``cmake`` or ``meson``) directly. This has several | ||
415 | advantages. | ||
416 | First of all, it is usually much faster to call for example ``cmake`` than | ||
417 | ``devtool build``. | ||
418 | It also allows to benefit from the very good integration that IDEs like | ||
419 | VSCode offer for tools like CMake or GDB. | ||
420 | |||
421 | However, supporting many programming languages and multiple | ||
422 | IDEs is quite an elaborate and constantly evolving thing. Support for IDEs | ||
423 | is therefore implemented as plugins. Plugins can also be provided by | ||
424 | optional layers. | ||
425 | |||
426 | So much about the introduction to the default mode of ``devtool sdk-ide`` which | ||
427 | is called the "modified" mode because it uses the workspace created by | ||
428 | ``devtool modify`` and the per recipe :term:`Sysroots <Sysroot>` of BitBake. | ||
429 | |||
430 | For some recipes and use cases, this default behavior of ``devtool ide-sdk`` | ||
431 | with full ``devtool`` and ``bitbake`` integration might not be suitable. | ||
432 | To offer full feature parity with the SDK and the eSDK, ``devtool ide-sdk`` has | ||
433 | a second mode called "shared" mode. | ||
434 | If ``devtool ide-sdk`` is called with the ``--mode=shared`` option, it | ||
435 | bootstraps an SDK directly from the BitBake environment, which offers the same | ||
436 | Environment Setup script as described in :ref:`running_the_ext_sdk_env`. | ||
437 | In addition to the (e)SDK installer-based setup, the IDE gets configured | ||
438 | to use the shared :term:`Sysroots <Sysroot>` and the tools from the SDK. | ||
439 | ``devtool ide-sdk --mode=shared`` is basically a wrapper for the setup of the | ||
440 | extensible SDK as described in :ref:`setting_up_ext_sdk_in_build`. | ||
441 | |||
442 | The use of ``devtool ide-sdk`` is an alternative to using one of the SDK | ||
443 | installers. | ||
444 | ``devtool ide-sdk`` allows the creation of SDKs that offer all the | ||
445 | functionality of the SDK and the eSDK installers. Compared to the installers, | ||
446 | however, the SDK created with ``devtool ide-sdk`` is much more flexible. | ||
447 | For example, it is very easy to change the :term:`MACHINE` in the | ||
448 | ``local.conf`` file, update the layer meta data and then regenerate the SDK. | ||
449 | |||
450 | Let's take a look at an example of how to use ``devtool ide-sdk`` in each of | ||
451 | the two modes: | ||
452 | |||
453 | #. *Modified mode*: | ||
454 | |||
455 | In order to use the ``devtool ide-sdk``, a few settings are needed. As a | ||
456 | starting example, the following lines of code can be added to the | ||
457 | ``local.conf`` file:: | ||
458 | |||
459 | # Build the companion debug file system | ||
460 | IMAGE_GEN_DEBUGFS = "1" | ||
461 | # Optimize build time: with devtool ide-sdk the dbg tar is not needed | ||
462 | IMAGE_FSTYPES_DEBUGFS = "" | ||
463 | # Without copying the binaries into roofs-dbg, GDB does not find all source files. | ||
464 | IMAGE_CLASSES += "image-combined-dbg" | ||
465 | |||
466 | # SSH is mandatory, no password simplifies the usage | ||
467 | EXTRA_IMAGE_FEATURES += "\ | ||
468 | ssh-server-openssh \ | ||
469 | allow-empty-password \ | ||
470 | allow-root-login \ | ||
471 | empty-root-password \ | ||
472 | " | ||
473 | |||
474 | # Remote debugging needs gdbserver on the target device | ||
475 | IMAGE_INSTALL:append = " gdbserver" | ||
476 | |||
477 | # Add the recipes which should be modified to the image | ||
478 | # Otherwise some dependencies might be missing. | ||
479 | IMAGE_INSTALL:append = " my-recipe" | ||
480 | |||
481 | Assuming the BitBake environment is set up correctly and a workspace has | ||
482 | been created for the recipe using ``devtool modify my-recipe`` or probably | ||
483 | even better by using ``devtool modify my-recipe --debug-build``, the | ||
484 | following command can create the SDK and the configuration for VSCode in | ||
485 | the recipe workspace:: | ||
486 | |||
487 | $ devtool ide-sdk my-recipe core-image-minimal --target root@192.168.7.2 | ||
488 | |||
489 | The command requires an image recipe (``core-image-minimal`` for this | ||
490 | example) that is used to create the SDK. | ||
491 | This firmware image should also be installed on the target device. | ||
492 | It is possible to pass multiple package recipes:: | ||
493 | |||
494 | $ devtool ide-sdk my-recipe-1 my-recipe-2 core-image-minimal --target root@192.168.7.2 | ||
495 | |||
496 | ``devtool ide-sdk`` tries to create an IDE configuration for all package | ||
497 | recipes. | ||
498 | |||
499 | What this command does exactly depends on the recipe, more precisely on the | ||
500 | build tool used by the recipe. The basic idea is to configure the IDE so | ||
501 | that it calls the build tool exactly as ``bitbake`` does. | ||
502 | |||
503 | For example, a CMake preset is created for a recipe that inherits | ||
504 | :ref:`ref-classes-cmake`. In the case of VSCode, CMake presets are supported | ||
505 | by the CMake Tools plugin. This is an example of how the build configuration | ||
506 | used by ``bitbake`` is exported to an IDE configuration that gives exactly | ||
507 | the same build results. | ||
508 | |||
509 | Support for remote debugging with seamless integration into the IDE is | ||
510 | important for a cross-SDK. ``devtool ide-sdk`` automatically generates the | ||
511 | necessary helper scripts for deploying the compiled artifacts to the target | ||
512 | device as well as the necessary configuration for the debugger and the IDE. | ||
513 | |||
514 | .. note:: | ||
515 | |||
516 | To ensure that the debug symbols on the build machine match the binaries | ||
517 | running on the target device, it is essential that the image built by | ||
518 | ``devtool ide-sdk`` is running on the target device. | ||
519 | |||
520 | The default IDE is VSCode. Some hints about using VSCode: | ||
521 | |||
522 | - VSCode can be used to work on the BitBake recipes or the application | ||
523 | source code. | ||
524 | Usually there is one instance of VSCode running in the folder where the | ||
525 | BitBake recipes are. This instance has the | ||
526 | `Yocto Project BitBake plugin <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`_ | ||
527 | running. | ||
528 | |||
529 | .. warning:: | ||
530 | |||
531 | Some VSCode plugins (Python, BitBake and others) need a reasonable | ||
532 | configuration to work as expected. Otherwise, some plugins try to | ||
533 | index the build directory of BitBake, which keeps your system quite | ||
534 | busy until an out of memory exception stops this nonsense. | ||
535 | Other plugins, such as the BitBake plugin, do not behave as expected. | ||
536 | |||
537 | To work around such issues, the ``oe-init-build-env`` script creates | ||
538 | an initial ``.vscode/settings.json`` file if ``code`` can be found | ||
539 | and the ``.vscode`` folder does not yet exist. | ||
540 | It is best to run ``oe-init-build-env`` once before starting VSCode. | ||
541 | An alternative approach is to use a build folder outside the layers, | ||
542 | e.g. ``oe-init-build-env ../build``. | ||
543 | |||
544 | The BitBake plugin also offers to create devtool workspaces and run | ||
545 | ``devtool ide-sdk`` with a few mouse clicks. | ||
546 | Of course, issuing commands in the terminal works as well. | ||
547 | |||
548 | - To work on the source code of a recipe another instance of VSCode is | ||
549 | started in the recipe's workspace. Example:: | ||
550 | |||
551 | code build/workspace/sources/my-recipe | ||
552 | |||
553 | This instance of VSCode uses plugins that are useful for the development | ||
554 | of the application. ``devtool ide-sdk`` generates the necessary | ||
555 | ``extensions.json``, ``settings.json``, ``tasks.json``and ``launch.json`` | ||
556 | configuration files for all the involved plugins. | ||
557 | |||
558 | When the source code folder present in the workspace folder is opened in | ||
559 | VSCode for the first time, a pop-up message recommends installing the | ||
560 | required plugins. | ||
561 | After accepting the installation of the plugins, working with the source | ||
562 | code or some debugging tasks should work as usual with VSCode. | ||
563 | |||
564 | Starting the VSCode instances in the recipe workspace folders can also be | ||
565 | done by a mouse click on the recipe workspaces in the first VSCode | ||
566 | instance. | ||
567 | |||
568 | - To work with CMake press ``Ctrl + Shift + p``, type ``cmake``. This will | ||
569 | show some possible commands like selecting a CMake preset, compiling or | ||
570 | running CTest. | ||
571 | |||
572 | For recipes inheriting :ref:`ref-classes-cmake-qemu` rather than | ||
573 | :ref:`ref-classes-cmake`, executing cross-compiled unit tests on the host | ||
574 | can be supported transparently with QEMU user-mode. | ||
575 | |||
576 | - To work with Meson press ``Ctrl + Shift + p``, type ``meson``. This will | ||
577 | show some possible commands like compiling or executing the unit tests. | ||
578 | |||
579 | A note on running cross-compiled unit tests on the host: Meson enables | ||
580 | support for QEMU user mode by default. It is expected that the execution | ||
581 | of the unit tests from the IDE will work without any additional steps, | ||
582 | given that the code is suitable for the execution on the host machine. | ||
583 | |||
584 | - For the deployment to the target device, just press ``Ctrl + Shift + p``, | ||
585 | type ``task``. Select ``install && deploy-target``. | ||
586 | |||
587 | - For remote debugging, switch to the debugging view by pressing the "play" | ||
588 | button with the ``bug icon`` on the left side. This will provide a green | ||
589 | play button with a drop-down list where a debug configuration can be | ||
590 | selected. After selecting one of the generated configurations, press the | ||
591 | "play" button. | ||
592 | |||
593 | Starting a remote debugging session automatically initiates the | ||
594 | deployment to the target device. If this is not desired, the | ||
595 | ``"dependsOn": ["install && deploy-target...]`` parameter of the tasks | ||
596 | with ``"label": "gdbserver start...`` can be removed from the | ||
597 | ``tasks.json`` file. | ||
598 | |||
599 | VSCode supports GDB with many different setups and configurations for | ||
600 | many different use cases. However, most of these setups have some | ||
601 | limitations when it comes to cross-development, support only a few target | ||
602 | architectures or require a high performance target device. Therefore | ||
603 | ``devtool ide-sdk`` supports the classic, generic setup with GDB on the | ||
604 | development host and gdbserver on the target device. | ||
605 | |||
606 | Roughly summarized, this means: | ||
607 | |||
608 | - The binaries are copied via SSH to the remote target device by a | ||
609 | script referred by ``tasks.json``. | ||
610 | |||
611 | - gdbserver is started on the remote target device via SSH by a script | ||
612 | referred by ``tasks.json``. | ||
613 | |||
614 | Changing the parameters that are passed to the debugging executable | ||
615 | requires modifying the generated script. The script is located at | ||
616 | ``oe-scripts/gdbserver_*``. Defining the parameters in the ``args`` | ||
617 | field in the ``launch.json`` file does not work. | ||
618 | |||
619 | - VSCode connects to gdbserver as documented in | ||
620 | `Remote debugging or debugging with a local debugger server | ||
621 | <https://code.visualstudio.com/docs/cpp/launch-json-reference#_remote-debugging-or-debugging-with-a-local-debugger-server>`__. | ||
622 | |||
623 | Additionally ``--ide=none`` is supported. With the ``none`` IDE parameter, | ||
624 | some generic configuration files like ``gdbinit`` files and some helper | ||
625 | scripts starting gdbserver remotely on the target device as well as the GDB | ||
626 | client on the host are generated. | ||
627 | |||
628 | Here is a usage example for the ``cmake-example`` recipe from the | ||
629 | ``meta-selftest`` layer which inherits :ref:`ref-classes-cmake-qemu`: | ||
630 | |||
631 | .. code-block:: sh | ||
632 | |||
633 | # Create the SDK | ||
634 | devtool modify cmake-example --debug-build | ||
635 | devtool ide-sdk cmake-example core-image-minimal -c --ide=none | ||
636 | |||
637 | # Install the firmware on a target device or start QEMU | ||
638 | runqemu | ||
639 | |||
640 | # From exploring the workspace of cmake-example | ||
641 | cd build/workspace/sources/cmake-example | ||
642 | |||
643 | # Find cmake-native and save the path into a variable | ||
644 | # Note: using just cmake instead of $CMAKE_NATIVE would work in many cases | ||
645 | CMAKE_NATIVE="$(jq -r '.configurePresets[0] | "\(.cmakeExecutable)"' CMakeUserPresets.json)" | ||
646 | |||
647 | # List available CMake presets | ||
648 | "$CMAKE_NATIVE" --list-presets | ||
649 | Available configure presets: | ||
650 | |||
651 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
652 | |||
653 | # Re-compile the already compiled sources | ||
654 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 | ||
655 | ninja: no work to do. | ||
656 | # Do a clean re-build | ||
657 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target clean | ||
658 | [1/1] Cleaning all built files... | ||
659 | Cleaning... 8 files. | ||
660 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target all | ||
661 | [7/7] Linking CXX executable cmake-example | ||
662 | |||
663 | # Run the cross-compiled unit tests with QEMU user-mode | ||
664 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target test | ||
665 | [0/1] Running tests... | ||
666 | Test project .../build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
667 | Start 1: test-cmake-example | ||
668 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
669 | |||
670 | 100% tests passed, 0 tests failed out of 1 | ||
671 | |||
672 | Total Test time (real) = 0.03 sec | ||
673 | |||
674 | # Using CTest directly is possible as well | ||
675 | CTEST_NATIVE="$(dirname "$CMAKE_NATIVE")/ctest" | ||
676 | |||
677 | # List available CMake presets | ||
678 | "$CTEST_NATIVE" --list-presets | ||
679 | Available test presets: | ||
680 | |||
681 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
682 | |||
683 | # Run the cross-compiled unit tests with QEMU user-mode | ||
684 | "$CTEST_NATIVE" --preset "cmake-example-cortexa57" | ||
685 | Test project ...build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
686 | Start 1: test-cmake-example | ||
687 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
688 | |||
689 | 100% tests passed, 0 tests failed out of 1 | ||
690 | |||
691 | Total Test time (real) = 0.03 sec | ||
692 | |||
693 | # Deploying the new build to the target device (default is QEUM at 192.168.7.2) | ||
694 | oe-scripts/install_and_deploy_cmake-example-cortexa57 | ||
695 | |||
696 | # Start a remote debugging session with gdbserver on the target and GDB on the host | ||
697 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m | ||
698 | oe-scripts/gdb_1234_usr-bin-cmake-example | ||
699 | break main | ||
700 | run | ||
701 | step | ||
702 | stepi | ||
703 | continue | ||
704 | quit | ||
705 | |||
706 | # Stop gdbserver on the target device | ||
707 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m stop | ||
708 | |||
709 | #. *Shared sysroots mode* | ||
710 | |||
711 | Creating an SDK with shared :term:`Sysroots <Sysroot>` that contains all the | ||
712 | dependencies needed to work with ``my-recipe`` is possible with the following | ||
713 | example command:: | ||
714 | |||
715 | $ devtool ide-sdk --mode=shared my-recipe | ||
716 | |||
717 | For VSCode the cross-toolchain is exposed as a CMake kit. CMake kits are | ||
718 | defined in ``~/.local/share/CMakeTools/cmake-tools-kits.json``. | ||
719 | The following example shows how the cross-toolchain can be selected in | ||
720 | VSCode. First of all we need a folder containing a CMake project. | ||
721 | For this example, let's create a CMake project and start VSCode:: | ||
722 | |||
723 | mkdir kit-test | ||
724 | echo "project(foo VERSION 1.0)" > kit-test/CMakeLists.txt | ||
725 | code kit-test | ||
726 | |||
727 | If there is a CMake project in the workspace, cross-compilation is | ||
728 | supported: | ||
729 | |||
730 | - Press ``Ctrl + Shift + P``, type ``CMake: Scan for Kits`` | ||
731 | - Press ``Ctrl + Shift + P``, type ``CMake: Select a Kit`` | ||
732 | |||
733 | Finally most of the features provided by CMake and the IDE should be | ||
734 | available. | ||
735 | |||
736 | Other IDEs than VSCode are supported as well. However, | ||
737 | ``devtool ide-sdk --mode=shared --ide=none my-recipe`` is currently | ||
738 | just a simple wrapper for the setup of the extensible SDK, as described in | ||
739 | :ref:`setting_up_ext_sdk_in_build`. | ||
740 | |||
741 | Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software | ||
742 | ======================================================================================================= | ||
743 | |||
744 | The ``devtool upgrade`` command upgrades an existing recipe to that of a | ||
745 | more up-to-date version found upstream. Throughout the life of software, | ||
746 | recipes continually undergo version upgrades by their upstream | ||
747 | publishers. You can use the ``devtool upgrade`` workflow to make sure | ||
748 | your recipes you are using for builds are up-to-date with their upstream | ||
749 | counterparts. | ||
750 | |||
751 | .. note:: | ||
752 | |||
753 | Several methods exist by which you can upgrade recipes --- | ||
754 | ``devtool upgrade`` happens to be one. You can read about all the methods by | ||
755 | which you can upgrade recipes in the | ||
756 | :ref:`dev-manual/upgrading-recipes:upgrading recipes` section of the Yocto | ||
757 | Project Development Tasks Manual. | ||
758 | |||
759 | The ``devtool upgrade`` command is flexible enough to allow you to specify | ||
760 | source code revision and versioning schemes, extract code into or out of the | ||
761 | ``devtool`` :ref:`devtool-the-workspace-layer-structure`, and work with any | ||
762 | source file forms that the | ||
763 | :ref:`bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support. | ||
764 | |||
765 | The following diagram shows the common development flow used with the | ||
766 | ``devtool upgrade`` command: | ||
767 | |||
768 | .. image:: figures/devtool-upgrade-flow.png | ||
769 | :width: 100% | ||
770 | |||
771 | #. *Initiate the Upgrade*: The top part of the flow shows the typical | ||
772 | scenario by which you use the ``devtool upgrade`` command. The | ||
773 | following conditions exist: | ||
774 | |||
775 | - The recipe exists in a local layer external to the ``devtool`` | ||
776 | workspace. | ||
777 | |||
778 | - The source files for the new release exist in the same location | ||
779 | pointed to by :term:`SRC_URI` | ||
780 | in the recipe (e.g. a tarball with the new version number in the | ||
781 | name, or as a different revision in the upstream Git repository). | ||
782 | |||
783 | A common situation is where third-party software has undergone a | ||
784 | revision so that it has been upgraded. The recipe you have access to | ||
785 | is likely in your own layer. Thus, you need to upgrade the recipe to | ||
786 | use the newer version of the software:: | ||
787 | |||
788 | $ devtool upgrade -V version recipe | ||
789 | |||
790 | By default, the ``devtool upgrade`` command extracts source | ||
791 | code into the ``sources`` directory in the | ||
792 | :ref:`devtool-the-workspace-layer-structure`. | ||
793 | If you want the code extracted to any other location, you need to | ||
794 | provide the ``srctree`` positional argument with the command as follows:: | ||
795 | |||
796 | $ devtool upgrade -V version recipe srctree | ||
797 | |||
798 | .. note:: | ||
799 | |||
800 | In this example, the "-V" option specifies the new version. If you | ||
801 | don't use "-V", the command upgrades the recipe to the latest | ||
802 | version. | ||
803 | |||
804 | If the source files pointed to by the :term:`SRC_URI` statement in the | ||
805 | recipe are in a Git repository, you must provide the "-S" option and | ||
806 | specify a revision for the software. | ||
807 | |||
808 | Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable | ||
809 | to locate the source code and any local patch files from other | ||
810 | developers. The result is that the command sets up the source code, | ||
811 | the new version of the recipe, and an append file all within the | ||
812 | workspace. | ||
813 | |||
814 | Additionally, if you have any non-patch local files (i.e. files | ||
815 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
816 | excluding ``*.patch/`` or ``*.diff``), these files are copied to an | ||
817 | ``oe-local-files`` folder under the newly created source tree. | ||
818 | Copying the files here gives you a convenient area from which you can | ||
819 | modify the files. Any changes or additions you make to those files | ||
820 | are incorporated into the build the next time you build the software | ||
821 | just as are other changes you might have made to the source. | ||
822 | |||
823 | #. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen | ||
824 | after upgrading the software to a new version. Conflicts occur | ||
825 | if your recipe specifies some patch files in :term:`SRC_URI` that | ||
826 | conflict with changes made in the new version of the software. For | ||
827 | such cases, you need to resolve the conflicts by editing the source | ||
828 | and following the normal ``git rebase`` conflict resolution process. | ||
829 | |||
830 | Before moving onto the next step, be sure to resolve any such | ||
831 | conflicts created through use of a newer or different version of the | ||
832 | software. | ||
833 | |||
834 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
835 | depends on what you are going to do with the new code. | ||
836 | |||
837 | If you need to eventually move the build output to the target | ||
838 | hardware, use the following ``devtool`` command:: | ||
839 | |||
840 | $ devtool build recipe | ||
841 | |||
842 | On the other hand, if you want an image to contain the recipe's | ||
843 | packages from the workspace for immediate deployment onto a device | ||
844 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
845 | command:: | ||
846 | |||
847 | $ devtool build-image image | ||
848 | |||
849 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
850 | or ``bitbake`` to build your recipe, you probably want to see if the | ||
851 | resulting build output works as expected on target hardware. | ||
852 | |||
853 | .. note:: | ||
854 | |||
855 | This step assumes you have a previously built image that is | ||
856 | already either running in QEMU or running on actual hardware. | ||
857 | Also, it is assumed that for deployment of the image to the | ||
858 | target, SSH is installed in the image and if the image is running | ||
859 | on real hardware that you have network access to and from your | ||
860 | development machine. | ||
861 | |||
862 | You can deploy your build output to that target hardware by using the | ||
863 | ``devtool deploy-target`` command:: | ||
864 | |||
865 | $ devtool deploy-target recipe target | ||
866 | |||
867 | The target is a live target machine running as an SSH server. | ||
868 | |||
869 | You can, of course, also deploy the image you build using the | ||
870 | ``devtool build-image`` command to actual hardware. However, | ||
871 | ``devtool`` does not provide a specific command that allows you to do | ||
872 | this. | ||
873 | |||
874 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
875 | creates any patches corresponding to commits in the local Git | ||
876 | repository, moves the new recipe to a more permanent layer, and then | ||
877 | resets the recipe so that the recipe is built normally rather than | ||
878 | from the workspace. | ||
879 | |||
880 | Any work you did in the ``oe-local-files`` directory is preserved in | ||
881 | the original files next to the recipe during the ``devtool finish`` | ||
882 | command. | ||
883 | |||
884 | If you specify a destination layer that is the same as the original | ||
885 | source, then the old version of the recipe and associated files are | ||
886 | removed prior to adding the new version:: | ||
887 | |||
888 | $ devtool finish recipe layer | ||
889 | |||
890 | .. note:: | ||
891 | |||
892 | Any changes you want to turn into patches must be committed to the | ||
893 | Git repository in the source tree. | ||
894 | |||
895 | As a final process of the ``devtool finish`` command, the state of | ||
896 | the standard layers and the upstream source is restored so that you | ||
897 | can build the recipe from those areas rather than the workspace. | ||
898 | |||
899 | .. note:: | ||
900 | |||
901 | You can use the ``devtool reset`` command to put things back should you | ||
902 | decide you do not want to proceed with your work. If you do use this | ||
903 | command, realize that the source tree is preserved. | ||
904 | |||
905 | A Closer Look at ``devtool add`` | ||
906 | ================================ | ||
907 | |||
908 | The ``devtool add`` command automatically creates a recipe based on the | ||
909 | source tree you provide with the command. Currently, the command has | ||
910 | support for the following: | ||
911 | |||
912 | - Autotools (``autoconf`` and ``automake``) | ||
913 | |||
914 | - CMake | ||
915 | |||
916 | - Scons | ||
917 | |||
918 | - ``qmake`` | ||
919 | |||
920 | - Plain ``Makefile`` | ||
921 | |||
922 | - Out-of-tree kernel module | ||
923 | |||
924 | - Binary package (i.e. "-b" option) | ||
925 | |||
926 | - Node.js module | ||
927 | |||
928 | - Python modules that use ``setuptools`` or ``distutils`` | ||
929 | |||
930 | Apart from binary packages, the determination of how a source tree | ||
931 | should be treated is automatic based on the files present within that | ||
932 | source tree. For example, if a ``CMakeLists.txt`` file is found, then | ||
933 | the source tree is assumed to be using CMake and is treated accordingly. | ||
934 | |||
935 | .. note:: | ||
936 | |||
937 | In most cases, you need to edit the automatically generated recipe in | ||
938 | order to make it build properly. Typically, you would go through | ||
939 | several edit and build cycles until the recipe successfully builds. | ||
940 | Once the recipe builds, you could use possible further iterations to | ||
941 | test the recipe on the target device. | ||
942 | |||
943 | The remainder of this section covers specifics regarding how parts of | ||
944 | the recipe are generated. | ||
945 | |||
946 | Name and Version | ||
947 | ---------------- | ||
948 | |||
949 | If you do not specify a name and version on the command line, | ||
950 | ``devtool add`` uses various metadata within the source tree in an | ||
951 | attempt to determine the name and version of the software being built. | ||
952 | Based on what the tool determines, ``devtool`` sets the name of the | ||
953 | created recipe file accordingly. | ||
954 | |||
955 | If ``devtool`` cannot determine the name and version, the command prints | ||
956 | an error. For such cases, you must re-run the command and provide the | ||
957 | name and version, just the name, or just the version as part of the | ||
958 | command line. | ||
959 | |||
960 | Sometimes the name or version determined from the source tree might be | ||
961 | incorrect. For such a case, you must reset the recipe:: | ||
962 | |||
963 | $ devtool reset -n recipename | ||
964 | |||
965 | After running the ``devtool reset`` command, you need to | ||
966 | run ``devtool add`` again and provide the name or the version. | ||
967 | |||
968 | Dependency Detection and Mapping | ||
969 | -------------------------------- | ||
970 | |||
971 | The ``devtool add`` command attempts to detect build-time dependencies and map | ||
972 | them to other recipes in the system. During this mapping, the command fills in | ||
973 | the names of those recipes as part of the :term:`DEPENDS` variable within the | ||
974 | recipe. If a dependency cannot be mapped, ``devtool`` places a comment | ||
975 | in the recipe indicating such. The inability to map a dependency can | ||
976 | result from naming not being recognized or because the dependency simply | ||
977 | is not available. For cases where the dependency is not available, you | ||
978 | must use the ``devtool add`` command to add an additional recipe that | ||
979 | satisfies the dependency. Once you add that recipe, you need to update | ||
980 | the :term:`DEPENDS` variable in the original recipe to include the new | ||
981 | recipe. | ||
982 | |||
983 | If you need to add runtime dependencies, you can do so by adding the | ||
984 | following to your recipe:: | ||
985 | |||
986 | RDEPENDS:${PN} += "dependency1 dependency2 ..." | ||
987 | |||
988 | .. note:: | ||
989 | |||
990 | The ``devtool add`` command often cannot distinguish between mandatory and | ||
991 | optional dependencies. Consequently, some of the detected dependencies might | ||
992 | in fact be optional. When in doubt, consult the documentation or the | ||
993 | configure script for the software the recipe is building for further | ||
994 | details. In some cases, you might find you can substitute the | ||
995 | dependency with an option that disables the associated functionality | ||
996 | passed to the configure script. | ||
997 | |||
998 | License Detection | ||
999 | ----------------- | ||
1000 | |||
1001 | The ``devtool add`` command attempts to determine if the software you are | ||
1002 | adding is able to be distributed under a common, open-source license. If | ||
1003 | so, the command sets the :term:`LICENSE` value accordingly. | ||
1004 | You should double-check the value added by the command against the | ||
1005 | documentation or source files for the software you are building and, if | ||
1006 | necessary, update that :term:`LICENSE` value. | ||
1007 | |||
1008 | The ``devtool add`` command also sets the :term:`LIC_FILES_CHKSUM` | ||
1009 | value to point to all files that appear to be license-related. Realize | ||
1010 | that license statements often appear in comments at the top of source | ||
1011 | files or within the documentation. In such cases, the command does not | ||
1012 | recognize those license statements. Consequently, you might need to | ||
1013 | amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those | ||
1014 | comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly | ||
1015 | important for third-party software. The mechanism attempts to ensure | ||
1016 | correct licensing should you upgrade the recipe to a newer upstream | ||
1017 | version in future. Any change in licensing is detected and you receive | ||
1018 | an error prompting you to check the license text again. | ||
1019 | |||
1020 | If the ``devtool add`` command cannot determine licensing information, | ||
1021 | ``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the | ||
1022 | :term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue | ||
1023 | with development even though the settings are unlikely to be correct in | ||
1024 | all cases. You should check the documentation or source files for the | ||
1025 | software you are building to determine the actual license. | ||
1026 | |||
1027 | Adding Makefile-Only Software | ||
1028 | ----------------------------- | ||
1029 | |||
1030 | The use of Make by itself is very common in both proprietary and | ||
1031 | open-source software. Unfortunately, Makefiles are often not written | ||
1032 | with cross-compilation in mind. Thus, ``devtool add`` often cannot do | ||
1033 | very much to ensure that these Makefiles build correctly. It is very | ||
1034 | common, for example, to explicitly call ``gcc`` instead of using the | ||
1035 | :term:`CC` variable. Usually, in a | ||
1036 | cross-compilation environment, ``gcc`` is the compiler for the build | ||
1037 | host and the cross-compiler is named something similar to | ||
1038 | ``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to | ||
1039 | point to the associated sysroot for the target machine). | ||
1040 | |||
1041 | When writing a recipe for Makefile-only software, keep the following in | ||
1042 | mind: | ||
1043 | |||
1044 | - You probably need to patch the Makefile to use variables instead of | ||
1045 | hardcoding tools within the toolchain such as ``gcc`` and ``g++``. | ||
1046 | |||
1047 | - The environment in which Make runs is set up with various standard | ||
1048 | variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a | ||
1049 | similar manner to the environment set up by the SDK's environment | ||
1050 | setup script. One easy way to see these variables is to run the | ||
1051 | ``devtool build`` command on the recipe and then look in | ||
1052 | ``oe-logs/run.do_compile``. Towards the top of this file, there is | ||
1053 | a list of environment variables that are set. You can take | ||
1054 | advantage of these variables within the Makefile. | ||
1055 | |||
1056 | - If the Makefile sets a default for a variable using "=", that default | ||
1057 | overrides the value set in the environment, which is usually not | ||
1058 | desirable. For this case, you can either patch the Makefile so it | ||
1059 | sets the default using the "?=" operator, or you can alternatively | ||
1060 | force the value on the ``make`` command line. To force the value on | ||
1061 | the command line, add the variable setting to | ||
1062 | :term:`EXTRA_OEMAKE` or | ||
1063 | :term:`PACKAGECONFIG_CONFARGS` | ||
1064 | within the recipe. Here is an example using :term:`EXTRA_OEMAKE`:: | ||
1065 | |||
1066 | EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" | ||
1067 | |||
1068 | In the above example, | ||
1069 | single quotes are used around the variable settings as the values are | ||
1070 | likely to contain spaces because required default options are passed | ||
1071 | to the compiler. | ||
1072 | |||
1073 | - Hardcoding paths inside Makefiles is often problematic in a | ||
1074 | cross-compilation environment. This is particularly true because | ||
1075 | those hardcoded paths often point to locations on the build host and | ||
1076 | thus will either be read-only or will introduce contamination into | ||
1077 | the cross-compilation because they are specific to the build host | ||
1078 | rather than the target. Patching the Makefile to use prefix variables | ||
1079 | or other path variables is usually the way to handle this situation. | ||
1080 | |||
1081 | - Sometimes a Makefile runs target-specific commands such as | ||
1082 | ``ldconfig``. For such cases, you might be able to apply patches that | ||
1083 | remove these commands from the Makefile. | ||
1084 | |||
1085 | Adding Native Tools | ||
1086 | ------------------- | ||
1087 | |||
1088 | Often, you need to build additional tools that run on the :term:`Build Host` | ||
1089 | as opposed to the target. You should indicate this requirement by using one of | ||
1090 | the following methods when you run ``devtool add``: | ||
1091 | |||
1092 | - Specify the name of the recipe such that it ends with "-native". | ||
1093 | Specifying the name like this produces a recipe that only builds for | ||
1094 | the build host. | ||
1095 | |||
1096 | - Specify the "--also-native" option with the ``devtool add`` | ||
1097 | command. Specifying this option creates a recipe file that still | ||
1098 | builds for the target but also creates a variant with a "-native" | ||
1099 | suffix that builds for the build host. | ||
1100 | |||
1101 | .. note:: | ||
1102 | |||
1103 | If you need to add a tool that is shipped as part of a source tree | ||
1104 | that builds code for the target, you can typically accomplish this by | ||
1105 | building the native and target parts separately rather than within | ||
1106 | the same compilation process. Realize though that with the | ||
1107 | "--also-native" option, you can add the tool using just one | ||
1108 | recipe file. | ||
1109 | |||
1110 | Adding Node.js Modules | ||
1111 | ---------------------- | ||
1112 | |||
1113 | You can use the ``devtool add`` command two different ways to add | ||
1114 | Node.js modules: through ``npm`` or from a repository or local source. | ||
1115 | |||
1116 | Use the following form to add Node.js modules through ``npm``:: | ||
1117 | |||
1118 | $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" | ||
1119 | |||
1120 | The name and | ||
1121 | version parameters are mandatory. Lockdown and shrinkwrap files are | ||
1122 | generated and pointed to by the recipe in order to freeze the version | ||
1123 | that is fetched for the dependencies according to the first time. This | ||
1124 | also saves checksums that are verified on future fetches. Together, | ||
1125 | these behaviors ensure the reproducibility and integrity of the build. | ||
1126 | |||
1127 | .. note:: | ||
1128 | |||
1129 | - You must use quotes around the URL. ``devtool add`` does not | ||
1130 | require the quotes, but the shell considers ";" as a splitter | ||
1131 | between multiple commands. Thus, without the quotes, | ||
1132 | ``devtool add`` does not receive the other parts, which results in | ||
1133 | several "command not found" errors. | ||
1134 | |||
1135 | - In order to support adding Node.js modules, a ``nodejs`` recipe | ||
1136 | must be part of your SDK. | ||
1137 | |||
1138 | As mentioned earlier, you can also add Node.js modules directly from a | ||
1139 | repository or local source tree. To add modules this way, use | ||
1140 | ``devtool add`` in the following form:: | ||
1141 | |||
1142 | $ devtool add https://github.com/diversario/node-ssdp | ||
1143 | |||
1144 | In this example, ``devtool`` fetches the specified Git repository, detects the | ||
1145 | code as Node.js code, fetches dependencies using ``npm``, and sets | ||
1146 | :term:`SRC_URI` accordingly. | ||
1147 | |||
1148 | Working With Recipes | ||
1149 | ==================== | ||
1150 | |||
1151 | When building a recipe using the ``devtool build`` command, the typical | ||
1152 | build progresses as follows: | ||
1153 | |||
1154 | #. Fetch the source | ||
1155 | |||
1156 | #. Unpack the source | ||
1157 | |||
1158 | #. Configure the source | ||
1159 | |||
1160 | #. Compile the source | ||
1161 | |||
1162 | #. Install the build output | ||
1163 | |||
1164 | #. Package the installed output | ||
1165 | |||
1166 | For recipes in the workspace, fetching and unpacking is disabled as the | ||
1167 | source tree has already been prepared and is persistent. Each of these | ||
1168 | build steps is defined as a function (task), usually with a "do\_" prefix | ||
1169 | (e.g. :ref:`ref-tasks-fetch`, | ||
1170 | :ref:`ref-tasks-unpack`, and so | ||
1171 | forth). These functions are typically shell scripts but can instead be | ||
1172 | written in Python. | ||
1173 | |||
1174 | If you look at the contents of a recipe, you will see that the recipe | ||
1175 | does not include complete instructions for building the software. | ||
1176 | Instead, common functionality is encapsulated in classes inherited with | ||
1177 | the ``inherit`` directive. This technique leaves the recipe to describe | ||
1178 | just the things that are specific to the software being built. There is | ||
1179 | a :ref:`ref-classes-base` class that is implicitly inherited by all recipes | ||
1180 | and provides the functionality that most recipes typically need. | ||
1181 | |||
1182 | The remainder of this section presents information useful when working | ||
1183 | with recipes. | ||
1184 | |||
1185 | Finding Logs and Work Files | ||
1186 | --------------------------- | ||
1187 | |||
1188 | After the first run of the ``devtool build`` command, recipes that were | ||
1189 | previously created using the ``devtool add`` command or whose sources | ||
1190 | were modified using the ``devtool modify`` command contain symbolic | ||
1191 | links created within the source tree: | ||
1192 | |||
1193 | - ``oe-logs``: This link points to the directory in which log files and | ||
1194 | run scripts for each build step are created. | ||
1195 | |||
1196 | - ``oe-workdir``: This link points to the temporary work area for the | ||
1197 | recipe. The following locations under ``oe-workdir`` are particularly | ||
1198 | useful: | ||
1199 | |||
1200 | - ``image/``: Contains all of the files installed during the | ||
1201 | :ref:`ref-tasks-install` stage. | ||
1202 | Within a recipe, this directory is referred to by the expression | ||
1203 | ``${``\ :term:`D`\ ``}``. | ||
1204 | |||
1205 | - ``sysroot-destdir/``: Contains a subset of files installed within | ||
1206 | :ref:`ref-tasks-install` that have been put into the shared sysroot. For | ||
1207 | more information, see the | ||
1208 | ":ref:`dev-manual/new-recipe:sharing files between recipes`" section. | ||
1209 | |||
1210 | - ``packages-split/``: Contains subdirectories for each package | ||
1211 | produced by the recipe. For more information, see the | ||
1212 | ":ref:`dev-manual/devtool:packaging`" section. | ||
1213 | |||
1214 | You can use these links to get more information on what is happening at | ||
1215 | each build step. | ||
1216 | |||
1217 | Setting Configure Arguments | ||
1218 | --------------------------- | ||
1219 | |||
1220 | If the software your recipe is building uses GNU autoconf, then a fixed | ||
1221 | set of arguments is passed to it to enable cross-compilation plus any | ||
1222 | extras specified by :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS` | ||
1223 | set within the recipe. If you wish to pass additional options, add them | ||
1224 | to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build | ||
1225 | tools have similar variables (e.g. :term:`EXTRA_OECMAKE` for CMake, | ||
1226 | :term:`EXTRA_OESCONS` for Scons, and so forth). If you need to pass anything on | ||
1227 | the ``make`` command line, you can use :term:`EXTRA_OEMAKE` or the | ||
1228 | :term:`PACKAGECONFIG_CONFARGS` variables to do so. | ||
1229 | |||
1230 | You can use the ``devtool configure-help`` command to help you set the | ||
1231 | arguments listed in the previous paragraph. The command determines the | ||
1232 | exact options being passed, and shows them to you along with any custom | ||
1233 | arguments specified through :term:`EXTRA_OECONF` or | ||
1234 | :term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you | ||
1235 | the output of the configure script's "--help" option as a | ||
1236 | reference. | ||
1237 | |||
1238 | Sharing Files Between Recipes | ||
1239 | ----------------------------- | ||
1240 | |||
1241 | Recipes often need to use files provided by other recipes on the | ||
1242 | :term:`Build Host`. For example, | ||
1243 | an application linking to a common library needs access to the library | ||
1244 | itself and its associated headers. The way this access is accomplished | ||
1245 | within the extensible SDK is through the sysroot. There is one sysroot per | ||
1246 | "machine" for which the SDK is being built. In practical terms, this | ||
1247 | means there is a sysroot for the target machine, and a sysroot for | ||
1248 | the build host. | ||
1249 | |||
1250 | Recipes should never write files directly into the sysroot. Instead, | ||
1251 | files should be installed into standard locations during the | ||
1252 | :ref:`ref-tasks-install` task within the ``${``\ :term:`D`\ ``}`` directory. A | ||
1253 | subset of these files automatically goes into the sysroot. The reason | ||
1254 | for this limitation is that almost all files that go into the sysroot | ||
1255 | are cataloged in manifests in order to ensure they can be removed later | ||
1256 | when a recipe is modified or removed. Thus, the sysroot is able to | ||
1257 | remain free from stale files. | ||
1258 | |||
1259 | Packaging | ||
1260 | --------- | ||
1261 | |||
1262 | Packaging is not always particularly relevant within the extensible SDK. | ||
1263 | However, if you examine how build output gets into the final image on | ||
1264 | the target device, it is important to understand packaging because the | ||
1265 | contents of the image are expressed in terms of packages and not | ||
1266 | recipes. | ||
1267 | |||
1268 | During the :ref:`ref-tasks-package` task, files installed during the | ||
1269 | :ref:`ref-tasks-install` task are split into one main package, which is almost | ||
1270 | always named the same as the recipe, and into several other packages. This | ||
1271 | separation exists because not all of those installed files are useful in every | ||
1272 | image. For example, you probably do not need any of the documentation installed | ||
1273 | in a production image. Consequently, for each recipe the documentation | ||
1274 | files are separated into a ``-doc`` package. Recipes that package | ||
1275 | software containing optional modules or plugins might undergo additional | ||
1276 | package splitting as well. | ||
1277 | |||
1278 | After building a recipe, you can see where files have gone by looking in | ||
1279 | the ``oe-workdir/packages-split`` directory, which contains a | ||
1280 | subdirectory for each package. Apart from some advanced cases, the | ||
1281 | :term:`PACKAGES` and :term:`FILES` variables controls | ||
1282 | splitting. The :term:`PACKAGES` variable lists all of the packages to be | ||
1283 | produced, while the :term:`FILES` variable specifies which files to include | ||
1284 | in each package by using an override to specify the package. For | ||
1285 | example, ``FILES:${PN}`` specifies the files to go into the main package | ||
1286 | (i.e. the main package has the same name as the recipe and | ||
1287 | ``${``\ :term:`PN`\ ``}`` evaluates to the | ||
1288 | recipe name). The order of the :term:`PACKAGES` value is significant. For | ||
1289 | each installed file, the first package whose :term:`FILES` value matches the | ||
1290 | file is the package into which the file goes. Both the :term:`PACKAGES` and | ||
1291 | :term:`FILES` variables have default values. Consequently, you might find | ||
1292 | you do not even need to set these variables in your recipe unless the | ||
1293 | software the recipe is building installs files into non-standard | ||
1294 | locations. | ||
1295 | |||
1296 | Restoring the Target Device to its Original State | ||
1297 | ================================================= | ||
1298 | |||
1299 | If you use the ``devtool deploy-target`` command to write a recipe's | ||
1300 | build output to the target, and you are working on an existing component | ||
1301 | of the system, then you might find yourself in a situation where you | ||
1302 | need to restore the original files that existed prior to running the | ||
1303 | ``devtool deploy-target`` command. Because the ``devtool deploy-target`` | ||
1304 | command backs up any files it overwrites, you can use the | ||
1305 | ``devtool undeploy-target`` command to restore those files and remove | ||
1306 | any other files the recipe deployed. Consider the following example:: | ||
1307 | |||
1308 | $ devtool undeploy-target lighttpd root@192.168.7.2 | ||
1309 | |||
1310 | If you have deployed | ||
1311 | multiple applications, you can remove them all using the "-a" option | ||
1312 | thus restoring the target device to its original state:: | ||
1313 | |||
1314 | $ devtool undeploy-target -a root@192.168.7.2 | ||
1315 | |||
1316 | Information about files deployed to | ||
1317 | the target as well as any backed up files are stored on the target | ||
1318 | itself. This storage, of course, requires some additional space on the | ||
1319 | target machine. | ||
1320 | |||
1321 | .. note:: | ||
1322 | |||
1323 | The ``devtool deploy-target`` and ``devtool undeploy-target`` commands do | ||
1324 | not currently interact with any package management system on the target | ||
1325 | device (e.g. RPM or OPKG). Consequently, you should not intermingle | ||
1326 | ``devtool deploy-target`` and package manager operations on the target | ||
1327 | device. Doing so could result in a conflicting set of files. | ||
diff --git a/documentation/sdk-manual/figures/sdk-devtool-add-flow.png b/documentation/dev-manual/figures/devtool-add-flow.png index e7d6173d2d..e7d6173d2d 100644 --- a/documentation/sdk-manual/figures/sdk-devtool-add-flow.png +++ b/documentation/dev-manual/figures/devtool-add-flow.png | |||
Binary files differ | |||
diff --git a/documentation/sdk-manual/figures/sdk-devtool-modify-flow.png b/documentation/dev-manual/figures/devtool-modify-flow.png index 18ba8b7e65..18ba8b7e65 100644 --- a/documentation/sdk-manual/figures/sdk-devtool-modify-flow.png +++ b/documentation/dev-manual/figures/devtool-modify-flow.png | |||
Binary files differ | |||
diff --git a/documentation/sdk-manual/figures/sdk-devtool-upgrade-flow.png b/documentation/dev-manual/figures/devtool-upgrade-flow.png index 7d4f395e24..7d4f395e24 100644 --- a/documentation/sdk-manual/figures/sdk-devtool-upgrade-flow.png +++ b/documentation/dev-manual/figures/devtool-upgrade-flow.png | |||
Binary files differ | |||
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst index 8243c0f4cb..760108f605 100644 --- a/documentation/dev-manual/index.rst +++ b/documentation/dev-manual/index.rst | |||
@@ -23,6 +23,7 @@ Yocto Project Development Tasks Manual | |||
23 | speeding-up-build | 23 | speeding-up-build |
24 | libraries | 24 | libraries |
25 | prebuilt-libraries | 25 | prebuilt-libraries |
26 | devtool | ||
26 | x32-psabi | 27 | x32-psabi |
27 | gobject-introspection | 28 | gobject-introspection |
28 | external-toolchain | 29 | external-toolchain |
diff --git a/documentation/dev-manual/new-recipe.rst b/documentation/dev-manual/new-recipe.rst index 944b9627f9..4a65d70243 100644 --- a/documentation/dev-manual/new-recipe.rst +++ b/documentation/dev-manual/new-recipe.rst | |||
@@ -56,7 +56,7 @@ necessary when adding a recipe to build a new piece of software to be | |||
56 | included in a build. | 56 | included in a build. |
57 | 57 | ||
58 | You can find a complete description of the ``devtool add`` command in | 58 | You can find a complete description of the ``devtool add`` command in |
59 | the ":ref:`sdk-manual/extensible:a closer look at \`\`devtool add\`\``" section | 59 | the ":ref:`dev-manual/devtool:a closer look at \`\`devtool add\`\``" section |
60 | in the Yocto Project Application Development and the Extensible Software | 60 | in the Yocto Project Application Development and the Extensible Software |
61 | Development Kit (eSDK) manual. | 61 | Development Kit (eSDK) manual. |
62 | 62 | ||
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst index 4fac78bdfb..35901a0ea8 100644 --- a/documentation/dev-manual/upgrading-recipes.rst +++ b/documentation/dev-manual/upgrading-recipes.rst | |||
@@ -203,7 +203,7 @@ As mentioned earlier, an alternative method for upgrading recipes to | |||
203 | newer versions is to use | 203 | newer versions is to use |
204 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. | 204 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. |
205 | You can read about ``devtool upgrade`` in general in the | 205 | You can read about ``devtool upgrade`` in general in the |
206 | ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" | 206 | ":ref:`dev-manual/devtool:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" |
207 | section in the Yocto Project Application Development and the Extensible | 207 | section in the Yocto Project Application Development and the Extensible |
208 | Software Development Kit (eSDK) Manual. | 208 | Software Development Kit (eSDK) Manual. |
209 | 209 | ||
diff --git a/documentation/migration-guides/migration-2.1.rst b/documentation/migration-guides/migration-2.1.rst index 18b05b52cc..4d7aa15af5 100644 --- a/documentation/migration-guides/migration-2.1.rst +++ b/documentation/migration-guides/migration-2.1.rst | |||
@@ -382,7 +382,7 @@ These additional changes exist: | |||
382 | removed at runtime). | 382 | removed at runtime). |
383 | 383 | ||
384 | - The | 384 | - The |
385 | :ref:`devtool modify <sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component>` | 385 | :ref:`devtool modify <dev-manual/devtool:use \`\`devtool modify\`\` to modify the source of an existing component>` |
386 | command now defaults to extracting the source since that is most | 386 | command now defaults to extracting the source since that is most |
387 | commonly expected. The ``-x`` or ``--extract`` options are now no-ops. If | 387 | commonly expected. The ``-x`` or ``--extract`` options are now no-ops. If |
388 | you wish to provide your own existing source tree, you will now need | 388 | you wish to provide your own existing source tree, you will now need |
diff --git a/documentation/overview-manual/concepts.rst b/documentation/overview-manual/concepts.rst index 70766559f8..d1fd6befc8 100644 --- a/documentation/overview-manual/concepts.rst +++ b/documentation/overview-manual/concepts.rst | |||
@@ -792,7 +792,7 @@ processes patches, see the | |||
792 | ":ref:`dev-manual/new-recipe:patching code`" | 792 | ":ref:`dev-manual/new-recipe:patching code`" |
793 | section in the | 793 | section in the |
794 | Yocto Project Development Tasks Manual. You can also see the | 794 | Yocto Project Development Tasks Manual. You can also see the |
795 | ":ref:`sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component`" | 795 | ":ref:`dev-manual/devtool:use \`\`devtool modify\`\` to modify the source of an existing component`" |
796 | section in the Yocto Project Application Development and the Extensible | 796 | section in the Yocto Project Application Development and the Extensible |
797 | Software Development Kit (SDK) manual and the | 797 | Software Development Kit (SDK) manual and the |
798 | ":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`" | 798 | ":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`" |
diff --git a/documentation/ref-manual/devtool-reference.rst b/documentation/ref-manual/devtool-reference.rst index a6ea00c210..e173720e1a 100644 --- a/documentation/ref-manual/devtool-reference.rst +++ b/documentation/ref-manual/devtool-reference.rst | |||
@@ -432,7 +432,7 @@ revision to which you want to upgrade (i.e. the | |||
432 | forth. | 432 | forth. |
433 | 433 | ||
434 | You can read more on the ``devtool upgrade`` workflow in the | 434 | You can read more on the ``devtool upgrade`` workflow in the |
435 | ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" | 435 | ":ref:`dev-manual/devtool:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" |
436 | section in the Yocto Project Application Development and the Extensible | 436 | section in the Yocto Project Application Development and the Extensible |
437 | Software Development Kit (eSDK) manual. You can also see an example of | 437 | Software Development Kit (eSDK) manual. You can also see an example of |
438 | how to use ``devtool upgrade`` in the ":ref:`dev-manual/upgrading-recipes:using \`\`devtool upgrade\`\``" | 438 | how to use ``devtool upgrade`` in the ":ref:`dev-manual/upgrading-recipes:using \`\`devtool upgrade\`\``" |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 47d4e814fd..86023195d3 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
@@ -8642,7 +8642,7 @@ system and gives an overview of their function and contents. | |||
8642 | 8642 | ||
8643 | For information on how staging for recipe-specific sysroots occurs, | 8643 | For information on how staging for recipe-specific sysroots occurs, |
8644 | see the :ref:`ref-tasks-populate_sysroot` | 8644 | see the :ref:`ref-tasks-populate_sysroot` |
8645 | task, the ":ref:`sdk-manual/extensible:sharing files between recipes`" | 8645 | task, the ":ref:`dev-manual/devtool:sharing files between recipes`" |
8646 | section in the Yocto Project Development Tasks Manual, the | 8646 | section in the Yocto Project Development Tasks Manual, the |
8647 | ":ref:`overview-manual/concepts:configuration, compilation, and staging`" | 8647 | ":ref:`overview-manual/concepts:configuration, compilation, and staging`" |
8648 | section in the Yocto Project Overview and Concepts Manual, and the | 8648 | section in the Yocto Project Overview and Concepts Manual, and the |
diff --git a/documentation/sdk-manual/extensible.rst b/documentation/sdk-manual/extensible.rst index 8483951bef..dcb02c4e00 100644 --- a/documentation/sdk-manual/extensible.rst +++ b/documentation/sdk-manual/extensible.rst | |||
@@ -257,1326 +257,9 @@ the recipe a source tree that is under your control is used in order to | |||
257 | allow you to make changes to the source as desired. By default, new | 257 | allow you to make changes to the source as desired. By default, new |
258 | recipes and the source go into a "workspace" directory under the SDK. | 258 | recipes and the source go into a "workspace" directory under the SDK. |
259 | 259 | ||
260 | The remainder of this section presents the ``devtool add``, | 260 | To learn how to use ``devtool`` to add, modify, upgrade recipes and more, see |
261 | ``devtool modify``, and ``devtool upgrade`` workflows. | 261 | the :ref:`dev-manual/devtool:Using the \`\`devtool\`\` command-line tool` |
262 | 262 | section of the Yocto Project Development Tasks Manual. | |
263 | Use ``devtool add`` to Add an Application | ||
264 | ----------------------------------------- | ||
265 | |||
266 | The ``devtool add`` command generates a new recipe based on existing | ||
267 | source code. This command takes advantage of the | ||
268 | :ref:`devtool-the-workspace-layer-structure` | ||
269 | layer that many ``devtool`` commands use. The command is flexible enough | ||
270 | to allow you to extract source code into both the workspace or a | ||
271 | separate local Git repository and to use existing code that does not | ||
272 | need to be extracted. | ||
273 | |||
274 | Depending on your particular scenario, the arguments and options you use | ||
275 | with ``devtool add`` form different combinations. The following diagram | ||
276 | shows common development flows you would use with the ``devtool add`` | ||
277 | command: | ||
278 | |||
279 | .. image:: figures/sdk-devtool-add-flow.png | ||
280 | :width: 100% | ||
281 | |||
282 | #. *Generating the New Recipe*: The top part of the flow shows three | ||
283 | scenarios by which you could use ``devtool add`` to generate a recipe | ||
284 | based on existing source code. | ||
285 | |||
286 | In a shared development environment, it is typical for other | ||
287 | developers to be responsible for various areas of source code. As a | ||
288 | developer, you are probably interested in using that source code as | ||
289 | part of your development within the Yocto Project. All you need is | ||
290 | access to the code, a recipe, and a controlled area in which to do | ||
291 | your work. | ||
292 | |||
293 | Within the diagram, three possible scenarios feed into the | ||
294 | ``devtool add`` workflow: | ||
295 | |||
296 | - *Left*: The left scenario in the figure represents a common | ||
297 | situation where the source code does not exist locally and needs | ||
298 | to be extracted. In this situation, the source code is extracted | ||
299 | to the default workspace --- you do not want the files in some | ||
300 | specific location outside of the workspace. Thus, everything you | ||
301 | need will be located in the workspace:: | ||
302 | |||
303 | $ devtool add recipe fetchuri | ||
304 | |||
305 | With this command, ``devtool`` extracts the upstream | ||
306 | source files into a local Git repository within the ``sources`` | ||
307 | folder. The command then creates a recipe named recipe and a | ||
308 | corresponding append file in the workspace. If you do not provide | ||
309 | recipe, the command makes an attempt to determine the recipe name. | ||
310 | |||
311 | - *Middle*: The middle scenario in the figure also represents a | ||
312 | situation where the source code does not exist locally. In this | ||
313 | case, the code is again upstream and needs to be extracted to some | ||
314 | local area --- this time outside of the default workspace. | ||
315 | |||
316 | .. note:: | ||
317 | |||
318 | If required, ``devtool`` always creates a Git repository locally | ||
319 | during the extraction. | ||
320 | |||
321 | Furthermore, the first positional argument ``srctree`` in this case | ||
322 | identifies where the ``devtool add`` command will locate the | ||
323 | extracted code outside of the workspace. You need to specify an | ||
324 | empty directory:: | ||
325 | |||
326 | $ devtool add recipe srctree fetchuri | ||
327 | |||
328 | In summary, the source code is pulled from fetchuri and extracted into the | ||
329 | location defined by ``srctree`` as a local Git repository. | ||
330 | |||
331 | Within workspace, ``devtool`` creates a recipe named recipe along | ||
332 | with an associated append file. | ||
333 | |||
334 | - *Right*: The right scenario in the figure represents a situation | ||
335 | where the ``srctree`` has been previously prepared outside of the | ||
336 | ``devtool`` workspace. | ||
337 | |||
338 | The following command provides a new recipe name and identifies | ||
339 | the existing source tree location:: | ||
340 | |||
341 | $ devtool add recipe srctree | ||
342 | |||
343 | The command examines the source code and creates a recipe named | ||
344 | recipe for the code and places the recipe into the workspace. | ||
345 | |||
346 | Because the extracted source code already exists, ``devtool`` does | ||
347 | not try to relocate the source code into the workspace --- only the | ||
348 | new recipe is placed in the workspace. | ||
349 | |||
350 | Aside from a recipe folder, the command also creates an associated | ||
351 | append folder and places an initial ``*.bbappend`` file within. | ||
352 | |||
353 | #. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the | ||
354 | editor as defined by the ``$EDITOR`` environment variable and modify | ||
355 | the file:: | ||
356 | |||
357 | $ devtool edit-recipe recipe | ||
358 | |||
359 | From within the editor, you can make modifications to the recipe that | ||
360 | take effect when you build it later. | ||
361 | |||
362 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
363 | depends on what you are going to do with the new code. | ||
364 | |||
365 | If you need to eventually move the build output to the target | ||
366 | hardware, use the following ``devtool`` command:: | ||
367 | |||
368 | $ devtool build recipe | ||
369 | |||
370 | On the other hand, if you want an image to contain the recipe's | ||
371 | packages from the workspace for immediate deployment onto a device | ||
372 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
373 | command:: | ||
374 | |||
375 | $ devtool build-image image | ||
376 | |||
377 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
378 | to build out your recipe, you probably want to see if the resulting | ||
379 | build output works as expected on the target hardware. | ||
380 | |||
381 | .. note:: | ||
382 | |||
383 | This step assumes you have a previously built image that is | ||
384 | already either running in QEMU or is running on actual hardware. | ||
385 | Also, it is assumed that for deployment of the image to the | ||
386 | target, SSH is installed in the image and, if the image is running | ||
387 | on real hardware, you have network access to and from your | ||
388 | development machine. | ||
389 | |||
390 | You can deploy your build output to that target hardware by using the | ||
391 | ``devtool deploy-target`` command:: | ||
392 | |||
393 | $ devtool deploy-target recipe target | ||
394 | |||
395 | The target is a live target machine running as an SSH server. | ||
396 | |||
397 | You can, of course, also deploy the image you build to actual | ||
398 | hardware by using the ``devtool build-image`` command. However, | ||
399 | ``devtool`` does not provide a specific command that allows you to | ||
400 | deploy the image to actual hardware. | ||
401 | |||
402 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
403 | creates any patches corresponding to commits in the local Git | ||
404 | repository, moves the new recipe to a more permanent layer, and then | ||
405 | resets the recipe so that the recipe is built normally rather than | ||
406 | from the workspace:: | ||
407 | |||
408 | $ devtool finish recipe layer | ||
409 | |||
410 | .. note:: | ||
411 | |||
412 | Any changes you want to turn into patches must be committed to the | ||
413 | Git repository in the source tree. | ||
414 | |||
415 | As mentioned, the ``devtool finish`` command moves the final recipe | ||
416 | to its permanent layer. | ||
417 | |||
418 | As a final process of the ``devtool finish`` command, the state of | ||
419 | the standard layers and the upstream source is restored so that you | ||
420 | can build the recipe from those areas rather than the workspace. | ||
421 | |||
422 | .. note:: | ||
423 | |||
424 | You can use the ``devtool reset`` command to put things back should you | ||
425 | decide you do not want to proceed with your work. If you do use this | ||
426 | command, realize that the source tree is preserved. | ||
427 | |||
428 | Use ``devtool modify`` to Modify the Source of an Existing Component | ||
429 | -------------------------------------------------------------------- | ||
430 | |||
431 | The ``devtool modify`` command prepares the way to work on existing code | ||
432 | that already has a local recipe in place that is used to build the | ||
433 | software. The command is flexible enough to allow you to extract code | ||
434 | from an upstream source, specify the existing recipe, and keep track of | ||
435 | and gather any patch files from other developers that are associated | ||
436 | with the code. | ||
437 | |||
438 | Depending on your particular scenario, the arguments and options you use | ||
439 | with ``devtool modify`` form different combinations. The following | ||
440 | diagram shows common development flows for the ``devtool modify`` | ||
441 | command: | ||
442 | |||
443 | .. image:: figures/sdk-devtool-modify-flow.png | ||
444 | :width: 100% | ||
445 | |||
446 | #. *Preparing to Modify the Code*: The top part of the flow shows three | ||
447 | scenarios by which you could use ``devtool modify`` to prepare to | ||
448 | work on source files. Each scenario assumes the following: | ||
449 | |||
450 | - The recipe exists locally in a layer external to the ``devtool`` | ||
451 | workspace. | ||
452 | |||
453 | - The source files exist either upstream in an un-extracted state or | ||
454 | locally in a previously extracted state. | ||
455 | |||
456 | The typical situation is where another developer has created a layer | ||
457 | for use with the Yocto Project and their recipe already resides in | ||
458 | that layer. Furthermore, their source code is readily available | ||
459 | either upstream or locally. | ||
460 | |||
461 | - *Left*: The left scenario in the figure represents a common | ||
462 | situation where the source code does not exist locally and it | ||
463 | needs to be extracted from an upstream source. In this situation, | ||
464 | the source is extracted into the default ``devtool`` workspace | ||
465 | location. The recipe, in this scenario, is in its own layer | ||
466 | outside the workspace (i.e. ``meta-``\ layername). | ||
467 | |||
468 | The following command identifies the recipe and, by default, | ||
469 | extracts the source files:: | ||
470 | |||
471 | $ devtool modify recipe | ||
472 | |||
473 | Once ``devtool`` locates the recipe, ``devtool`` uses the recipe's | ||
474 | :term:`SRC_URI` statements to locate the source code and any local | ||
475 | patch files from other developers. | ||
476 | |||
477 | With this scenario, there is no ``srctree`` argument. Consequently, the | ||
478 | default behavior of the ``devtool modify`` command is to extract | ||
479 | the source files pointed to by the :term:`SRC_URI` statements into a | ||
480 | local Git structure. Furthermore, the location for the extracted | ||
481 | source is the default area within the ``devtool`` workspace. The | ||
482 | result is that the command sets up both the source code and an | ||
483 | append file within the workspace while the recipe remains in its | ||
484 | original location. | ||
485 | |||
486 | Additionally, if you have any non-patch local files (i.e. files | ||
487 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
488 | excluding ``*.patch/`` or ``*.diff``), these files are copied to | ||
489 | an ``oe-local-files`` folder under the newly created source tree. | ||
490 | Copying the files here gives you a convenient area from which you | ||
491 | can modify the files. Any changes or additions you make to those | ||
492 | files are incorporated into the build the next time you build the | ||
493 | software just as are other changes you might have made to the | ||
494 | source. | ||
495 | |||
496 | - *Middle*: The middle scenario in the figure represents a situation | ||
497 | where the source code also does not exist locally. In this case, | ||
498 | the code is again upstream and needs to be extracted to some local | ||
499 | area as a Git repository. The recipe, in this scenario, is again | ||
500 | local and in its own layer outside the workspace. | ||
501 | |||
502 | The following command tells ``devtool`` the recipe with which to | ||
503 | work and, in this case, identifies a local area for the extracted | ||
504 | source files that exists outside of the default ``devtool`` | ||
505 | workspace:: | ||
506 | |||
507 | $ devtool modify recipe srctree | ||
508 | |||
509 | .. note:: | ||
510 | |||
511 | You cannot provide a URL for ``srctree`` using the ``devtool`` command. | ||
512 | |||
513 | As with all extractions, the command uses the recipe's :term:`SRC_URI` | ||
514 | statements to locate the source files and any associated patch | ||
515 | files. Non-patch files are copied to an ``oe-local-files`` folder | ||
516 | under the newly created source tree. | ||
517 | |||
518 | Once the files are located, the command by default extracts them | ||
519 | into ``srctree``. | ||
520 | |||
521 | Within workspace, ``devtool`` creates an append file for the | ||
522 | recipe. The recipe remains in its original location but the source | ||
523 | files are extracted to the location you provide with ``srctree``. | ||
524 | |||
525 | - *Right*: The right scenario in the figure represents a situation | ||
526 | where the source tree (``srctree``) already exists locally as a | ||
527 | previously extracted Git structure outside of the ``devtool`` | ||
528 | workspace. In this example, the recipe also exists elsewhere | ||
529 | locally in its own layer. | ||
530 | |||
531 | The following command tells ``devtool`` the recipe with which to | ||
532 | work, uses the "-n" option to indicate source does not need to be | ||
533 | extracted, and uses ``srctree`` to point to the previously extracted | ||
534 | source files:: | ||
535 | |||
536 | $ devtool modify -n recipe srctree | ||
537 | |||
538 | If an ``oe-local-files`` subdirectory happens to exist and it | ||
539 | contains non-patch files, the files are used. However, if the | ||
540 | subdirectory does not exist and you run the ``devtool finish`` | ||
541 | command, any non-patch files that might exist next to the recipe | ||
542 | are removed because it appears to ``devtool`` that you have | ||
543 | deleted those files. | ||
544 | |||
545 | Once the ``devtool modify`` command finishes, it creates only an | ||
546 | append file for the recipe in the ``devtool`` workspace. The | ||
547 | recipe and the source code remain in their original locations. | ||
548 | |||
549 | #. *Edit the Source*: Once you have used the ``devtool modify`` command, | ||
550 | you are free to make changes to the source files. You can use any | ||
551 | editor you like to make and save your source code modifications. | ||
552 | |||
553 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
554 | depends on what you are going to do with the new code. | ||
555 | |||
556 | If you need to eventually move the build output to the target | ||
557 | hardware, use the following ``devtool`` command:: | ||
558 | |||
559 | $ devtool build recipe | ||
560 | |||
561 | On the other hand, if you want an image to contain the recipe's | ||
562 | packages from the workspace for immediate deployment onto a device | ||
563 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
564 | command:: | ||
565 | |||
566 | $ devtool build-image image | ||
567 | |||
568 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
569 | to build out your recipe, you probably want to see if the resulting | ||
570 | build output works as expected on target hardware. | ||
571 | |||
572 | .. note:: | ||
573 | |||
574 | This step assumes you have a previously built image that is | ||
575 | already either running in QEMU or running on actual hardware. | ||
576 | Also, it is assumed that for deployment of the image to the | ||
577 | target, SSH is installed in the image and if the image is running | ||
578 | on real hardware that you have network access to and from your | ||
579 | development machine. | ||
580 | |||
581 | You can deploy your build output to that target hardware by using the | ||
582 | ``devtool deploy-target`` command:: | ||
583 | |||
584 | $ devtool deploy-target recipe target | ||
585 | |||
586 | The target is a live target machine running as an SSH server. | ||
587 | |||
588 | You can, of course, use other methods to deploy the image you built | ||
589 | using the ``devtool build-image`` command to actual hardware. | ||
590 | ``devtool`` does not provide a specific command to deploy the image | ||
591 | to actual hardware. | ||
592 | |||
593 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
594 | creates any patches corresponding to commits in the local Git | ||
595 | repository, updates the recipe to point to them (or creates a | ||
596 | ``.bbappend`` file to do so, depending on the specified destination | ||
597 | layer), and then resets the recipe so that the recipe is built | ||
598 | normally rather than from the workspace:: | ||
599 | |||
600 | $ devtool finish recipe layer | ||
601 | |||
602 | .. note:: | ||
603 | |||
604 | Any changes you want to turn into patches must be staged and | ||
605 | committed within the local Git repository before you use the | ||
606 | ``devtool finish`` command. | ||
607 | |||
608 | Because there is no need to move the recipe, ``devtool finish`` | ||
609 | either updates the original recipe in the original layer or the | ||
610 | command creates a ``.bbappend`` file in a different layer as provided | ||
611 | by layer. Any work you did in the ``oe-local-files`` directory is | ||
612 | preserved in the original files next to the recipe during the | ||
613 | ``devtool finish`` command. | ||
614 | |||
615 | As a final process of the ``devtool finish`` command, the state of | ||
616 | the standard layers and the upstream source is restored so that you | ||
617 | can build the recipe from those areas rather than from the workspace. | ||
618 | |||
619 | .. note:: | ||
620 | |||
621 | You can use the ``devtool reset`` command to put things back should you | ||
622 | decide you do not want to proceed with your work. If you do use this | ||
623 | command, realize that the source tree is preserved. | ||
624 | |||
625 | ``devtool ide-sdk`` configures IDEs and bootstraps SDKs | ||
626 | ------------------------------------------------------- | ||
627 | |||
628 | The ``devtool ide-sdk`` command can provide an IDE configuration for IDEs when | ||
629 | working on the source code of one or more recipes. | ||
630 | Depending on the programming language, and the build system used by the recipe, | ||
631 | the tools required for cross-development and remote debugging are different. | ||
632 | For example: | ||
633 | |||
634 | - A C/C++ project usually uses CMake or Meson. | ||
635 | |||
636 | - A Python project uses setuptools or one of its successors. | ||
637 | |||
638 | - A Rust project uses Cargo. | ||
639 | |||
640 | Also, the IDE plugins needed for the integration of a build system with the | ||
641 | IDE and the corresponding settings are usually specific to these build-systems. | ||
642 | To hide all these details from the user, ``devtool ide-sdk`` does two things: | ||
643 | |||
644 | - It generates any kind of SDK needed for cross-development and remote | ||
645 | debugging of the specified recipes. | ||
646 | |||
647 | - It generates the configuration for the IDE (and the IDE plugins) for using | ||
648 | the cross-toolchain and remote debugging tools provided by the SDK directly | ||
649 | from the IDE. | ||
650 | |||
651 | For supported build systems the configurations generated by ``devtool ide-sdk`` | ||
652 | combine the advantages of the ``devtool modify`` based workflow | ||
653 | (see :ref:`using_devtool`) with the advantages of the simple Environment Setup | ||
654 | script based workflow (see :ref:`running_the_ext_sdk_env`) provided by Yocto's | ||
655 | SDK or eSDK: | ||
656 | |||
657 | - The source code of the recipe is in the workspace created by | ||
658 | ``devtool modify`` or ``devtool add``. | ||
659 | Using ``devtool build``, ``devtool build-image``, | ||
660 | ``devtool deploy-target`` or ``bitbake`` is possible. | ||
661 | Also ``devtool ide-sdk`` can be used to update the SDK and the IDE | ||
662 | configuration at any time. | ||
663 | |||
664 | - ``devtool ide-sdk`` aims to support multiple programming languages and | ||
665 | multiple IDEs natively. "Natively" means that the IDE is configured to call | ||
666 | the build tool (e.g. ``cmake`` or ``meson``) directly. This has several | ||
667 | advantages. | ||
668 | First of all, it is usually much faster to call for example ``cmake`` than | ||
669 | ``devtool build``. | ||
670 | It also allows to benefit from the very good integration that IDEs like | ||
671 | VSCode offer for tools like CMake or GDB. | ||
672 | |||
673 | However, supporting many programming languages and multiple | ||
674 | IDEs is quite an elaborate and constantly evolving thing. Support for IDEs | ||
675 | is therefore implemented as plugins. Plugins can also be provided by | ||
676 | optional layers. | ||
677 | |||
678 | So much about the introduction to the default mode of ``devtool sdk-ide`` which | ||
679 | is called the "modified" mode because it uses the workspace created by | ||
680 | ``devtool modify`` and the per recipe :term:`Sysroots <Sysroot>` of BitBake. | ||
681 | |||
682 | For some recipes and use cases, this default behavior of ``devtool ide-sdk`` | ||
683 | with full ``devtool`` and ``bitbake`` integration might not be suitable. | ||
684 | To offer full feature parity with the SDK and the eSDK, ``devtool ide-sdk`` has | ||
685 | a second mode called "shared" mode. | ||
686 | If ``devtool ide-sdk`` is called with the ``--mode=shared`` option, it | ||
687 | bootstraps an SDK directly from the BitBake environment, which offers the same | ||
688 | Environment Setup script as described in :ref:`running_the_ext_sdk_env`. | ||
689 | In addition to the (e)SDK installer-based setup, the IDE gets configured | ||
690 | to use the shared :term:`Sysroots <Sysroot>` and the tools from the SDK. | ||
691 | ``devtool ide-sdk --mode=shared`` is basically a wrapper for the setup of the | ||
692 | extensible SDK as described in :ref:`setting_up_ext_sdk_in_build`. | ||
693 | |||
694 | The use of ``devtool ide-sdk`` is an alternative to using one of the SDK | ||
695 | installers. | ||
696 | ``devtool ide-sdk`` allows the creation of SDKs that offer all the | ||
697 | functionality of the SDK and the eSDK installers. Compared to the installers, | ||
698 | however, the SDK created with ``devtool ide-sdk`` is much more flexible. | ||
699 | For example, it is very easy to change the :term:`MACHINE` in the | ||
700 | ``local.conf`` file, update the layer meta data and then regenerate the SDK. | ||
701 | |||
702 | Let's take a look at an example of how to use ``devtool ide-sdk`` in each of | ||
703 | the two modes: | ||
704 | |||
705 | #. *Modified mode*: | ||
706 | |||
707 | In order to use the ``devtool ide-sdk``, a few settings are needed. As a | ||
708 | starting example, the following lines of code can be added to the | ||
709 | ``local.conf`` file:: | ||
710 | |||
711 | # Build the companion debug file system | ||
712 | IMAGE_GEN_DEBUGFS = "1" | ||
713 | # Optimize build time: with devtool ide-sdk the dbg tar is not needed | ||
714 | IMAGE_FSTYPES_DEBUGFS = "" | ||
715 | # Without copying the binaries into roofs-dbg, GDB does not find all source files. | ||
716 | IMAGE_CLASSES += "image-combined-dbg" | ||
717 | |||
718 | # SSH is mandatory, no password simplifies the usage | ||
719 | EXTRA_IMAGE_FEATURES += "\ | ||
720 | ssh-server-openssh \ | ||
721 | allow-empty-password \ | ||
722 | allow-root-login \ | ||
723 | empty-root-password \ | ||
724 | " | ||
725 | |||
726 | # Remote debugging needs gdbserver on the target device | ||
727 | IMAGE_INSTALL:append = " gdbserver" | ||
728 | |||
729 | # Add the recipes which should be modified to the image | ||
730 | # Otherwise some dependencies might be missing. | ||
731 | IMAGE_INSTALL:append = " my-recipe" | ||
732 | |||
733 | Assuming the BitBake environment is set up correctly and a workspace has | ||
734 | been created for the recipe using ``devtool modify my-recipe`` or probably | ||
735 | even better by using ``devtool modify my-recipe --debug-build``, the | ||
736 | following command can create the SDK and the configuration for VSCode in | ||
737 | the recipe workspace:: | ||
738 | |||
739 | $ devtool ide-sdk my-recipe core-image-minimal --target root@192.168.7.2 | ||
740 | |||
741 | The command requires an image recipe (``core-image-minimal`` for this | ||
742 | example) that is used to create the SDK. | ||
743 | This firmware image should also be installed on the target device. | ||
744 | It is possible to pass multiple package recipes:: | ||
745 | |||
746 | $ devtool ide-sdk my-recipe-1 my-recipe-2 core-image-minimal --target root@192.168.7.2 | ||
747 | |||
748 | ``devtool ide-sdk`` tries to create an IDE configuration for all package | ||
749 | recipes. | ||
750 | |||
751 | What this command does exactly depends on the recipe, more precisely on the | ||
752 | build tool used by the recipe. The basic idea is to configure the IDE so | ||
753 | that it calls the build tool exactly as ``bitbake`` does. | ||
754 | |||
755 | For example, a CMake preset is created for a recipe that inherits | ||
756 | :ref:`ref-classes-cmake`. In the case of VSCode, CMake presets are supported | ||
757 | by the CMake Tools plugin. This is an example of how the build configuration | ||
758 | used by ``bitbake`` is exported to an IDE configuration that gives exactly | ||
759 | the same build results. | ||
760 | |||
761 | Support for remote debugging with seamless integration into the IDE is | ||
762 | important for a cross-SDK. ``devtool ide-sdk`` automatically generates the | ||
763 | necessary helper scripts for deploying the compiled artifacts to the target | ||
764 | device as well as the necessary configuration for the debugger and the IDE. | ||
765 | |||
766 | .. note:: | ||
767 | |||
768 | To ensure that the debug symbols on the build machine match the binaries | ||
769 | running on the target device, it is essential that the image built by | ||
770 | ``devtool ide-sdk`` is running on the target device. | ||
771 | |||
772 | The default IDE is VSCode. Some hints about using VSCode: | ||
773 | |||
774 | - VSCode can be used to work on the BitBake recipes or the application | ||
775 | source code. | ||
776 | Usually there is one instance of VSCode running in the folder where the | ||
777 | BitBake recipes are. This instance has the | ||
778 | `Yocto Project BitBake plugin <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`_ | ||
779 | running. | ||
780 | |||
781 | .. warning:: | ||
782 | |||
783 | Some VSCode plugins (Python, BitBake and others) need a reasonable | ||
784 | configuration to work as expected. Otherwise, some plugins try to | ||
785 | index the build directory of BitBake, which keeps your system quite | ||
786 | busy until an out of memory exception stops this nonsense. | ||
787 | Other plugins, such as the BitBake plugin, do not behave as expected. | ||
788 | |||
789 | To work around such issues, the ``oe-init-build-env`` script creates | ||
790 | an initial ``.vscode/settings.json`` file if ``code`` can be found | ||
791 | and the ``.vscode`` folder does not yet exist. | ||
792 | It is best to run ``oe-init-build-env`` once before starting VSCode. | ||
793 | An alternative approach is to use a build folder outside the layers, | ||
794 | e.g. ``oe-init-build-env ../build``. | ||
795 | |||
796 | The BitBake plugin also offers to create devtool workspaces and run | ||
797 | ``devtool ide-sdk`` with a few mouse clicks. | ||
798 | Of course, issuing commands in the terminal works as well. | ||
799 | |||
800 | - To work on the source code of a recipe another instance of VSCode is | ||
801 | started in the recipe's workspace. Example:: | ||
802 | |||
803 | code build/workspace/sources/my-recipe | ||
804 | |||
805 | This instance of VSCode uses plugins that are useful for the development | ||
806 | of the application. ``devtool ide-sdk`` generates the necessary | ||
807 | ``extensions.json``, ``settings.json``, ``tasks.json``and ``launch.json`` | ||
808 | configuration files for all the involved plugins. | ||
809 | |||
810 | When the source code folder present in the workspace folder is opened in | ||
811 | VSCode for the first time, a pop-up message recommends installing the | ||
812 | required plugins. | ||
813 | After accepting the installation of the plugins, working with the source | ||
814 | code or some debugging tasks should work as usual with VSCode. | ||
815 | |||
816 | Starting the VSCode instances in the recipe workspace folders can also be | ||
817 | done by a mouse click on the recipe workspaces in the first VSCode | ||
818 | instance. | ||
819 | |||
820 | - To work with CMake press ``Ctrl + Shift + p``, type ``cmake``. This will | ||
821 | show some possible commands like selecting a CMake preset, compiling or | ||
822 | running CTest. | ||
823 | |||
824 | For recipes inheriting :ref:`ref-classes-cmake-qemu` rather than | ||
825 | :ref:`ref-classes-cmake`, executing cross-compiled unit tests on the host | ||
826 | can be supported transparently with QEMU user-mode. | ||
827 | |||
828 | - To work with Meson press ``Ctrl + Shift + p``, type ``meson``. This will | ||
829 | show some possible commands like compiling or executing the unit tests. | ||
830 | |||
831 | A note on running cross-compiled unit tests on the host: Meson enables | ||
832 | support for QEMU user mode by default. It is expected that the execution | ||
833 | of the unit tests from the IDE will work without any additional steps, | ||
834 | given that the code is suitable for the execution on the host machine. | ||
835 | |||
836 | - For the deployment to the target device, just press ``Ctrl + Shift + p``, | ||
837 | type ``task``. Select ``install && deploy-target``. | ||
838 | |||
839 | - For remote debugging, switch to the debugging view by pressing the "play" | ||
840 | button with the ``bug icon`` on the left side. This will provide a green | ||
841 | play button with a drop-down list where a debug configuration can be | ||
842 | selected. After selecting one of the generated configurations, press the | ||
843 | "play" button. | ||
844 | |||
845 | Starting a remote debugging session automatically initiates the | ||
846 | deployment to the target device. If this is not desired, the | ||
847 | ``"dependsOn": ["install && deploy-target...]`` parameter of the tasks | ||
848 | with ``"label": "gdbserver start...`` can be removed from the | ||
849 | ``tasks.json`` file. | ||
850 | |||
851 | VSCode supports GDB with many different setups and configurations for | ||
852 | many different use cases. However, most of these setups have some | ||
853 | limitations when it comes to cross-development, support only a few target | ||
854 | architectures or require a high performance target device. Therefore | ||
855 | ``devtool ide-sdk`` supports the classic, generic setup with GDB on the | ||
856 | development host and gdbserver on the target device. | ||
857 | |||
858 | Roughly summarized, this means: | ||
859 | |||
860 | - The binaries are copied via SSH to the remote target device by a | ||
861 | script referred by ``tasks.json``. | ||
862 | |||
863 | - gdbserver is started on the remote target device via SSH by a script | ||
864 | referred by ``tasks.json``. | ||
865 | |||
866 | Changing the parameters that are passed to the debugging executable | ||
867 | requires modifying the generated script. The script is located at | ||
868 | ``oe-scripts/gdbserver_*``. Defining the parameters in the ``args`` | ||
869 | field in the ``launch.json`` file does not work. | ||
870 | |||
871 | - VSCode connects to gdbserver as documented in | ||
872 | `Remote debugging or debugging with a local debugger server | ||
873 | <https://code.visualstudio.com/docs/cpp/launch-json-reference#_remote-debugging-or-debugging-with-a-local-debugger-server>`__. | ||
874 | |||
875 | Additionally ``--ide=none`` is supported. With the ``none`` IDE parameter, | ||
876 | some generic configuration files like ``gdbinit`` files and some helper | ||
877 | scripts starting gdbserver remotely on the target device as well as the GDB | ||
878 | client on the host are generated. | ||
879 | |||
880 | Here is a usage example for the ``cmake-example`` recipe from the | ||
881 | ``meta-selftest`` layer which inherits :ref:`ref-classes-cmake-qemu`: | ||
882 | |||
883 | .. code-block:: sh | ||
884 | |||
885 | # Create the SDK | ||
886 | devtool modify cmake-example --debug-build | ||
887 | devtool ide-sdk cmake-example core-image-minimal -c --ide=none | ||
888 | |||
889 | # Install the firmware on a target device or start QEMU | ||
890 | runqemu | ||
891 | |||
892 | # From exploring the workspace of cmake-example | ||
893 | cd build/workspace/sources/cmake-example | ||
894 | |||
895 | # Find cmake-native and save the path into a variable | ||
896 | # Note: using just cmake instead of $CMAKE_NATIVE would work in many cases | ||
897 | CMAKE_NATIVE="$(jq -r '.configurePresets[0] | "\(.cmakeExecutable)"' CMakeUserPresets.json)" | ||
898 | |||
899 | # List available CMake presets | ||
900 | "$CMAKE_NATIVE" --list-presets | ||
901 | Available configure presets: | ||
902 | |||
903 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
904 | |||
905 | # Re-compile the already compiled sources | ||
906 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 | ||
907 | ninja: no work to do. | ||
908 | # Do a clean re-build | ||
909 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target clean | ||
910 | [1/1] Cleaning all built files... | ||
911 | Cleaning... 8 files. | ||
912 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target all | ||
913 | [7/7] Linking CXX executable cmake-example | ||
914 | |||
915 | # Run the cross-compiled unit tests with QEMU user-mode | ||
916 | "$CMAKE_NATIVE" --build --preset cmake-example-cortexa57 --target test | ||
917 | [0/1] Running tests... | ||
918 | Test project .../build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
919 | Start 1: test-cmake-example | ||
920 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
921 | |||
922 | 100% tests passed, 0 tests failed out of 1 | ||
923 | |||
924 | Total Test time (real) = 0.03 sec | ||
925 | |||
926 | # Using CTest directly is possible as well | ||
927 | CTEST_NATIVE="$(dirname "$CMAKE_NATIVE")/ctest" | ||
928 | |||
929 | # List available CMake presets | ||
930 | "$CTEST_NATIVE" --list-presets | ||
931 | Available test presets: | ||
932 | |||
933 | "cmake-example-cortexa57" - cmake-example: cortexa57 | ||
934 | |||
935 | # Run the cross-compiled unit tests with QEMU user-mode | ||
936 | "$CTEST_NATIVE" --preset "cmake-example-cortexa57" | ||
937 | Test project ...build/tmp/work/cortexa57-poky-linux/cmake-example/1.0/cmake-example-1.0 | ||
938 | Start 1: test-cmake-example | ||
939 | 1/1 Test #1: test-cmake-example ............... Passed 0.03 sec | ||
940 | |||
941 | 100% tests passed, 0 tests failed out of 1 | ||
942 | |||
943 | Total Test time (real) = 0.03 sec | ||
944 | |||
945 | # Deploying the new build to the target device (default is QEUM at 192.168.7.2) | ||
946 | oe-scripts/install_and_deploy_cmake-example-cortexa57 | ||
947 | |||
948 | # Start a remote debugging session with gdbserver on the target and GDB on the host | ||
949 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m | ||
950 | oe-scripts/gdb_1234_usr-bin-cmake-example | ||
951 | break main | ||
952 | run | ||
953 | step | ||
954 | stepi | ||
955 | continue | ||
956 | quit | ||
957 | |||
958 | # Stop gdbserver on the target device | ||
959 | oe-scripts/gdbserver_1234_usr-bin-cmake-example_m stop | ||
960 | |||
961 | #. *Shared sysroots mode* | ||
962 | |||
963 | Creating an SDK with shared :term:`Sysroots <Sysroot>` that contains all the | ||
964 | dependencies needed to work with ``my-recipe`` is possible with the following | ||
965 | example command:: | ||
966 | |||
967 | $ devtool ide-sdk --mode=shared my-recipe | ||
968 | |||
969 | For VSCode the cross-toolchain is exposed as a CMake kit. CMake kits are | ||
970 | defined in ``~/.local/share/CMakeTools/cmake-tools-kits.json``. | ||
971 | The following example shows how the cross-toolchain can be selected in | ||
972 | VSCode. First of all we need a folder containing a CMake project. | ||
973 | For this example, let's create a CMake project and start VSCode:: | ||
974 | |||
975 | mkdir kit-test | ||
976 | echo "project(foo VERSION 1.0)" > kit-test/CMakeLists.txt | ||
977 | code kit-test | ||
978 | |||
979 | If there is a CMake project in the workspace, cross-compilation is | ||
980 | supported: | ||
981 | |||
982 | - Press ``Ctrl + Shift + P``, type ``CMake: Scan for Kits`` | ||
983 | - Press ``Ctrl + Shift + P``, type ``CMake: Select a Kit`` | ||
984 | |||
985 | Finally most of the features provided by CMake and the IDE should be | ||
986 | available. | ||
987 | |||
988 | Other IDEs than VSCode are supported as well. However, | ||
989 | ``devtool ide-sdk --mode=shared --ide=none my-recipe`` is currently | ||
990 | just a simple wrapper for the setup of the extensible SDK, as described in | ||
991 | :ref:`setting_up_ext_sdk_in_build`. | ||
992 | |||
993 | Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software | ||
994 | ------------------------------------------------------------------------------------------------------- | ||
995 | |||
996 | The ``devtool upgrade`` command upgrades an existing recipe to that of a | ||
997 | more up-to-date version found upstream. Throughout the life of software, | ||
998 | recipes continually undergo version upgrades by their upstream | ||
999 | publishers. You can use the ``devtool upgrade`` workflow to make sure | ||
1000 | your recipes you are using for builds are up-to-date with their upstream | ||
1001 | counterparts. | ||
1002 | |||
1003 | .. note:: | ||
1004 | |||
1005 | Several methods exist by which you can upgrade recipes --- | ||
1006 | ``devtool upgrade`` happens to be one. You can read about all the methods by | ||
1007 | which you can upgrade recipes in the | ||
1008 | :ref:`dev-manual/upgrading-recipes:upgrading recipes` section of the Yocto | ||
1009 | Project Development Tasks Manual. | ||
1010 | |||
1011 | The ``devtool upgrade`` command is flexible enough to allow you to specify | ||
1012 | source code revision and versioning schemes, extract code into or out of the | ||
1013 | ``devtool`` :ref:`devtool-the-workspace-layer-structure`, and work with any | ||
1014 | source file forms that the | ||
1015 | :ref:`bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support. | ||
1016 | |||
1017 | The following diagram shows the common development flow used with the | ||
1018 | ``devtool upgrade`` command: | ||
1019 | |||
1020 | .. image:: figures/sdk-devtool-upgrade-flow.png | ||
1021 | :width: 100% | ||
1022 | |||
1023 | #. *Initiate the Upgrade*: The top part of the flow shows the typical | ||
1024 | scenario by which you use the ``devtool upgrade`` command. The | ||
1025 | following conditions exist: | ||
1026 | |||
1027 | - The recipe exists in a local layer external to the ``devtool`` | ||
1028 | workspace. | ||
1029 | |||
1030 | - The source files for the new release exist in the same location | ||
1031 | pointed to by :term:`SRC_URI` | ||
1032 | in the recipe (e.g. a tarball with the new version number in the | ||
1033 | name, or as a different revision in the upstream Git repository). | ||
1034 | |||
1035 | A common situation is where third-party software has undergone a | ||
1036 | revision so that it has been upgraded. The recipe you have access to | ||
1037 | is likely in your own layer. Thus, you need to upgrade the recipe to | ||
1038 | use the newer version of the software:: | ||
1039 | |||
1040 | $ devtool upgrade -V version recipe | ||
1041 | |||
1042 | By default, the ``devtool upgrade`` command extracts source | ||
1043 | code into the ``sources`` directory in the | ||
1044 | :ref:`devtool-the-workspace-layer-structure`. | ||
1045 | If you want the code extracted to any other location, you need to | ||
1046 | provide the ``srctree`` positional argument with the command as follows:: | ||
1047 | |||
1048 | $ devtool upgrade -V version recipe srctree | ||
1049 | |||
1050 | .. note:: | ||
1051 | |||
1052 | In this example, the "-V" option specifies the new version. If you | ||
1053 | don't use "-V", the command upgrades the recipe to the latest | ||
1054 | version. | ||
1055 | |||
1056 | If the source files pointed to by the :term:`SRC_URI` statement in the | ||
1057 | recipe are in a Git repository, you must provide the "-S" option and | ||
1058 | specify a revision for the software. | ||
1059 | |||
1060 | Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable | ||
1061 | to locate the source code and any local patch files from other | ||
1062 | developers. The result is that the command sets up the source code, | ||
1063 | the new version of the recipe, and an append file all within the | ||
1064 | workspace. | ||
1065 | |||
1066 | Additionally, if you have any non-patch local files (i.e. files | ||
1067 | referred to with ``file://`` entries in :term:`SRC_URI` statement | ||
1068 | excluding ``*.patch/`` or ``*.diff``), these files are copied to an | ||
1069 | ``oe-local-files`` folder under the newly created source tree. | ||
1070 | Copying the files here gives you a convenient area from which you can | ||
1071 | modify the files. Any changes or additions you make to those files | ||
1072 | are incorporated into the build the next time you build the software | ||
1073 | just as are other changes you might have made to the source. | ||
1074 | |||
1075 | #. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen | ||
1076 | after upgrading the software to a new version. Conflicts occur | ||
1077 | if your recipe specifies some patch files in :term:`SRC_URI` that | ||
1078 | conflict with changes made in the new version of the software. For | ||
1079 | such cases, you need to resolve the conflicts by editing the source | ||
1080 | and following the normal ``git rebase`` conflict resolution process. | ||
1081 | |||
1082 | Before moving onto the next step, be sure to resolve any such | ||
1083 | conflicts created through use of a newer or different version of the | ||
1084 | software. | ||
1085 | |||
1086 | #. *Build the Recipe or Rebuild the Image*: The next step you take | ||
1087 | depends on what you are going to do with the new code. | ||
1088 | |||
1089 | If you need to eventually move the build output to the target | ||
1090 | hardware, use the following ``devtool`` command:: | ||
1091 | |||
1092 | $ devtool build recipe | ||
1093 | |||
1094 | On the other hand, if you want an image to contain the recipe's | ||
1095 | packages from the workspace for immediate deployment onto a device | ||
1096 | (e.g. for testing purposes), you can use the ``devtool build-image`` | ||
1097 | command:: | ||
1098 | |||
1099 | $ devtool build-image image | ||
1100 | |||
1101 | #. *Deploy the Build Output*: When you use the ``devtool build`` command | ||
1102 | or ``bitbake`` to build your recipe, you probably want to see if the | ||
1103 | resulting build output works as expected on target hardware. | ||
1104 | |||
1105 | .. note:: | ||
1106 | |||
1107 | This step assumes you have a previously built image that is | ||
1108 | already either running in QEMU or running on actual hardware. | ||
1109 | Also, it is assumed that for deployment of the image to the | ||
1110 | target, SSH is installed in the image and if the image is running | ||
1111 | on real hardware that you have network access to and from your | ||
1112 | development machine. | ||
1113 | |||
1114 | You can deploy your build output to that target hardware by using the | ||
1115 | ``devtool deploy-target`` command:: | ||
1116 | |||
1117 | $ devtool deploy-target recipe target | ||
1118 | |||
1119 | The target is a live target machine running as an SSH server. | ||
1120 | |||
1121 | You can, of course, also deploy the image you build using the | ||
1122 | ``devtool build-image`` command to actual hardware. However, | ||
1123 | ``devtool`` does not provide a specific command that allows you to do | ||
1124 | this. | ||
1125 | |||
1126 | #. *Finish Your Work With the Recipe*: The ``devtool finish`` command | ||
1127 | creates any patches corresponding to commits in the local Git | ||
1128 | repository, moves the new recipe to a more permanent layer, and then | ||
1129 | resets the recipe so that the recipe is built normally rather than | ||
1130 | from the workspace. | ||
1131 | |||
1132 | Any work you did in the ``oe-local-files`` directory is preserved in | ||
1133 | the original files next to the recipe during the ``devtool finish`` | ||
1134 | command. | ||
1135 | |||
1136 | If you specify a destination layer that is the same as the original | ||
1137 | source, then the old version of the recipe and associated files are | ||
1138 | removed prior to adding the new version:: | ||
1139 | |||
1140 | $ devtool finish recipe layer | ||
1141 | |||
1142 | .. note:: | ||
1143 | |||
1144 | Any changes you want to turn into patches must be committed to the | ||
1145 | Git repository in the source tree. | ||
1146 | |||
1147 | As a final process of the ``devtool finish`` command, the state of | ||
1148 | the standard layers and the upstream source is restored so that you | ||
1149 | can build the recipe from those areas rather than the workspace. | ||
1150 | |||
1151 | .. note:: | ||
1152 | |||
1153 | You can use the ``devtool reset`` command to put things back should you | ||
1154 | decide you do not want to proceed with your work. If you do use this | ||
1155 | command, realize that the source tree is preserved. | ||
1156 | |||
1157 | A Closer Look at ``devtool add`` | ||
1158 | ================================ | ||
1159 | |||
1160 | The ``devtool add`` command automatically creates a recipe based on the | ||
1161 | source tree you provide with the command. Currently, the command has | ||
1162 | support for the following: | ||
1163 | |||
1164 | - Autotools (``autoconf`` and ``automake``) | ||
1165 | |||
1166 | - CMake | ||
1167 | |||
1168 | - Scons | ||
1169 | |||
1170 | - ``qmake`` | ||
1171 | |||
1172 | - Plain ``Makefile`` | ||
1173 | |||
1174 | - Out-of-tree kernel module | ||
1175 | |||
1176 | - Binary package (i.e. "-b" option) | ||
1177 | |||
1178 | - Node.js module | ||
1179 | |||
1180 | - Python modules that use ``setuptools`` or ``distutils`` | ||
1181 | |||
1182 | Apart from binary packages, the determination of how a source tree | ||
1183 | should be treated is automatic based on the files present within that | ||
1184 | source tree. For example, if a ``CMakeLists.txt`` file is found, then | ||
1185 | the source tree is assumed to be using CMake and is treated accordingly. | ||
1186 | |||
1187 | .. note:: | ||
1188 | |||
1189 | In most cases, you need to edit the automatically generated recipe in | ||
1190 | order to make it build properly. Typically, you would go through | ||
1191 | several edit and build cycles until the recipe successfully builds. | ||
1192 | Once the recipe builds, you could use possible further iterations to | ||
1193 | test the recipe on the target device. | ||
1194 | |||
1195 | The remainder of this section covers specifics regarding how parts of | ||
1196 | the recipe are generated. | ||
1197 | |||
1198 | Name and Version | ||
1199 | ---------------- | ||
1200 | |||
1201 | If you do not specify a name and version on the command line, | ||
1202 | ``devtool add`` uses various metadata within the source tree in an | ||
1203 | attempt to determine the name and version of the software being built. | ||
1204 | Based on what the tool determines, ``devtool`` sets the name of the | ||
1205 | created recipe file accordingly. | ||
1206 | |||
1207 | If ``devtool`` cannot determine the name and version, the command prints | ||
1208 | an error. For such cases, you must re-run the command and provide the | ||
1209 | name and version, just the name, or just the version as part of the | ||
1210 | command line. | ||
1211 | |||
1212 | Sometimes the name or version determined from the source tree might be | ||
1213 | incorrect. For such a case, you must reset the recipe:: | ||
1214 | |||
1215 | $ devtool reset -n recipename | ||
1216 | |||
1217 | After running the ``devtool reset`` command, you need to | ||
1218 | run ``devtool add`` again and provide the name or the version. | ||
1219 | |||
1220 | Dependency Detection and Mapping | ||
1221 | -------------------------------- | ||
1222 | |||
1223 | The ``devtool add`` command attempts to detect build-time dependencies and map | ||
1224 | them to other recipes in the system. During this mapping, the command fills in | ||
1225 | the names of those recipes as part of the :term:`DEPENDS` variable within the | ||
1226 | recipe. If a dependency cannot be mapped, ``devtool`` places a comment | ||
1227 | in the recipe indicating such. The inability to map a dependency can | ||
1228 | result from naming not being recognized or because the dependency simply | ||
1229 | is not available. For cases where the dependency is not available, you | ||
1230 | must use the ``devtool add`` command to add an additional recipe that | ||
1231 | satisfies the dependency. Once you add that recipe, you need to update | ||
1232 | the :term:`DEPENDS` variable in the original recipe to include the new | ||
1233 | recipe. | ||
1234 | |||
1235 | If you need to add runtime dependencies, you can do so by adding the | ||
1236 | following to your recipe:: | ||
1237 | |||
1238 | RDEPENDS:${PN} += "dependency1 dependency2 ..." | ||
1239 | |||
1240 | .. note:: | ||
1241 | |||
1242 | The ``devtool add`` command often cannot distinguish between mandatory and | ||
1243 | optional dependencies. Consequently, some of the detected dependencies might | ||
1244 | in fact be optional. When in doubt, consult the documentation or the | ||
1245 | configure script for the software the recipe is building for further | ||
1246 | details. In some cases, you might find you can substitute the | ||
1247 | dependency with an option that disables the associated functionality | ||
1248 | passed to the configure script. | ||
1249 | |||
1250 | License Detection | ||
1251 | ----------------- | ||
1252 | |||
1253 | The ``devtool add`` command attempts to determine if the software you are | ||
1254 | adding is able to be distributed under a common, open-source license. If | ||
1255 | so, the command sets the :term:`LICENSE` value accordingly. | ||
1256 | You should double-check the value added by the command against the | ||
1257 | documentation or source files for the software you are building and, if | ||
1258 | necessary, update that :term:`LICENSE` value. | ||
1259 | |||
1260 | The ``devtool add`` command also sets the :term:`LIC_FILES_CHKSUM` | ||
1261 | value to point to all files that appear to be license-related. Realize | ||
1262 | that license statements often appear in comments at the top of source | ||
1263 | files or within the documentation. In such cases, the command does not | ||
1264 | recognize those license statements. Consequently, you might need to | ||
1265 | amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those | ||
1266 | comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly | ||
1267 | important for third-party software. The mechanism attempts to ensure | ||
1268 | correct licensing should you upgrade the recipe to a newer upstream | ||
1269 | version in future. Any change in licensing is detected and you receive | ||
1270 | an error prompting you to check the license text again. | ||
1271 | |||
1272 | If the ``devtool add`` command cannot determine licensing information, | ||
1273 | ``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the | ||
1274 | :term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue | ||
1275 | with development even though the settings are unlikely to be correct in | ||
1276 | all cases. You should check the documentation or source files for the | ||
1277 | software you are building to determine the actual license. | ||
1278 | |||
1279 | Adding Makefile-Only Software | ||
1280 | ----------------------------- | ||
1281 | |||
1282 | The use of Make by itself is very common in both proprietary and | ||
1283 | open-source software. Unfortunately, Makefiles are often not written | ||
1284 | with cross-compilation in mind. Thus, ``devtool add`` often cannot do | ||
1285 | very much to ensure that these Makefiles build correctly. It is very | ||
1286 | common, for example, to explicitly call ``gcc`` instead of using the | ||
1287 | :term:`CC` variable. Usually, in a | ||
1288 | cross-compilation environment, ``gcc`` is the compiler for the build | ||
1289 | host and the cross-compiler is named something similar to | ||
1290 | ``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to | ||
1291 | point to the associated sysroot for the target machine). | ||
1292 | |||
1293 | When writing a recipe for Makefile-only software, keep the following in | ||
1294 | mind: | ||
1295 | |||
1296 | - You probably need to patch the Makefile to use variables instead of | ||
1297 | hardcoding tools within the toolchain such as ``gcc`` and ``g++``. | ||
1298 | |||
1299 | - The environment in which Make runs is set up with various standard | ||
1300 | variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a | ||
1301 | similar manner to the environment set up by the SDK's environment | ||
1302 | setup script. One easy way to see these variables is to run the | ||
1303 | ``devtool build`` command on the recipe and then look in | ||
1304 | ``oe-logs/run.do_compile``. Towards the top of this file, there is | ||
1305 | a list of environment variables that are set. You can take | ||
1306 | advantage of these variables within the Makefile. | ||
1307 | |||
1308 | - If the Makefile sets a default for a variable using "=", that default | ||
1309 | overrides the value set in the environment, which is usually not | ||
1310 | desirable. For this case, you can either patch the Makefile so it | ||
1311 | sets the default using the "?=" operator, or you can alternatively | ||
1312 | force the value on the ``make`` command line. To force the value on | ||
1313 | the command line, add the variable setting to | ||
1314 | :term:`EXTRA_OEMAKE` or | ||
1315 | :term:`PACKAGECONFIG_CONFARGS` | ||
1316 | within the recipe. Here is an example using :term:`EXTRA_OEMAKE`:: | ||
1317 | |||
1318 | EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" | ||
1319 | |||
1320 | In the above example, | ||
1321 | single quotes are used around the variable settings as the values are | ||
1322 | likely to contain spaces because required default options are passed | ||
1323 | to the compiler. | ||
1324 | |||
1325 | - Hardcoding paths inside Makefiles is often problematic in a | ||
1326 | cross-compilation environment. This is particularly true because | ||
1327 | those hardcoded paths often point to locations on the build host and | ||
1328 | thus will either be read-only or will introduce contamination into | ||
1329 | the cross-compilation because they are specific to the build host | ||
1330 | rather than the target. Patching the Makefile to use prefix variables | ||
1331 | or other path variables is usually the way to handle this situation. | ||
1332 | |||
1333 | - Sometimes a Makefile runs target-specific commands such as | ||
1334 | ``ldconfig``. For such cases, you might be able to apply patches that | ||
1335 | remove these commands from the Makefile. | ||
1336 | |||
1337 | Adding Native Tools | ||
1338 | ------------------- | ||
1339 | |||
1340 | Often, you need to build additional tools that run on the :term:`Build Host` | ||
1341 | as opposed to the target. You should indicate this requirement by using one of | ||
1342 | the following methods when you run ``devtool add``: | ||
1343 | |||
1344 | - Specify the name of the recipe such that it ends with "-native". | ||
1345 | Specifying the name like this produces a recipe that only builds for | ||
1346 | the build host. | ||
1347 | |||
1348 | - Specify the "--also-native" option with the ``devtool add`` | ||
1349 | command. Specifying this option creates a recipe file that still | ||
1350 | builds for the target but also creates a variant with a "-native" | ||
1351 | suffix that builds for the build host. | ||
1352 | |||
1353 | .. note:: | ||
1354 | |||
1355 | If you need to add a tool that is shipped as part of a source tree | ||
1356 | that builds code for the target, you can typically accomplish this by | ||
1357 | building the native and target parts separately rather than within | ||
1358 | the same compilation process. Realize though that with the | ||
1359 | "--also-native" option, you can add the tool using just one | ||
1360 | recipe file. | ||
1361 | |||
1362 | Adding Node.js Modules | ||
1363 | ---------------------- | ||
1364 | |||
1365 | You can use the ``devtool add`` command two different ways to add | ||
1366 | Node.js modules: through ``npm`` or from a repository or local source. | ||
1367 | |||
1368 | Use the following form to add Node.js modules through ``npm``:: | ||
1369 | |||
1370 | $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" | ||
1371 | |||
1372 | The name and | ||
1373 | version parameters are mandatory. Lockdown and shrinkwrap files are | ||
1374 | generated and pointed to by the recipe in order to freeze the version | ||
1375 | that is fetched for the dependencies according to the first time. This | ||
1376 | also saves checksums that are verified on future fetches. Together, | ||
1377 | these behaviors ensure the reproducibility and integrity of the build. | ||
1378 | |||
1379 | .. note:: | ||
1380 | |||
1381 | - You must use quotes around the URL. ``devtool add`` does not | ||
1382 | require the quotes, but the shell considers ";" as a splitter | ||
1383 | between multiple commands. Thus, without the quotes, | ||
1384 | ``devtool add`` does not receive the other parts, which results in | ||
1385 | several "command not found" errors. | ||
1386 | |||
1387 | - In order to support adding Node.js modules, a ``nodejs`` recipe | ||
1388 | must be part of your SDK. | ||
1389 | |||
1390 | As mentioned earlier, you can also add Node.js modules directly from a | ||
1391 | repository or local source tree. To add modules this way, use | ||
1392 | ``devtool add`` in the following form:: | ||
1393 | |||
1394 | $ devtool add https://github.com/diversario/node-ssdp | ||
1395 | |||
1396 | In this example, ``devtool`` fetches the specified Git repository, detects the | ||
1397 | code as Node.js code, fetches dependencies using ``npm``, and sets | ||
1398 | :term:`SRC_URI` accordingly. | ||
1399 | |||
1400 | Working With Recipes | ||
1401 | ==================== | ||
1402 | |||
1403 | When building a recipe using the ``devtool build`` command, the typical | ||
1404 | build progresses as follows: | ||
1405 | |||
1406 | #. Fetch the source | ||
1407 | |||
1408 | #. Unpack the source | ||
1409 | |||
1410 | #. Configure the source | ||
1411 | |||
1412 | #. Compile the source | ||
1413 | |||
1414 | #. Install the build output | ||
1415 | |||
1416 | #. Package the installed output | ||
1417 | |||
1418 | For recipes in the workspace, fetching and unpacking is disabled as the | ||
1419 | source tree has already been prepared and is persistent. Each of these | ||
1420 | build steps is defined as a function (task), usually with a "do\_" prefix | ||
1421 | (e.g. :ref:`ref-tasks-fetch`, | ||
1422 | :ref:`ref-tasks-unpack`, and so | ||
1423 | forth). These functions are typically shell scripts but can instead be | ||
1424 | written in Python. | ||
1425 | |||
1426 | If you look at the contents of a recipe, you will see that the recipe | ||
1427 | does not include complete instructions for building the software. | ||
1428 | Instead, common functionality is encapsulated in classes inherited with | ||
1429 | the ``inherit`` directive. This technique leaves the recipe to describe | ||
1430 | just the things that are specific to the software being built. There is | ||
1431 | a :ref:`ref-classes-base` class that is implicitly inherited by all recipes | ||
1432 | and provides the functionality that most recipes typically need. | ||
1433 | |||
1434 | The remainder of this section presents information useful when working | ||
1435 | with recipes. | ||
1436 | |||
1437 | Finding Logs and Work Files | ||
1438 | --------------------------- | ||
1439 | |||
1440 | After the first run of the ``devtool build`` command, recipes that were | ||
1441 | previously created using the ``devtool add`` command or whose sources | ||
1442 | were modified using the ``devtool modify`` command contain symbolic | ||
1443 | links created within the source tree: | ||
1444 | |||
1445 | - ``oe-logs``: This link points to the directory in which log files and | ||
1446 | run scripts for each build step are created. | ||
1447 | |||
1448 | - ``oe-workdir``: This link points to the temporary work area for the | ||
1449 | recipe. The following locations under ``oe-workdir`` are particularly | ||
1450 | useful: | ||
1451 | |||
1452 | - ``image/``: Contains all of the files installed during the | ||
1453 | :ref:`ref-tasks-install` stage. | ||
1454 | Within a recipe, this directory is referred to by the expression | ||
1455 | ``${``\ :term:`D`\ ``}``. | ||
1456 | |||
1457 | - ``sysroot-destdir/``: Contains a subset of files installed within | ||
1458 | :ref:`ref-tasks-install` that have been put into the shared sysroot. For | ||
1459 | more information, see the | ||
1460 | ":ref:`dev-manual/new-recipe:sharing files between recipes`" section. | ||
1461 | |||
1462 | - ``packages-split/``: Contains subdirectories for each package | ||
1463 | produced by the recipe. For more information, see the | ||
1464 | ":ref:`sdk-manual/extensible:packaging`" section. | ||
1465 | |||
1466 | You can use these links to get more information on what is happening at | ||
1467 | each build step. | ||
1468 | |||
1469 | Setting Configure Arguments | ||
1470 | --------------------------- | ||
1471 | |||
1472 | If the software your recipe is building uses GNU autoconf, then a fixed | ||
1473 | set of arguments is passed to it to enable cross-compilation plus any | ||
1474 | extras specified by :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS` | ||
1475 | set within the recipe. If you wish to pass additional options, add them | ||
1476 | to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build | ||
1477 | tools have similar variables (e.g. :term:`EXTRA_OECMAKE` for CMake, | ||
1478 | :term:`EXTRA_OESCONS` for Scons, and so forth). If you need to pass anything on | ||
1479 | the ``make`` command line, you can use :term:`EXTRA_OEMAKE` or the | ||
1480 | :term:`PACKAGECONFIG_CONFARGS` variables to do so. | ||
1481 | |||
1482 | You can use the ``devtool configure-help`` command to help you set the | ||
1483 | arguments listed in the previous paragraph. The command determines the | ||
1484 | exact options being passed, and shows them to you along with any custom | ||
1485 | arguments specified through :term:`EXTRA_OECONF` or | ||
1486 | :term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you | ||
1487 | the output of the configure script's "--help" option as a | ||
1488 | reference. | ||
1489 | |||
1490 | Sharing Files Between Recipes | ||
1491 | ----------------------------- | ||
1492 | |||
1493 | Recipes often need to use files provided by other recipes on the | ||
1494 | :term:`Build Host`. For example, | ||
1495 | an application linking to a common library needs access to the library | ||
1496 | itself and its associated headers. The way this access is accomplished | ||
1497 | within the extensible SDK is through the sysroot. There is one sysroot per | ||
1498 | "machine" for which the SDK is being built. In practical terms, this | ||
1499 | means there is a sysroot for the target machine, and a sysroot for | ||
1500 | the build host. | ||
1501 | |||
1502 | Recipes should never write files directly into the sysroot. Instead, | ||
1503 | files should be installed into standard locations during the | ||
1504 | :ref:`ref-tasks-install` task within the ``${``\ :term:`D`\ ``}`` directory. A | ||
1505 | subset of these files automatically goes into the sysroot. The reason | ||
1506 | for this limitation is that almost all files that go into the sysroot | ||
1507 | are cataloged in manifests in order to ensure they can be removed later | ||
1508 | when a recipe is modified or removed. Thus, the sysroot is able to | ||
1509 | remain free from stale files. | ||
1510 | |||
1511 | Packaging | ||
1512 | --------- | ||
1513 | |||
1514 | Packaging is not always particularly relevant within the extensible SDK. | ||
1515 | However, if you examine how build output gets into the final image on | ||
1516 | the target device, it is important to understand packaging because the | ||
1517 | contents of the image are expressed in terms of packages and not | ||
1518 | recipes. | ||
1519 | |||
1520 | During the :ref:`ref-tasks-package` task, files installed during the | ||
1521 | :ref:`ref-tasks-install` task are split into one main package, which is almost | ||
1522 | always named the same as the recipe, and into several other packages. This | ||
1523 | separation exists because not all of those installed files are useful in every | ||
1524 | image. For example, you probably do not need any of the documentation installed | ||
1525 | in a production image. Consequently, for each recipe the documentation | ||
1526 | files are separated into a ``-doc`` package. Recipes that package | ||
1527 | software containing optional modules or plugins might undergo additional | ||
1528 | package splitting as well. | ||
1529 | |||
1530 | After building a recipe, you can see where files have gone by looking in | ||
1531 | the ``oe-workdir/packages-split`` directory, which contains a | ||
1532 | subdirectory for each package. Apart from some advanced cases, the | ||
1533 | :term:`PACKAGES` and :term:`FILES` variables controls | ||
1534 | splitting. The :term:`PACKAGES` variable lists all of the packages to be | ||
1535 | produced, while the :term:`FILES` variable specifies which files to include | ||
1536 | in each package by using an override to specify the package. For | ||
1537 | example, ``FILES:${PN}`` specifies the files to go into the main package | ||
1538 | (i.e. the main package has the same name as the recipe and | ||
1539 | ``${``\ :term:`PN`\ ``}`` evaluates to the | ||
1540 | recipe name). The order of the :term:`PACKAGES` value is significant. For | ||
1541 | each installed file, the first package whose :term:`FILES` value matches the | ||
1542 | file is the package into which the file goes. Both the :term:`PACKAGES` and | ||
1543 | :term:`FILES` variables have default values. Consequently, you might find | ||
1544 | you do not even need to set these variables in your recipe unless the | ||
1545 | software the recipe is building installs files into non-standard | ||
1546 | locations. | ||
1547 | |||
1548 | Restoring the Target Device to its Original State | ||
1549 | ================================================= | ||
1550 | |||
1551 | If you use the ``devtool deploy-target`` command to write a recipe's | ||
1552 | build output to the target, and you are working on an existing component | ||
1553 | of the system, then you might find yourself in a situation where you | ||
1554 | need to restore the original files that existed prior to running the | ||
1555 | ``devtool deploy-target`` command. Because the ``devtool deploy-target`` | ||
1556 | command backs up any files it overwrites, you can use the | ||
1557 | ``devtool undeploy-target`` command to restore those files and remove | ||
1558 | any other files the recipe deployed. Consider the following example:: | ||
1559 | |||
1560 | $ devtool undeploy-target lighttpd root@192.168.7.2 | ||
1561 | |||
1562 | If you have deployed | ||
1563 | multiple applications, you can remove them all using the "-a" option | ||
1564 | thus restoring the target device to its original state:: | ||
1565 | |||
1566 | $ devtool undeploy-target -a root@192.168.7.2 | ||
1567 | |||
1568 | Information about files deployed to | ||
1569 | the target as well as any backed up files are stored on the target | ||
1570 | itself. This storage, of course, requires some additional space on the | ||
1571 | target machine. | ||
1572 | |||
1573 | .. note:: | ||
1574 | |||
1575 | The ``devtool deploy-target`` and ``devtool undeploy-target`` commands do | ||
1576 | not currently interact with any package management system on the target | ||
1577 | device (e.g. RPM or OPKG). Consequently, you should not intermingle | ||
1578 | ``devtool deploy-target`` and package manager operations on the target | ||
1579 | device. Doing so could result in a conflicting set of files. | ||
1580 | 263 | ||
1581 | Installing Additional Items Into the Extensible SDK | 264 | Installing Additional Items Into the Extensible SDK |
1582 | =================================================== | 265 | =================================================== |
diff --git a/documentation/what-i-wish-id-known.rst b/documentation/what-i-wish-id-known.rst index 5bc55804f6..836097910d 100644 --- a/documentation/what-i-wish-id-known.rst +++ b/documentation/what-i-wish-id-known.rst | |||
@@ -136,7 +136,7 @@ contact us with other suggestions. | |||
136 | valuable links: :ref:`dev-manual/development-shell:Using a Development | 136 | valuable links: :ref:`dev-manual/development-shell:Using a Development |
137 | Shell` for information on how to build and run a specific task using | 137 | Shell` for information on how to build and run a specific task using |
138 | devshell. Also, the :ref:`SDK manual shows how to build out a specific recipe | 138 | devshell. Also, the :ref:`SDK manual shows how to build out a specific recipe |
139 | <sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component>`. | 139 | <dev-manual/devtool:use \`\`devtool modify\`\` to modify the source of an existing component>`. |
140 | 140 | ||
141 | #. **An ambiguous definition: Package vs Recipe:** | 141 | #. **An ambiguous definition: Package vs Recipe:** |
142 | A recipe contains instructions the build system uses to create | 142 | A recipe contains instructions the build system uses to create |