diff options
Diffstat (limited to 'documentation/dev-manual/building.rst')
-rw-r--r-- | documentation/dev-manual/building.rst | 942 |
1 files changed, 942 insertions, 0 deletions
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst new file mode 100644 index 0000000000..fe502690dd --- /dev/null +++ b/documentation/dev-manual/building.rst | |||
@@ -0,0 +1,942 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Building | ||
4 | ******** | ||
5 | |||
6 | This section describes various build procedures, such as the steps | ||
7 | needed for a simple build, building a target for multiple configurations, | ||
8 | generating an image for more than one machine, and so forth. | ||
9 | |||
10 | Building a Simple Image | ||
11 | ======================= | ||
12 | |||
13 | In the development environment, you need to build an image whenever you | ||
14 | change hardware support, add or change system libraries, or add or | ||
15 | change services that have dependencies. There are several methods that allow | ||
16 | you to build an image within the Yocto Project. This section presents | ||
17 | the basic steps you need to build a simple image using BitBake from a | ||
18 | build host running Linux. | ||
19 | |||
20 | .. note:: | ||
21 | |||
22 | - For information on how to build an image using | ||
23 | :term:`Toaster`, see the | ||
24 | :doc:`/toaster-manual/index`. | ||
25 | |||
26 | - For information on how to use ``devtool`` to build images, see the | ||
27 | ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
28 | section in the Yocto Project Application Development and the | ||
29 | Extensible Software Development Kit (eSDK) manual. | ||
30 | |||
31 | - For a quick example on how to build an image using the | ||
32 | OpenEmbedded build system, see the | ||
33 | :doc:`/brief-yoctoprojectqs/index` document. | ||
34 | |||
35 | - You can also use the `Yocto Project BitBake | ||
36 | <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`__ | ||
37 | extension for Visual Studio Code to build images. | ||
38 | |||
39 | The build process creates an entire Linux distribution from source and | ||
40 | places it in your :term:`Build Directory` under ``tmp/deploy/images``. For | ||
41 | detailed information on the build process using BitBake, see the | ||
42 | ":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview | ||
43 | and Concepts Manual. | ||
44 | |||
45 | The following figure and list overviews the build process: | ||
46 | |||
47 | .. image:: figures/bitbake-build-flow.png | ||
48 | :width: 100% | ||
49 | |||
50 | #. *Set up Your Host Development System to Support Development Using the | ||
51 | Yocto Project*: See the ":doc:`start`" section for options on how to get a | ||
52 | build host ready to use the Yocto Project. | ||
53 | |||
54 | #. *Initialize the Build Environment:* Initialize the build environment | ||
55 | by sourcing the build environment script (i.e. | ||
56 | :ref:`structure-core-script`):: | ||
57 | |||
58 | $ source oe-init-build-env [build_dir] | ||
59 | |||
60 | When you use the initialization script, the OpenEmbedded build system | ||
61 | uses ``build`` as the default :term:`Build Directory` in your current work | ||
62 | directory. You can use a `build_dir` argument with the script to | ||
63 | specify a different :term:`Build Directory`. | ||
64 | |||
65 | .. note:: | ||
66 | |||
67 | A common practice is to use a different :term:`Build Directory` for | ||
68 | different targets; for example, ``~/build/x86`` for a ``qemux86`` | ||
69 | target, and ``~/build/arm`` for a ``qemuarm`` target. In any | ||
70 | event, it's typically cleaner to locate the :term:`Build Directory` | ||
71 | somewhere outside of your source directory. | ||
72 | |||
73 | #. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the | ||
74 | ``conf/local.conf`` configuration file, which is found in the | ||
75 | :term:`Build Directory`, is set up how you want it. This file defines many | ||
76 | aspects of the build environment including the target machine architecture | ||
77 | through the :term:`MACHINE` variable, the packaging format used during | ||
78 | the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download | ||
79 | directory through the :term:`DL_DIR` variable. | ||
80 | |||
81 | #. *Build the Image:* Build the image using the ``bitbake`` command:: | ||
82 | |||
83 | $ bitbake target | ||
84 | |||
85 | .. note:: | ||
86 | |||
87 | For information on BitBake, see the :doc:`bitbake:index`. | ||
88 | |||
89 | The target is the name of the recipe you want to build. Common | ||
90 | targets are the images in ``meta/recipes-core/images``, | ||
91 | ``meta/recipes-sato/images``, and so forth all found in the | ||
92 | :term:`Source Directory`. Alternatively, the target | ||
93 | can be the name of a recipe for a specific piece of software such as | ||
94 | BusyBox. For more details about the images the OpenEmbedded build | ||
95 | system supports, see the | ||
96 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
97 | Project Reference Manual. | ||
98 | |||
99 | As an example, the following command builds the | ||
100 | ``core-image-minimal`` image:: | ||
101 | |||
102 | $ bitbake core-image-minimal | ||
103 | |||
104 | Once an | ||
105 | image has been built, it often needs to be installed. The images and | ||
106 | kernels built by the OpenEmbedded build system are placed in the | ||
107 | :term:`Build Directory` in ``tmp/deploy/images``. For information on how to | ||
108 | run pre-built images such as ``qemux86`` and ``qemuarm``, see the | ||
109 | :doc:`/sdk-manual/index` manual. For | ||
110 | information about how to install these images, see the documentation | ||
111 | for your particular board or machine. | ||
112 | |||
113 | Building Images for Multiple Targets Using Multiple Configurations | ||
114 | ================================================================== | ||
115 | |||
116 | You can use a single ``bitbake`` command to build multiple images or | ||
117 | packages for different targets where each image or package requires a | ||
118 | different configuration (multiple configuration builds). The builds, in | ||
119 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
120 | section uses that term throughout. | ||
121 | |||
122 | This section describes how to set up for multiple configuration builds | ||
123 | and how to account for cross-build dependencies between the | ||
124 | multiconfigs. | ||
125 | |||
126 | Setting Up and Running a Multiple Configuration Build | ||
127 | ----------------------------------------------------- | ||
128 | |||
129 | To accomplish a multiple configuration build, you must define each | ||
130 | target's configuration separately using a parallel configuration file in | ||
131 | the :term:`Build Directory` or configuration directory within a layer, and you | ||
132 | must follow a required file hierarchy. Additionally, you must enable the | ||
133 | multiple configuration builds in your ``local.conf`` file. | ||
134 | |||
135 | Follow these steps to set up and execute multiple configuration builds: | ||
136 | |||
137 | - *Create Separate Configuration Files*: You need to create a single | ||
138 | configuration file for each build target (each multiconfig). | ||
139 | The configuration definitions are implementation dependent but often | ||
140 | each configuration file will define the machine and the | ||
141 | temporary directory BitBake uses for the build. Whether the same | ||
142 | temporary directory (:term:`TMPDIR`) can be shared will depend on what is | ||
143 | similar and what is different between the configurations. Multiple MACHINE | ||
144 | targets can share the same (:term:`TMPDIR`) as long as the rest of the | ||
145 | configuration is the same, multiple :term:`DISTRO` settings would need separate | ||
146 | (:term:`TMPDIR`) directories. | ||
147 | |||
148 | For example, consider a scenario with two different multiconfigs for the same | ||
149 | :term:`MACHINE`: "qemux86" built | ||
150 | for two distributions such as "poky" and "poky-lsb". In this case, | ||
151 | you would need to use the different :term:`TMPDIR`. | ||
152 | |||
153 | Here is an example showing the minimal statements needed in a | ||
154 | configuration file for a "qemux86" target whose temporary build | ||
155 | directory is ``tmpmultix86``:: | ||
156 | |||
157 | MACHINE = "qemux86" | ||
158 | TMPDIR = "${TOPDIR}/tmpmultix86" | ||
159 | |||
160 | The location for these multiconfig configuration files is specific. | ||
161 | They must reside in the current :term:`Build Directory` in a sub-directory of | ||
162 | ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory | ||
163 | under a directory named ``multiconfig``. Here is an example that defines | ||
164 | two configuration files for the "x86" and "arm" multiconfigs: | ||
165 | |||
166 | .. image:: figures/multiconfig_files.png | ||
167 | :align: center | ||
168 | :width: 50% | ||
169 | |||
170 | The usual :term:`BBPATH` search path is used to locate multiconfig files in | ||
171 | a similar way to other conf files. | ||
172 | |||
173 | - *Add the BitBake Multi-configuration Variable to the Local | ||
174 | Configuration File*: Use the | ||
175 | :term:`BBMULTICONFIG` | ||
176 | variable in your ``conf/local.conf`` configuration file to specify | ||
177 | each multiconfig. Continuing with the example from the previous | ||
178 | figure, the :term:`BBMULTICONFIG` variable needs to enable two | ||
179 | multiconfigs: "x86" and "arm" by specifying each configuration file:: | ||
180 | |||
181 | BBMULTICONFIG = "x86 arm" | ||
182 | |||
183 | .. note:: | ||
184 | |||
185 | A "default" configuration already exists by definition. This | ||
186 | configuration is named: "" (i.e. empty string) and is defined by | ||
187 | the variables coming from your ``local.conf`` | ||
188 | file. Consequently, the previous example actually adds two | ||
189 | additional configurations to your build: "arm" and "x86" along | ||
190 | with "". | ||
191 | |||
192 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
193 | the multiple configuration build:: | ||
194 | |||
195 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
196 | |||
197 | For the example in this section, the following command applies:: | ||
198 | |||
199 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
200 | |||
201 | The previous BitBake command builds a ``core-image-minimal`` image | ||
202 | that is configured through the ``x86.conf`` configuration file, a | ||
203 | ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
204 | configuration file and a ``core-image-base`` that is configured | ||
205 | through your ``local.conf`` configuration file. | ||
206 | |||
207 | .. note:: | ||
208 | |||
209 | Support for multiple configuration builds in the Yocto Project &DISTRO; | ||
210 | (&DISTRO_NAME;) Release does not include Shared State (sstate) | ||
211 | optimizations. Consequently, if a build uses the same object twice | ||
212 | in, for example, two different :term:`TMPDIR` | ||
213 | directories, the build either loads from an existing sstate cache for | ||
214 | that build at the start or builds the object fresh. | ||
215 | |||
216 | Enabling Multiple Configuration Build Dependencies | ||
217 | -------------------------------------------------- | ||
218 | |||
219 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
220 | multiple configuration build. For example, suppose that in order to | ||
221 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
222 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
223 | essentially that the | ||
224 | :ref:`ref-tasks-image` task in the | ||
225 | ``core-image-sato`` recipe depends on the completion of the | ||
226 | :ref:`ref-tasks-rootfs` task of the | ||
227 | ``core-image-minimal`` recipe. | ||
228 | |||
229 | To enable dependencies in a multiple configuration build, you must | ||
230 | declare the dependencies in the recipe using the following statement | ||
231 | form:: | ||
232 | |||
233 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
234 | |||
235 | To better show how to use this statement, consider the example scenario | ||
236 | from the first paragraph of this section. The following statement needs | ||
237 | to be added to the recipe that builds the ``core-image-sato`` image:: | ||
238 | |||
239 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
240 | |||
241 | In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The | ||
242 | task on which the :ref:`ref-tasks-image` task in the recipe depends is the | ||
243 | :ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated | ||
244 | with the "arm" multiconfig. | ||
245 | |||
246 | Once you set up this dependency, you can build the "x86" multiconfig | ||
247 | using a BitBake command as follows:: | ||
248 | |||
249 | $ bitbake mc:x86:core-image-sato | ||
250 | |||
251 | This command executes all the tasks needed to create the | ||
252 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
253 | dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the | ||
254 | "arm" multiconfig build. | ||
255 | |||
256 | Having a recipe depend on the root filesystem of another build might not | ||
257 | seem that useful. Consider this change to the statement in the | ||
258 | ``core-image-sato`` recipe:: | ||
259 | |||
260 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
261 | |||
262 | In this case, BitBake must | ||
263 | create the ``core-image-minimal`` image for the "arm" build since the | ||
264 | "x86" build depends on it. | ||
265 | |||
266 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
267 | and have separate configuration files, BitBake places the artifacts for | ||
268 | each build in the respective temporary build directories (i.e. | ||
269 | :term:`TMPDIR`). | ||
270 | |||
271 | Building an Initial RAM Filesystem (Initramfs) Image | ||
272 | ==================================================== | ||
273 | |||
274 | An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root | ||
275 | filesystem used for early system initialization, typically providing tools and | ||
276 | loading modules needed to locate and mount the final root filesystem. | ||
277 | |||
278 | Follow these steps to create an :term:`Initramfs` image: | ||
279 | |||
280 | #. *Create the Initramfs Image Recipe:* You can reference the | ||
281 | ``core-image-minimal-initramfs.bb`` recipe found in the | ||
282 | ``meta/recipes-core`` directory of the :term:`Source Directory` | ||
283 | as an example from which to work. | ||
284 | |||
285 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel | ||
286 | Image:* If you want the :term:`Initramfs` image that is built to be bundled | ||
287 | in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE` | ||
288 | variable to ``"1"`` in your ``local.conf`` configuration file and set the | ||
289 | :term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image. | ||
290 | |||
291 | Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs` | ||
292 | image to be unpacked into the ``${B}/usr/`` directory. The unpacked | ||
293 | :term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the | ||
294 | :term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs` | ||
295 | image to be built into the kernel normally. | ||
296 | |||
297 | #. *Optionally Add Items to the Initramfs Image Through the Initramfs | ||
298 | Image Recipe:* If you add items to the :term:`Initramfs` image by way of its | ||
299 | recipe, you should use :term:`PACKAGE_INSTALL` rather than | ||
300 | :term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of | ||
301 | what is added to the image as compared to the defaults you might not | ||
302 | necessarily want that are set by the :ref:`ref-classes-image` | ||
303 | or :ref:`ref-classes-core-image` classes. | ||
304 | |||
305 | #. *Build the Kernel Image and the Initramfs Image:* Build your kernel | ||
306 | image using BitBake. Because the :term:`Initramfs` image recipe is a | ||
307 | dependency of the kernel image, the :term:`Initramfs` image is built as well | ||
308 | and bundled with the kernel image if you used the | ||
309 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. | ||
310 | |||
311 | Bundling an Initramfs Image From a Separate Multiconfig | ||
312 | ------------------------------------------------------- | ||
313 | |||
314 | There may be a case where we want to build an :term:`Initramfs` image which does not | ||
315 | inherit the same distro policy as our main image, for example, we may want | ||
316 | our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs` | ||
317 | image to keep a smaller footprint. However, by performing the steps mentioned | ||
318 | above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us | ||
319 | to override it. | ||
320 | |||
321 | To achieve this, you need to perform some additional steps: | ||
322 | |||
323 | #. *Create a multiconfig for your Initramfs image:* You can perform the steps | ||
324 | on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig. | ||
325 | For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and | ||
326 | contains the variables:: | ||
327 | |||
328 | TMPDIR="${TOPDIR}/tmp-initramfscfg" | ||
329 | TCLIBC="musl" | ||
330 | |||
331 | #. *Set additional Initramfs variables on your main configuration:* | ||
332 | Additionally, on your main configuration (``local.conf``) you need to set the | ||
333 | variables:: | ||
334 | |||
335 | INITRAMFS_MULTICONFIG = "initramfscfg" | ||
336 | INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}" | ||
337 | |||
338 | The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
339 | are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE` | ||
340 | to be built coming from the ``initramfscfg`` multiconfig, and to let the | ||
341 | buildsystem know where the :term:`INITRAMFS_IMAGE` will be located. | ||
342 | |||
343 | Building a system with such configuration will build the kernel using the | ||
344 | main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the | ||
345 | selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
346 | instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel | ||
347 | but a glibc based main image. | ||
348 | |||
349 | The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE` | ||
350 | or to build a different :term:`DISTRO` for it such as ``poky-tiny``. | ||
351 | |||
352 | |||
353 | Building a Tiny System | ||
354 | ====================== | ||
355 | |||
356 | Very small distributions have some significant advantages such as | ||
357 | requiring less on-die or in-package memory (cheaper), better performance | ||
358 | through efficient cache usage, lower power requirements due to less | ||
359 | memory, faster boot times, and reduced development overhead. Some | ||
360 | real-world examples where a very small distribution gives you distinct | ||
361 | advantages are digital cameras, medical devices, and small headless | ||
362 | systems. | ||
363 | |||
364 | This section presents information that shows you how you can trim your | ||
365 | distribution to even smaller sizes than the ``poky-tiny`` distribution, | ||
366 | which is around 5 Mbytes, that can be built out-of-the-box using the | ||
367 | Yocto Project. | ||
368 | |||
369 | Tiny System Overview | ||
370 | -------------------- | ||
371 | |||
372 | The following list presents the overall steps you need to consider and | ||
373 | perform to create distributions with smaller root filesystems, achieve | ||
374 | faster boot times, maintain your critical functionality, and avoid | ||
375 | initial RAM disks: | ||
376 | |||
377 | - :ref:`Determine your goals and guiding principles | ||
378 | <dev-manual/building:goals and guiding principles>` | ||
379 | |||
380 | - :ref:`dev-manual/building:understand what contributes to your image size` | ||
381 | |||
382 | - :ref:`Reduce the size of the root filesystem | ||
383 | <dev-manual/building:trim the root filesystem>` | ||
384 | |||
385 | - :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>` | ||
386 | |||
387 | - :ref:`dev-manual/building:remove package management requirements` | ||
388 | |||
389 | - :ref:`dev-manual/building:look for other ways to minimize size` | ||
390 | |||
391 | - :ref:`dev-manual/building:iterate on the process` | ||
392 | |||
393 | Goals and Guiding Principles | ||
394 | ---------------------------- | ||
395 | |||
396 | Before you can reach your destination, you need to know where you are | ||
397 | going. Here is an example list that you can use as a guide when creating | ||
398 | very small distributions: | ||
399 | |||
400 | - Determine how much space you need (e.g. a kernel that is 1 Mbyte or | ||
401 | less and a root filesystem that is 3 Mbytes or less). | ||
402 | |||
403 | - Find the areas that are currently taking 90% of the space and | ||
404 | concentrate on reducing those areas. | ||
405 | |||
406 | - Do not create any difficult "hacks" to achieve your goals. | ||
407 | |||
408 | - Leverage the device-specific options. | ||
409 | |||
410 | - Work in a separate layer so that you keep changes isolated. For | ||
411 | information on how to create layers, see the | ||
412 | ":ref:`dev-manual/layers:understanding and creating layers`" section. | ||
413 | |||
414 | Understand What Contributes to Your Image Size | ||
415 | ---------------------------------------------- | ||
416 | |||
417 | It is easiest to have something to start with when creating your own | ||
418 | distribution. You can use the Yocto Project out-of-the-box to create the | ||
419 | ``poky-tiny`` distribution. Ultimately, you will want to make changes in | ||
420 | your own distribution that are likely modeled after ``poky-tiny``. | ||
421 | |||
422 | .. note:: | ||
423 | |||
424 | To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your | ||
425 | ``local.conf`` file to "poky-tiny" as described in the | ||
426 | ":ref:`dev-manual/custom-distribution:creating your own distribution`" | ||
427 | section. | ||
428 | |||
429 | Understanding some memory concepts will help you reduce the system size. | ||
430 | Memory consists of static, dynamic, and temporary memory. Static memory | ||
431 | is the TEXT (code), DATA (initialized data in the code), and BSS | ||
432 | (uninitialized data) sections. Dynamic memory represents memory that is | ||
433 | allocated at runtime: stacks, hash tables, and so forth. Temporary | ||
434 | memory is recovered after the boot process. This memory consists of | ||
435 | memory used for decompressing the kernel and for the ``__init__`` | ||
436 | functions. | ||
437 | |||
438 | To help you see where you currently are with kernel and root filesystem | ||
439 | sizes, you can use two tools found in the :term:`Source Directory` | ||
440 | in the | ||
441 | ``scripts/tiny/`` directory: | ||
442 | |||
443 | - ``ksize.py``: Reports component sizes for the kernel build objects. | ||
444 | |||
445 | - ``dirsize.py``: Reports component sizes for the root filesystem. | ||
446 | |||
447 | This next tool and command help you organize configuration fragments and | ||
448 | view file dependencies in a human-readable form: | ||
449 | |||
450 | - ``merge_config.sh``: Helps you manage configuration files and | ||
451 | fragments within the kernel. With this tool, you can merge individual | ||
452 | configuration fragments together. The tool allows you to make | ||
453 | overrides and warns you of any missing configuration options. The | ||
454 | tool is ideal for allowing you to iterate on configurations, create | ||
455 | minimal configurations, and create configuration files for different | ||
456 | machines without having to duplicate your process. | ||
457 | |||
458 | The ``merge_config.sh`` script is part of the Linux Yocto kernel Git | ||
459 | repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, | ||
460 | ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` | ||
461 | directory. | ||
462 | |||
463 | For more information on configuration fragments, see the | ||
464 | ":ref:`kernel-dev/common:creating configuration fragments`" | ||
465 | section in the Yocto Project Linux Kernel Development Manual. | ||
466 | |||
467 | - ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command | ||
468 | with these options brings up a Dependency Explorer from which you can | ||
469 | view file dependencies. Understanding these dependencies allows you | ||
470 | to make informed decisions when cutting out various pieces of the | ||
471 | kernel and root filesystem. | ||
472 | |||
473 | Trim the Root Filesystem | ||
474 | ------------------------ | ||
475 | |||
476 | The root filesystem is made up of packages for booting, libraries, and | ||
477 | applications. To change things, you can configure how the packaging | ||
478 | happens, which changes the way you build them. You can also modify the | ||
479 | filesystem itself or select a different filesystem. | ||
480 | |||
481 | First, find out what is hogging your root filesystem by running the | ||
482 | ``dirsize.py`` script from your root directory:: | ||
483 | |||
484 | $ cd root-directory-of-image | ||
485 | $ dirsize.py 100000 > dirsize-100k.log | ||
486 | $ cat dirsize-100k.log | ||
487 | |||
488 | You can apply a filter to the script to ignore files | ||
489 | under a certain size. The previous example filters out any files below | ||
490 | 100 Kbytes. The sizes reported by the tool are uncompressed, and thus | ||
491 | will be smaller by a relatively constant factor in a compressed root | ||
492 | filesystem. When you examine your log file, you can focus on areas of | ||
493 | the root filesystem that take up large amounts of memory. | ||
494 | |||
495 | You need to be sure that what you eliminate does not cripple the | ||
496 | functionality you need. One way to see how packages relate to each other | ||
497 | is by using the Dependency Explorer UI with the BitBake command:: | ||
498 | |||
499 | $ cd image-directory | ||
500 | $ bitbake -u taskexp -g image | ||
501 | |||
502 | Use the interface to | ||
503 | select potential packages you wish to eliminate and see their dependency | ||
504 | relationships. | ||
505 | |||
506 | When deciding how to reduce the size, get rid of packages that result in | ||
507 | minimal impact on the feature set. For example, you might not need a VGA | ||
508 | display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` | ||
509 | instead of ``udev``. | ||
510 | |||
511 | Use your ``local.conf`` file to make changes. For example, to eliminate | ||
512 | ``udev`` and ``glib``, set the following in the local configuration | ||
513 | file:: | ||
514 | |||
515 | VIRTUAL-RUNTIME_dev_manager = "" | ||
516 | |||
517 | Finally, you should consider exactly the type of root filesystem you | ||
518 | need to meet your needs while also reducing its size. For example, | ||
519 | consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an | ||
520 | :term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1 | ||
521 | Mbyte journal. If you are okay with running read-only, you do not need | ||
522 | this journal. | ||
523 | |||
524 | .. note:: | ||
525 | |||
526 | After each round of elimination, you need to rebuild your system and | ||
527 | then use the tools to see the effects of your reductions. | ||
528 | |||
529 | Trim the Kernel | ||
530 | --------------- | ||
531 | |||
532 | The kernel is built by including policies for hardware-independent | ||
533 | aspects. What subsystems do you enable? For what architecture are you | ||
534 | building? Which drivers do you build by default? | ||
535 | |||
536 | .. note:: | ||
537 | |||
538 | You can modify the kernel source if you want to help with boot time. | ||
539 | |||
540 | Run the ``ksize.py`` script from the top-level Linux build directory to | ||
541 | get an idea of what is making up the kernel:: | ||
542 | |||
543 | $ cd top-level-linux-build-directory | ||
544 | $ ksize.py > ksize.log | ||
545 | $ cat ksize.log | ||
546 | |||
547 | When you examine the log, you will see how much space is taken up with | ||
548 | the built-in ``.o`` files for drivers, networking, core kernel files, | ||
549 | filesystem, sound, and so forth. The sizes reported by the tool are | ||
550 | uncompressed, and thus will be smaller by a relatively constant factor | ||
551 | in a compressed kernel image. Look to reduce the areas that are large | ||
552 | and taking up around the "90% rule." | ||
553 | |||
554 | To examine, or drill down, into any particular area, use the ``-d`` | ||
555 | option with the script:: | ||
556 | |||
557 | $ ksize.py -d > ksize.log | ||
558 | |||
559 | Using this option | ||
560 | breaks out the individual file information for each area of the kernel | ||
561 | (e.g. drivers, networking, and so forth). | ||
562 | |||
563 | Use your log file to see what you can eliminate from the kernel based on | ||
564 | features you can let go. For example, if you are not going to need | ||
565 | sound, you do not need any drivers that support sound. | ||
566 | |||
567 | After figuring out what to eliminate, you need to reconfigure the kernel | ||
568 | to reflect those changes during the next build. You could run | ||
569 | ``menuconfig`` and make all your changes at once. However, that makes it | ||
570 | difficult to see the effects of your individual eliminations and also | ||
571 | makes it difficult to replicate the changes for perhaps another target | ||
572 | device. A better method is to start with no configurations using | ||
573 | ``allnoconfig``, create configuration fragments for individual changes, | ||
574 | and then manage the fragments into a single configuration file using | ||
575 | ``merge_config.sh``. The tool makes it easy for you to iterate using the | ||
576 | configuration change and build cycle. | ||
577 | |||
578 | Each time you make configuration changes, you need to rebuild the kernel | ||
579 | and check to see what impact your changes had on the overall size. | ||
580 | |||
581 | Remove Package Management Requirements | ||
582 | -------------------------------------- | ||
583 | |||
584 | Packaging requirements add size to the image. One way to reduce the size | ||
585 | of the image is to remove all the packaging requirements from the image. | ||
586 | This reduction includes both removing the package manager and its unique | ||
587 | dependencies as well as removing the package management data itself. | ||
588 | |||
589 | To eliminate all the packaging requirements for an image, be sure that | ||
590 | "package-management" is not part of your | ||
591 | :term:`IMAGE_FEATURES` | ||
592 | statement for the image. When you remove this feature, you are removing | ||
593 | the package manager as well as its dependencies from the root | ||
594 | filesystem. | ||
595 | |||
596 | Look for Other Ways to Minimize Size | ||
597 | ------------------------------------ | ||
598 | |||
599 | Depending on your particular circumstances, other areas that you can | ||
600 | trim likely exist. The key to finding these areas is through tools and | ||
601 | methods described here combined with experimentation and iteration. Here | ||
602 | are a couple of areas to experiment with: | ||
603 | |||
604 | - ``glibc``: In general, follow this process: | ||
605 | |||
606 | #. Remove ``glibc`` features from | ||
607 | :term:`DISTRO_FEATURES` | ||
608 | that you think you do not need. | ||
609 | |||
610 | #. Build your distribution. | ||
611 | |||
612 | #. If the build fails due to missing symbols in a package, determine | ||
613 | if you can reconfigure the package to not need those features. For | ||
614 | example, change the configuration to not support wide character | ||
615 | support as is done for ``ncurses``. Or, if support for those | ||
616 | characters is needed, determine what ``glibc`` features provide | ||
617 | the support and restore the configuration. | ||
618 | |||
619 | 4. Rebuild and repeat the process. | ||
620 | |||
621 | - ``busybox``: For BusyBox, use a process similar as described for | ||
622 | ``glibc``. A difference is you will need to boot the resulting system | ||
623 | to see if you are able to do everything you expect from the running | ||
624 | system. You need to be sure to integrate configuration fragments into | ||
625 | Busybox because BusyBox handles its own core features and then allows | ||
626 | you to add configuration fragments on top. | ||
627 | |||
628 | Iterate on the Process | ||
629 | ---------------------- | ||
630 | |||
631 | If you have not reached your goals on system size, you need to iterate | ||
632 | on the process. The process is the same. Use the tools and see just what | ||
633 | is taking up 90% of the root filesystem and the kernel. Decide what you | ||
634 | can eliminate without limiting your device beyond what you need. | ||
635 | |||
636 | Depending on your system, a good place to look might be Busybox, which | ||
637 | provides a stripped down version of Unix tools in a single, executable | ||
638 | file. You might be able to drop virtual terminal services or perhaps | ||
639 | ipv6. | ||
640 | |||
641 | Building Images for More than One Machine | ||
642 | ========================================= | ||
643 | |||
644 | A common scenario developers face is creating images for several | ||
645 | different machines that use the same software environment. In this | ||
646 | situation, it is tempting to set the tunings and optimization flags for | ||
647 | each build specifically for the targeted hardware (i.e. "maxing out" the | ||
648 | tunings). Doing so can considerably add to build times and package feed | ||
649 | maintenance collectively for the machines. For example, selecting tunes | ||
650 | that are extremely specific to a CPU core used in a system might enable | ||
651 | some micro optimizations in GCC for that particular system but would | ||
652 | otherwise not gain you much of a performance difference across the other | ||
653 | systems as compared to using a more general tuning across all the builds | ||
654 | (e.g. setting :term:`DEFAULTTUNE` | ||
655 | specifically for each machine's build). Rather than "max out" each | ||
656 | build's tunings, you can take steps that cause the OpenEmbedded build | ||
657 | system to reuse software across the various machines where it makes | ||
658 | sense. | ||
659 | |||
660 | If build speed and package feed maintenance are considerations, you | ||
661 | should consider the points in this section that can help you optimize | ||
662 | your tunings to best consider build times and package feed maintenance. | ||
663 | |||
664 | - *Share the :term:`Build Directory`:* If at all possible, share the | ||
665 | :term:`TMPDIR` across builds. The Yocto Project supports switching between | ||
666 | different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice | ||
667 | is well supported and regularly used by developers when building for | ||
668 | multiple machines. When you use the same :term:`TMPDIR` for multiple | ||
669 | machine builds, the OpenEmbedded build system can reuse the existing native | ||
670 | and often cross-recipes for multiple machines. Thus, build time decreases. | ||
671 | |||
672 | .. note:: | ||
673 | |||
674 | If :term:`DISTRO` settings change or fundamental configuration settings | ||
675 | such as the filesystem layout, you need to work with a clean :term:`TMPDIR`. | ||
676 | Sharing :term:`TMPDIR` under these circumstances might work but since it is | ||
677 | not guaranteed, you should use a clean :term:`TMPDIR`. | ||
678 | |||
679 | - *Enable the Appropriate Package Architecture:* By default, the | ||
680 | OpenEmbedded build system enables three levels of package | ||
681 | architectures: "all", "tune" or "package", and "machine". Any given | ||
682 | recipe usually selects one of these package architectures (types) for | ||
683 | its output. Depending for what a given recipe creates packages, | ||
684 | making sure you enable the appropriate package architecture can | ||
685 | directly impact the build time. | ||
686 | |||
687 | A recipe that just generates scripts can enable "all" architecture | ||
688 | because there are no binaries to build. To specifically enable "all" | ||
689 | architecture, be sure your recipe inherits the | ||
690 | :ref:`ref-classes-allarch` class. | ||
691 | This class is useful for "all" architectures because it configures | ||
692 | many variables so packages can be used across multiple architectures. | ||
693 | |||
694 | If your recipe needs to generate packages that are machine-specific | ||
695 | or when one of the build or runtime dependencies is already | ||
696 | machine-architecture dependent, which makes your recipe also | ||
697 | machine-architecture dependent, make sure your recipe enables the | ||
698 | "machine" package architecture through the | ||
699 | :term:`MACHINE_ARCH` | ||
700 | variable:: | ||
701 | |||
702 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
703 | |||
704 | When you do not | ||
705 | specifically enable a package architecture through the | ||
706 | :term:`PACKAGE_ARCH`, The | ||
707 | OpenEmbedded build system defaults to the | ||
708 | :term:`TUNE_PKGARCH` setting:: | ||
709 | |||
710 | PACKAGE_ARCH = "${TUNE_PKGARCH}" | ||
711 | |||
712 | - *Choose a Generic Tuning File if Possible:* Some tunes are more | ||
713 | generic and can run on multiple targets (e.g. an ``armv5`` set of | ||
714 | packages could run on ``armv6`` and ``armv7`` processors in most | ||
715 | cases). Similarly, ``i486`` binaries could work on ``i586`` and | ||
716 | higher processors. You should realize, however, that advances on | ||
717 | newer processor versions would not be used. | ||
718 | |||
719 | If you select the same tune for several different machines, the | ||
720 | OpenEmbedded build system reuses software previously built, thus | ||
721 | speeding up the overall build time. Realize that even though a new | ||
722 | sysroot for each machine is generated, the software is not recompiled | ||
723 | and only one package feed exists. | ||
724 | |||
725 | - *Manage Granular Level Packaging:* Sometimes there are cases where | ||
726 | injecting another level of package architecture beyond the three | ||
727 | higher levels noted earlier can be useful. For example, consider how | ||
728 | NXP (formerly Freescale) allows for the easy reuse of binary packages | ||
729 | in their layer | ||
730 | :yocto_git:`meta-freescale </meta-freescale/>`. | ||
731 | In this example, the | ||
732 | :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` | ||
733 | class shares GPU packages for i.MX53 boards because all boards share | ||
734 | the AMD GPU. The i.MX6-based boards can do the same because all | ||
735 | boards share the Vivante GPU. This class inspects the BitBake | ||
736 | datastore to identify if the package provides or depends on one of | ||
737 | the sub-architecture values. If so, the class sets the | ||
738 | :term:`PACKAGE_ARCH` value | ||
739 | based on the ``MACHINE_SUBARCH`` value. If the package does not | ||
740 | provide or depend on one of the sub-architecture values but it | ||
741 | matches a value in the machine-specific filter, it sets | ||
742 | :term:`MACHINE_ARCH`. This | ||
743 | behavior reduces the number of packages built and saves build time by | ||
744 | reusing binaries. | ||
745 | |||
746 | - *Use Tools to Debug Issues:* Sometimes you can run into situations | ||
747 | where software is being rebuilt when you think it should not be. For | ||
748 | example, the OpenEmbedded build system might not be using shared | ||
749 | state between machines when you think it should be. These types of | ||
750 | situations are usually due to references to machine-specific | ||
751 | variables such as :term:`MACHINE`, | ||
752 | :term:`SERIAL_CONSOLES`, | ||
753 | :term:`XSERVER`, | ||
754 | :term:`MACHINE_FEATURES`, | ||
755 | and so forth in code that is supposed to only be tune-specific or | ||
756 | when the recipe depends | ||
757 | (:term:`DEPENDS`, | ||
758 | :term:`RDEPENDS`, | ||
759 | :term:`RRECOMMENDS`, | ||
760 | :term:`RSUGGESTS`, and so forth) | ||
761 | on some other recipe that already has | ||
762 | :term:`PACKAGE_ARCH` defined | ||
763 | as "${MACHINE_ARCH}". | ||
764 | |||
765 | .. note:: | ||
766 | |||
767 | Patches to fix any issues identified are most welcome as these | ||
768 | issues occasionally do occur. | ||
769 | |||
770 | For such cases, you can use some tools to help you sort out the | ||
771 | situation: | ||
772 | |||
773 | - ``state-diff-machines.sh``*:* You can find this tool in the | ||
774 | ``scripts`` directory of the Source Repositories. See the comments | ||
775 | in the script for information on how to use the tool. | ||
776 | |||
777 | - *BitBake's "-S printdiff" Option:* Using this option causes | ||
778 | BitBake to try to establish the most recent signature match | ||
779 | (e.g. in the shared state cache) and then compare matched signatures | ||
780 | to determine the stamps and delta where these two stamp trees diverge. | ||
781 | |||
782 | Building Software from an External Source | ||
783 | ========================================= | ||
784 | |||
785 | By default, the OpenEmbedded build system uses the :term:`Build Directory` | ||
786 | when building source code. The build process involves fetching the source | ||
787 | files, unpacking them, and then patching them if necessary before the build | ||
788 | takes place. | ||
789 | |||
790 | There are situations where you might want to build software from source | ||
791 | files that are external to and thus outside of the OpenEmbedded build | ||
792 | system. For example, suppose you have a project that includes a new BSP | ||
793 | with a heavily customized kernel. And, you want to minimize exposing the | ||
794 | build system to the development team so that they can focus on their | ||
795 | project and maintain everyone's workflow as much as possible. In this | ||
796 | case, you want a kernel source directory on the development machine | ||
797 | where the development occurs. You want the recipe's | ||
798 | :term:`SRC_URI` variable to point to | ||
799 | the external directory and use it as is, not copy it. | ||
800 | |||
801 | To build from software that comes from an external source, all you need to do | ||
802 | is inherit the :ref:`ref-classes-externalsrc` class and then set | ||
803 | the :term:`EXTERNALSRC` variable to point to your external source code. Here | ||
804 | are the statements to put in your ``local.conf`` file:: | ||
805 | |||
806 | INHERIT += "externalsrc" | ||
807 | EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree" | ||
808 | |||
809 | This next example shows how to accomplish the same thing by setting | ||
810 | :term:`EXTERNALSRC` in the recipe itself or in the recipe's append file:: | ||
811 | |||
812 | EXTERNALSRC = "path" | ||
813 | EXTERNALSRC_BUILD = "path" | ||
814 | |||
815 | .. note:: | ||
816 | |||
817 | In order for these settings to take effect, you must globally or | ||
818 | locally inherit the :ref:`ref-classes-externalsrc` class. | ||
819 | |||
820 | By default, :ref:`ref-classes-externalsrc` builds the source code in a | ||
821 | directory separate from the external source directory as specified by | ||
822 | :term:`EXTERNALSRC`. If you need | ||
823 | to have the source built in the same directory in which it resides, or | ||
824 | some other nominated directory, you can set | ||
825 | :term:`EXTERNALSRC_BUILD` | ||
826 | to point to that directory:: | ||
827 | |||
828 | EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree" | ||
829 | |||
830 | Replicating a Build Offline | ||
831 | =========================== | ||
832 | |||
833 | It can be useful to take a "snapshot" of upstream sources used in a | ||
834 | build and then use that "snapshot" later to replicate the build offline. | ||
835 | To do so, you need to first prepare and populate your downloads | ||
836 | directory your "snapshot" of files. Once your downloads directory is | ||
837 | ready, you can use it at any time and from any machine to replicate your | ||
838 | build. | ||
839 | |||
840 | Follow these steps to populate your Downloads directory: | ||
841 | |||
842 | #. *Create a Clean Downloads Directory:* Start with an empty downloads | ||
843 | directory (:term:`DL_DIR`). You | ||
844 | start with an empty downloads directory by either removing the files | ||
845 | in the existing directory or by setting :term:`DL_DIR` to point to either | ||
846 | an empty location or one that does not yet exist. | ||
847 | |||
848 | #. *Generate Tarballs of the Source Git Repositories:* Edit your | ||
849 | ``local.conf`` configuration file as follows:: | ||
850 | |||
851 | DL_DIR = "/home/your-download-dir/" | ||
852 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
853 | |||
854 | During | ||
855 | the fetch process in the next step, BitBake gathers the source files | ||
856 | and creates tarballs in the directory pointed to by :term:`DL_DIR`. See | ||
857 | the | ||
858 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
859 | variable for more information. | ||
860 | |||
861 | #. *Populate Your Downloads Directory Without Building:* Use BitBake to | ||
862 | fetch your sources but inhibit the build:: | ||
863 | |||
864 | $ bitbake target --runonly=fetch | ||
865 | |||
866 | The downloads directory (i.e. ``${DL_DIR}``) now has | ||
867 | a "snapshot" of the source files in the form of tarballs, which can | ||
868 | be used for the build. | ||
869 | |||
870 | #. *Optionally Remove Any Git or other SCM Subdirectories From the | ||
871 | Downloads Directory:* If you want, you can clean up your downloads | ||
872 | directory by removing any Git or other Source Control Management | ||
873 | (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs | ||
874 | already contain these subdirectories. | ||
875 | |||
876 | Once your downloads directory has everything it needs regarding source | ||
877 | files, you can create your "own-mirror" and build your target. | ||
878 | Understand that you can use the files to build the target offline from | ||
879 | any machine and at any time. | ||
880 | |||
881 | Follow these steps to build your target using the files in the downloads | ||
882 | directory: | ||
883 | |||
884 | #. *Using Local Files Only:* Inside your ``local.conf`` file, add the | ||
885 | :term:`SOURCE_MIRROR_URL` variable, inherit the | ||
886 | :ref:`ref-classes-own-mirrors` class, and use the | ||
887 | :term:`BB_NO_NETWORK` variable to your ``local.conf``:: | ||
888 | |||
889 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | ||
890 | INHERIT += "own-mirrors" | ||
891 | BB_NO_NETWORK = "1" | ||
892 | |||
893 | The :term:`SOURCE_MIRROR_URL` and :ref:`ref-classes-own-mirrors` | ||
894 | class set up the system to use the downloads directory as your "own | ||
895 | mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that | ||
896 | BitBake's fetching process in step 3 stays local, which means files | ||
897 | from your "own-mirror" are used. | ||
898 | |||
899 | #. *Start With a Clean Build:* You can start with a clean build by | ||
900 | removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new | ||
901 | :term:`Build Directory`. | ||
902 | |||
903 | #. *Build Your Target:* Use BitBake to build your target:: | ||
904 | |||
905 | $ bitbake target | ||
906 | |||
907 | The build completes using the known local "snapshot" of source | ||
908 | files from your mirror. The resulting tarballs for your "snapshot" of | ||
909 | source files are in the downloads directory. | ||
910 | |||
911 | .. note:: | ||
912 | |||
913 | The offline build does not work if recipes attempt to find the | ||
914 | latest version of software by setting | ||
915 | :term:`SRCREV` to | ||
916 | ``${``\ :term:`AUTOREV`\ ``}``:: | ||
917 | |||
918 | SRCREV = "${AUTOREV}" | ||
919 | |||
920 | When a recipe sets :term:`SRCREV` to | ||
921 | ``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an | ||
922 | attempt to determine the latest version of software from the SCM. | ||
923 | Typically, recipes that use :term:`AUTOREV` are custom or modified | ||
924 | recipes. Recipes that reside in public repositories usually do not | ||
925 | use :term:`AUTOREV`. | ||
926 | |||
927 | If you do have recipes that use :term:`AUTOREV`, you can take steps to | ||
928 | still use the recipes in an offline build. Do the following: | ||
929 | |||
930 | #. Use a configuration generated by enabling :ref:`build | ||
931 | history <dev-manual/build-quality:maintaining build output quality>`. | ||
932 | |||
933 | #. Use the ``buildhistory-collect-srcrevs`` command to collect the | ||
934 | stored :term:`SRCREV` values from the build's history. For more | ||
935 | information on collecting these values, see the | ||
936 | ":ref:`dev-manual/build-quality:build history package information`" | ||
937 | section. | ||
938 | |||
939 | #. Once you have the correct source revisions, you can modify | ||
940 | those recipes to set :term:`SRCREV` to specific versions of the | ||
941 | software. | ||
942 | |||