diff options
Diffstat (limited to 'documentation/dev-manual/building.rst')
-rw-r--r-- | documentation/dev-manual/building.rst | 940 |
1 files changed, 940 insertions, 0 deletions
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst new file mode 100644 index 0000000000..c2cf3a35f3 --- /dev/null +++ b/documentation/dev-manual/building.rst | |||
@@ -0,0 +1,940 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Building | ||
4 | ******** | ||
5 | |||
6 | This section describes various build procedures, such as the steps | ||
7 | needed for a simple build, building a target for multiple configurations, | ||
8 | generating an image for more than one machine, and so forth. | ||
9 | |||
10 | Building a Simple Image | ||
11 | ======================= | ||
12 | |||
13 | In the development environment, you need to build an image whenever you | ||
14 | change hardware support, add or change system libraries, or add or | ||
15 | change services that have dependencies. There are several methods that allow | ||
16 | you to build an image within the Yocto Project. This section presents | ||
17 | the basic steps you need to build a simple image using BitBake from a | ||
18 | build host running Linux. | ||
19 | |||
20 | .. note:: | ||
21 | |||
22 | - For information on how to build an image using | ||
23 | :term:`Toaster`, see the | ||
24 | :doc:`/toaster-manual/index`. | ||
25 | |||
26 | - For information on how to use ``devtool`` to build images, see the | ||
27 | ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
28 | section in the Yocto Project Application Development and the | ||
29 | Extensible Software Development Kit (eSDK) manual. | ||
30 | |||
31 | - For a quick example on how to build an image using the | ||
32 | OpenEmbedded build system, see the | ||
33 | :doc:`/brief-yoctoprojectqs/index` document. | ||
34 | |||
35 | The build process creates an entire Linux distribution from source and | ||
36 | places it in your :term:`Build Directory` under ``tmp/deploy/images``. For | ||
37 | detailed information on the build process using BitBake, see the | ||
38 | ":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview | ||
39 | and Concepts Manual. | ||
40 | |||
41 | The following figure and list overviews the build process: | ||
42 | |||
43 | .. image:: figures/bitbake-build-flow.png | ||
44 | :width: 100% | ||
45 | |||
46 | 1. *Set up Your Host Development System to Support Development Using the | ||
47 | Yocto Project*: See the ":doc:`start`" section for options on how to get a | ||
48 | build host ready to use the Yocto Project. | ||
49 | |||
50 | 2. *Initialize the Build Environment:* Initialize the build environment | ||
51 | by sourcing the build environment script (i.e. | ||
52 | :ref:`structure-core-script`):: | ||
53 | |||
54 | $ source oe-init-build-env [build_dir] | ||
55 | |||
56 | When you use the initialization script, the OpenEmbedded build system | ||
57 | uses ``build`` as the default :term:`Build Directory` in your current work | ||
58 | directory. You can use a `build_dir` argument with the script to | ||
59 | specify a different :term:`Build Directory`. | ||
60 | |||
61 | .. note:: | ||
62 | |||
63 | A common practice is to use a different :term:`Build Directory` for | ||
64 | different targets; for example, ``~/build/x86`` for a ``qemux86`` | ||
65 | target, and ``~/build/arm`` for a ``qemuarm`` target. In any | ||
66 | event, it's typically cleaner to locate the :term:`Build Directory` | ||
67 | somewhere outside of your source directory. | ||
68 | |||
69 | 3. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the | ||
70 | ``conf/local.conf`` configuration file, which is found in the | ||
71 | :term:`Build Directory`, is set up how you want it. This file defines many | ||
72 | aspects of the build environment including the target machine architecture | ||
73 | through the :term:`MACHINE` variable, the packaging format used during | ||
74 | the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download | ||
75 | directory through the :term:`DL_DIR` variable. | ||
76 | |||
77 | 4. *Build the Image:* Build the image using the ``bitbake`` command:: | ||
78 | |||
79 | $ bitbake target | ||
80 | |||
81 | .. note:: | ||
82 | |||
83 | For information on BitBake, see the :doc:`bitbake:index`. | ||
84 | |||
85 | The target is the name of the recipe you want to build. Common | ||
86 | targets are the images in ``meta/recipes-core/images``, | ||
87 | ``meta/recipes-sato/images``, and so forth all found in the | ||
88 | :term:`Source Directory`. Alternatively, the target | ||
89 | can be the name of a recipe for a specific piece of software such as | ||
90 | BusyBox. For more details about the images the OpenEmbedded build | ||
91 | system supports, see the | ||
92 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
93 | Project Reference Manual. | ||
94 | |||
95 | As an example, the following command builds the | ||
96 | ``core-image-minimal`` image:: | ||
97 | |||
98 | $ bitbake core-image-minimal | ||
99 | |||
100 | Once an | ||
101 | image has been built, it often needs to be installed. The images and | ||
102 | kernels built by the OpenEmbedded build system are placed in the | ||
103 | :term:`Build Directory` in ``tmp/deploy/images``. For information on how to | ||
104 | run pre-built images such as ``qemux86`` and ``qemuarm``, see the | ||
105 | :doc:`/sdk-manual/index` manual. For | ||
106 | information about how to install these images, see the documentation | ||
107 | for your particular board or machine. | ||
108 | |||
109 | Building Images for Multiple Targets Using Multiple Configurations | ||
110 | ================================================================== | ||
111 | |||
112 | You can use a single ``bitbake`` command to build multiple images or | ||
113 | packages for different targets where each image or package requires a | ||
114 | different configuration (multiple configuration builds). The builds, in | ||
115 | this scenario, are sometimes referred to as "multiconfigs", and this | ||
116 | section uses that term throughout. | ||
117 | |||
118 | This section describes how to set up for multiple configuration builds | ||
119 | and how to account for cross-build dependencies between the | ||
120 | multiconfigs. | ||
121 | |||
122 | Setting Up and Running a Multiple Configuration Build | ||
123 | ----------------------------------------------------- | ||
124 | |||
125 | To accomplish a multiple configuration build, you must define each | ||
126 | target's configuration separately using a parallel configuration file in | ||
127 | the :term:`Build Directory` or configuration directory within a layer, and you | ||
128 | must follow a required file hierarchy. Additionally, you must enable the | ||
129 | multiple configuration builds in your ``local.conf`` file. | ||
130 | |||
131 | Follow these steps to set up and execute multiple configuration builds: | ||
132 | |||
133 | - *Create Separate Configuration Files*: You need to create a single | ||
134 | configuration file for each build target (each multiconfig). | ||
135 | The configuration definitions are implementation dependent but often | ||
136 | each configuration file will define the machine and the | ||
137 | temporary directory BitBake uses for the build. Whether the same | ||
138 | temporary directory (:term:`TMPDIR`) can be shared will depend on what is | ||
139 | similar and what is different between the configurations. Multiple MACHINE | ||
140 | targets can share the same (:term:`TMPDIR`) as long as the rest of the | ||
141 | configuration is the same, multiple DISTRO settings would need separate | ||
142 | (:term:`TMPDIR`) directories. | ||
143 | |||
144 | For example, consider a scenario with two different multiconfigs for the same | ||
145 | :term:`MACHINE`: "qemux86" built | ||
146 | for two distributions such as "poky" and "poky-lsb". In this case, | ||
147 | you would need to use the different :term:`TMPDIR`. | ||
148 | |||
149 | Here is an example showing the minimal statements needed in a | ||
150 | configuration file for a "qemux86" target whose temporary build | ||
151 | directory is ``tmpmultix86``:: | ||
152 | |||
153 | MACHINE = "qemux86" | ||
154 | TMPDIR = "${TOPDIR}/tmpmultix86" | ||
155 | |||
156 | The location for these multiconfig configuration files is specific. | ||
157 | They must reside in the current :term:`Build Directory` in a sub-directory of | ||
158 | ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory | ||
159 | under a directory named ``multiconfig``. Following is an example that defines | ||
160 | two configuration files for the "x86" and "arm" multiconfigs: | ||
161 | |||
162 | .. image:: figures/multiconfig_files.png | ||
163 | :align: center | ||
164 | :width: 50% | ||
165 | |||
166 | The usual :term:`BBPATH` search path is used to locate multiconfig files in | ||
167 | a similar way to other conf files. | ||
168 | |||
169 | - *Add the BitBake Multi-configuration Variable to the Local | ||
170 | Configuration File*: Use the | ||
171 | :term:`BBMULTICONFIG` | ||
172 | variable in your ``conf/local.conf`` configuration file to specify | ||
173 | each multiconfig. Continuing with the example from the previous | ||
174 | figure, the :term:`BBMULTICONFIG` variable needs to enable two | ||
175 | multiconfigs: "x86" and "arm" by specifying each configuration file:: | ||
176 | |||
177 | BBMULTICONFIG = "x86 arm" | ||
178 | |||
179 | .. note:: | ||
180 | |||
181 | A "default" configuration already exists by definition. This | ||
182 | configuration is named: "" (i.e. empty string) and is defined by | ||
183 | the variables coming from your ``local.conf`` | ||
184 | file. Consequently, the previous example actually adds two | ||
185 | additional configurations to your build: "arm" and "x86" along | ||
186 | with "". | ||
187 | |||
188 | - *Launch BitBake*: Use the following BitBake command form to launch | ||
189 | the multiple configuration build:: | ||
190 | |||
191 | $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] | ||
192 | |||
193 | For the example in this section, the following command applies:: | ||
194 | |||
195 | $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base | ||
196 | |||
197 | The previous BitBake command builds a ``core-image-minimal`` image | ||
198 | that is configured through the ``x86.conf`` configuration file, a | ||
199 | ``core-image-sato`` image that is configured through the ``arm.conf`` | ||
200 | configuration file and a ``core-image-base`` that is configured | ||
201 | through your ``local.conf`` configuration file. | ||
202 | |||
203 | .. note:: | ||
204 | |||
205 | Support for multiple configuration builds in the Yocto Project &DISTRO; | ||
206 | (&DISTRO_NAME;) Release does not include Shared State (sstate) | ||
207 | optimizations. Consequently, if a build uses the same object twice | ||
208 | in, for example, two different :term:`TMPDIR` | ||
209 | directories, the build either loads from an existing sstate cache for | ||
210 | that build at the start or builds the object fresh. | ||
211 | |||
212 | Enabling Multiple Configuration Build Dependencies | ||
213 | -------------------------------------------------- | ||
214 | |||
215 | Sometimes dependencies can exist between targets (multiconfigs) in a | ||
216 | multiple configuration build. For example, suppose that in order to | ||
217 | build a ``core-image-sato`` image for an "x86" multiconfig, the root | ||
218 | filesystem of an "arm" multiconfig must exist. This dependency is | ||
219 | essentially that the | ||
220 | :ref:`ref-tasks-image` task in the | ||
221 | ``core-image-sato`` recipe depends on the completion of the | ||
222 | :ref:`ref-tasks-rootfs` task of the | ||
223 | ``core-image-minimal`` recipe. | ||
224 | |||
225 | To enable dependencies in a multiple configuration build, you must | ||
226 | declare the dependencies in the recipe using the following statement | ||
227 | form:: | ||
228 | |||
229 | task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" | ||
230 | |||
231 | To better show how to use this statement, consider the example scenario | ||
232 | from the first paragraph of this section. The following statement needs | ||
233 | to be added to the recipe that builds the ``core-image-sato`` image:: | ||
234 | |||
235 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" | ||
236 | |||
237 | In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The | ||
238 | task on which the :ref:`ref-tasks-image` task in the recipe depends is the | ||
239 | :ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated | ||
240 | with the "arm" multiconfig. | ||
241 | |||
242 | Once you set up this dependency, you can build the "x86" multiconfig | ||
243 | using a BitBake command as follows:: | ||
244 | |||
245 | $ bitbake mc:x86:core-image-sato | ||
246 | |||
247 | This command executes all the tasks needed to create the | ||
248 | ``core-image-sato`` image for the "x86" multiconfig. Because of the | ||
249 | dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the | ||
250 | "arm" multiconfig build. | ||
251 | |||
252 | Having a recipe depend on the root filesystem of another build might not | ||
253 | seem that useful. Consider this change to the statement in the | ||
254 | ``core-image-sato`` recipe:: | ||
255 | |||
256 | do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" | ||
257 | |||
258 | In this case, BitBake must | ||
259 | create the ``core-image-minimal`` image for the "arm" build since the | ||
260 | "x86" build depends on it. | ||
261 | |||
262 | Because "x86" and "arm" are enabled for multiple configuration builds | ||
263 | and have separate configuration files, BitBake places the artifacts for | ||
264 | each build in the respective temporary build directories (i.e. | ||
265 | :term:`TMPDIR`). | ||
266 | |||
267 | Building an Initial RAM Filesystem (Initramfs) Image | ||
268 | ==================================================== | ||
269 | |||
270 | An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root | ||
271 | filesystem used for early system initialization, typically providing tools and | ||
272 | loading modules needed to locate and mount the final root filesystem. | ||
273 | |||
274 | Follow these steps to create an :term:`Initramfs` image: | ||
275 | |||
276 | 1. *Create the Initramfs Image Recipe:* You can reference the | ||
277 | ``core-image-minimal-initramfs.bb`` recipe found in the | ||
278 | ``meta/recipes-core`` directory of the :term:`Source Directory` | ||
279 | as an example from which to work. | ||
280 | |||
281 | 2. *Decide if You Need to Bundle the Initramfs Image Into the Kernel | ||
282 | Image:* If you want the :term:`Initramfs` image that is built to be bundled | ||
283 | in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE` | ||
284 | variable to ``"1"`` in your ``local.conf`` configuration file and set the | ||
285 | :term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image. | ||
286 | |||
287 | Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs` | ||
288 | image to be unpacked into the ``${B}/usr/`` directory. The unpacked | ||
289 | :term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the | ||
290 | :term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs` | ||
291 | image to be built into the kernel normally. | ||
292 | |||
293 | 3. *Optionally Add Items to the Initramfs Image Through the Initramfs | ||
294 | Image Recipe:* If you add items to the :term:`Initramfs` image by way of its | ||
295 | recipe, you should use :term:`PACKAGE_INSTALL` rather than | ||
296 | :term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of | ||
297 | what is added to the image as compared to the defaults you might not | ||
298 | necessarily want that are set by the :ref:`image <ref-classes-image>` | ||
299 | or :ref:`core-image <ref-classes-core-image>` classes. | ||
300 | |||
301 | 4. *Build the Kernel Image and the Initramfs Image:* Build your kernel | ||
302 | image using BitBake. Because the :term:`Initramfs` image recipe is a | ||
303 | dependency of the kernel image, the :term:`Initramfs` image is built as well | ||
304 | and bundled with the kernel image if you used the | ||
305 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. | ||
306 | |||
307 | Bundling an Initramfs Image From a Separate Multiconfig | ||
308 | ------------------------------------------------------- | ||
309 | |||
310 | There may be a case where we want to build an :term:`Initramfs` image which does not | ||
311 | inherit the same distro policy as our main image, for example, we may want | ||
312 | our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs` | ||
313 | image to keep a smaller footprint. However, by performing the steps mentioned | ||
314 | above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us | ||
315 | to override it. | ||
316 | |||
317 | To achieve this, you need to perform some additional steps: | ||
318 | |||
319 | 1. *Create a multiconfig for your Initramfs image:* You can perform the steps | ||
320 | on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig. | ||
321 | For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and | ||
322 | contains the variables:: | ||
323 | |||
324 | TMPDIR="${TOPDIR}/tmp-initramfscfg" | ||
325 | TCLIBC="musl" | ||
326 | |||
327 | 2. *Set additional Initramfs variables on your main configuration:* | ||
328 | Additionally, on your main configuration (``local.conf``) you need to set the | ||
329 | variables:: | ||
330 | |||
331 | INITRAMFS_MULTICONFIG = "initramfscfg" | ||
332 | INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}" | ||
333 | |||
334 | The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
335 | are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE` | ||
336 | to be built coming from the ``initramfscfg`` multiconfig, and to let the | ||
337 | buildsystem know where the :term:`INITRAMFS_IMAGE` will be located. | ||
338 | |||
339 | Building a system with such configuration will build the kernel using the | ||
340 | main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the | ||
341 | selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
342 | instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel | ||
343 | but a glibc based main image. | ||
344 | |||
345 | The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE` | ||
346 | or to build a different :term:`DISTRO` for it such as ``poky-tiny``. | ||
347 | |||
348 | |||
349 | Building a Tiny System | ||
350 | ====================== | ||
351 | |||
352 | Very small distributions have some significant advantages such as | ||
353 | requiring less on-die or in-package memory (cheaper), better performance | ||
354 | through efficient cache usage, lower power requirements due to less | ||
355 | memory, faster boot times, and reduced development overhead. Some | ||
356 | real-world examples where a very small distribution gives you distinct | ||
357 | advantages are digital cameras, medical devices, and small headless | ||
358 | systems. | ||
359 | |||
360 | This section presents information that shows you how you can trim your | ||
361 | distribution to even smaller sizes than the ``poky-tiny`` distribution, | ||
362 | which is around 5 Mbytes, that can be built out-of-the-box using the | ||
363 | Yocto Project. | ||
364 | |||
365 | Tiny System Overview | ||
366 | -------------------- | ||
367 | |||
368 | The following list presents the overall steps you need to consider and | ||
369 | perform to create distributions with smaller root filesystems, achieve | ||
370 | faster boot times, maintain your critical functionality, and avoid | ||
371 | initial RAM disks: | ||
372 | |||
373 | - :ref:`Determine your goals and guiding principles | ||
374 | <dev-manual/building:goals and guiding principles>` | ||
375 | |||
376 | - :ref:`dev-manual/building:understand what contributes to your image size` | ||
377 | |||
378 | - :ref:`Reduce the size of the root filesystem | ||
379 | <dev-manual/building:trim the root filesystem>` | ||
380 | |||
381 | - :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>` | ||
382 | |||
383 | - :ref:`dev-manual/building:remove package management requirements` | ||
384 | |||
385 | - :ref:`dev-manual/building:look for other ways to minimize size` | ||
386 | |||
387 | - :ref:`dev-manual/building:iterate on the process` | ||
388 | |||
389 | Goals and Guiding Principles | ||
390 | ---------------------------- | ||
391 | |||
392 | Before you can reach your destination, you need to know where you are | ||
393 | going. Here is an example list that you can use as a guide when creating | ||
394 | very small distributions: | ||
395 | |||
396 | - Determine how much space you need (e.g. a kernel that is 1 Mbyte or | ||
397 | less and a root filesystem that is 3 Mbytes or less). | ||
398 | |||
399 | - Find the areas that are currently taking 90% of the space and | ||
400 | concentrate on reducing those areas. | ||
401 | |||
402 | - Do not create any difficult "hacks" to achieve your goals. | ||
403 | |||
404 | - Leverage the device-specific options. | ||
405 | |||
406 | - Work in a separate layer so that you keep changes isolated. For | ||
407 | information on how to create layers, see the | ||
408 | ":ref:`dev-manual/layers:understanding and creating layers`" section. | ||
409 | |||
410 | Understand What Contributes to Your Image Size | ||
411 | ---------------------------------------------- | ||
412 | |||
413 | It is easiest to have something to start with when creating your own | ||
414 | distribution. You can use the Yocto Project out-of-the-box to create the | ||
415 | ``poky-tiny`` distribution. Ultimately, you will want to make changes in | ||
416 | your own distribution that are likely modeled after ``poky-tiny``. | ||
417 | |||
418 | .. note:: | ||
419 | |||
420 | To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your | ||
421 | ``local.conf`` file to "poky-tiny" as described in the | ||
422 | ":ref:`dev-manual/custom-distribution:creating your own distribution`" | ||
423 | section. | ||
424 | |||
425 | Understanding some memory concepts will help you reduce the system size. | ||
426 | Memory consists of static, dynamic, and temporary memory. Static memory | ||
427 | is the TEXT (code), DATA (initialized data in the code), and BSS | ||
428 | (uninitialized data) sections. Dynamic memory represents memory that is | ||
429 | allocated at runtime: stacks, hash tables, and so forth. Temporary | ||
430 | memory is recovered after the boot process. This memory consists of | ||
431 | memory used for decompressing the kernel and for the ``__init__`` | ||
432 | functions. | ||
433 | |||
434 | To help you see where you currently are with kernel and root filesystem | ||
435 | sizes, you can use two tools found in the :term:`Source Directory` | ||
436 | in the | ||
437 | ``scripts/tiny/`` directory: | ||
438 | |||
439 | - ``ksize.py``: Reports component sizes for the kernel build objects. | ||
440 | |||
441 | - ``dirsize.py``: Reports component sizes for the root filesystem. | ||
442 | |||
443 | This next tool and command help you organize configuration fragments and | ||
444 | view file dependencies in a human-readable form: | ||
445 | |||
446 | - ``merge_config.sh``: Helps you manage configuration files and | ||
447 | fragments within the kernel. With this tool, you can merge individual | ||
448 | configuration fragments together. The tool allows you to make | ||
449 | overrides and warns you of any missing configuration options. The | ||
450 | tool is ideal for allowing you to iterate on configurations, create | ||
451 | minimal configurations, and create configuration files for different | ||
452 | machines without having to duplicate your process. | ||
453 | |||
454 | The ``merge_config.sh`` script is part of the Linux Yocto kernel Git | ||
455 | repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, | ||
456 | ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` | ||
457 | directory. | ||
458 | |||
459 | For more information on configuration fragments, see the | ||
460 | ":ref:`kernel-dev/common:creating configuration fragments`" | ||
461 | section in the Yocto Project Linux Kernel Development Manual. | ||
462 | |||
463 | - ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command | ||
464 | with these options brings up a Dependency Explorer from which you can | ||
465 | view file dependencies. Understanding these dependencies allows you | ||
466 | to make informed decisions when cutting out various pieces of the | ||
467 | kernel and root filesystem. | ||
468 | |||
469 | Trim the Root Filesystem | ||
470 | ------------------------ | ||
471 | |||
472 | The root filesystem is made up of packages for booting, libraries, and | ||
473 | applications. To change things, you can configure how the packaging | ||
474 | happens, which changes the way you build them. You can also modify the | ||
475 | filesystem itself or select a different filesystem. | ||
476 | |||
477 | First, find out what is hogging your root filesystem by running the | ||
478 | ``dirsize.py`` script from your root directory:: | ||
479 | |||
480 | $ cd root-directory-of-image | ||
481 | $ dirsize.py 100000 > dirsize-100k.log | ||
482 | $ cat dirsize-100k.log | ||
483 | |||
484 | You can apply a filter to the script to ignore files | ||
485 | under a certain size. The previous example filters out any files below | ||
486 | 100 Kbytes. The sizes reported by the tool are uncompressed, and thus | ||
487 | will be smaller by a relatively constant factor in a compressed root | ||
488 | filesystem. When you examine your log file, you can focus on areas of | ||
489 | the root filesystem that take up large amounts of memory. | ||
490 | |||
491 | You need to be sure that what you eliminate does not cripple the | ||
492 | functionality you need. One way to see how packages relate to each other | ||
493 | is by using the Dependency Explorer UI with the BitBake command:: | ||
494 | |||
495 | $ cd image-directory | ||
496 | $ bitbake -u taskexp -g image | ||
497 | |||
498 | Use the interface to | ||
499 | select potential packages you wish to eliminate and see their dependency | ||
500 | relationships. | ||
501 | |||
502 | When deciding how to reduce the size, get rid of packages that result in | ||
503 | minimal impact on the feature set. For example, you might not need a VGA | ||
504 | display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` | ||
505 | instead of ``udev``. | ||
506 | |||
507 | Use your ``local.conf`` file to make changes. For example, to eliminate | ||
508 | ``udev`` and ``glib``, set the following in the local configuration | ||
509 | file:: | ||
510 | |||
511 | VIRTUAL-RUNTIME_dev_manager = "" | ||
512 | |||
513 | Finally, you should consider exactly the type of root filesystem you | ||
514 | need to meet your needs while also reducing its size. For example, | ||
515 | consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an | ||
516 | :term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1 | ||
517 | Mbyte journal. If you are okay with running read-only, you do not need | ||
518 | this journal. | ||
519 | |||
520 | .. note:: | ||
521 | |||
522 | After each round of elimination, you need to rebuild your system and | ||
523 | then use the tools to see the effects of your reductions. | ||
524 | |||
525 | Trim the Kernel | ||
526 | --------------- | ||
527 | |||
528 | The kernel is built by including policies for hardware-independent | ||
529 | aspects. What subsystems do you enable? For what architecture are you | ||
530 | building? Which drivers do you build by default? | ||
531 | |||
532 | .. note:: | ||
533 | |||
534 | You can modify the kernel source if you want to help with boot time. | ||
535 | |||
536 | Run the ``ksize.py`` script from the top-level Linux build directory to | ||
537 | get an idea of what is making up the kernel:: | ||
538 | |||
539 | $ cd top-level-linux-build-directory | ||
540 | $ ksize.py > ksize.log | ||
541 | $ cat ksize.log | ||
542 | |||
543 | When you examine the log, you will see how much space is taken up with | ||
544 | the built-in ``.o`` files for drivers, networking, core kernel files, | ||
545 | filesystem, sound, and so forth. The sizes reported by the tool are | ||
546 | uncompressed, and thus will be smaller by a relatively constant factor | ||
547 | in a compressed kernel image. Look to reduce the areas that are large | ||
548 | and taking up around the "90% rule." | ||
549 | |||
550 | To examine, or drill down, into any particular area, use the ``-d`` | ||
551 | option with the script:: | ||
552 | |||
553 | $ ksize.py -d > ksize.log | ||
554 | |||
555 | Using this option | ||
556 | breaks out the individual file information for each area of the kernel | ||
557 | (e.g. drivers, networking, and so forth). | ||
558 | |||
559 | Use your log file to see what you can eliminate from the kernel based on | ||
560 | features you can let go. For example, if you are not going to need | ||
561 | sound, you do not need any drivers that support sound. | ||
562 | |||
563 | After figuring out what to eliminate, you need to reconfigure the kernel | ||
564 | to reflect those changes during the next build. You could run | ||
565 | ``menuconfig`` and make all your changes at once. However, that makes it | ||
566 | difficult to see the effects of your individual eliminations and also | ||
567 | makes it difficult to replicate the changes for perhaps another target | ||
568 | device. A better method is to start with no configurations using | ||
569 | ``allnoconfig``, create configuration fragments for individual changes, | ||
570 | and then manage the fragments into a single configuration file using | ||
571 | ``merge_config.sh``. The tool makes it easy for you to iterate using the | ||
572 | configuration change and build cycle. | ||
573 | |||
574 | Each time you make configuration changes, you need to rebuild the kernel | ||
575 | and check to see what impact your changes had on the overall size. | ||
576 | |||
577 | Remove Package Management Requirements | ||
578 | -------------------------------------- | ||
579 | |||
580 | Packaging requirements add size to the image. One way to reduce the size | ||
581 | of the image is to remove all the packaging requirements from the image. | ||
582 | This reduction includes both removing the package manager and its unique | ||
583 | dependencies as well as removing the package management data itself. | ||
584 | |||
585 | To eliminate all the packaging requirements for an image, be sure that | ||
586 | "package-management" is not part of your | ||
587 | :term:`IMAGE_FEATURES` | ||
588 | statement for the image. When you remove this feature, you are removing | ||
589 | the package manager as well as its dependencies from the root | ||
590 | filesystem. | ||
591 | |||
592 | Look for Other Ways to Minimize Size | ||
593 | ------------------------------------ | ||
594 | |||
595 | Depending on your particular circumstances, other areas that you can | ||
596 | trim likely exist. The key to finding these areas is through tools and | ||
597 | methods described here combined with experimentation and iteration. Here | ||
598 | are a couple of areas to experiment with: | ||
599 | |||
600 | - ``glibc``: In general, follow this process: | ||
601 | |||
602 | 1. Remove ``glibc`` features from | ||
603 | :term:`DISTRO_FEATURES` | ||
604 | that you think you do not need. | ||
605 | |||
606 | 2. Build your distribution. | ||
607 | |||
608 | 3. If the build fails due to missing symbols in a package, determine | ||
609 | if you can reconfigure the package to not need those features. For | ||
610 | example, change the configuration to not support wide character | ||
611 | support as is done for ``ncurses``. Or, if support for those | ||
612 | characters is needed, determine what ``glibc`` features provide | ||
613 | the support and restore the configuration. | ||
614 | |||
615 | 4. Rebuild and repeat the process. | ||
616 | |||
617 | - ``busybox``: For BusyBox, use a process similar as described for | ||
618 | ``glibc``. A difference is you will need to boot the resulting system | ||
619 | to see if you are able to do everything you expect from the running | ||
620 | system. You need to be sure to integrate configuration fragments into | ||
621 | Busybox because BusyBox handles its own core features and then allows | ||
622 | you to add configuration fragments on top. | ||
623 | |||
624 | Iterate on the Process | ||
625 | ---------------------- | ||
626 | |||
627 | If you have not reached your goals on system size, you need to iterate | ||
628 | on the process. The process is the same. Use the tools and see just what | ||
629 | is taking up 90% of the root filesystem and the kernel. Decide what you | ||
630 | can eliminate without limiting your device beyond what you need. | ||
631 | |||
632 | Depending on your system, a good place to look might be Busybox, which | ||
633 | provides a stripped down version of Unix tools in a single, executable | ||
634 | file. You might be able to drop virtual terminal services or perhaps | ||
635 | ipv6. | ||
636 | |||
637 | Building Images for More than One Machine | ||
638 | ========================================= | ||
639 | |||
640 | A common scenario developers face is creating images for several | ||
641 | different machines that use the same software environment. In this | ||
642 | situation, it is tempting to set the tunings and optimization flags for | ||
643 | each build specifically for the targeted hardware (i.e. "maxing out" the | ||
644 | tunings). Doing so can considerably add to build times and package feed | ||
645 | maintenance collectively for the machines. For example, selecting tunes | ||
646 | that are extremely specific to a CPU core used in a system might enable | ||
647 | some micro optimizations in GCC for that particular system but would | ||
648 | otherwise not gain you much of a performance difference across the other | ||
649 | systems as compared to using a more general tuning across all the builds | ||
650 | (e.g. setting :term:`DEFAULTTUNE` | ||
651 | specifically for each machine's build). Rather than "max out" each | ||
652 | build's tunings, you can take steps that cause the OpenEmbedded build | ||
653 | system to reuse software across the various machines where it makes | ||
654 | sense. | ||
655 | |||
656 | If build speed and package feed maintenance are considerations, you | ||
657 | should consider the points in this section that can help you optimize | ||
658 | your tunings to best consider build times and package feed maintenance. | ||
659 | |||
660 | - *Share the :term:`Build Directory`:* If at all possible, share the | ||
661 | :term:`TMPDIR` across builds. The Yocto Project supports switching between | ||
662 | different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice | ||
663 | is well supported and regularly used by developers when building for | ||
664 | multiple machines. When you use the same :term:`TMPDIR` for multiple | ||
665 | machine builds, the OpenEmbedded build system can reuse the existing native | ||
666 | and often cross-recipes for multiple machines. Thus, build time decreases. | ||
667 | |||
668 | .. note:: | ||
669 | |||
670 | If :term:`DISTRO` settings change or fundamental configuration settings | ||
671 | such as the filesystem layout, you need to work with a clean :term:`TMPDIR`. | ||
672 | Sharing :term:`TMPDIR` under these circumstances might work but since it is | ||
673 | not guaranteed, you should use a clean :term:`TMPDIR`. | ||
674 | |||
675 | - *Enable the Appropriate Package Architecture:* By default, the | ||
676 | OpenEmbedded build system enables three levels of package | ||
677 | architectures: "all", "tune" or "package", and "machine". Any given | ||
678 | recipe usually selects one of these package architectures (types) for | ||
679 | its output. Depending for what a given recipe creates packages, | ||
680 | making sure you enable the appropriate package architecture can | ||
681 | directly impact the build time. | ||
682 | |||
683 | A recipe that just generates scripts can enable "all" architecture | ||
684 | because there are no binaries to build. To specifically enable "all" | ||
685 | architecture, be sure your recipe inherits the | ||
686 | :ref:`allarch <ref-classes-allarch>` class. | ||
687 | This class is useful for "all" architectures because it configures | ||
688 | many variables so packages can be used across multiple architectures. | ||
689 | |||
690 | If your recipe needs to generate packages that are machine-specific | ||
691 | or when one of the build or runtime dependencies is already | ||
692 | machine-architecture dependent, which makes your recipe also | ||
693 | machine-architecture dependent, make sure your recipe enables the | ||
694 | "machine" package architecture through the | ||
695 | :term:`MACHINE_ARCH` | ||
696 | variable:: | ||
697 | |||
698 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
699 | |||
700 | When you do not | ||
701 | specifically enable a package architecture through the | ||
702 | :term:`PACKAGE_ARCH`, The | ||
703 | OpenEmbedded build system defaults to the | ||
704 | :term:`TUNE_PKGARCH` setting:: | ||
705 | |||
706 | PACKAGE_ARCH = "${TUNE_PKGARCH}" | ||
707 | |||
708 | - *Choose a Generic Tuning File if Possible:* Some tunes are more | ||
709 | generic and can run on multiple targets (e.g. an ``armv5`` set of | ||
710 | packages could run on ``armv6`` and ``armv7`` processors in most | ||
711 | cases). Similarly, ``i486`` binaries could work on ``i586`` and | ||
712 | higher processors. You should realize, however, that advances on | ||
713 | newer processor versions would not be used. | ||
714 | |||
715 | If you select the same tune for several different machines, the | ||
716 | OpenEmbedded build system reuses software previously built, thus | ||
717 | speeding up the overall build time. Realize that even though a new | ||
718 | sysroot for each machine is generated, the software is not recompiled | ||
719 | and only one package feed exists. | ||
720 | |||
721 | - *Manage Granular Level Packaging:* Sometimes there are cases where | ||
722 | injecting another level of package architecture beyond the three | ||
723 | higher levels noted earlier can be useful. For example, consider how | ||
724 | NXP (formerly Freescale) allows for the easy reuse of binary packages | ||
725 | in their layer | ||
726 | :yocto_git:`meta-freescale </meta-freescale/>`. | ||
727 | In this example, the | ||
728 | :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` | ||
729 | class shares GPU packages for i.MX53 boards because all boards share | ||
730 | the AMD GPU. The i.MX6-based boards can do the same because all | ||
731 | boards share the Vivante GPU. This class inspects the BitBake | ||
732 | datastore to identify if the package provides or depends on one of | ||
733 | the sub-architecture values. If so, the class sets the | ||
734 | :term:`PACKAGE_ARCH` value | ||
735 | based on the ``MACHINE_SUBARCH`` value. If the package does not | ||
736 | provide or depend on one of the sub-architecture values but it | ||
737 | matches a value in the machine-specific filter, it sets | ||
738 | :term:`MACHINE_ARCH`. This | ||
739 | behavior reduces the number of packages built and saves build time by | ||
740 | reusing binaries. | ||
741 | |||
742 | - *Use Tools to Debug Issues:* Sometimes you can run into situations | ||
743 | where software is being rebuilt when you think it should not be. For | ||
744 | example, the OpenEmbedded build system might not be using shared | ||
745 | state between machines when you think it should be. These types of | ||
746 | situations are usually due to references to machine-specific | ||
747 | variables such as :term:`MACHINE`, | ||
748 | :term:`SERIAL_CONSOLES`, | ||
749 | :term:`XSERVER`, | ||
750 | :term:`MACHINE_FEATURES`, | ||
751 | and so forth in code that is supposed to only be tune-specific or | ||
752 | when the recipe depends | ||
753 | (:term:`DEPENDS`, | ||
754 | :term:`RDEPENDS`, | ||
755 | :term:`RRECOMMENDS`, | ||
756 | :term:`RSUGGESTS`, and so forth) | ||
757 | on some other recipe that already has | ||
758 | :term:`PACKAGE_ARCH` defined | ||
759 | as "${MACHINE_ARCH}". | ||
760 | |||
761 | .. note:: | ||
762 | |||
763 | Patches to fix any issues identified are most welcome as these | ||
764 | issues occasionally do occur. | ||
765 | |||
766 | For such cases, you can use some tools to help you sort out the | ||
767 | situation: | ||
768 | |||
769 | - ``state-diff-machines.sh``*:* You can find this tool in the | ||
770 | ``scripts`` directory of the Source Repositories. See the comments | ||
771 | in the script for information on how to use the tool. | ||
772 | |||
773 | - *BitBake's "-S printdiff" Option:* Using this option causes | ||
774 | BitBake to try to establish the closest signature match it can | ||
775 | (e.g. in the shared state cache) and then run ``bitbake-diffsigs`` | ||
776 | over the matches to determine the stamps and delta where these two | ||
777 | stamp trees diverge. | ||
778 | |||
779 | Building Software from an External Source | ||
780 | ========================================= | ||
781 | |||
782 | By default, the OpenEmbedded build system uses the :term:`Build Directory` | ||
783 | when building source code. The build process involves fetching the source | ||
784 | files, unpacking them, and then patching them if necessary before the build | ||
785 | takes place. | ||
786 | |||
787 | There are situations where you might want to build software from source | ||
788 | files that are external to and thus outside of the OpenEmbedded build | ||
789 | system. For example, suppose you have a project that includes a new BSP | ||
790 | with a heavily customized kernel. And, you want to minimize exposing the | ||
791 | build system to the development team so that they can focus on their | ||
792 | project and maintain everyone's workflow as much as possible. In this | ||
793 | case, you want a kernel source directory on the development machine | ||
794 | where the development occurs. You want the recipe's | ||
795 | :term:`SRC_URI` variable to point to | ||
796 | the external directory and use it as is, not copy it. | ||
797 | |||
798 | To build from software that comes from an external source, all you need to do | ||
799 | is inherit the :ref:`externalsrc <ref-classes-externalsrc>` class and then set | ||
800 | the :term:`EXTERNALSRC` variable to point to your external source code. Here | ||
801 | are the statements to put in your ``local.conf`` file:: | ||
802 | |||
803 | INHERIT += "externalsrc" | ||
804 | EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree" | ||
805 | |||
806 | This next example shows how to accomplish the same thing by setting | ||
807 | :term:`EXTERNALSRC` in the recipe itself or in the recipe's append file:: | ||
808 | |||
809 | EXTERNALSRC = "path" | ||
810 | EXTERNALSRC_BUILD = "path" | ||
811 | |||
812 | .. note:: | ||
813 | |||
814 | In order for these settings to take effect, you must globally or | ||
815 | locally inherit the :ref:`externalsrc <ref-classes-externalsrc>` | ||
816 | class. | ||
817 | |||
818 | By default, :ref:`ref-classes-externalsrc` builds the source code in a | ||
819 | directory separate from the external source directory as specified by | ||
820 | :term:`EXTERNALSRC`. If you need | ||
821 | to have the source built in the same directory in which it resides, or | ||
822 | some other nominated directory, you can set | ||
823 | :term:`EXTERNALSRC_BUILD` | ||
824 | to point to that directory:: | ||
825 | |||
826 | EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree" | ||
827 | |||
828 | Replicating a Build Offline | ||
829 | =========================== | ||
830 | |||
831 | It can be useful to take a "snapshot" of upstream sources used in a | ||
832 | build and then use that "snapshot" later to replicate the build offline. | ||
833 | To do so, you need to first prepare and populate your downloads | ||
834 | directory your "snapshot" of files. Once your downloads directory is | ||
835 | ready, you can use it at any time and from any machine to replicate your | ||
836 | build. | ||
837 | |||
838 | Follow these steps to populate your Downloads directory: | ||
839 | |||
840 | 1. *Create a Clean Downloads Directory:* Start with an empty downloads | ||
841 | directory (:term:`DL_DIR`). You | ||
842 | start with an empty downloads directory by either removing the files | ||
843 | in the existing directory or by setting :term:`DL_DIR` to point to either | ||
844 | an empty location or one that does not yet exist. | ||
845 | |||
846 | 2. *Generate Tarballs of the Source Git Repositories:* Edit your | ||
847 | ``local.conf`` configuration file as follows:: | ||
848 | |||
849 | DL_DIR = "/home/your-download-dir/" | ||
850 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
851 | |||
852 | During | ||
853 | the fetch process in the next step, BitBake gathers the source files | ||
854 | and creates tarballs in the directory pointed to by :term:`DL_DIR`. See | ||
855 | the | ||
856 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
857 | variable for more information. | ||
858 | |||
859 | 3. *Populate Your Downloads Directory Without Building:* Use BitBake to | ||
860 | fetch your sources but inhibit the build:: | ||
861 | |||
862 | $ bitbake target --runonly=fetch | ||
863 | |||
864 | The downloads directory (i.e. ``${DL_DIR}``) now has | ||
865 | a "snapshot" of the source files in the form of tarballs, which can | ||
866 | be used for the build. | ||
867 | |||
868 | 4. *Optionally Remove Any Git or other SCM Subdirectories From the | ||
869 | Downloads Directory:* If you want, you can clean up your downloads | ||
870 | directory by removing any Git or other Source Control Management | ||
871 | (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs | ||
872 | already contain these subdirectories. | ||
873 | |||
874 | Once your downloads directory has everything it needs regarding source | ||
875 | files, you can create your "own-mirror" and build your target. | ||
876 | Understand that you can use the files to build the target offline from | ||
877 | any machine and at any time. | ||
878 | |||
879 | Follow these steps to build your target using the files in the downloads | ||
880 | directory: | ||
881 | |||
882 | 1. *Using Local Files Only:* Inside your ``local.conf`` file, add the | ||
883 | :term:`SOURCE_MIRROR_URL` variable, inherit the | ||
884 | :ref:`own-mirrors <ref-classes-own-mirrors>` class, and use the | ||
885 | :term:`BB_NO_NETWORK` variable to your ``local.conf``:: | ||
886 | |||
887 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | ||
888 | INHERIT += "own-mirrors" | ||
889 | BB_NO_NETWORK = "1" | ||
890 | |||
891 | The :term:`SOURCE_MIRROR_URL` and :ref:`own-mirrors <ref-classes-own-mirrors>` | ||
892 | class set up the system to use the downloads directory as your "own | ||
893 | mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that | ||
894 | BitBake's fetching process in step 3 stays local, which means files | ||
895 | from your "own-mirror" are used. | ||
896 | |||
897 | 2. *Start With a Clean Build:* You can start with a clean build by | ||
898 | removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new | ||
899 | :term:`Build Directory`. | ||
900 | |||
901 | 3. *Build Your Target:* Use BitBake to build your target:: | ||
902 | |||
903 | $ bitbake target | ||
904 | |||
905 | The build completes using the known local "snapshot" of source | ||
906 | files from your mirror. The resulting tarballs for your "snapshot" of | ||
907 | source files are in the downloads directory. | ||
908 | |||
909 | .. note:: | ||
910 | |||
911 | The offline build does not work if recipes attempt to find the | ||
912 | latest version of software by setting | ||
913 | :term:`SRCREV` to | ||
914 | ``${``\ :term:`AUTOREV`\ ``}``:: | ||
915 | |||
916 | SRCREV = "${AUTOREV}" | ||
917 | |||
918 | When a recipe sets :term:`SRCREV` to | ||
919 | ``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an | ||
920 | attempt to determine the latest version of software from the SCM. | ||
921 | Typically, recipes that use :term:`AUTOREV` are custom or modified | ||
922 | recipes. Recipes that reside in public repositories usually do not | ||
923 | use :term:`AUTOREV`. | ||
924 | |||
925 | If you do have recipes that use :term:`AUTOREV`, you can take steps to | ||
926 | still use the recipes in an offline build. Do the following: | ||
927 | |||
928 | 1. Use a configuration generated by enabling :ref:`build | ||
929 | history <dev-manual/build-quality:maintaining build output quality>`. | ||
930 | |||
931 | 2. Use the ``buildhistory-collect-srcrevs`` command to collect the | ||
932 | stored :term:`SRCREV` values from the build's history. For more | ||
933 | information on collecting these values, see the | ||
934 | ":ref:`dev-manual/build-quality:build history package information`" | ||
935 | section. | ||
936 | |||
937 | 3. Once you have the correct source revisions, you can modify | ||
938 | those recipes to set :term:`SRCREV` to specific versions of the | ||
939 | software. | ||
940 | |||