diff options
Diffstat (limited to 'documentation/dev-manual/building.rst')
-rw-r--r-- | documentation/dev-manual/building.rst | 872 |
1 files changed, 872 insertions, 0 deletions
diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst new file mode 100644 index 0000000000..32c7aa5da0 --- /dev/null +++ b/documentation/dev-manual/building.rst | |||
@@ -0,0 +1,872 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Building | ||
4 | ******** | ||
5 | |||
6 | This section describes various build procedures, such as the steps | ||
7 | needed for a simple build, building a target for multiple configurations, | ||
8 | generating an image for more than one machine, and so forth. | ||
9 | |||
10 | Building a Simple Image | ||
11 | ======================= | ||
12 | |||
13 | In the development environment, you need to build an image whenever you | ||
14 | change hardware support, add or change system libraries, or add or | ||
15 | change services that have dependencies. There are several methods that allow | ||
16 | you to build an image within the Yocto Project. This section presents | ||
17 | the basic steps you need to build a simple image using BitBake from a | ||
18 | build host running Linux. | ||
19 | |||
20 | .. note:: | ||
21 | |||
22 | - For information on how to build an image using | ||
23 | :term:`Toaster`, see the | ||
24 | :doc:`/toaster-manual/index`. | ||
25 | |||
26 | - For information on how to use ``devtool`` to build images, see the | ||
27 | ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
28 | section in the Yocto Project Application Development and the | ||
29 | Extensible Software Development Kit (eSDK) manual. | ||
30 | |||
31 | - For a quick example on how to build an image using the | ||
32 | OpenEmbedded build system, see the | ||
33 | :doc:`/brief-yoctoprojectqs/index` document. | ||
34 | |||
35 | - You can also use the `Yocto Project BitBake | ||
36 | <https://marketplace.visualstudio.com/items?itemName=yocto-project.yocto-bitbake>`__ | ||
37 | extension for Visual Studio Code to build images. | ||
38 | |||
39 | The build process creates an entire Linux distribution from source and | ||
40 | places it in your :term:`Build Directory` under ``tmp/deploy/images``. For | ||
41 | detailed information on the build process using BitBake, see the | ||
42 | ":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview | ||
43 | and Concepts Manual. | ||
44 | |||
45 | The following figure and list overviews the build process: | ||
46 | |||
47 | .. image:: figures/bitbake-build-flow.png | ||
48 | :width: 100% | ||
49 | |||
50 | #. *Set up Your Host Development System to Support Development Using the | ||
51 | Yocto Project*: See the ":doc:`/dev-manual/start`" section for options on how to get a | ||
52 | build host ready to use the Yocto Project. | ||
53 | |||
54 | #. *Initialize the Build Environment:* Initialize the build environment | ||
55 | by sourcing the build environment script (i.e. | ||
56 | :ref:`structure-core-script`):: | ||
57 | |||
58 | $ source oe-init-build-env [build_dir] | ||
59 | |||
60 | When you use the initialization script, the OpenEmbedded build system | ||
61 | uses ``build`` as the default :term:`Build Directory` in your current work | ||
62 | directory. You can use a `build_dir` argument with the script to | ||
63 | specify a different :term:`Build Directory`. | ||
64 | |||
65 | .. note:: | ||
66 | |||
67 | A common practice is to use a different :term:`Build Directory` for | ||
68 | different targets; for example, ``~/build/x86`` for a ``qemux86`` | ||
69 | target, and ``~/build/arm`` for a ``qemuarm`` target. In any | ||
70 | event, it's typically cleaner to locate the :term:`Build Directory` | ||
71 | somewhere outside of your source directory. | ||
72 | |||
73 | #. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the | ||
74 | ``conf/local.conf`` configuration file, which is found in the | ||
75 | :term:`Build Directory`, is set up how you want it. This file defines many | ||
76 | aspects of the build environment including the target machine architecture | ||
77 | through the :term:`MACHINE` variable, the packaging format used during | ||
78 | the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download | ||
79 | directory through the :term:`DL_DIR` variable. | ||
80 | |||
81 | #. *Build the Image:* Build the image using the ``bitbake`` command:: | ||
82 | |||
83 | $ bitbake target | ||
84 | |||
85 | .. note:: | ||
86 | |||
87 | For information on BitBake, see the :doc:`bitbake:index`. | ||
88 | |||
89 | The target is the name of the recipe you want to build. Common | ||
90 | targets are the images in ``meta/recipes-core/images``, | ||
91 | ``meta/recipes-sato/images``, and so forth all found in the | ||
92 | :term:`Source Directory`. Alternatively, the target | ||
93 | can be the name of a recipe for a specific piece of software such as | ||
94 | BusyBox. For more details about the images the OpenEmbedded build | ||
95 | system supports, see the | ||
96 | ":ref:`ref-manual/images:Images`" chapter in the Yocto | ||
97 | Project Reference Manual. | ||
98 | |||
99 | As an example, the following command builds the | ||
100 | ``core-image-minimal`` image:: | ||
101 | |||
102 | $ bitbake core-image-minimal | ||
103 | |||
104 | Once an | ||
105 | image has been built, it often needs to be installed. The images and | ||
106 | kernels built by the OpenEmbedded build system are placed in the | ||
107 | :term:`Build Directory` in ``tmp/deploy/images``. For information on how to | ||
108 | run pre-built images such as ``qemux86`` and ``qemuarm``, see the | ||
109 | :doc:`/sdk-manual/index` manual. For | ||
110 | information about how to install these images, see the documentation | ||
111 | for your particular board or machine. | ||
112 | |||
113 | Building Images for Multiple Targets Using Multiple Configurations | ||
114 | ================================================================== | ||
115 | |||
116 | See the :doc:`/dev-manual/multiconfig` section of the Yocto Project Development Tasks | ||
117 | Manual. | ||
118 | |||
119 | Building an Initial RAM Filesystem (Initramfs) Image | ||
120 | ==================================================== | ||
121 | |||
122 | An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root | ||
123 | filesystem used for early system initialization, typically providing tools and | ||
124 | loading modules needed to locate and mount the final root filesystem. | ||
125 | |||
126 | Follow these steps to create an :term:`Initramfs` image: | ||
127 | |||
128 | #. *Create the Initramfs Image Recipe:* You can reference the | ||
129 | ``core-image-minimal-initramfs.bb`` recipe found in the | ||
130 | ``meta/recipes-core`` directory of the :term:`Source Directory` | ||
131 | as an example from which to work. The ``core-image-minimal-initramfs`` recipe | ||
132 | is based on the :ref:`initramfs-framework <dev-manual/building:Customizing an | ||
133 | Initramfs using \`\`initramfs-framework\`\`>` recipe described below. | ||
134 | |||
135 | #. *Decide if You Need to Bundle the Initramfs Image Into the Kernel | ||
136 | Image:* If you want the :term:`Initramfs` image that is built to be bundled | ||
137 | in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE` | ||
138 | variable to ``"1"`` in your ``local.conf`` configuration file and set the | ||
139 | :term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image. | ||
140 | |||
141 | Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs` | ||
142 | image to be unpacked into the ``${B}/usr/`` directory. The unpacked | ||
143 | :term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the | ||
144 | :term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs` | ||
145 | image to be built into the kernel normally. | ||
146 | |||
147 | #. *Optionally Add Items to the Initramfs Image Through the Initramfs | ||
148 | Image Recipe:* If you add items to the :term:`Initramfs` image by way of its | ||
149 | recipe, you should use :term:`PACKAGE_INSTALL` rather than | ||
150 | :term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of | ||
151 | what is added to the image as compared to the defaults you might not | ||
152 | necessarily want that are set by the :ref:`ref-classes-image` | ||
153 | or :ref:`ref-classes-core-image` classes. | ||
154 | |||
155 | #. *Build the Kernel Image and the Initramfs Image:* Build your kernel | ||
156 | image using BitBake. Because the :term:`Initramfs` image recipe is a | ||
157 | dependency of the kernel image, the :term:`Initramfs` image is built as well | ||
158 | and bundled with the kernel image if you used the | ||
159 | :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. | ||
160 | |||
161 | Customizing an Initramfs using ``initramfs-framework`` | ||
162 | ------------------------------------------------------ | ||
163 | |||
164 | The ``core-image-minimal-initramfs.bb`` recipe found in | ||
165 | :oe_git:`meta/recipes-core/images | ||
166 | </openembedded-core/tree/meta/recipes-core/images>` uses the | ||
167 | :oe_git:`initramfs-framework_1.0.bb | ||
168 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb>` | ||
169 | recipe as its base component. The goal of the ``initramfs-framework`` recipe is | ||
170 | to provide the building blocks to build a customized :term:`Initramfs`. | ||
171 | |||
172 | The ``initramfs-framework`` recipe relies on shell initialization scripts | ||
173 | defined in :oe_git:`meta/recipes-core/initrdscripts/initramfs-framework | ||
174 | </openembedded-core/tree/meta/recipes-core/initrdscripts/initramfs-framework>`. Since some of | ||
175 | these scripts do not apply for all use cases, the ``initramfs-framework`` recipe | ||
176 | defines different packages: | ||
177 | |||
178 | - ``initramfs-framework-base``: this package installs the basic components of | ||
179 | an :term:`Initramfs`, such as the ``init`` script or the ``/dev/console`` | ||
180 | character special file. As this package is a runtime dependency of all | ||
181 | modules listed below, it is automatically pulled in when one of the modules | ||
182 | is installed in the image. | ||
183 | - ``initramfs-module-exec``: support for execution of applications. | ||
184 | - ``initramfs-module-mdev``: support for `mdev | ||
185 | <https://wiki.gentoo.org/wiki/Mdev>`__. | ||
186 | - ``initramfs-module-udev``: support for :wikipedia:`Udev <Udev>`. | ||
187 | - ``initramfs-module-e2fs``: support for :wikipedia:`ext4/ext3/ext2 | ||
188 | <Extended_file_system>` filesystems. | ||
189 | - ``initramfs-module-nfsrootfs``: support for locating and mounting the root | ||
190 | partition via :wikipedia:`NFS <Network_File_System>`. | ||
191 | - ``initramfs-module-rootfs``: support for locating and mounting the root | ||
192 | partition. | ||
193 | - ``initramfs-module-debug``: dynamic debug support. | ||
194 | - ``initramfs-module-lvm``: :wikipedia:`LVM <Logical_volume_management>` rootfs support. | ||
195 | - ``initramfs-module-overlayroot``: support for mounting a read-write overlay | ||
196 | on top of a read-only root filesystem. | ||
197 | |||
198 | In addition to the packages defined by the ``initramfs-framework`` recipe | ||
199 | itself, the following packages are defined by the recipes present in | ||
200 | :oe_git:`meta/recipes-core/initrdscripts </openembedded-core/tree/meta/recipes-core/initrdscripts>`: | ||
201 | |||
202 | - ``initramfs-module-install``: module to create and install a partition layout | ||
203 | on a selected block device. | ||
204 | - ``initramfs-module-install-efi``: module to create and install an EFI | ||
205 | partition layout on a selected block device. | ||
206 | - ``initramfs-module-setup-live``: module to start a shell in the | ||
207 | :term:`Initramfs` if ``root=/dev/ram0`` in passed in the `Kernel command-line | ||
208 | <https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html>`__ | ||
209 | or the ``root=`` parameter was not passed. | ||
210 | |||
211 | To customize the :term:`Initramfs`, you can add or remove packages listed | ||
212 | earlier from the :term:`PACKAGE_INSTALL` variable with a :ref:`bbappend | ||
213 | <dev-manual/layers:Appending Other Layers Metadata With Your Layer>` on the | ||
214 | ``core-image-minimal-initramfs`` recipe, or create a custom recipe for the | ||
215 | :term:`Initramfs` taking ``core-image-minimal-initramfs`` as example. | ||
216 | |||
217 | Custom scripts can be added to the :term:`Initramfs` by writing your own | ||
218 | recipes. The recipes are conventionally named ``initramfs-module-<module name>`` | ||
219 | where ``<module name>`` is the name of the module. The recipe should set its | ||
220 | :term:`RDEPENDS` package-specific variables to include | ||
221 | ``initramfs-framework-base`` and the other packages on which the module depends | ||
222 | at runtime. | ||
223 | |||
224 | The recipe must install shell initialization scripts in :term:`${D} <D>`\ | ||
225 | ``/init.d`` and must follow the ``<number>-<script name>`` naming scheme where: | ||
226 | |||
227 | - ``<number>`` is a *two-digit* number that affects the execution order of the | ||
228 | script compared to others. For example, the script ``80-setup-live`` would be | ||
229 | executed after ``01-udev`` because 80 is greater than 01. | ||
230 | |||
231 | This number being two-digits is important here as the scripts are executed | ||
232 | alphabetically. For example, the script ``10-script`` would be executed | ||
233 | before the script ``8-script``, because ``1`` is inferior to ``8``. | ||
234 | Therefore, the script should be named ``08-script``. | ||
235 | |||
236 | - ``<script name>`` is the script name which you can choose freely. | ||
237 | |||
238 | If two script use the same ``<number>``, they are sorted alphabetically based | ||
239 | on ``<script name>``. | ||
240 | |||
241 | Bundling an Initramfs Image From a Separate Multiconfig | ||
242 | ------------------------------------------------------- | ||
243 | |||
244 | There may be a case where we want to build an :term:`Initramfs` image which does not | ||
245 | inherit the same distro policy as our main image, for example, we may want | ||
246 | our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs` | ||
247 | image to keep a smaller footprint. However, by performing the steps mentioned | ||
248 | above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us | ||
249 | to override it. | ||
250 | |||
251 | To achieve this, you need to perform some additional steps: | ||
252 | |||
253 | #. *Create a multiconfig for your Initramfs image:* You can perform the steps | ||
254 | on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig. | ||
255 | For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and | ||
256 | contains the variables:: | ||
257 | |||
258 | TMPDIR="${TOPDIR}/tmp-initramfscfg" | ||
259 | TCLIBC="musl" | ||
260 | |||
261 | #. *Set additional Initramfs variables on your main configuration:* | ||
262 | Additionally, on your main configuration (``local.conf``) you need to set the | ||
263 | variables:: | ||
264 | |||
265 | INITRAMFS_MULTICONFIG = "initramfscfg" | ||
266 | INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}" | ||
267 | |||
268 | The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
269 | are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE` | ||
270 | to be built coming from the ``initramfscfg`` multiconfig, and to let the | ||
271 | buildsystem know where the :term:`INITRAMFS_IMAGE` will be located. | ||
272 | |||
273 | Building a system with such configuration will build the kernel using the | ||
274 | main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the | ||
275 | selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE` | ||
276 | instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel | ||
277 | but a glibc based main image. | ||
278 | |||
279 | The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE` | ||
280 | or to build a different :term:`DISTRO` for it such as ``poky-tiny``. | ||
281 | |||
282 | |||
283 | Building a Tiny System | ||
284 | ====================== | ||
285 | |||
286 | Very small distributions have some significant advantages such as | ||
287 | requiring less on-die or in-package memory (cheaper), better performance | ||
288 | through efficient cache usage, lower power requirements due to less | ||
289 | memory, faster boot times, and reduced development overhead. Some | ||
290 | real-world examples where a very small distribution gives you distinct | ||
291 | advantages are digital cameras, medical devices, and small headless | ||
292 | systems. | ||
293 | |||
294 | This section presents information that shows you how you can trim your | ||
295 | distribution to even smaller sizes than the ``poky-tiny`` distribution, | ||
296 | which is around 5 Mbytes, that can be built out-of-the-box using the | ||
297 | Yocto Project. | ||
298 | |||
299 | Tiny System Overview | ||
300 | -------------------- | ||
301 | |||
302 | The following list presents the overall steps you need to consider and | ||
303 | perform to create distributions with smaller root filesystems, achieve | ||
304 | faster boot times, maintain your critical functionality, and avoid | ||
305 | initial RAM disks: | ||
306 | |||
307 | - :ref:`Determine your goals and guiding principles | ||
308 | <dev-manual/building:goals and guiding principles>` | ||
309 | |||
310 | - :ref:`dev-manual/building:understand what contributes to your image size` | ||
311 | |||
312 | - :ref:`Reduce the size of the root filesystem | ||
313 | <dev-manual/building:trim the root filesystem>` | ||
314 | |||
315 | - :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>` | ||
316 | |||
317 | - :ref:`dev-manual/building:remove package management requirements` | ||
318 | |||
319 | - :ref:`dev-manual/building:look for other ways to minimize size` | ||
320 | |||
321 | - :ref:`dev-manual/building:iterate on the process` | ||
322 | |||
323 | Goals and Guiding Principles | ||
324 | ---------------------------- | ||
325 | |||
326 | Before you can reach your destination, you need to know where you are | ||
327 | going. Here is an example list that you can use as a guide when creating | ||
328 | very small distributions: | ||
329 | |||
330 | - Determine how much space you need (e.g. a kernel that is 1 Mbyte or | ||
331 | less and a root filesystem that is 3 Mbytes or less). | ||
332 | |||
333 | - Find the areas that are currently taking 90% of the space and | ||
334 | concentrate on reducing those areas. | ||
335 | |||
336 | - Do not create any difficult "hacks" to achieve your goals. | ||
337 | |||
338 | - Leverage the device-specific options. | ||
339 | |||
340 | - Work in a separate layer so that you keep changes isolated. For | ||
341 | information on how to create layers, see the | ||
342 | ":ref:`dev-manual/layers:understanding and creating layers`" section. | ||
343 | |||
344 | Understand What Contributes to Your Image Size | ||
345 | ---------------------------------------------- | ||
346 | |||
347 | It is easiest to have something to start with when creating your own | ||
348 | distribution. You can use the Yocto Project out-of-the-box to create the | ||
349 | ``poky-tiny`` distribution. Ultimately, you will want to make changes in | ||
350 | your own distribution that are likely modeled after ``poky-tiny``. | ||
351 | |||
352 | .. note:: | ||
353 | |||
354 | To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your | ||
355 | ``local.conf`` file to "poky-tiny" as described in the | ||
356 | ":ref:`dev-manual/custom-distribution:creating your own distribution`" | ||
357 | section. | ||
358 | |||
359 | Understanding some memory concepts will help you reduce the system size. | ||
360 | Memory consists of static, dynamic, and temporary memory. Static memory | ||
361 | is the TEXT (code), DATA (initialized data in the code), and BSS | ||
362 | (uninitialized data) sections. Dynamic memory represents memory that is | ||
363 | allocated at runtime: stacks, hash tables, and so forth. Temporary | ||
364 | memory is recovered after the boot process. This memory consists of | ||
365 | memory used for decompressing the kernel and for the ``__init__`` | ||
366 | functions. | ||
367 | |||
368 | To help you see where you currently are with kernel and root filesystem | ||
369 | sizes, you can use two tools found in the :term:`Source Directory` | ||
370 | in the | ||
371 | ``scripts/tiny/`` directory: | ||
372 | |||
373 | - ``ksize.py``: Reports component sizes for the kernel build objects. | ||
374 | |||
375 | - ``dirsize.py``: Reports component sizes for the root filesystem. | ||
376 | |||
377 | This next tool and command help you organize configuration fragments and | ||
378 | view file dependencies in a human-readable form: | ||
379 | |||
380 | - ``merge_config.sh``: Helps you manage configuration files and | ||
381 | fragments within the kernel. With this tool, you can merge individual | ||
382 | configuration fragments together. The tool allows you to make | ||
383 | overrides and warns you of any missing configuration options. The | ||
384 | tool is ideal for allowing you to iterate on configurations, create | ||
385 | minimal configurations, and create configuration files for different | ||
386 | machines without having to duplicate your process. | ||
387 | |||
388 | The ``merge_config.sh`` script is part of the Linux Yocto kernel Git | ||
389 | repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, | ||
390 | ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` | ||
391 | directory. | ||
392 | |||
393 | For more information on configuration fragments, see the | ||
394 | ":ref:`kernel-dev/common:creating configuration fragments`" | ||
395 | section in the Yocto Project Linux Kernel Development Manual. | ||
396 | |||
397 | - ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command | ||
398 | with these options brings up a Dependency Explorer from which you can | ||
399 | view file dependencies. Understanding these dependencies allows you | ||
400 | to make informed decisions when cutting out various pieces of the | ||
401 | kernel and root filesystem. | ||
402 | |||
403 | Trim the Root Filesystem | ||
404 | ------------------------ | ||
405 | |||
406 | The root filesystem is made up of packages for booting, libraries, and | ||
407 | applications. To change things, you can configure how the packaging | ||
408 | happens, which changes the way you build them. You can also modify the | ||
409 | filesystem itself or select a different filesystem. | ||
410 | |||
411 | First, find out what is hogging your root filesystem by running the | ||
412 | ``dirsize.py`` script from your root directory:: | ||
413 | |||
414 | $ cd root-directory-of-image | ||
415 | $ dirsize.py 100000 > dirsize-100k.log | ||
416 | $ cat dirsize-100k.log | ||
417 | |||
418 | You can apply a filter to the script to ignore files | ||
419 | under a certain size. The previous example filters out any files below | ||
420 | 100 Kbytes. The sizes reported by the tool are uncompressed, and thus | ||
421 | will be smaller by a relatively constant factor in a compressed root | ||
422 | filesystem. When you examine your log file, you can focus on areas of | ||
423 | the root filesystem that take up large amounts of memory. | ||
424 | |||
425 | You need to be sure that what you eliminate does not cripple the | ||
426 | functionality you need. One way to see how packages relate to each other | ||
427 | is by using the Dependency Explorer UI with the BitBake command:: | ||
428 | |||
429 | $ cd image-directory | ||
430 | $ bitbake -u taskexp -g image | ||
431 | |||
432 | Use the interface to | ||
433 | select potential packages you wish to eliminate and see their dependency | ||
434 | relationships. | ||
435 | |||
436 | When deciding how to reduce the size, get rid of packages that result in | ||
437 | minimal impact on the feature set. For example, you might not need a VGA | ||
438 | display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` | ||
439 | instead of ``udev``. | ||
440 | |||
441 | Use your ``local.conf`` file to make changes. For example, to eliminate | ||
442 | ``udev`` and ``glib``, set the following in the local configuration | ||
443 | file:: | ||
444 | |||
445 | VIRTUAL-RUNTIME_dev_manager = "" | ||
446 | |||
447 | Finally, you should consider exactly the type of root filesystem you | ||
448 | need to meet your needs while also reducing its size. For example, | ||
449 | consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an | ||
450 | :term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1 | ||
451 | Mbyte journal. If you are okay with running read-only, you do not need | ||
452 | this journal. | ||
453 | |||
454 | .. note:: | ||
455 | |||
456 | After each round of elimination, you need to rebuild your system and | ||
457 | then use the tools to see the effects of your reductions. | ||
458 | |||
459 | Trim the Kernel | ||
460 | --------------- | ||
461 | |||
462 | The kernel is built by including policies for hardware-independent | ||
463 | aspects. What subsystems do you enable? For what architecture are you | ||
464 | building? Which drivers do you build by default? | ||
465 | |||
466 | .. note:: | ||
467 | |||
468 | You can modify the kernel source if you want to help with boot time. | ||
469 | |||
470 | Run the ``ksize.py`` script from the top-level Linux build directory to | ||
471 | get an idea of what is making up the kernel:: | ||
472 | |||
473 | $ cd top-level-linux-build-directory | ||
474 | $ ksize.py > ksize.log | ||
475 | $ cat ksize.log | ||
476 | |||
477 | When you examine the log, you will see how much space is taken up with | ||
478 | the built-in ``.o`` files for drivers, networking, core kernel files, | ||
479 | filesystem, sound, and so forth. The sizes reported by the tool are | ||
480 | uncompressed, and thus will be smaller by a relatively constant factor | ||
481 | in a compressed kernel image. Look to reduce the areas that are large | ||
482 | and taking up around the "90% rule." | ||
483 | |||
484 | To examine, or drill down, into any particular area, use the ``-d`` | ||
485 | option with the script:: | ||
486 | |||
487 | $ ksize.py -d > ksize.log | ||
488 | |||
489 | Using this option | ||
490 | breaks out the individual file information for each area of the kernel | ||
491 | (e.g. drivers, networking, and so forth). | ||
492 | |||
493 | Use your log file to see what you can eliminate from the kernel based on | ||
494 | features you can let go. For example, if you are not going to need | ||
495 | sound, you do not need any drivers that support sound. | ||
496 | |||
497 | After figuring out what to eliminate, you need to reconfigure the kernel | ||
498 | to reflect those changes during the next build. You could run | ||
499 | ``menuconfig`` and make all your changes at once. However, that makes it | ||
500 | difficult to see the effects of your individual eliminations and also | ||
501 | makes it difficult to replicate the changes for perhaps another target | ||
502 | device. A better method is to start with no configurations using | ||
503 | ``allnoconfig``, create configuration fragments for individual changes, | ||
504 | and then manage the fragments into a single configuration file using | ||
505 | ``merge_config.sh``. The tool makes it easy for you to iterate using the | ||
506 | configuration change and build cycle. | ||
507 | |||
508 | Each time you make configuration changes, you need to rebuild the kernel | ||
509 | and check to see what impact your changes had on the overall size. | ||
510 | |||
511 | Remove Package Management Requirements | ||
512 | -------------------------------------- | ||
513 | |||
514 | Packaging requirements add size to the image. One way to reduce the size | ||
515 | of the image is to remove all the packaging requirements from the image. | ||
516 | This reduction includes both removing the package manager and its unique | ||
517 | dependencies as well as removing the package management data itself. | ||
518 | |||
519 | To eliminate all the packaging requirements for an image, be sure that | ||
520 | "package-management" is not part of your | ||
521 | :term:`IMAGE_FEATURES` | ||
522 | statement for the image. When you remove this feature, you are removing | ||
523 | the package manager as well as its dependencies from the root | ||
524 | filesystem. | ||
525 | |||
526 | Look for Other Ways to Minimize Size | ||
527 | ------------------------------------ | ||
528 | |||
529 | Depending on your particular circumstances, other areas that you can | ||
530 | trim likely exist. The key to finding these areas is through tools and | ||
531 | methods described here combined with experimentation and iteration. Here | ||
532 | are a couple of areas to experiment with: | ||
533 | |||
534 | - ``glibc``: In general, follow this process: | ||
535 | |||
536 | #. Remove ``glibc`` features from | ||
537 | :term:`DISTRO_FEATURES` | ||
538 | that you think you do not need. | ||
539 | |||
540 | #. Build your distribution. | ||
541 | |||
542 | #. If the build fails due to missing symbols in a package, determine | ||
543 | if you can reconfigure the package to not need those features. For | ||
544 | example, change the configuration to not support wide character | ||
545 | support as is done for ``ncurses``. Or, if support for those | ||
546 | characters is needed, determine what ``glibc`` features provide | ||
547 | the support and restore the configuration. | ||
548 | |||
549 | 4. Rebuild and repeat the process. | ||
550 | |||
551 | - ``busybox``: For BusyBox, use a process similar as described for | ||
552 | ``glibc``. A difference is you will need to boot the resulting system | ||
553 | to see if you are able to do everything you expect from the running | ||
554 | system. You need to be sure to integrate configuration fragments into | ||
555 | Busybox because BusyBox handles its own core features and then allows | ||
556 | you to add configuration fragments on top. | ||
557 | |||
558 | Iterate on the Process | ||
559 | ---------------------- | ||
560 | |||
561 | If you have not reached your goals on system size, you need to iterate | ||
562 | on the process. The process is the same. Use the tools and see just what | ||
563 | is taking up 90% of the root filesystem and the kernel. Decide what you | ||
564 | can eliminate without limiting your device beyond what you need. | ||
565 | |||
566 | Depending on your system, a good place to look might be Busybox, which | ||
567 | provides a stripped down version of Unix tools in a single, executable | ||
568 | file. You might be able to drop virtual terminal services or perhaps | ||
569 | ipv6. | ||
570 | |||
571 | Building Images for More than One Machine | ||
572 | ========================================= | ||
573 | |||
574 | A common scenario developers face is creating images for several | ||
575 | different machines that use the same software environment. In this | ||
576 | situation, it is tempting to set the tunings and optimization flags for | ||
577 | each build specifically for the targeted hardware (i.e. "maxing out" the | ||
578 | tunings). Doing so can considerably add to build times and package feed | ||
579 | maintenance collectively for the machines. For example, selecting tunes | ||
580 | that are extremely specific to a CPU core used in a system might enable | ||
581 | some micro optimizations in GCC for that particular system but would | ||
582 | otherwise not gain you much of a performance difference across the other | ||
583 | systems as compared to using a more general tuning across all the builds | ||
584 | (e.g. setting :term:`DEFAULTTUNE` | ||
585 | specifically for each machine's build). Rather than "max out" each | ||
586 | build's tunings, you can take steps that cause the OpenEmbedded build | ||
587 | system to reuse software across the various machines where it makes | ||
588 | sense. | ||
589 | |||
590 | If build speed and package feed maintenance are considerations, you | ||
591 | should consider the points in this section that can help you optimize | ||
592 | your tunings to best consider build times and package feed maintenance. | ||
593 | |||
594 | - *Share the* :term:`Build Directory` *:* If at all possible, share the | ||
595 | :term:`TMPDIR` across builds. The Yocto Project supports switching between | ||
596 | different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice | ||
597 | is well supported and regularly used by developers when building for | ||
598 | multiple machines. When you use the same :term:`TMPDIR` for multiple | ||
599 | machine builds, the OpenEmbedded build system can reuse the existing native | ||
600 | and often cross-recipes for multiple machines. Thus, build time decreases. | ||
601 | |||
602 | .. note:: | ||
603 | |||
604 | If :term:`DISTRO` settings change or fundamental configuration settings | ||
605 | such as the filesystem layout, you need to work with a clean :term:`TMPDIR`. | ||
606 | Sharing :term:`TMPDIR` under these circumstances might work but since it is | ||
607 | not guaranteed, you should use a clean :term:`TMPDIR`. | ||
608 | |||
609 | - *Enable the Appropriate Package Architecture:* By default, the | ||
610 | OpenEmbedded build system enables three levels of package | ||
611 | architectures: "all", "tune" or "package", and "machine". Any given | ||
612 | recipe usually selects one of these package architectures (types) for | ||
613 | its output. Depending for what a given recipe creates packages, | ||
614 | making sure you enable the appropriate package architecture can | ||
615 | directly impact the build time. | ||
616 | |||
617 | A recipe that just generates scripts can enable "all" architecture | ||
618 | because there are no binaries to build. To specifically enable "all" | ||
619 | architecture, be sure your recipe inherits the | ||
620 | :ref:`ref-classes-allarch` class. | ||
621 | This class is useful for "all" architectures because it configures | ||
622 | many variables so packages can be used across multiple architectures. | ||
623 | |||
624 | If your recipe needs to generate packages that are machine-specific | ||
625 | or when one of the build or runtime dependencies is already | ||
626 | machine-architecture dependent, which makes your recipe also | ||
627 | machine-architecture dependent, make sure your recipe enables the | ||
628 | "machine" package architecture through the | ||
629 | :term:`MACHINE_ARCH` | ||
630 | variable:: | ||
631 | |||
632 | PACKAGE_ARCH = "${MACHINE_ARCH}" | ||
633 | |||
634 | When you do not | ||
635 | specifically enable a package architecture through the | ||
636 | :term:`PACKAGE_ARCH`, The | ||
637 | OpenEmbedded build system defaults to the | ||
638 | :term:`TUNE_PKGARCH` setting:: | ||
639 | |||
640 | PACKAGE_ARCH = "${TUNE_PKGARCH}" | ||
641 | |||
642 | - *Choose a Generic Tuning File if Possible:* Some tunes are more | ||
643 | generic and can run on multiple targets (e.g. an ``armv5`` set of | ||
644 | packages could run on ``armv6`` and ``armv7`` processors in most | ||
645 | cases). Similarly, ``i486`` binaries could work on ``i586`` and | ||
646 | higher processors. You should realize, however, that advances on | ||
647 | newer processor versions would not be used. | ||
648 | |||
649 | If you select the same tune for several different machines, the | ||
650 | OpenEmbedded build system reuses software previously built, thus | ||
651 | speeding up the overall build time. Realize that even though a new | ||
652 | sysroot for each machine is generated, the software is not recompiled | ||
653 | and only one package feed exists. | ||
654 | |||
655 | - *Manage Granular Level Packaging:* Sometimes there are cases where | ||
656 | injecting another level of package architecture beyond the three | ||
657 | higher levels noted earlier can be useful. For example, consider how | ||
658 | NXP (formerly Freescale) allows for the easy reuse of binary packages | ||
659 | in their layer | ||
660 | :yocto_git:`meta-freescale </meta-freescale/>`. | ||
661 | In this example, the | ||
662 | :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` | ||
663 | class shares GPU packages for i.MX53 boards because all boards share | ||
664 | the AMD GPU. The i.MX6-based boards can do the same because all | ||
665 | boards share the Vivante GPU. This class inspects the BitBake | ||
666 | datastore to identify if the package provides or depends on one of | ||
667 | the sub-architecture values. If so, the class sets the | ||
668 | :term:`PACKAGE_ARCH` value | ||
669 | based on the ``MACHINE_SUBARCH`` value. If the package does not | ||
670 | provide or depend on one of the sub-architecture values but it | ||
671 | matches a value in the machine-specific filter, it sets | ||
672 | :term:`MACHINE_ARCH`. This | ||
673 | behavior reduces the number of packages built and saves build time by | ||
674 | reusing binaries. | ||
675 | |||
676 | - *Use Tools to Debug Issues:* Sometimes you can run into situations | ||
677 | where software is being rebuilt when you think it should not be. For | ||
678 | example, the OpenEmbedded build system might not be using shared | ||
679 | state between machines when you think it should be. These types of | ||
680 | situations are usually due to references to machine-specific | ||
681 | variables such as :term:`MACHINE`, | ||
682 | :term:`SERIAL_CONSOLES`, | ||
683 | :term:`XSERVER`, | ||
684 | :term:`MACHINE_FEATURES`, | ||
685 | and so forth in code that is supposed to only be tune-specific or | ||
686 | when the recipe depends | ||
687 | (:term:`DEPENDS`, | ||
688 | :term:`RDEPENDS`, | ||
689 | :term:`RRECOMMENDS`, | ||
690 | :term:`RSUGGESTS`, and so forth) | ||
691 | on some other recipe that already has | ||
692 | :term:`PACKAGE_ARCH` defined | ||
693 | as "${MACHINE_ARCH}". | ||
694 | |||
695 | .. note:: | ||
696 | |||
697 | Patches to fix any issues identified are most welcome as these | ||
698 | issues occasionally do occur. | ||
699 | |||
700 | For such cases, you can use some tools to help you sort out the | ||
701 | situation: | ||
702 | |||
703 | - ``state-diff-machines.sh``*:* You can find this tool in the | ||
704 | ``scripts`` directory of the Source Repositories. See the comments | ||
705 | in the script for information on how to use the tool. | ||
706 | |||
707 | - *BitBake's "-S printdiff" Option:* Using this option causes | ||
708 | BitBake to try to establish the most recent signature match | ||
709 | (e.g. in the shared state cache) and then compare matched signatures | ||
710 | to determine the stamps and delta where these two stamp trees diverge. | ||
711 | |||
712 | Building Software from an External Source | ||
713 | ========================================= | ||
714 | |||
715 | By default, the OpenEmbedded build system uses the :term:`Build Directory` | ||
716 | when building source code. The build process involves fetching the source | ||
717 | files, unpacking them, and then patching them if necessary before the build | ||
718 | takes place. | ||
719 | |||
720 | There are situations where you might want to build software from source | ||
721 | files that are external to and thus outside of the OpenEmbedded build | ||
722 | system. For example, suppose you have a project that includes a new BSP | ||
723 | with a heavily customized kernel. And, you want to minimize exposing the | ||
724 | build system to the development team so that they can focus on their | ||
725 | project and maintain everyone's workflow as much as possible. In this | ||
726 | case, you want a kernel source directory on the development machine | ||
727 | where the development occurs. You want the recipe's | ||
728 | :term:`SRC_URI` variable to point to | ||
729 | the external directory and use it as is, not copy it. | ||
730 | |||
731 | To build from software that comes from an external source, all you need to do | ||
732 | is inherit the :ref:`ref-classes-externalsrc` class and then set | ||
733 | the :term:`EXTERNALSRC` variable to point to your external source code. Here | ||
734 | are the statements to put in your ``local.conf`` file:: | ||
735 | |||
736 | INHERIT += "externalsrc" | ||
737 | EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree" | ||
738 | |||
739 | This next example shows how to accomplish the same thing by setting | ||
740 | :term:`EXTERNALSRC` in the recipe itself or in the recipe's append file:: | ||
741 | |||
742 | EXTERNALSRC = "path" | ||
743 | EXTERNALSRC_BUILD = "path" | ||
744 | |||
745 | .. note:: | ||
746 | |||
747 | In order for these settings to take effect, you must globally or | ||
748 | locally inherit the :ref:`ref-classes-externalsrc` class. | ||
749 | |||
750 | By default, :ref:`ref-classes-externalsrc` builds the source code in a | ||
751 | directory separate from the external source directory as specified by | ||
752 | :term:`EXTERNALSRC`. If you need | ||
753 | to have the source built in the same directory in which it resides, or | ||
754 | some other nominated directory, you can set | ||
755 | :term:`EXTERNALSRC_BUILD` | ||
756 | to point to that directory:: | ||
757 | |||
758 | EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree" | ||
759 | |||
760 | Replicating a Build Offline | ||
761 | =========================== | ||
762 | |||
763 | It can be useful to take a "snapshot" of upstream sources used in a | ||
764 | build and then use that "snapshot" later to replicate the build offline. | ||
765 | To do so, you need to first prepare and populate your downloads | ||
766 | directory your "snapshot" of files. Once your downloads directory is | ||
767 | ready, you can use it at any time and from any machine to replicate your | ||
768 | build. | ||
769 | |||
770 | Follow these steps to populate your Downloads directory: | ||
771 | |||
772 | #. *Create a Clean Downloads Directory:* Start with an empty downloads | ||
773 | directory (:term:`DL_DIR`). You | ||
774 | start with an empty downloads directory by either removing the files | ||
775 | in the existing directory or by setting :term:`DL_DIR` to point to either | ||
776 | an empty location or one that does not yet exist. | ||
777 | |||
778 | #. *Generate Tarballs of the Source Git Repositories:* Edit your | ||
779 | ``local.conf`` configuration file as follows:: | ||
780 | |||
781 | DL_DIR = "/home/your-download-dir/" | ||
782 | BB_GENERATE_MIRROR_TARBALLS = "1" | ||
783 | |||
784 | During | ||
785 | the fetch process in the next step, BitBake gathers the source files | ||
786 | and creates tarballs in the directory pointed to by :term:`DL_DIR`. See | ||
787 | the | ||
788 | :term:`BB_GENERATE_MIRROR_TARBALLS` | ||
789 | variable for more information. | ||
790 | |||
791 | #. *Populate Your Downloads Directory Without Building:* Use BitBake to | ||
792 | fetch your sources but inhibit the build:: | ||
793 | |||
794 | $ bitbake target --runonly=fetch | ||
795 | |||
796 | The downloads directory (i.e. ``${DL_DIR}``) now has | ||
797 | a "snapshot" of the source files in the form of tarballs, which can | ||
798 | be used for the build. | ||
799 | |||
800 | #. *Optionally Remove Any Git or other SCM Subdirectories From the | ||
801 | Downloads Directory:* If you want, you can clean up your downloads | ||
802 | directory by removing any Git or other Source Control Management | ||
803 | (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs | ||
804 | already contain these subdirectories. | ||
805 | |||
806 | Once your downloads directory has everything it needs regarding source | ||
807 | files, you can create your "own-mirror" and build your target. | ||
808 | Understand that you can use the files to build the target offline from | ||
809 | any machine and at any time. | ||
810 | |||
811 | Follow these steps to build your target using the files in the downloads | ||
812 | directory: | ||
813 | |||
814 | #. *Using Local Files Only:* Inside your ``local.conf`` file, add the | ||
815 | :term:`SOURCE_MIRROR_URL` variable, inherit the | ||
816 | :ref:`ref-classes-own-mirrors` class, and add the | ||
817 | :term:`BB_NO_NETWORK` variable to your ``local.conf``:: | ||
818 | |||
819 | SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" | ||
820 | INHERIT += "own-mirrors" | ||
821 | BB_NO_NETWORK = "1" | ||
822 | |||
823 | The :term:`SOURCE_MIRROR_URL` and :ref:`ref-classes-own-mirrors` | ||
824 | class set up the system to use the downloads directory as your "own | ||
825 | mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that | ||
826 | BitBake's fetching process in step 3 stays local, which means files | ||
827 | from your "own-mirror" are used. | ||
828 | |||
829 | #. *Start With a Clean Build:* You can start with a clean build by | ||
830 | removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new | ||
831 | :term:`Build Directory`. | ||
832 | |||
833 | #. *Build Your Target:* Use BitBake to build your target:: | ||
834 | |||
835 | $ bitbake target | ||
836 | |||
837 | The build completes using the known local "snapshot" of source | ||
838 | files from your mirror. The resulting tarballs for your "snapshot" of | ||
839 | source files are in the downloads directory. | ||
840 | |||
841 | .. note:: | ||
842 | |||
843 | The offline build does not work if recipes attempt to find the | ||
844 | latest version of software by setting | ||
845 | :term:`SRCREV` to | ||
846 | ``${``\ :term:`AUTOREV`\ ``}``:: | ||
847 | |||
848 | SRCREV = "${AUTOREV}" | ||
849 | |||
850 | When a recipe sets :term:`SRCREV` to | ||
851 | ``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an | ||
852 | attempt to determine the latest version of software from the SCM. | ||
853 | Typically, recipes that use :term:`AUTOREV` are custom or modified | ||
854 | recipes. Recipes that reside in public repositories usually do not | ||
855 | use :term:`AUTOREV`. | ||
856 | |||
857 | If you do have recipes that use :term:`AUTOREV`, you can take steps to | ||
858 | still use the recipes in an offline build. Do the following: | ||
859 | |||
860 | #. Use a configuration generated by enabling :ref:`build | ||
861 | history <dev-manual/build-quality:maintaining build output quality>`. | ||
862 | |||
863 | #. Use the ``buildhistory-collect-srcrevs`` command to collect the | ||
864 | stored :term:`SRCREV` values from the build's history. For more | ||
865 | information on collecting these values, see the | ||
866 | ":ref:`dev-manual/build-quality:build history package information`" | ||
867 | section. | ||
868 | |||
869 | #. Once you have the correct source revisions, you can modify | ||
870 | those recipes to set :term:`SRCREV` to specific versions of the | ||
871 | software. | ||
872 | |||