summaryrefslogtreecommitdiffstats
path: root/documentation/kernel-dev
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/kernel-dev')
-rw-r--r--documentation/kernel-dev/kernel-dev-advanced.rst762
-rw-r--r--documentation/kernel-dev/kernel-dev-common.rst1728
-rw-r--r--documentation/kernel-dev/kernel-dev-concepts-appx.rst409
-rw-r--r--documentation/kernel-dev/kernel-dev-faq.rst43
-rw-r--r--documentation/kernel-dev/kernel-dev-intro.rst178
-rw-r--r--documentation/kernel-dev/kernel-dev-maint-appx.rst226
-rw-r--r--documentation/kernel-dev/kernel-dev.rst14
7 files changed, 3360 insertions, 0 deletions
diff --git a/documentation/kernel-dev/kernel-dev-advanced.rst b/documentation/kernel-dev/kernel-dev-advanced.rst
new file mode 100644
index 0000000000..be76bd7b52
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-advanced.rst
@@ -0,0 +1,762 @@
1*******************************************************
2Working with Advanced Metadata (``yocto-kernel-cache``)
3*******************************************************
4
5.. _kernel-dev-advanced-overview:
6
7Overview
8========
9
10In addition to supporting configuration fragments and patches, the Yocto
11Project kernel tools also support rich
12`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ that you can use to define
13complex policies and Board Support Package (BSP) support. The purpose of
14the Metadata and the tools that manage it is to help you manage the
15complexity of the configuration and sources used to support multiple
16BSPs and Linux kernel types.
17
18Kernel Metadata exists in many places. One area in the Yocto Project
19`Source Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ is the
20``yocto-kernel-cache`` Git repository. You can find this repository
21grouped under the "Yocto Linux Kernel" heading in the `Yocto Project
22Source Repositories <&YOCTO_GIT_URL;>`__.
23
24Kernel development tools ("kern-tools") exist also in the Yocto Project
25Source Repositories under the "Yocto Linux Kernel" heading in the
26``yocto-kernel-tools`` Git repository. The recipe that builds these
27tools is ``meta/recipes-kernel/kern-tools/kern-tools-native_git.bb`` in
28the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
29``poky``).
30
31Using Kernel Metadata in a Recipe
32=================================
33
34As mentioned in the introduction, the Yocto Project contains kernel
35Metadata, which is located in the ``yocto-kernel-cache`` Git repository.
36This Metadata defines Board Support Packages (BSPs) that correspond to
37definitions in linux-yocto recipes for corresponding BSPs. A BSP
38consists of an aggregation of kernel policy and enabled
39hardware-specific features. The BSP can be influenced from within the
40linux-yocto recipe.
41
42.. note::
43
44 A Linux kernel recipe that contains kernel Metadata (e.g. inherits
45 from the
46 linux-yocto.inc
47 file) is said to be a "linux-yocto style" recipe.
48
49Every linux-yocto style recipe must define the
50```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__ variable. This
51variable is typically set to the same value as the ``MACHINE`` variable,
52which is used by `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__.
53However, in some cases, the variable might instead refer to the
54underlying platform of the ``MACHINE``.
55
56Multiple BSPs can reuse the same ``KMACHINE`` name if they are built
57using the same BSP description. Multiple Corei7-based BSPs could share
58the same "intel-corei7-64" value for ``KMACHINE``. It is important to
59realize that ``KMACHINE`` is just for kernel mapping, while ``MACHINE``
60is the machine type within a BSP Layer. Even with this distinction,
61however, these two variables can hold the same value. See the `BSP
62Descriptions <#bsp-descriptions>`__ section for more information.
63
64Every linux-yocto style recipe must also indicate the Linux kernel
65source repository branch used to build the Linux kernel. The
66```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable must be set
67to indicate the branch.
68
69.. note::
70
71 You can use the
72 KBRANCH
73 value to define an alternate branch typically with a machine override
74 as shown here from the
75 meta-yocto-bsp
76 layer:
77 ::
78
79 KBRANCH_edgerouter = "standard/edgerouter"
80
81
82The linux-yocto style recipes can optionally define the following
83variables: KERNEL_FEATURES LINUX_KERNEL_TYPE
84
85```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
86defines the kernel type to be used in assembling the configuration. If
87you do not specify a ``LINUX_KERNEL_TYPE``, it defaults to "standard".
88Together with ``KMACHINE``, ``LINUX_KERNEL_TYPE`` defines the search
89arguments used by the kernel tools to find the appropriate description
90within the kernel Metadata with which to build out the sources and
91configuration. The linux-yocto recipes define "standard", "tiny", and
92"preempt-rt" kernel types. See the "`Kernel Types <#kernel-types>`__"
93section for more information on kernel types.
94
95During the build, the kern-tools search for the BSP description file
96that most closely matches the ``KMACHINE`` and ``LINUX_KERNEL_TYPE``
97variables passed in from the recipe. The tools use the first BSP
98description it finds that match both variables. If the tools cannot find
99a match, they issue a warning.
100
101The tools first search for the ``KMACHINE`` and then for the
102``LINUX_KERNEL_TYPE``. If the tools cannot find a partial match, they
103will use the sources from the ``KBRANCH`` and any configuration
104specified in the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__.
105
106You can use the
107```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
108variable to include features (configuration fragments, patches, or both)
109that are not already included by the ``KMACHINE`` and
110``LINUX_KERNEL_TYPE`` variable combination. For example, to include a
111feature specified as "features/netfilter/netfilter.scc", specify:
112KERNEL_FEATURES += "features/netfilter/netfilter.scc" To include a
113feature called "cfg/sound.scc" just for the ``qemux86`` machine,
114specify: KERNEL_FEATURES_append_qemux86 = " cfg/sound.scc" The value of
115the entries in ``KERNEL_FEATURES`` are dependent on their location
116within the kernel Metadata itself. The examples here are taken from the
117``yocto-kernel-cache`` repository. Each branch of this repository
118contains "features" and "cfg" subdirectories at the top-level. For more
119information, see the "`Kernel Metadata
120Syntax <#kernel-metadata-syntax>`__" section.
121
122Kernel Metadata Syntax
123======================
124
125The kernel Metadata consists of three primary types of files: ``scc``
126 [1]_ description files, configuration fragments, and patches. The
127``scc`` files define variables and include or otherwise reference any of
128the three file types. The description files are used to aggregate all
129types of kernel Metadata into what ultimately describes the sources and
130the configuration required to build a Linux kernel tailored to a
131specific machine.
132
133The ``scc`` description files are used to define two fundamental types
134of kernel Metadata:
135
136- Features
137
138- Board Support Packages (BSPs)
139
140Features aggregate sources in the form of patches and configuration
141fragments into a modular reusable unit. You can use features to
142implement conceptually separate kernel Metadata descriptions such as
143pure configuration fragments, simple patches, complex features, and
144kernel types. `Kernel types <#kernel-types>`__ define general kernel
145features and policy to be reused in the BSPs.
146
147BSPs define hardware-specific features and aggregate them with kernel
148types to form the final description of what will be assembled and built.
149
150While the kernel Metadata syntax does not enforce any logical separation
151of configuration fragments, patches, features or kernel types, best
152practices dictate a logical separation of these types of Metadata. The
153following Metadata file hierarchy is recommended: base/ bsp/ cfg/
154features/ ktypes/ patches/
155
156The ``bsp`` directory contains the `BSP
157descriptions <#bsp-descriptions>`__. The remaining directories all
158contain "features". Separating ``bsp`` from the rest of the structure
159aids conceptualizing intended usage.
160
161Use these guidelines to help place your ``scc`` description files within
162the structure:
163
164- If your file contains only configuration fragments, place the file in
165 the ``cfg`` directory.
166
167- If your file contains only source-code fixes, place the file in the
168 ``patches`` directory.
169
170- If your file encapsulates a major feature, often combining sources
171 and configurations, place the file in ``features`` directory.
172
173- If your file aggregates non-hardware configuration and patches in
174 order to define a base kernel policy or major kernel type to be
175 reused across multiple BSPs, place the file in ``ktypes`` directory.
176
177These distinctions can easily become blurred - especially as out-of-tree
178features slowly merge upstream over time. Also, remember that how the
179description files are placed is a purely logical organization and has no
180impact on the functionality of the kernel Metadata. There is no impact
181because all of ``cfg``, ``features``, ``patches``, and ``ktypes``,
182contain "features" as far as the kernel tools are concerned.
183
184Paths used in kernel Metadata files are relative to base, which is
185either
186```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__ if
187you are creating Metadata in `recipe-space <#recipe-space-metadata>`__,
188or the top level of
189```yocto-kernel-cache`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/>`__
190if you are creating `Metadata outside of the
191recipe-space <#metadata-outside-the-recipe-space>`__.
192
193Configuration
194-------------
195
196The simplest unit of kernel Metadata is the configuration-only feature.
197This feature consists of one or more Linux kernel configuration
198parameters in a configuration fragment file (``.cfg``) and a ``.scc``
199file that describes the fragment.
200
201As an example, consider the Symmetric Multi-Processing (SMP) fragment
202used with the ``linux-yocto-4.12`` kernel as defined outside of the
203recipe space (i.e. ``yocto-kernel-cache``). This Metadata consists of
204two files: ``smp.scc`` and ``smp.cfg``. You can find these files in the
205``cfg`` directory of the ``yocto-4.12`` branch in the
206``yocto-kernel-cache`` Git repository: cfg/smp.scc: define
207KFEATURE_DESCRIPTION "Enable SMP for 32 bit builds" define
208KFEATURE_COMPATIBILITY all kconf hardware smp.cfg cfg/smp.cfg:
209CONFIG_SMP=y CONFIG_SCHED_SMT=y # Increase default NR_CPUS from 8 to 64
210so that platform with # more than 8 processors can be all activated at
211boot time CONFIG_NR_CPUS=64 # The following is needed when setting
212NR_CPUS to something # greater than 8 on x86 architectures, it should be
213automatically # disregarded by Kconfig when using a different arch
214CONFIG_X86_BIGSMP=y You can find general information on configuration
215fragment files in the "`Creating Configuration
216Fragments <#creating-config-fragments>`__" section.
217
218Within the ``smp.scc`` file, the
219```KFEATURE_DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-KFEATURE_DESCRIPTION>`__
220statement provides a short description of the fragment. Higher level
221kernel tools use this description.
222
223Also within the ``smp.scc`` file, the ``kconf`` command includes the
224actual configuration fragment in an ``.scc`` file, and the "hardware"
225keyword identifies the fragment as being hardware enabling, as opposed
226to general policy, which would use the "non-hardware" keyword. The
227distinction is made for the benefit of the configuration validation
228tools, which warn you if a hardware fragment overrides a policy set by a
229non-hardware fragment.
230
231.. note::
232
233 The description file can include multiple
234 kconf
235 statements, one per fragment.
236
237As described in the "`Validating
238Configuration <#validating-configuration>`__" section, you can use the
239following BitBake command to audit your configuration: $ bitbake
240linux-yocto -c kernel_configcheck -f
241
242Patches
243-------
244
245Patch descriptions are very similar to configuration fragment
246descriptions, which are described in the previous section. However,
247instead of a ``.cfg`` file, these descriptions work with source patches
248(i.e. ``.patch`` files).
249
250A typical patch includes a description file and the patch itself. As an
251example, consider the build patches used with the ``linux-yocto-4.12``
252kernel as defined outside of the recipe space (i.e.
253``yocto-kernel-cache``). This Metadata consists of several files:
254``build.scc`` and a set of ``*.patch`` files. You can find these files
255in the ``patches/build`` directory of the ``yocto-4.12`` branch in the
256``yocto-kernel-cache`` Git repository.
257
258The following listings show the ``build.scc`` file and part of the
259``modpost-mask-trivial-warnings.patch`` file: patches/build/build.scc:
260patch arm-serialize-build-targets.patch patch
261powerpc-serialize-image-targets.patch patch
262kbuild-exclude-meta-directory-from-distclean-processi.patch # applied by
263kgit # patch kbuild-add-meta-files-to-the-ignore-li.patch patch
264modpost-mask-trivial-warnings.patch patch
265menuconfig-check-lxdiaglog.sh-Allow-specification-of.patch
266patches/build/modpost-mask-trivial-warnings.patch: From
267bd48931bc142bdd104668f3a062a1f22600aae61 Mon Sep 17 00:00:00 2001 From:
268Paul Gortmaker <paul.gortmaker@windriver.com> Date: Sun, 25 Jan 2009
26917:58:09 -0500 Subject: [PATCH] modpost: mask trivial warnings Newer
270HOSTCC will complain about various stdio fcns because . . . char
271\*dump_write = NULL, \*files_source = NULL; int opt; -- 2.10.1 generated
272by cgit v0.10.2 at 2017-09-28 15:23:23 (GMT) The description file can
273include multiple patch statements where each statement handles a single
274patch. In the example ``build.scc`` file, five patch statements exist
275for the five patches in the directory.
276
277You can create a typical ``.patch`` file using ``diff -Nurp`` or
278``git format-patch`` commands. For information on how to create patches,
279see the "`Using ``devtool`` to Patch the
280Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
281Kernel Development to Patch the
282Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
283sections.
284
285Features
286--------
287
288Features are complex kernel Metadata types that consist of configuration
289fragments, patches, and possibly other feature description files. As an
290example, consider the following generic listing: features/myfeature.scc
291define KFEATURE_DESCRIPTION "Enable myfeature" patch
2920001-myfeature-core.patch patch 0002-myfeature-interface.patch include
293cfg/myfeature_dependency.scc kconf non-hardware myfeature.cfg This
294example shows how the ``patch`` and ``kconf`` commands are used as well
295as how an additional feature description file is included with the
296``include`` command.
297
298Typically, features are less granular than configuration fragments and
299are more likely than configuration fragments and patches to be the types
300of things you want to specify in the ``KERNEL_FEATURES`` variable of the
301Linux kernel recipe. See the "`Using Kernel Metadata in a
302Recipe <#using-kernel-metadata-in-a-recipe>`__" section earlier in the
303manual.
304
305Kernel Types
306------------
307
308A kernel type defines a high-level kernel policy by aggregating
309non-hardware configuration fragments with patches you want to use when
310building a Linux kernel of a specific type (e.g. a real-time kernel).
311Syntactically, kernel types are no different than features as described
312in the "`Features <#features>`__" section. The
313```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
314variable in the kernel recipe selects the kernel type. For example, in
315the ``linux-yocto_4.12.bb`` kernel recipe found in
316``poky/meta/recipes-kernel/linux``, a
317```require`` <&YOCTO_DOCS_BB_URL;#require-inclusion>`__ directive
318includes the ``poky/meta/recipes-kernel/linux/linux-yocto.inc`` file,
319which has the following statement that defines the default kernel type:
320LINUX_KERNEL_TYPE ??= "standard"
321
322Another example would be the real-time kernel (i.e.
323``linux-yocto-rt_4.12.bb``). This kernel recipe directly sets the kernel
324type as follows: LINUX_KERNEL_TYPE = "preempt-rt"
325
326.. note::
327
328 You can find kernel recipes in the
329 meta/recipes-kernel/linux
330 directory of the
331 Source Directory
332 (e.g.
333 poky/meta/recipes-kernel/linux/linux-yocto_4.12.bb
334 ). See the "
335 Using Kernel Metadata in a Recipe
336 " section for more information.
337
338Three kernel types ("standard", "tiny", and "preempt-rt") are supported
339for Linux Yocto kernels:
340
341- "standard": Includes the generic Linux kernel policy of the Yocto
342 Project linux-yocto kernel recipes. This policy includes, among other
343 things, which file systems, networking options, core kernel features,
344 and debugging and tracing options are supported.
345
346- "preempt-rt": Applies the ``PREEMPT_RT`` patches and the
347 configuration options required to build a real-time Linux kernel.
348 This kernel type inherits from the "standard" kernel type.
349
350- "tiny": Defines a bare minimum configuration meant to serve as a base
351 for very small Linux kernels. The "tiny" kernel type is independent
352 from the "standard" configuration. Although the "tiny" kernel type
353 does not currently include any source changes, it might in the
354 future.
355
356For any given kernel type, the Metadata is defined by the ``.scc`` (e.g.
357``standard.scc``). Here is a partial listing for the ``standard.scc``
358file, which is found in the ``ktypes/standard`` directory of the
359``yocto-kernel-cache`` Git repository: # Include this kernel type
360fragment to get the standard features and # configuration values. #
361Note: if only the features are desired, but not the configuration # then
362this should be included as: # include ktypes/standard/standard.scc nocfg
363# if no chained configuration is desired, include it as: # include
364ktypes/standard/standard.scc nocfg inherit include ktypes/base/base.scc
365branch standard kconf non-hardware standard.cfg include
366features/kgdb/kgdb.scc . . . include cfg/net/ip6_nf.scc include
367cfg/net/bridge.scc include cfg/systemd.scc include
368features/rfkill/rfkill.scc
369
370As with any ``.scc`` file, a kernel type definition can aggregate other
371``.scc`` files with ``include`` commands. These definitions can also
372directly pull in configuration fragments and patches with the ``kconf``
373and ``patch`` commands, respectively.
374
375.. note::
376
377 It is not strictly necessary to create a kernel type
378 .scc
379 file. The Board Support Package (BSP) file can implicitly define the
380 kernel type using a
381 define
382 KTYPE
383 myktype
384 line. See the "
385 BSP Descriptions
386 " section for more information.
387
388BSP Descriptions
389----------------
390
391BSP descriptions (i.e. ``*.scc`` files) combine kernel types with
392hardware-specific features. The hardware-specific Metadata is typically
393defined independently in the BSP layer, and then aggregated with each
394supported kernel type.
395
396.. note::
397
398 For BSPs supported by the Yocto Project, the BSP description files
399 are located in the
400 bsp
401 directory of the
402 yocto-kernel-cache
403 repository organized under the "Yocto Linux Kernel" heading in the
404 Yocto Project Source Repositories
405 .
406
407This section overviews the BSP description structure, the aggregation
408concepts, and presents a detailed example using a BSP supported by the
409Yocto Project (i.e. BeagleBone Board). For complete information on BSP
410layer file hierarchy, see the `Yocto Project Board Support Package (BSP)
411Developer's Guide <&YOCTO_DOCS_BSP_URL;>`__.
412
413.. _bsp-description-file-overview:
414
415Overview
416~~~~~~~~
417
418For simplicity, consider the following root BSP layer description files
419for the BeagleBone board. These files employ both a structure and naming
420convention for consistency. The naming convention for the file is as
421follows: bsp_root_name-kernel_type.scc Here are some example root layer
422BSP filenames for the BeagleBone Board BSP, which is supported by the
423Yocto Project: beaglebone-standard.scc beaglebone-preempt-rt.scc Each
424file uses the root name (i.e "beaglebone") BSP name followed by the
425kernel type.
426
427Examine the ``beaglebone-standard.scc`` file: define KMACHINE beaglebone
428define KTYPE standard define KARCH arm include
429ktypes/standard/standard.scc branch beaglebone include beaglebone.scc #
430default policy for standard kernels include
431features/latencytop/latencytop.scc include
432features/profiling/profiling.scc Every top-level BSP description file
433should define the ```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
434```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__, and
435```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__ variables. These
436variables allow the OpenEmbedded build system to identify the
437description as meeting the criteria set by the recipe being built. This
438example supports the "beaglebone" machine for the "standard" kernel and
439the "arm" architecture.
440
441Be aware that a hard link between the ``KTYPE`` variable and a kernel
442type description file does not exist. Thus, if you do not have the
443kernel type defined in your kernel Metadata as it is here, you only need
444to ensure that the
445```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
446variable in the kernel recipe and the ``KTYPE`` variable in the BSP
447description file match.
448
449To separate your kernel policy from your hardware configuration, you
450include a kernel type (``ktype``), such as "standard". In the previous
451example, this is done using the following: include
452ktypes/standard/standard.scc This file aggregates all the configuration
453fragments, patches, and features that make up your standard kernel
454policy. See the "`Kernel Types <#kernel-types>`__" section for more
455information.
456
457To aggregate common configurations and features specific to the kernel
458for mybsp, use the following: include mybsp.scc You can see that in the
459BeagleBone example with the following: include beaglebone.scc For
460information on how to break a complete ``.config`` file into the various
461configuration fragments, see the "`Creating Configuration
462Fragments <#creating-config-fragments>`__" section.
463
464Finally, if you have any configurations specific to the hardware that
465are not in a ``*.scc`` file, you can include them as follows: kconf
466hardware mybsp-extra.cfg The BeagleBone example does not include these
467types of configurations. However, the Malta 32-bit board does
468("mti-malta32"). Here is the ``mti-malta32-le-standard.scc`` file:
469define KMACHINE mti-malta32-le define KMACHINE qemumipsel define KTYPE
470standard define KARCH mips include ktypes/standard/standard.scc branch
471mti-malta32 include mti-malta32.scc kconf hardware mti-malta32-le.cfg
472
473.. _bsp-description-file-example-minnow:
474
475Example
476~~~~~~~
477
478Many real-world examples are more complex. Like any other ``.scc`` file,
479BSP descriptions can aggregate features. Consider the Minnow BSP
480definition given the ``linux-yocto-4.4`` branch of the
481``yocto-kernel-cache`` (i.e.
482``yocto-kernel-cache/bsp/minnow/minnow.scc``):
483
484.. note::
485
486 Although the Minnow Board BSP is unused, the Metadata remains and is
487 being used here just as an example.
488
489include cfg/x86.scc include features/eg20t/eg20t.scc include
490cfg/dmaengine.scc include features/power/intel.scc include cfg/efi.scc
491include features/usb/ehci-hcd.scc include features/usb/ohci-hcd.scc
492include features/usb/usb-gadgets.scc include
493features/usb/touchscreen-composite.scc include cfg/timer/hpet.scc
494include features/leds/leds.scc include features/spi/spidev.scc include
495features/i2c/i2cdev.scc include features/mei/mei-txe.scc # Earlyprintk
496and port debug requires 8250 kconf hardware cfg/8250.cfg kconf hardware
497minnow.cfg kconf hardware minnow-dev.cfg
498
499The ``minnow.scc`` description file includes a hardware configuration
500fragment (``minnow.cfg``) specific to the Minnow BSP as well as several
501more general configuration fragments and features enabling hardware
502found on the machine. This ``minnow.scc`` description file is then
503included in each of the three "minnow" description files for the
504supported kernel types (i.e. "standard", "preempt-rt", and "tiny").
505Consider the "minnow" description for the "standard" kernel type (i.e.
506``minnow-standard.scc``: define KMACHINE minnow define KTYPE standard
507define KARCH i386 include ktypes/standard include minnow.scc # Extra
508minnow configs above the minimal defined in minnow.scc include
509cfg/efi-ext.scc include features/media/media-all.scc include
510features/sound/snd_hda_intel.scc # The following should really be in
511standard.scc # USB live-image support include cfg/usb-mass-storage.scc
512include cfg/boot-live.scc # Basic profiling include
513features/latencytop/latencytop.scc include
514features/profiling/profiling.scc # Requested drivers that don't have an
515existing scc kconf hardware minnow-drivers-extra.cfg The ``include``
516command midway through the file includes the ``minnow.scc`` description
517that defines all enabled hardware for the BSP that is common to all
518kernel types. Using this command significantly reduces duplication.
519
520Now consider the "minnow" description for the "tiny" kernel type (i.e.
521``minnow-tiny.scc``): define KMACHINE minnow define KTYPE tiny define
522KARCH i386 include ktypes/tiny include minnow.scc As you might expect,
523the "tiny" description includes quite a bit less. In fact, it includes
524only the minimal policy defined by the "tiny" kernel type and the
525hardware-specific configuration required for booting the machine along
526with the most basic functionality of the system as defined in the base
527"minnow" description file.
528
529Notice again the three critical variables:
530```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
531```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__, and
532```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__. Of these variables, only
533``KTYPE`` has changed to specify the "tiny" kernel type.
534
535Kernel Metadata Location
536========================
537
538Kernel Metadata always exists outside of the kernel tree either defined
539in a kernel recipe (recipe-space) or outside of the recipe. Where you
540choose to define the Metadata depends on what you want to do and how you
541intend to work. Regardless of where you define the kernel Metadata, the
542syntax used applies equally.
543
544If you are unfamiliar with the Linux kernel and only wish to apply a
545configuration and possibly a couple of patches provided to you by
546others, the recipe-space method is recommended. This method is also a
547good approach if you are working with Linux kernel sources you do not
548control or if you just do not want to maintain a Linux kernel Git
549repository on your own. For partial information on how you can define
550kernel Metadata in the recipe-space, see the "`Modifying an Existing
551Recipe <#modifying-an-existing-recipe>`__" section.
552
553Conversely, if you are actively developing a kernel and are already
554maintaining a Linux kernel Git repository of your own, you might find it
555more convenient to work with kernel Metadata kept outside the
556recipe-space. Working with Metadata in this area can make iterative
557development of the Linux kernel more efficient outside of the BitBake
558environment.
559
560Recipe-Space Metadata
561---------------------
562
563When stored in recipe-space, the kernel Metadata files reside in a
564directory hierarchy below
565```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__. For
566a linux-yocto recipe or for a Linux kernel recipe derived by copying and
567modifying
568``oe-core/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb`` to
569a recipe in your layer, ``FILESEXTRAPATHS`` is typically set to
570``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``.
571See the "`Modifying an Existing
572Recipe <#modifying-an-existing-recipe>`__" section for more information.
573
574Here is an example that shows a trivial tree of kernel Metadata stored
575in recipe-space within a BSP layer: meta-my_bsp_layer/ \`--
576recipes-kernel \`-- linux \`-- linux-yocto \|-- bsp-standard.scc \|--
577bsp.cfg \`-- standard.cfg
578
579When the Metadata is stored in recipe-space, you must take steps to
580ensure BitBake has the necessary information to decide what files to
581fetch and when they need to be fetched again. It is only necessary to
582specify the ``.scc`` files on the
583```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__. BitBake parses them
584and fetches any files referenced in the ``.scc`` files by the
585``include``, ``patch``, or ``kconf`` commands. Because of this, it is
586necessary to bump the recipe ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__
587value when changing the content of files not explicitly listed in the
588``SRC_URI``.
589
590If the BSP description is in recipe space, you cannot simply list the
591``*.scc`` in the ``SRC_URI`` statement. You need to use the following
592form from your kernel append file: SRC_URI_append_myplatform = " \\
593file://myplatform;type=kmeta;destsuffix=myplatform \\ "
594
595Metadata Outside the Recipe-Space
596---------------------------------
597
598When stored outside of the recipe-space, the kernel Metadata files
599reside in a separate repository. The OpenEmbedded build system adds the
600Metadata to the build as a "type=kmeta" repository through the
601```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. As an
602example, consider the following ``SRC_URI`` statement from the
603``linux-yocto_4.12.bb`` kernel recipe: SRC_URI =
604"git://git.yoctoproject.org/linux-yocto-4.12.git;name=machine;branch=${KBRANCH};
605\\
606git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
607``${KMETA}``, in this context, is simply used to name the directory into
608which the Git fetcher places the Metadata. This behavior is no different
609than any multi-repository ``SRC_URI`` statement used in a recipe (e.g.
610see the previous section).
611
612You can keep kernel Metadata in a "kernel-cache", which is a directory
613containing configuration fragments. As with any Metadata kept outside
614the recipe-space, you simply need to use the ``SRC_URI`` statement with
615the "type=kmeta" attribute. Doing so makes the kernel Metadata available
616during the configuration phase.
617
618If you modify the Metadata, you must not forget to update the ``SRCREV``
619statements in the kernel's recipe. In particular, you need to update the
620``SRCREV_meta`` variable to match the commit in the ``KMETA`` branch you
621wish to use. Changing the data in these branches and not updating the
622``SRCREV`` statements to match will cause the build to fetch an older
623commit.
624
625Organizing Your Source
626======================
627
628Many recipes based on the ``linux-yocto-custom.bb`` recipe use Linux
629kernel sources that have only a single branch - "master". This type of
630repository structure is fine for linear development supporting a single
631machine and architecture. However, if you work with multiple boards and
632architectures, a kernel source repository with multiple branches is more
633efficient. For example, suppose you need a series of patches for one
634board to boot. Sometimes, these patches are works-in-progress or
635fundamentally wrong, yet they are still necessary for specific boards.
636In these situations, you most likely do not want to include these
637patches in every kernel you build (i.e. have the patches as part of the
638lone "master" branch). It is situations like these that give rise to
639multiple branches used within a Linux kernel sources Git repository.
640
641Repository organization strategies exist that maximize source reuse,
642remove redundancy, and logically order your changes. This section
643presents strategies for the following cases:
644
645- Encapsulating patches in a feature description and only including the
646 patches in the BSP descriptions of the applicable boards.
647
648- Creating a machine branch in your kernel source repository and
649 applying the patches on that branch only.
650
651- Creating a feature branch in your kernel source repository and
652 merging that branch into your BSP when needed.
653
654The approach you take is entirely up to you and depends on what works
655best for your development model.
656
657Encapsulating Patches
658---------------------
659
660if you are reusing patches from an external tree and are not working on
661the patches, you might find the encapsulated feature to be appropriate.
662Given this scenario, you do not need to create any branches in the
663source repository. Rather, you just take the static patches you need and
664encapsulate them within a feature description. Once you have the feature
665description, you simply include that into the BSP description as
666described in the "`BSP Descriptions <#bsp-descriptions>`__" section.
667
668You can find information on how to create patches and BSP descriptions
669in the "`Patches <#patches>`__" and "`BSP
670Descriptions <#bsp-descriptions>`__" sections.
671
672Machine Branches
673----------------
674
675When you have multiple machines and architectures to support, or you are
676actively working on board support, it is more efficient to create
677branches in the repository based on individual machines. Having machine
678branches allows common source to remain in the "master" branch with any
679features specific to a machine stored in the appropriate machine branch.
680This organization method frees you from continually reintegrating your
681patches into a feature.
682
683Once you have a new branch, you can set up your kernel Metadata to use
684the branch a couple different ways. In the recipe, you can specify the
685new branch as the ``KBRANCH`` to use for the board as follows: KBRANCH =
686"mynewbranch" Another method is to use the ``branch`` command in the BSP
687description: mybsp.scc: define KMACHINE mybsp define KTYPE standard
688define KARCH i386 include standard.scc branch mynewbranch include
689mybsp-hw.scc
690
691If you find yourself with numerous branches, you might consider using a
692hierarchical branching system similar to what the Yocto Linux Kernel Git
693repositories use: common/kernel_type/machine
694
695If you had two kernel types, "standard" and "small" for instance, three
696machines, and common as ``mydir``, the branches in your Git repository
697might look like this: mydir/base mydir/standard/base
698mydir/standard/machine_a mydir/standard/machine_b
699mydir/standard/machine_c mydir/small/base mydir/small/machine_a
700
701This organization can help clarify the branch relationships. In this
702case, ``mydir/standard/machine_a`` includes everything in ``mydir/base``
703and ``mydir/standard/base``. The "standard" and "small" branches add
704sources specific to those kernel types that for whatever reason are not
705appropriate for the other branches.
706
707.. note::
708
709 The "base" branches are an artifact of the way Git manages its data
710 internally on the filesystem: Git will not allow you to use
711 mydir/standard
712 and
713 mydir/standard/machine_a
714 because it would have to create a file and a directory named
715 "standard".
716
717Feature Branches
718----------------
719
720When you are actively developing new features, it can be more efficient
721to work with that feature as a branch, rather than as a set of patches
722that have to be regularly updated. The Yocto Project Linux kernel tools
723provide for this with the ``git merge`` command.
724
725To merge a feature branch into a BSP, insert the ``git merge`` command
726after any ``branch`` commands: mybsp.scc: define KMACHINE mybsp define
727KTYPE standard define KARCH i386 include standard.scc branch mynewbranch
728git merge myfeature include mybsp-hw.scc
729
730.. _scc-reference:
731
732SCC Description File Reference
733==============================
734
735This section provides a brief reference for the commands you can use
736within an SCC description file (``.scc``):
737
738- ``branch [ref]``: Creates a new branch relative to the current branch
739 (typically ``${KTYPE}``) using the currently checked-out branch, or
740 "ref" if specified.
741
742- ``define``: Defines variables, such as
743 ```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
744 ```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__,
745 ```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__, and
746 ```KFEATURE_DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-KFEATURE_DESCRIPTION>`__.
747
748- ``include SCC_FILE``: Includes an SCC file in the current file. The
749 file is parsed as if you had inserted it inline.
750
751- ``kconf [hardware|non-hardware] CFG_FILE``: Queues a configuration
752 fragment for merging into the final Linux ``.config`` file.
753
754- ``git merge GIT_BRANCH``: Merges the feature branch into the current
755 branch.
756
757- ``patch PATCH_FILE``: Applies the patch to the current Git branch.
758
759.. [1]
760 ``scc`` stands for Series Configuration Control, but the naming has
761 less significance in the current implementation of the tooling than
762 it had in the past. Consider ``scc`` files to be description files.
diff --git a/documentation/kernel-dev/kernel-dev-common.rst b/documentation/kernel-dev/kernel-dev-common.rst
new file mode 100644
index 0000000000..bf87edbdc4
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-common.rst
@@ -0,0 +1,1728 @@
1************
2Common Tasks
3************
4
5This chapter presents several common tasks you perform when you work
6with the Yocto Project Linux kernel. These tasks include preparing your
7host development system for kernel development, preparing a layer,
8modifying an existing recipe, patching the kernel, configuring the
9kernel, iterative development, working with your own sources, and
10incorporating out-of-tree modules.
11
12.. note::
13
14 The examples presented in this chapter work with the Yocto Project
15 2.4 Release and forward.
16
17Preparing the Build Host to Work on the Kernel
18==============================================
19
20Before you can do any kernel development, you need to be sure your build
21host is set up to use the Yocto Project. For information on how to get
22set up, see the "`Preparing the Build
23Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section in
24the Yocto Project Development Tasks Manual. Part of preparing the system
25is creating a local Git repository of the `Source
26Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (``poky``) on your
27system. Follow the steps in the "`Cloning the ``poky``
28Repository <&YOCTO_DOCS_DEV_URL;#cloning-the-poky-repository>`__"
29section in the Yocto Project Development Tasks Manual to set up your
30Source Directory.
31
32.. note::
33
34 Be sure you check out the appropriate development branch or you
35 create your local branch by checking out a specific tag to get the
36 desired version of Yocto Project. See the "
37 Checking Out by Branch in Poky
38 " and "
39 Checking Out by Tag in Poky
40 " sections in the Yocto Project Development Tasks Manual for more
41 information.
42
43Kernel development is best accomplished using
44```devtool`` <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__
45and not through traditional kernel workflow methods. The remainder of
46this section provides information for both scenarios.
47
48Getting Ready to Develop Using ``devtool``
49------------------------------------------
50
51Follow these steps to prepare to update the kernel image using
52``devtool``. Completing this procedure leaves you with a clean kernel
53image and ready to make modifications as described in the "`Using
54``devtool`` to Patch the Kernel <#using-devtool-to-patch-the-kernel>`__"
55section:
56
571. *Initialize the BitBake Environment:* Before building an extensible
58 SDK, you need to initialize the BitBake build environment by sourcing
59 the build environment script (i.e.
60 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__):
61 $ cd ~/poky $ source oe-init-build-env
62
63 .. note::
64
65 The previous commands assume the
66 Source Repositories
67 (i.e.
68 poky
69 ) have been cloned using Git and the local repository is named
70 "poky".
71
722. *Prepare Your ``local.conf`` File:* By default, the
73 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable is set to
74 "qemux86-64", which is fine if you are building for the QEMU emulator
75 in 64-bit mode. However, if you are not, you need to set the
76 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
77 found in the `Build
78 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (i.e.
79 ``~/poky/build`` in this example).
80
81 Also, since you are preparing to work on the kernel image, you need
82 to set the
83 ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
84 variable to include kernel modules.
85
86 In this example we wish to build for qemux86 so we must set the
87 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
88 As described we do this by appending to ``conf/local.conf``: MACHINE
89 = "qemux86" MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
90
913. *Create a Layer for Patches:* You need to create a layer to hold
92 patches created for the kernel image. You can use the
93 ``bitbake-layers create-layer`` command as follows: $ cd ~/poky/build
94 $ bitbake-layers create-layer ../../meta-mylayer NOTE: Starting
95 bitbake server... Add your new layer with 'bitbake-layers add-layer
96 ../../meta-mylayer' $
97
98 .. note::
99
100 For background information on working with common and BSP layers,
101 see the "
102 Understanding and Creating Layers
103 " section in the Yocto Project Development Tasks Manual and the "
104 BSP Layers
105 " section in the Yocto Project Board Support (BSP) Developer's
106 Guide, respectively. For information on how to use the
107 bitbake-layers create-layer
108 command to quickly set up a layer, see the "
109 Creating a General Layer Using the
110 bitbake-layers
111 Script
112 " section in the Yocto Project Development Tasks Manual.
113
1144. *Inform the BitBake Build Environment About Your Layer:* As directed
115 when you created your layer, you need to add the layer to the
116 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable in the
117 ``bblayers.conf`` file as follows: $ cd ~/poky/build $ bitbake-layers
118 add-layer ../../meta-mylayer NOTE: Starting bitbake server... $
119
1205. *Build the Extensible SDK:* Use BitBake to build the extensible SDK
121 specifically for use with images to be run using QEMU: $ cd
122 ~/poky/build $ bitbake core-image-minimal -c populate_sdk_ext Once
123 the build finishes, you can find the SDK installer file (i.e.
124 ``*.sh`` file) in the following directory:
125 ~/poky/build/tmp/deploy/sdk For this example, the installer file is
126 named
127 ``poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-DISTRO.sh``
128
1296. *Install the Extensible SDK:* Use the following command to install
130 the SDK. For this example, install the SDK in the default
131 ``~/poky_sdk`` directory: $ cd ~/poky/build/tmp/deploy/sdk $
132 ./poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-DISTRO.sh
133 Poky (Yocto Project Reference Distro) Extensible SDK installer
134 version DISTRO
135 ============================================================================
136 Enter target directory for SDK (default: ~/poky_sdk): You are about
137 to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
138 Extracting SDK......................................done Setting it
139 up... Extracting buildtools... Preparing build system... Parsing
140 recipes: 100%
141 \|#################################################################\|
142 Time: 0:00:52 Initializing tasks: 100% \|##############
143 ###############################################\| Time: 0:00:04
144 Checking sstate mirror object availability: 100%
145 \|######################################\| Time: 0:00:00 Parsing
146 recipes: 100%
147 \|#################################################################\|
148 Time: 0:00:33 Initializing tasks: 100%
149 \|##############################################################\|
150 Time: 0:00:00 done SDK has been successfully set up and is ready to
151 be used. Each time you wish to use the SDK in a new shell session,
152 you need to source the environment setup script e.g. $ .
153 /home/scottrif/poky_sdk/environment-setup-i586-poky-linux
154
1557. *Set Up a New Terminal to Work With the Extensible SDK:* You must set
156 up a new terminal to work with the SDK. You cannot use the same
157 BitBake shell used to build the installer.
158
159 After opening a new shell, run the SDK environment setup script as
160 directed by the output from installing the SDK: $ source
161 ~/poky_sdk/environment-setup-i586-poky-linux "SDK environment now set
162 up; additionally you may now run devtool to perform development
163 tasks. Run devtool --help for further details.
164
165 .. note::
166
167 If you get a warning about attempting to use the extensible SDK in
168 an environment set up to run BitBake, you did not use a new shell.
169
1708. *Build the Clean Image:* The final step in preparing to work on the
171 kernel is to build an initial image using ``devtool`` in the new
172 terminal you just set up and initialized for SDK work: $ devtool
173 build-image Parsing recipes: 100%
174 \|##########################################\| Time: 0:00:05 Parsing
175 of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47
176 skipped, 0 masked, 0 errors. WARNING: No packages to add, building
177 image core-image-minimal unmodified Loading cache: 100%
178 \|############################################\| Time: 0:00:00 Loaded
179 1299 entries from dependency cache. NOTE: Resolving any missing task
180 queue dependencies Initializing tasks: 100%
181 \|#######################################\| Time: 0:00:07 Checking
182 sstate mirror object availability: 100% \|###############\| Time:
183 0:00:00 NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks
184 NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need
185 to be rerun and all succeeded. NOTE: Successfully built
186 core-image-minimal. You can find output files in
187 /home/scottrif/poky_sdk/tmp/deploy/images/qemux86 If you were
188 building for actual hardware and not for emulation, you could flash
189 the image to a USB stick on ``/dev/sdd`` and boot your device. For an
190 example that uses a Minnowboard, see the
191 `TipsAndTricks/KernelDevelopmentWithEsdk <https://wiki.yoctoproject.org/wiki/TipsAndTricks/KernelDevelopmentWithEsdk>`__
192 Wiki page.
193
194At this point you have set up to start making modifications to the
195kernel by using the extensible SDK. For a continued example, see the
196"`Using ``devtool`` to Patch the
197Kernel <#using-devtool-to-patch-the-kernel>`__" section.
198
199Getting Ready for Traditional Kernel Development
200------------------------------------------------
201
202Getting ready for traditional kernel development using the Yocto Project
203involves many of the same steps as described in the previous section.
204However, you need to establish a local copy of the kernel source since
205you will be editing these files.
206
207Follow these steps to prepare to update the kernel image using
208traditional kernel development flow with the Yocto Project. Completing
209this procedure leaves you ready to make modifications to the kernel
210source as described in the "`Using Traditional Kernel Development to
211Patch the
212Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
213section:
214
2151. *Initialize the BitBake Environment:* Before you can do anything
216 using BitBake, you need to initialize the BitBake build environment
217 by sourcing the build environment script (i.e.
218 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__).
219 Also, for this example, be sure that the local branch you have
220 checked out for ``poky`` is the Yocto Project DISTRO_NAME branch. If
221 you need to checkout out the DISTRO_NAME branch, see the "`Checking
222 out by Branch in
223 Poky <&YOCTO_DOCS_DEV_URL;#checking-out-by-branch-in-poky>`__"
224 section in the Yocto Project Development Tasks Manual. $ cd ~/poky $
225 git branch master \* DISTRO_NAME $ source oe-init-build-env
226
227 .. note::
228
229 The previous commands assume the
230 Source Repositories
231 (i.e.
232 poky
233 ) have been cloned using Git and the local repository is named
234 "poky".
235
2362. *Prepare Your ``local.conf`` File:* By default, the
237 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable is set to
238 "qemux86-64", which is fine if you are building for the QEMU emulator
239 in 64-bit mode. However, if you are not, you need to set the
240 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
241 found in the `Build
242 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (i.e.
243 ``~/poky/build`` in this example).
244
245 Also, since you are preparing to work on the kernel image, you need
246 to set the
247 ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
248 variable to include kernel modules.
249
250 In this example we wish to build for qemux86 so we must set the
251 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
252 As described we do this by appending to ``conf/local.conf``: MACHINE
253 = "qemux86" MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
254
2553. *Create a Layer for Patches:* You need to create a layer to hold
256 patches created for the kernel image. You can use the
257 ``bitbake-layers create-layer`` command as follows: $ cd ~/poky/build
258 $ bitbake-layers create-layer ../../meta-mylayer NOTE: Starting
259 bitbake server... Add your new layer with 'bitbake-layers add-layer
260 ../../meta-mylayer'
261
262 .. note::
263
264 For background information on working with common and BSP layers,
265 see the "
266 Understanding and Creating Layers
267 " section in the Yocto Project Development Tasks Manual and the "
268 BSP Layers
269 " section in the Yocto Project Board Support (BSP) Developer's
270 Guide, respectively. For information on how to use the
271 bitbake-layers create-layer
272 command to quickly set up a layer, see the "
273 Creating a General Layer Using the
274 bitbake-layers
275 Script
276 " section in the Yocto Project Development Tasks Manual.
277
2784. *Inform the BitBake Build Environment About Your Layer:* As directed
279 when you created your layer, you need to add the layer to the
280 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable in the
281 ``bblayers.conf`` file as follows: $ cd ~/poky/build $ bitbake-layers
282 add-layer ../../meta-mylayer NOTE: Starting bitbake server ... $
283
2845. *Create a Local Copy of the Kernel Git Repository:* You can find Git
285 repositories of supported Yocto Project kernels organized under
286 "Yocto Linux Kernel" in the Yocto Project Source Repositories at
287 ` <&YOCTO_GIT_URL;>`__.
288
289 For simplicity, it is recommended that you create your copy of the
290 kernel Git repository outside of the `Source
291 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__, which is
292 usually named ``poky``. Also, be sure you are in the
293 ``standard/base`` branch.
294
295 The following commands show how to create a local copy of the
296 ``linux-yocto-4.12`` kernel and be in the ``standard/base`` branch.
297
298 .. note::
299
300 The
301 linux-yocto-4.12
302 kernel can be used with the Yocto Project 2.4 release and forward.
303 You cannot use the
304 linux-yocto-4.12
305 kernel with releases prior to Yocto Project 2.4:
306
307 $ cd ~ $ git clone git://git.yoctoproject.org/linux-yocto-4.12
308 --branch standard/base Cloning into 'linux-yocto-4.12'... remote:
309 Counting objects: 6097195, done. remote: Compressing objects: 100%
310 (901026/901026), done. remote: Total 6097195 (delta 5152604), reused
311 6096847 (delta 5152256) Receiving objects: 100% (6097195/6097195),
312 1.24 GiB \| 7.81 MiB/s, done. Resolving deltas: 100%
313 (5152604/5152604), done. Checking connectivity... done. Checking out
314 files: 100% (59846/59846), done.
315
3166. *Create a Local Copy of the Kernel Cache Git Repository:* For
317 simplicity, it is recommended that you create your copy of the kernel
318 cache Git repository outside of the `Source
319 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__, which is
320 usually named ``poky``. Also, for this example, be sure you are in
321 the ``yocto-4.12`` branch.
322
323 The following commands show how to create a local copy of the
324 ``yocto-kernel-cache`` and be in the ``yocto-4.12`` branch: $ cd ~ $
325 git clone git://git.yoctoproject.org/yocto-kernel-cache --branch
326 yocto-4.12 Cloning into 'yocto-kernel-cache'... remote: Counting
327 objects: 22639, done. remote: Compressing objects: 100% (9761/9761),
328 done. remote: Total 22639 (delta 12400), reused 22586 (delta 12347)
329 Receiving objects: 100% (22639/22639), 22.34 MiB \| 6.27 MiB/s, done.
330 Resolving deltas: 100% (12400/12400), done. Checking connectivity...
331 done.
332
333At this point, you are ready to start making modifications to the kernel
334using traditional kernel development steps. For a continued example, see
335the "`Using Traditional Kernel Development to Patch the
336Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
337section.
338
339Creating and Preparing a Layer
340==============================
341
342If you are going to be modifying kernel recipes, it is recommended that
343you create and prepare your own layer in which to do your work. Your
344layer contains its own `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__
345append files (``.bbappend``) and provides a convenient mechanism to
346create your own recipe files (``.bb``) as well as store and use kernel
347patch files. For background information on working with layers, see the
348"`Understanding and Creating
349Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
350section in the Yocto Project Development Tasks Manual.
351
352.. note::
353
354 The Yocto Project comes with many tools that simplify tasks you need
355 to perform. One such tool is the
356 bitbake-layers create-layer
357 command, which simplifies creating a new layer. See the "
358 Creating a General Layer Using the
359 bitbake-layers
360 Script
361 " section in the Yocto Project Development Tasks Manual for
362 information on how to use this script to quick set up a new layer.
363
364To better understand the layer you create for kernel development, the
365following section describes how to create a layer without the aid of
366tools. These steps assume creation of a layer named ``mylayer`` in your
367home directory:
368
3691. *Create Structure*: Create the layer's structure: $ cd $HOME $ mkdir
370 meta-mylayer $ mkdir meta-mylayer/conf $ mkdir
371 meta-mylayer/recipes-kernel $ mkdir meta-mylayer/recipes-kernel/linux
372 $ mkdir meta-mylayer/recipes-kernel/linux/linux-yocto The ``conf``
373 directory holds your configuration files, while the
374 ``recipes-kernel`` directory holds your append file and eventual
375 patch files.
376
3772. *Create the Layer Configuration File*: Move to the
378 ``meta-mylayer/conf`` directory and create the ``layer.conf`` file as
379 follows: # We have a conf and classes directory, add to BBPATH BBPATH
380 .= ":${LAYERDIR}" # We have recipes-\* directories, add to BBFILES
381 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \\
382 ${LAYERDIR}/recipes-*/*/*.bbappend" BBFILE_COLLECTIONS += "mylayer"
383 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/" BBFILE_PRIORITY_mylayer =
384 "5" Notice ``mylayer`` as part of the last three statements.
385
3863. *Create the Kernel Recipe Append File*: Move to the
387 ``meta-mylayer/recipes-kernel/linux`` directory and create the
388 kernel's append file. This example uses the ``linux-yocto-4.12``
389 kernel. Thus, the name of the append file is
390 ``linux-yocto_4.12.bbappend``: FILESEXTRAPATHS_prepend :=
391 "${THISDIR}/${PN}:" SRC_URI_append = " file://patch-file-one"
392 SRC_URI_append = " file://patch-file-two" SRC_URI_append = "
393 file://patch-file-three" The
394 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
395 and ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
396 enable the OpenEmbedded build system to find patch files. For more
397 information on using append files, see the "`Using .bbappend Files in
398 Your Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in
399 the Yocto Project Development Tasks Manual.
400
401Modifying an Existing Recipe
402============================
403
404In many cases, you can customize an existing linux-yocto recipe to meet
405the needs of your project. Each release of the Yocto Project provides a
406few Linux kernel recipes from which you can choose. These are located in
407the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ in
408``meta/recipes-kernel/linux``.
409
410Modifying an existing recipe can consist of the following:
411
412- Creating the append file
413
414- Applying patches
415
416- Changing the configuration
417
418Before modifying an existing recipe, be sure that you have created a
419minimal, custom layer from which you can work. See the "`Creating and
420Preparing a Layer <#creating-and-preparing-a-layer>`__" section for
421information.
422
423Creating the Append File
424------------------------
425
426You create this file in your custom layer. You also name it accordingly
427based on the linux-yocto recipe you are using. For example, if you are
428modifying the ``meta/recipes-kernel/linux/linux-yocto_4.12.bb`` recipe,
429the append file will typically be located as follows within your custom
430layer: your-layer/recipes-kernel/linux/linux-yocto_4.12.bbappend The
431append file should initially extend the
432```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ search path by
433prepending the directory that contains your files to the
434```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
435variable as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" The
436path
437``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``
438expands to "linux-yocto" in the current directory for this example. If
439you add any new files that modify the kernel recipe and you have
440extended ``FILESPATH`` as described above, you must place the files in
441your layer in the following area:
442your-layer/recipes-kernel/linux/linux-yocto/
443
444.. note::
445
446 If you are working on a new machine Board Support Package (BSP), be
447 sure to refer to the
448 Yocto Project Board Support Package (BSP) Developer's Guide
449 .
450
451As an example, consider the following append file used by the BSPs in
452``meta-yocto-bsp``:
453meta-yocto-bsp/recipes-kernel/linux/linux-yocto_4.12.bbappend The
454following listing shows the file. Be aware that the actual commit ID
455strings in this example listing might be different than the actual
456strings in the file from the ``meta-yocto-bsp`` layer upstream.
457KBRANCH_genericx86 = "standard/base" KBRANCH_genericx86-64 =
458"standard/base" KMACHINE_genericx86 ?= "common-pc"
459KMACHINE_genericx86-64 ?= "common-pc-64" KBRANCH_edgerouter =
460"standard/edgerouter" KBRANCH_beaglebone = "standard/beaglebone"
461SRCREV_machine_genericx86 ?= "d09f2ce584d60ecb7890550c22a80c48b83c2e19"
462SRCREV_machine_genericx86-64 ?=
463"d09f2ce584d60ecb7890550c22a80c48b83c2e19" SRCREV_machine_edgerouter ?=
464"b5c8cfda2dfe296410d51e131289fb09c69e1e7d" SRCREV_machine_beaglebone ?=
465"b5c8cfda2dfe296410d51e131289fb09c69e1e7d" COMPATIBLE_MACHINE_genericx86
466= "genericx86" COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
467COMPATIBLE_MACHINE_edgerouter = "edgerouter"
468COMPATIBLE_MACHINE_beaglebone = "beaglebone" LINUX_VERSION_genericx86 =
469"4.12.7" LINUX_VERSION_genericx86-64 = "4.12.7" LINUX_VERSION_edgerouter
470= "4.12.10" LINUX_VERSION_beaglebone = "4.12.10" This append file
471contains statements used to support several BSPs that ship with the
472Yocto Project. The file defines machines using the
473```COMPATIBLE_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE>`__
474variable and uses the
475```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__ variable to ensure
476the machine name used by the OpenEmbedded build system maps to the
477machine name used by the Linux Yocto kernel. The file also uses the
478optional ```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable to
479ensure the build process uses the appropriate kernel branch.
480
481Although this particular example does not use it, the
482```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
483variable could be used to enable features specific to the kernel. The
484append file points to specific commits in the `Source
485Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ Git repository and
486the ``meta`` Git repository branches to identify the exact kernel needed
487to build the BSP.
488
489One thing missing in this particular BSP, which you will typically need
490when developing a BSP, is the kernel configuration file (``.config``)
491for your BSP. When developing a BSP, you probably have a kernel
492configuration file or a set of kernel configuration files that, when
493taken together, define the kernel configuration for your BSP. You can
494accomplish this definition by putting the configurations in a file or a
495set of files inside a directory located at the same level as your
496kernel's append file and having the same name as the kernel's main
497recipe file. With all these conditions met, simply reference those files
498in the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement in
499the append file.
500
501For example, suppose you had some configuration options in a file called
502``network_configs.cfg``. You can place that file inside a directory
503named ``linux-yocto`` and then add a ``SRC_URI`` statement such as the
504following to the append file. When the OpenEmbedded build system builds
505the kernel, the configuration options are picked up and applied. SRC_URI
506+= "file://network_configs.cfg"
507
508To group related configurations into multiple files, you perform a
509similar procedure. Here is an example that groups separate
510configurations specifically for Ethernet and graphics into their own
511files and adds the configurations by using a ``SRC_URI`` statement like
512the following in your append file: SRC_URI += "file://myconfig.cfg \\
513file://eth.cfg \\ file://gfx.cfg"
514
515Another variable you can use in your kernel recipe append file is the
516```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
517variable. When you use this statement, you are extending the locations
518used by the OpenEmbedded system to look for files and patches as the
519recipe is processed.
520
521.. note::
522
523 Other methods exist to accomplish grouping and defining configuration
524 options. For example, if you are working with a local clone of the
525 kernel repository, you could checkout the kernel's ``meta`` branch,
526 make your changes, and then push the changes to the local bare clone
527 of the kernel. The result is that you directly add configuration
528 options to the ``meta`` branch for your BSP. The configuration
529 options will likely end up in that location anyway if the BSP gets
530 added to the Yocto Project.
531
532 In general, however, the Yocto Project maintainers take care of
533 moving the ``SRC_URI``-specified configuration options to the
534 kernel's ``meta`` branch. Not only is it easier for BSP developers to
535 not have to worry about putting those configurations in the branch,
536 but having the maintainers do it allows them to apply 'global'
537 knowledge about the kinds of common configuration options multiple
538 BSPs in the tree are typically using. This allows for promotion of
539 common configurations into common features.
540
541Applying Patches
542----------------
543
544If you have a single patch or a small series of patches that you want to
545apply to the Linux kernel source, you can do so just as you would with
546any other recipe. You first copy the patches to the path added to
547```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__ in
548your ``.bbappend`` file as described in the previous section, and then
549reference them in ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
550statements.
551
552For example, you can apply a three-patch series by adding the following
553lines to your linux-yocto ``.bbappend`` file in your layer: SRC_URI +=
554"file://0001-first-change.patch" SRC_URI +=
555"file://0002-second-change.patch" SRC_URI +=
556"file://0003-third-change.patch" The next time you run BitBake to build
557the Linux kernel, BitBake detects the change in the recipe and fetches
558and applies the patches before building the kernel.
559
560For a detailed example showing how to patch the kernel using
561``devtool``, see the "`Using ``devtool`` to Patch the
562Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
563Kernel Development to Patch the
564Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
565sections.
566
567Changing the Configuration
568--------------------------
569
570You can make wholesale or incremental changes to the final ``.config``
571file used for the eventual Linux kernel configuration by including a
572``defconfig`` file and by specifying configuration fragments in the
573```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ to be applied to that
574file.
575
576If you have a complete, working Linux kernel ``.config`` file you want
577to use for the configuration, as before, copy that file to the
578appropriate ``${PN}`` directory in your layer's ``recipes-kernel/linux``
579directory, and rename the copied file to "defconfig". Then, add the
580following lines to the linux-yocto ``.bbappend`` file in your layer:
581FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
582"file://defconfig" The ``SRC_URI`` tells the build system how to search
583for the file, while the
584```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
585extends the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
586variable (search directories) to include the ``${PN}`` directory you
587created to hold the configuration changes.
588
589.. note::
590
591 The build system applies the configurations from the
592 defconfig
593 file before applying any subsequent configuration fragments. The
594 final kernel configuration is a combination of the configurations in
595 the
596 defconfig
597 file and any configuration fragments you provide. You need to realize
598 that if you have any configuration fragments, the build system
599 applies these on top of and after applying the existing
600 defconfig
601 file configurations.
602
603Generally speaking, the preferred approach is to determine the
604incremental change you want to make and add that as a configuration
605fragment. For example, if you want to add support for a basic serial
606console, create a file named ``8250.cfg`` in the ``${PN}`` directory
607with the following content (without indentation): CONFIG_SERIAL_8250=y
608CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_PCI=y
609CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_8250_RUNTIME_UARTS=4
610CONFIG_SERIAL_CORE=y CONFIG_SERIAL_CORE_CONSOLE=y Next, include this
611configuration fragment and extend the ``FILESPATH`` variable in your
612``.bbappend`` file: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
613SRC_URI += "file://8250.cfg" The next time you run BitBake to build the
614Linux kernel, BitBake detects the change in the recipe and fetches and
615applies the new configuration before building the kernel.
616
617For a detailed example showing how to configure the kernel, see the
618"`Configuring the Kernel <#configuring-the-kernel>`__" section.
619
620Using an "In-Tree"  ``defconfig`` File
621--------------------------------------
622
623It might be desirable to have kernel configuration fragment support
624through a ``defconfig`` file that is pulled from the kernel source tree
625for the configured machine. By default, the OpenEmbedded build system
626looks for ``defconfig`` files in the layer used for Metadata, which is
627"out-of-tree", and then configures them using the following: SRC_URI +=
628"file://defconfig" If you do not want to maintain copies of
629``defconfig`` files in your layer but would rather allow users to use
630the default configuration from the kernel tree and still be able to add
631configuration fragments to the
632```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ through, for example,
633append files, you can direct the OpenEmbedded build system to use a
634``defconfig`` file that is "in-tree".
635
636To specify an "in-tree" ``defconfig`` file, use the following statement
637form: KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file Here is an example
638that assigns the ``KBUILD_DEFCONFIG`` variable based on "raspberrypi2"
639and provides the path to the "in-tree" ``defconfig`` file to be used for
640a Raspberry Pi 2, which is based on the Broadcom 2708/2709 chipset:
641KBUILD_DEFCONFIG_raspberrypi2 ?= "bcm2709_defconfig"
642
643Aside from modifying your kernel recipe and providing your own
644``defconfig`` file, you need to be sure no files or statements set
645``SRC_URI`` to use a ``defconfig`` other than your "in-tree" file (e.g.
646a kernel's ``linux-``\ machine\ ``.inc`` file). In other words, if the
647build system detects a statement that identifies an "out-of-tree"
648``defconfig`` file, that statement will override your
649``KBUILD_DEFCONFIG`` variable.
650
651See the
652```KBUILD_DEFCONFIG`` <&YOCTO_DOCS_REF_URL;#var-KBUILD_DEFCONFIG>`__
653variable description for more information.
654
655Using ``devtool`` to Patch the Kernel
656=====================================
657
658The steps in this procedure show you how you can patch the kernel using
659the extensible SDK and ``devtool``.
660
661.. note::
662
663 Before attempting this procedure, be sure you have performed the
664 steps to get ready for updating the kernel as described in the "
665 Getting Ready to Develop Using
666 devtool
667 " section.
668
669Patching the kernel involves changing or adding configurations to an
670existing kernel, changing or adding recipes to the kernel that are
671needed to support specific hardware features, or even altering the
672source code itself.
673
674This example creates a simple patch by adding some QEMU emulator console
675output at boot time through ``printk`` statements in the kernel's
676``calibrate.c`` source code file. Applying the patch and booting the
677modified image causes the added messages to appear on the emulator's
678console. The example is a continuation of the setup procedure found in
679the "`Getting Ready to Develop Using
680``devtool`` <#getting-ready-to-develop-using-devtool>`__" Section.
681
6821. *Check Out the Kernel Source Files:* First you must use ``devtool``
683 to checkout the kernel source code in its workspace. Be sure you are
684 in the terminal set up to do work with the extensible SDK.
685
686 .. note::
687
688 See this
689 step
690 in the "
691 Getting Ready to Develop Using
692 devtool
693 " section for more information.
694
695 Use the following ``devtool`` command to check out the code: $
696 devtool modify linux-yocto
697
698 .. note::
699
700 During the checkout operation, a bug exists that could cause
701 errors such as the following to appear:
702 ::
703
704 ERROR: Taskhash mismatch 2c793438c2d9f8c3681fd5f7bc819efa versus
705 be3a89ce7c47178880ba7bf6293d7404 for
706 /path/to/esdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_4.10.bb.do_unpack
707
708
709 You can safely ignore these messages. The source code is correctly
710 checked out.
711
7122. *Edit the Source Files* Follow these steps to make some simple
713 changes to the source files:
714
715 1. *Change the working directory*: In the previous step, the output
716 noted where you can find the source files (e.g.
717 ``~/poky_sdk/workspace/sources/linux-yocto``). Change to where the
718 kernel source code is before making your edits to the
719 ``calibrate.c`` file: $ cd
720 ~/poky_sdk/workspace/sources/linux-yocto
721
722 2. *Edit the source file*: Edit the ``init/calibrate.c`` file to have
723 the following changes: void calibrate_delay(void) { unsigned long
724 lpj; static bool printed; int this_cpu = smp_processor_id();
725 printk("*************************************\n"); printk("\*
726 \*\n"); printk("\* HELLO YOCTO KERNEL \*\n"); printk("\* \*\n");
727 printk("*************************************\n"); if
728 (per_cpu(cpu_loops_per_jiffy, this_cpu)) { . . .
729
7303. *Build the Updated Kernel Source:* To build the updated kernel
731 source, use ``devtool``: $ devtool build linux-yocto
732
7334. *Create the Image With the New Kernel:* Use the
734 ``devtool build-image`` command to create a new image that has the
735 new kernel.
736
737 .. note::
738
739 If the image you originally created resulted in a Wic file, you
740 can use an alternate method to create the new image with the
741 updated kernel. For an example, see the steps in the
742 TipsAndTricks/KernelDevelopmentWithEsdk
743 Wiki Page.
744
745 $ cd ~ $ devtool build-image core-image-minimal
746
7475. *Test the New Image:* For this example, you can run the new image
748 using QEMU to verify your changes:
749
750 1. *Boot the image*: Boot the modified image in the QEMU emulator
751 using this command: $ runqemu qemux86
752
753 2. *Verify the changes*: Log into the machine using ``root`` with no
754 password and then use the following shell command to scroll
755 through the console's boot output. # dmesg \| less You should see
756 the results of your ``printk`` statements as part of the output
757 when you scroll down the console window.
758
7596. *Stage and commit your changes*: Within your eSDK terminal, change
760 your working directory to where you modified the ``calibrate.c`` file
761 and use these Git commands to stage and commit your changes: $ cd
762 ~/poky_sdk/workspace/sources/linux-yocto $ git status $ git add
763 init/calibrate.c $ git commit -m "calibrate: Add printk example"
764
7657. *Export the Patches and Create an Append File:* To export your
766 commits as patches and create a ``.bbappend`` file, use the following
767 command in the terminal used to work with the extensible SDK. This
768 example uses the previously established layer named ``meta-mylayer``.
769
770 .. note::
771
772 See Step 3 of the "
773 Getting Ready to Develop Using devtool
774 " section for information on setting up this layer.
775
776 $ devtool finish linux-yocto ~/meta-mylayer Once the command
777 finishes, the patches and the ``.bbappend`` file are located in the
778 ``~/meta-mylayer/recipes-kernel/linux`` directory.
779
7808. *Build the Image With Your Modified Kernel:* You can now build an
781 image that includes your kernel patches. Execute the following
782 command from your `Build
783 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in the terminal
784 set up to run BitBake: $ cd ~/poky/build $ bitbake core-image-minimal
785
786Using Traditional Kernel Development to Patch the Kernel
787========================================================
788
789The steps in this procedure show you how you can patch the kernel using
790traditional kernel development (i.e. not using ``devtool`` and the
791extensible SDK as described in the "`Using ``devtool`` to Patch the
792Kernel <#using-devtool-to-patch-the-kernel>`__" section).
793
794.. note::
795
796 Before attempting this procedure, be sure you have performed the
797 steps to get ready for updating the kernel as described in the "
798 Getting Ready for Traditional Kernel Development
799 " section.
800
801Patching the kernel involves changing or adding configurations to an
802existing kernel, changing or adding recipes to the kernel that are
803needed to support specific hardware features, or even altering the
804source code itself.
805
806The example in this section creates a simple patch by adding some QEMU
807emulator console output at boot time through ``printk`` statements in
808the kernel's ``calibrate.c`` source code file. Applying the patch and
809booting the modified image causes the added messages to appear on the
810emulator's console. The example is a continuation of the setup procedure
811found in the "`Getting Ready for Traditional Kernel
812Development <#getting-ready-for-traditional-kernel-development>`__"
813Section.
814
8151. *Edit the Source Files* Prior to this step, you should have used Git
816 to create a local copy of the repository for your kernel. Assuming
817 you created the repository as directed in the "`Getting Ready for
818 Traditional Kernel
819 Development <#getting-ready-for-traditional-kernel-development>`__"
820 section, use the following commands to edit the ``calibrate.c`` file:
821
822 1. *Change the working directory*: You need to locate the source
823 files in the local copy of the kernel Git repository: Change to
824 where the kernel source code is before making your edits to the
825 ``calibrate.c`` file: $ cd ~/linux-yocto-4.12/init
826
827 2. *Edit the source file*: Edit the ``calibrate.c`` file to have the
828 following changes: void calibrate_delay(void) { unsigned long lpj;
829 static bool printed; int this_cpu = smp_processor_id();
830 printk("*************************************\n"); printk("\*
831 \*\n"); printk("\* HELLO YOCTO KERNEL \*\n"); printk("\* \*\n");
832 printk("*************************************\n"); if
833 (per_cpu(cpu_loops_per_jiffy, this_cpu)) { . . .
834
8352. *Stage and Commit Your Changes:* Use standard Git commands to stage
836 and commit the changes you just made: $ git add calibrate.c $ git
837 commit -m "calibrate.c - Added some printk statements" If you do not
838 stage and commit your changes, the OpenEmbedded Build System will not
839 pick up the changes.
840
8413. *Update Your ``local.conf`` File to Point to Your Source Files:* In
842 addition to your ``local.conf`` file specifying to use
843 "kernel-modules" and the "qemux86" machine, it must also point to the
844 updated kernel source files. Add
845 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ and
846 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ statements similar
847 to the following to your ``local.conf``: $ cd ~/poky/build/conf Add
848 the following to the ``local.conf``: SRC_URI_pn-linux-yocto =
849 "git:///path-to/linux-yocto-4.12;protocol=file;name=machine;branch=standard/base;
850 \\
851 git:///path-to/yocto-kernel-cache;protocol=file;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
852 SRCREV_meta_qemux86 = "${AUTOREV}" SRCREV_machine_qemux86 =
853 "${AUTOREV}"
854
855 .. note::
856
857 Be sure to replace
858 path-to
859 with the pathname to your local Git repositories. Also, you must
860 be sure to specify the correct branch and machine types. For this
861 example, the branch is
862 standard/base
863 and the machine is "qemux86".
864
8654. *Build the Image:* With the source modified, your changes staged and
866 committed, and the ``local.conf`` file pointing to the kernel files,
867 you can now use BitBake to build the image: $ cd ~/poky/build $
868 bitbake core-image-minimal
869
8705. *Boot the image*: Boot the modified image in the QEMU emulator using
871 this command. When prompted to login to the QEMU console, use "root"
872 with no password: $ cd ~/poky/build $ runqemu qemux86
873
8746. *Look for Your Changes:* As QEMU booted, you might have seen your
875 changes rapidly scroll by. If not, use these commands to see your
876 changes: # dmesg \| less You should see the results of your
877 ``printk`` statements as part of the output when you scroll down the
878 console window.
879
8807. *Generate the Patch File:* Once you are sure that your patch works
881 correctly, you can generate a ``*.patch`` file in the kernel source
882 repository: $ cd ~/linux-yocto-4.12/init $ git format-patch -1
883 0001-calibrate.c-Added-some-printk-statements.patch
884
8858. *Move the Patch File to Your Layer:* In order for subsequent builds
886 to pick up patches, you need to move the patch file you created in
887 the previous step to your layer ``meta-mylayer``. For this example,
888 the layer created earlier is located in your home directory as
889 ``meta-mylayer``. When the layer was created using the
890 ``yocto-create`` script, no additional hierarchy was created to
891 support patches. Before moving the patch file, you need to add
892 additional structure to your layer using the following commands: $ cd
893 ~/meta-mylayer $ mkdir recipes-kernel $ mkdir recipes-kernel/linux $
894 mkdir recipes-kernel/linux/linux-yocto Once you have created this
895 hierarchy in your layer, you can move the patch file using the
896 following command: $ mv
897 ~/linux-yocto-4.12/init/0001-calibrate.c-Added-some-printk-statements.patch
898 ~/meta-mylayer/recipes-kernel/linux/linux-yocto
899
9009. *Create the Append File:* Finally, you need to create the
901 ``linux-yocto_4.12.bbappend`` file and insert statements that allow
902 the OpenEmbedded build system to find the patch. The append file
903 needs to be in your layer's ``recipes-kernel/linux`` directory and it
904 must be named ``linux-yocto_4.12.bbappend`` and have the following
905 contents: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
906 SRC_URI_append = "
907 file://0001-calibrate.c-Added-some-printk-statements.patch" The
908 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
909 and ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
910 enable the OpenEmbedded build system to find the patch file.
911
912 For more information on append files and patches, see the "`Creating
913 the Append File <#creating-the-append-file>`__" and "`Applying
914 Patches <#applying-patches>`__" sections. You can also see the
915 "`Using .bbappend Files in Your
916 Layer" <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
917 Yocto Project Development Tasks Manual.
918
919 .. note::
920
921 To build
922 core-image-minimal
923 again and see the effects of your patch, you can essentially
924 eliminate the temporary source files saved in
925 poky/build/tmp/work/...
926 and residual effects of the build by entering the following
927 sequence of commands:
928 ::
929
930 $ cd ~/poky/build
931 $ bitbake -c cleanall yocto-linux
932 $ bitbake core-image-minimal -c cleanall
933 $ bitbake core-image-minimal
934 $ runqemu qemux86
935
936
937Configuring the Kernel
938======================
939
940Configuring the Yocto Project kernel consists of making sure the
941``.config`` file has all the right information in it for the image you
942are building. You can use the ``menuconfig`` tool and configuration
943fragments to make sure your ``.config`` file is just how you need it.
944You can also save known configurations in a ``defconfig`` file that the
945build system can use for kernel configuration.
946
947This section describes how to use ``menuconfig``, create and use
948configuration fragments, and how to interactively modify your
949``.config`` file to create the leanest kernel configuration file
950possible.
951
952For more information on kernel configuration, see the "`Changing the
953Configuration <#changing-the-configuration>`__" section.
954
955Using  ``menuconfig``
956---------------------
957
958The easiest way to define kernel configurations is to set them through
959the ``menuconfig`` tool. This tool provides an interactive method with
960which to set kernel configurations. For general information on
961``menuconfig``, see ` <http://en.wikipedia.org/wiki/Menuconfig>`__.
962
963To use the ``menuconfig`` tool in the Yocto Project development
964environment, you must do the following:
965
966- Because you launch ``menuconfig`` using BitBake, you must be sure to
967 set up your environment by running the
968 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ script found in
969 the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
970
971- You must be sure of the state of your build's configuration in the
972 `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
973
974- Your build host must have the following two packages installed:
975 libncurses5-dev libtinfo-dev
976
977The following commands initialize the BitBake environment, run the
978```do_kernel_configme`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configme>`__
979task, and launch ``menuconfig``. These commands assume the Source
980Directory's top-level folder is ``~/poky``: $ cd poky $ source
981oe-init-build-env $ bitbake linux-yocto -c kernel_configme -f $ bitbake
982linux-yocto -c menuconfig Once ``menuconfig`` comes up, its standard
983interface allows you to interactively examine and configure all the
984kernel configuration parameters. After making your changes, simply exit
985the tool and save your changes to create an updated version of the
986``.config`` configuration file.
987
988.. note::
989
990 You can use the entire
991 .config
992 file as the
993 defconfig
994 file. For information on
995 defconfig
996 files, see the "
997 Changing the Configuration
998 ", "
999 Using an In-Tree
1000 defconfig
1001 File
1002 , and "
1003 Creating a
1004 defconfig
1005 File
1006 " sections.
1007
1008Consider an example that configures the "CONFIG_SMP" setting for the
1009``linux-yocto-4.12`` kernel.
1010
1011.. note::
1012
1013 The OpenEmbedded build system recognizes this kernel as
1014 linux-yocto
1015 through Metadata (e.g.
1016 PREFERRED_VERSION
1017 \_linux-yocto ?= "12.4%"
1018 ).
1019
1020Once ``menuconfig`` launches, use the interface to navigate through the
1021selections to find the configuration settings in which you are
1022interested. For this example, you deselect "CONFIG_SMP" by clearing the
1023"Symmetric Multi-Processing Support" option. Using the interface, you
1024can find the option under "Processor Type and Features". To deselect
1025"CONFIG_SMP", use the arrow keys to highlight "Symmetric
1026Multi-Processing Support" and enter "N" to clear the asterisk. When you
1027are finished, exit out and save the change.
1028
1029Saving the selections updates the ``.config`` configuration file. This
1030is the file that the OpenEmbedded build system uses to configure the
1031kernel during the build. You can find and examine this file in the Build
1032Directory in ``tmp/work/``. The actual ``.config`` is located in the
1033area where the specific kernel is built. For example, if you were
1034building a Linux Yocto kernel based on the ``linux-yocto-4.12`` kernel
1035and you were building a QEMU image targeted for ``x86`` architecture,
1036the ``.config`` file would be:
1037poky/build/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+gitAUTOINC+eda4d18...
1038...967-r0/linux-qemux86-standard-build/.config
1039
1040.. note::
1041
1042 The previous example directory is artificially split and many of the
1043 characters in the actual filename are omitted in order to make it
1044 more readable. Also, depending on the kernel you are using, the exact
1045 pathname might differ.
1046
1047Within the ``.config`` file, you can see the kernel settings. For
1048example, the following entry shows that symmetric multi-processor
1049support is not set: # CONFIG_SMP is not set
1050
1051A good method to isolate changed configurations is to use a combination
1052of the ``menuconfig`` tool and simple shell commands. Before changing
1053configurations with ``menuconfig``, copy the existing ``.config`` and
1054rename it to something else, use ``menuconfig`` to make as many changes
1055as you want and save them, then compare the renamed configuration file
1056against the newly created file. You can use the resulting differences as
1057your base to create configuration fragments to permanently save in your
1058kernel layer.
1059
1060.. note::
1061
1062 Be sure to make a copy of the
1063 .config
1064 file and do not just rename it. The build system needs an existing
1065 .config
1066 file from which to work.
1067
1068Creating a  ``defconfig`` File
1069------------------------------
1070
1071A ``defconfig`` file in the context of the Yocto Project is often a
1072``.config`` file that is copied from a build or a ``defconfig`` taken
1073from the kernel tree and moved into recipe space. You can use a
1074``defconfig`` file to retain a known set of kernel configurations from
1075which the OpenEmbedded build system can draw to create the final
1076``.config`` file.
1077
1078.. note::
1079
1080 Out-of-the-box, the Yocto Project never ships a
1081 defconfig
1082 or
1083 .config
1084 file. The OpenEmbedded build system creates the final
1085 .config
1086 file used to configure the kernel.
1087
1088To create a ``defconfig``, start with a complete, working Linux kernel
1089``.config`` file. Copy that file to the appropriate
1090``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` directory in
1091your layer's ``recipes-kernel/linux`` directory, and rename the copied
1092file to "defconfig" (e.g.
1093``~/meta-mylayer/recipes-kernel/linux/linux-yocto/defconfig``). Then,
1094add the following lines to the linux-yocto ``.bbappend`` file in your
1095layer: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
1096"file://defconfig" The
1097```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ tells the build
1098system how to search for the file, while the
1099```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
1100extends the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
1101variable (search directories) to include the ``${PN}`` directory you
1102created to hold the configuration changes.
1103
1104.. note::
1105
1106 The build system applies the configurations from the
1107 defconfig
1108 file before applying any subsequent configuration fragments. The
1109 final kernel configuration is a combination of the configurations in
1110 the
1111 defconfig
1112 file and any configuration fragments you provide. You need to realize
1113 that if you have any configuration fragments, the build system
1114 applies these on top of and after applying the existing defconfig
1115 file configurations.
1116
1117For more information on configuring the kernel, see the "`Changing the
1118Configuration <#changing-the-configuration>`__" section.
1119
1120.. _creating-config-fragments:
1121
1122Creating Configuration Fragments
1123--------------------------------
1124
1125Configuration fragments are simply kernel options that appear in a file
1126placed where the OpenEmbedded build system can find and apply them. The
1127build system applies configuration fragments after applying
1128configurations from a ``defconfig`` file. Thus, the final kernel
1129configuration is a combination of the configurations in the
1130``defconfig`` file and then any configuration fragments you provide. The
1131build system applies fragments on top of and after applying the existing
1132defconfig file configurations.
1133
1134Syntactically, the configuration statement is identical to what would
1135appear in the ``.config`` file, which is in the `Build
1136Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
1137
1138.. note::
1139
1140 For more information about where the
1141 .config
1142 file is located, see the example in the "
1143 Using
1144 menuconfig
1145 " section.
1146
1147It is simple to create a configuration fragment. One method is to use
1148shell commands. For example, issuing the following from the shell
1149creates a configuration fragment file named ``my_smp.cfg`` that enables
1150multi-processor support within the kernel: $ echo "CONFIG_SMP=y" >>
1151my_smp.cfg
1152
1153.. note::
1154
1155 All configuration fragment files must use the
1156 .cfg
1157 extension in order for the OpenEmbedded build system to recognize
1158 them as a configuration fragment.
1159
1160Another method is to create a configuration fragment using the
1161differences between two configuration files: one previously created and
1162saved, and one freshly created using the ``menuconfig`` tool.
1163
1164To create a configuration fragment using this method, follow these
1165steps:
1166
11671. *Complete a Build Through Kernel Configuration:* Complete a build at
1168 least through the kernel configuration task as follows: $ bitbake
1169 linux-yocto -c kernel_configme -f This step ensures that you create a
1170 ``.config`` file from a known state. Because situations exist where
1171 your build state might become unknown, it is best to run this task
1172 prior to starting ``menuconfig``.
1173
11742. *Launch ``menuconfig``:* Run the ``menuconfig`` command: $ bitbake
1175 linux-yocto -c menuconfig
1176
11773. *Create the Configuration Fragment:* Run the ``diffconfig`` command
1178 to prepare a configuration fragment. The resulting file
1179 ``fragment.cfg`` is placed in the
1180 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``
1181 directory: $ bitbake linux-yocto -c diffconfig
1182
1183The ``diffconfig`` command creates a file that is a list of Linux kernel
1184``CONFIG_`` assignments. See the "`Changing the
1185Configuration <#changing-the-configuration>`__" section for additional
1186information on how to use the output as a configuration fragment.
1187
1188.. note::
1189
1190 You can also use this method to create configuration fragments for a
1191 BSP. See the "
1192 BSP Descriptions
1193 " section for more information.
1194
1195Where do you put your configuration fragment files? You can place these
1196files in an area pointed to by
1197```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ as directed by your
1198``bblayers.conf`` file, which is located in your layer. The OpenEmbedded
1199build system picks up the configuration and adds it to the kernel's
1200configuration. For example, suppose you had a set of configuration
1201options in a file called ``myconfig.cfg``. If you put that file inside a
1202directory named ``linux-yocto`` that resides in the same directory as
1203the kernel's append file within your layer and then add the following
1204statements to the kernel's append file, those configuration options will
1205be picked up and applied when the kernel is built:
1206FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
1207"file://myconfig.cfg"
1208
1209As mentioned earlier, you can group related configurations into multiple
1210files and name them all in the ``SRC_URI`` statement as well. For
1211example, you could group separate configurations specifically for
1212Ethernet and graphics into their own files and add those by using a
1213``SRC_URI`` statement like the following in your append file: SRC_URI +=
1214"file://myconfig.cfg \\ file://eth.cfg \\ file://gfx.cfg"
1215
1216Validating Configuration
1217------------------------
1218
1219You can use the
1220```do_kernel_configcheck`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configcheck>`__
1221task to provide configuration validation: $ bitbake linux-yocto -c
1222kernel_configcheck -f Running this task produces warnings for when a
1223requested configuration does not appear in the final ``.config`` file or
1224when you override a policy configuration in a hardware configuration
1225fragment.
1226
1227In order to run this task, you must have an existing ``.config`` file.
1228See the "`Using ``menuconfig`` <#using-menuconfig>`__" section for
1229information on how to create a configuration file.
1230
1231Following is sample output from the ``do_kernel_configcheck`` task:
1232Loading cache: 100%
1233\|########################################################\| Time:
12340:00:00 Loaded 1275 entries from dependency cache. NOTE: Resolving any
1235missing task queue dependencies Build Configuration: . . . NOTE:
1236Executing SetScene Tasks NOTE: Executing RunQueue Tasks WARNING:
1237linux-yocto-4.12.12+gitAUTOINC+eda4d18ce4_16de014967-r0
1238do_kernel_configcheck: [kernel config]: specified values did not make it
1239into the kernel's final configuration: ---------- CONFIG_X86_TSC
1240----------------- Config: CONFIG_X86_TSC From:
1241/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc-cpu.cfg
1242Requested value: CONFIG_X86_TSC=y Actual value: ----------
1243CONFIG_X86_BIGSMP ----------------- Config: CONFIG_X86_BIGSMP From:
1244/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1245/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1246Requested value: # CONFIG_X86_BIGSMP is not set Actual value: ----------
1247CONFIG_NR_CPUS ----------------- Config: CONFIG_NR_CPUS From:
1248/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1249/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc.cfg
1250/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1251Requested value: CONFIG_NR_CPUS=8 Actual value: CONFIG_NR_CPUS=1
1252---------- CONFIG_SCHED_SMT ----------------- Config: CONFIG_SCHED_SMT
1253From:
1254/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1255/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1256Requested value: CONFIG_SCHED_SMT=y Actual value: NOTE: Tasks Summary:
1257Attempted 288 tasks of which 285 didn't need to be rerun and all
1258succeeded. Summary: There were 3 WARNING messages shown.
1259
1260.. note::
1261
1262 The previous output example has artificial line breaks to make it
1263 more readable.
1264
1265The output describes the various problems that you can encounter along
1266with where to find the offending configuration items. You can use the
1267information in the logs to adjust your configuration files and then
1268repeat the
1269```do_kernel_configme`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configme>`__
1270and
1271```do_kernel_configcheck`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configcheck>`__
1272tasks until they produce no warnings.
1273
1274For more information on how to use the ``menuconfig`` tool, see the
1275"`Using ``menuconfig`` <#using-menuconfig>`__" section.
1276
1277Fine-Tuning the Kernel Configuration File
1278-----------------------------------------
1279
1280You can make sure the ``.config`` file is as lean or efficient as
1281possible by reading the output of the kernel configuration fragment
1282audit, noting any issues, making changes to correct the issues, and then
1283repeating.
1284
1285As part of the kernel build process, the ``do_kernel_configcheck`` task
1286runs. This task validates the kernel configuration by checking the final
1287``.config`` file against the input files. During the check, the task
1288produces warning messages for the following issues:
1289
1290- Requested options that did not make the final ``.config`` file.
1291
1292- Configuration items that appear twice in the same configuration
1293 fragment.
1294
1295- Configuration items tagged as "required" that were overridden.
1296
1297- A board overrides a non-board specific option.
1298
1299- Listed options not valid for the kernel being processed. In other
1300 words, the option does not appear anywhere.
1301
1302.. note::
1303
1304 The
1305 do_kernel_configcheck
1306 task can also optionally report if an option is overridden during
1307 processing.
1308
1309For each output warning, a message points to the file that contains a
1310list of the options and a pointer to the configuration fragment that
1311defines them. Collectively, the files are the key to streamlining the
1312configuration.
1313
1314To streamline the configuration, do the following:
1315
13161. *Use a Working Configuration:* Start with a full configuration that
1317 you know works. Be sure the configuration builds and boots
1318 successfully. Use this configuration file as your baseline.
1319
13202. *Run Configure and Check Tasks:* Separately run the
1321 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks: $ bitbake
1322 linux-yocto -c kernel_configme -f $ bitbake linux-yocto -c
1323 kernel_configcheck -f
1324
13253. *Process the Results:* Take the resulting list of files from the
1326 ``do_kernel_configcheck`` task warnings and do the following:
1327
1328 - Drop values that are redefined in the fragment but do not change
1329 the final ``.config`` file.
1330
1331 - Analyze and potentially drop values from the ``.config`` file that
1332 override required configurations.
1333
1334 - Analyze and potentially remove non-board specific options.
1335
1336 - Remove repeated and invalid options.
1337
13384. *Re-Run Configure and Check Tasks:* After you have worked through the
1339 output of the kernel configuration audit, you can re-run the
1340 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks to see the
1341 results of your changes. If you have more issues, you can deal with
1342 them as described in the previous step.
1343
1344Iteratively working through steps two through four eventually yields a
1345minimal, streamlined configuration file. Once you have the best
1346``.config``, you can build the Linux Yocto kernel.
1347
1348Expanding Variables
1349===================
1350
1351Sometimes it is helpful to determine what a variable expands to during a
1352build. You can do examine the values of variables by examining the
1353output of the ``bitbake -e`` command. The output is long and is more
1354easily managed in a text file, which allows for easy searches: $ bitbake
1355-e virtual/kernel > some_text_file Within the text file, you can see
1356exactly how each variable is expanded and used by the OpenEmbedded build
1357system.
1358
1359Working with a "Dirty" Kernel Version String
1360============================================
1361
1362If you build a kernel image and the version string has a "+" or a
1363"-dirty" at the end, uncommitted modifications exist in the kernel's
1364source directory. Follow these steps to clean up the version string:
1365
13661. *Discover the Uncommitted Changes:* Go to the kernel's locally cloned
1367 Git repository (source directory) and use the following Git command
1368 to list the files that have been changed, added, or removed: $ git
1369 status
1370
13712. *Commit the Changes:* You should commit those changes to the kernel
1372 source tree regardless of whether or not you will save, export, or
1373 use the changes: $ git add $ git commit -s -a -m "getting rid of
1374 -dirty"
1375
13763. *Rebuild the Kernel Image:* Once you commit the changes, rebuild the
1377 kernel.
1378
1379 Depending on your particular kernel development workflow, the
1380 commands you use to rebuild the kernel might differ. For information
1381 on building the kernel image when using ``devtool``, see the "`Using
1382 ``devtool`` to Patch the
1383 Kernel <#using-devtool-to-patch-the-kernel>`__" section. For
1384 information on building the kernel image when using Bitbake, see the
1385 "`Using Traditional Kernel Development to Patch the
1386 Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
1387 section.
1388
1389Working With Your Own Sources
1390=============================
1391
1392If you cannot work with one of the Linux kernel versions supported by
1393existing linux-yocto recipes, you can still make use of the Yocto
1394Project Linux kernel tooling by working with your own sources. When you
1395use your own sources, you will not be able to leverage the existing
1396kernel `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ and stabilization
1397work of the linux-yocto sources. However, you will be able to manage
1398your own Metadata in the same format as the linux-yocto sources.
1399Maintaining format compatibility facilitates converging with linux-yocto
1400on a future, mutually-supported kernel version.
1401
1402To help you use your own sources, the Yocto Project provides a
1403linux-yocto custom recipe (``linux-yocto-custom.bb``) that uses
1404``kernel.org`` sources and the Yocto Project Linux kernel tools for
1405managing kernel Metadata. You can find this recipe in the ``poky`` Git
1406repository of the Yocto Project `Source Repository <&YOCTO_GIT_URL;>`__
1407at: poky/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
1408
1409Here are some basic steps you can use to work with your own sources:
1410
14111. *Create a Copy of the Kernel Recipe:* Copy the
1412 ``linux-yocto-custom.bb`` recipe to your layer and give it a
1413 meaningful name. The name should include the version of the Yocto
1414 Linux kernel you are using (e.g. ``linux-yocto-myproject_4.12.bb``,
1415 where "4.12" is the base version of the Linux kernel with which you
1416 would be working).
1417
14182. *Create a Directory for Your Patches:* In the same directory inside
1419 your layer, create a matching directory to store your patches and
1420 configuration files (e.g. ``linux-yocto-myproject``).
1421
14223. *Ensure You Have Configurations:* Make sure you have either a
1423 ``defconfig`` file or configuration fragment files in your layer.
1424 When you use the ``linux-yocto-custom.bb`` recipe, you must specify a
1425 configuration. If you do not have a ``defconfig`` file, you can run
1426 the following: $ make defconfig After running the command, copy the
1427 resulting ``.config`` file to the ``files`` directory in your layer
1428 as "defconfig" and then add it to the
1429 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable in the
1430 recipe.
1431
1432 Running the ``make defconfig`` command results in the default
1433 configuration for your architecture as defined by your kernel.
1434 However, no guarantee exists that this configuration is valid for
1435 your use case, or that your board will even boot. This is
1436 particularly true for non-x86 architectures.
1437
1438 To use non-x86 ``defconfig`` files, you need to be more specific and
1439 find one that matches your board (i.e. for arm, you look in
1440 ``arch/arm/configs`` and use the one that is the best starting point
1441 for your board).
1442
14434. *Edit the Recipe:* Edit the following variables in your recipe as
1444 appropriate for your project:
1445
1446 - ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__: The
1447 ``SRC_URI`` should specify a Git repository that uses one of the
1448 supported Git fetcher protocols (i.e. ``file``, ``git``, ``http``,
1449 and so forth). The ``SRC_URI`` variable should also specify either
1450 a ``defconfig`` file or some configuration fragment files. The
1451 skeleton recipe provides an example ``SRC_URI`` as a syntax
1452 reference.
1453
1454 - ```LINUX_VERSION`` <&YOCTO_DOCS_REF_URL;#var-LINUX_VERSION>`__:
1455 The Linux kernel version you are using (e.g. "4.12").
1456
1457 - ```LINUX_VERSION_EXTENSION`` <&YOCTO_DOCS_REF_URL;#var-LINUX_VERSION_EXTENSION>`__:
1458 The Linux kernel ``CONFIG_LOCALVERSION`` that is compiled into the
1459 resulting kernel and visible through the ``uname`` command.
1460
1461 - ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__: The commit ID
1462 from which you want to build.
1463
1464 - ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__: Treat this variable the
1465 same as you would in any other recipe. Increment the variable to
1466 indicate to the OpenEmbedded build system that the recipe has
1467 changed.
1468
1469 - ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The default ``PV``
1470 assignment is typically adequate. It combines the
1471 ``LINUX_VERSION`` with the Source Control Manager (SCM) revision
1472 as derived from the ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__
1473 variable. The combined results are a string with the following
1474 form:
1475 3.19.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c60d32f0d770931e3cf343e2
1476 While lengthy, the extra verbosity in ``PV`` helps ensure you are
1477 using the exact sources from which you intend to build.
1478
1479 - ```COMPATIBLE_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE>`__:
1480 A list of the machines supported by your new recipe. This variable
1481 in the example recipe is set by default to a regular expression
1482 that matches only the empty string, "(^$)". This default setting
1483 triggers an explicit build failure. You must change it to match a
1484 list of the machines that your new recipe supports. For example,
1485 to support the ``qemux86`` and ``qemux86-64`` machines, use the
1486 following form: COMPATIBLE_MACHINE = "qemux86|qemux86-64"
1487
14885. *Customize Your Recipe as Needed:* Provide further customizations to
1489 your recipe as needed just as you would customize an existing
1490 linux-yocto recipe. See the "`Modifying an Existing
1491 Recipe <#modifying-an-existing-recipe>`__" section for information.
1492
1493Working with Out-of-Tree Modules
1494================================
1495
1496This section describes steps to build out-of-tree modules on your target
1497and describes how to incorporate out-of-tree modules in the build.
1498
1499Building Out-of-Tree Modules on the Target
1500------------------------------------------
1501
1502While the traditional Yocto Project development model would be to
1503include kernel modules as part of the normal build process, you might
1504find it useful to build modules on the target. This could be the case if
1505your target system is capable and powerful enough to handle the
1506necessary compilation. Before deciding to build on your target, however,
1507you should consider the benefits of using a proper cross-development
1508environment from your build host.
1509
1510If you want to be able to build out-of-tree modules on the target, there
1511are some steps you need to take on the target that is running your SDK
1512image. Briefly, the ``kernel-dev`` package is installed by default on
1513all ``*.sdk`` images and the ``kernel-devsrc`` package is installed on
1514many of the ``*.sdk`` images. However, you need to create some scripts
1515prior to attempting to build the out-of-tree modules on the target that
1516is running that image.
1517
1518Prior to attempting to build the out-of-tree modules, you need to be on
1519the target as root and you need to change to the ``/usr/src/kernel``
1520directory. Next, ``make`` the scripts: # cd /usr/src/kernel # make
1521scripts Because all SDK image recipes include ``dev-pkgs``, the
1522``kernel-dev`` packages will be installed as part of the SDK image and
1523the ``kernel-devsrc`` packages will be installed as part of applicable
1524SDK images. The SDK uses the scripts when building out-of-tree modules.
1525Once you have switched to that directory and created the scripts, you
1526should be able to build your out-of-tree modules on the target.
1527
1528Incorporating Out-of-Tree Modules
1529---------------------------------
1530
1531While it is always preferable to work with sources integrated into the
1532Linux kernel sources, if you need an external kernel module, the
1533``hello-mod.bb`` recipe is available as a template from which you can
1534create your own out-of-tree Linux kernel module recipe.
1535
1536This template recipe is located in the ``poky`` Git repository of the
1537Yocto Project `Source Repository <&YOCTO_GIT_URL;>`__ at:
1538poky/meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb
1539
1540To get started, copy this recipe to your layer and give it a meaningful
1541name (e.g. ``mymodule_1.0.bb``). In the same directory, create a new
1542directory named ``files`` where you can store any source files, patches,
1543or other files necessary for building the module that do not come with
1544the sources. Finally, update the recipe as needed for the module.
1545Typically, you will need to set the following variables:
1546
1547- ```DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-DESCRIPTION>`__
1548
1549- ```LICENSE*`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__
1550
1551- ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
1552
1553- ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__
1554
1555Depending on the build system used by the module sources, you might need
1556to make some adjustments. For example, a typical module ``Makefile``
1557looks much like the one provided with the ``hello-mod`` template: obj-m
1558:= hello.o SRC := $(shell pwd) all: $(MAKE) -C $(KERNEL_SRC) M=$(SRC)
1559modules_install: $(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install ...
1560
1561The important point to note here is the
1562```KERNEL_SRC`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_SRC>`__ variable. The
1563```module`` <&YOCTO_DOCS_REF_URL;#ref-classes-module>`__ class sets this
1564variable and the
1565```KERNEL_PATH`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_PATH>`__ variable to
1566``${STAGING_KERNEL_DIR}`` with the necessary Linux kernel build
1567information to build modules. If your module ``Makefile`` uses a
1568different variable, you might want to override the
1569```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ step, or
1570create a patch to the ``Makefile`` to work with the more typical
1571``KERNEL_SRC`` or ``KERNEL_PATH`` variables.
1572
1573After you have prepared your recipe, you will likely want to include the
1574module in your images. To do this, see the documentation for the
1575following variables in the Yocto Project Reference Manual and set one of
1576them appropriately for your machine configuration file:
1577
1578- ```MACHINE_ESSENTIAL_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RDEPENDS>`__
1579
1580- ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
1581
1582- ```MACHINE_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RDEPENDS>`__
1583
1584- ```MACHINE_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RRECOMMENDS>`__
1585
1586Modules are often not required for boot and can be excluded from certain
1587build configurations. The following allows for the most flexibility:
1588MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule" The value is
1589derived by appending the module filename without the ``.ko`` extension
1590to the string "kernel-module-".
1591
1592Because the variable is
1593```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ and not a
1594```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ variable, the build
1595will not fail if this module is not available to include in the image.
1596
1597Inspecting Changes and Commits
1598==============================
1599
1600A common question when working with a kernel is: "What changes have been
1601applied to this tree?" Rather than using "grep" across directories to
1602see what has changed, you can use Git to inspect or search the kernel
1603tree. Using Git is an efficient way to see what has changed in the tree.
1604
1605What Changed in a Kernel?
1606-------------------------
1607
1608Following are a few examples that show how to use Git commands to
1609examine changes. These examples are by no means the only way to see
1610changes.
1611
1612.. note::
1613
1614 In the following examples, unless you provide a commit range,
1615 kernel.org
1616 history is blended with Yocto Project kernel changes. You can form
1617 ranges by using branch names from the kernel tree as the upper and
1618 lower commit markers with the Git commands. You can see the branch
1619 names through the web interface to the Yocto Project source
1620 repositories at
1621 .
1622
1623To see a full range of the changes, use the ``git whatchanged`` command
1624and specify a commit range for the branch (commit\ ``..``\ commit).
1625
1626Here is an example that looks at what has changed in the ``emenlow``
1627branch of the ``linux-yocto-3.19`` kernel. The lower commit range is the
1628commit associated with the ``standard/base`` branch, while the upper
1629commit range is the commit associated with the ``standard/emenlow``
1630branch. $ git whatchanged origin/standard/base..origin/standard/emenlow
1631
1632To see short, one line summaries of changes use the ``git log`` command:
1633$ git log --oneline origin/standard/base..origin/standard/emenlow
1634
1635Use this command to see code differences for the changes: $ git diff
1636origin/standard/base..origin/standard/emenlow
1637
1638Use this command to see the commit log messages and the text
1639differences: $ git show origin/standard/base..origin/standard/emenlow
1640
1641Use this command to create individual patches for each change. Here is
1642an example that that creates patch files for each commit and places them
1643in your ``Documents`` directory: $ git format-patch -o $HOME/Documents
1644origin/standard/base..origin/standard/emenlow
1645
1646Showing a Particular Feature or Branch Change
1647---------------------------------------------
1648
1649Tags in the Yocto Project kernel tree divide changes for significant
1650features or branches. The ``git show`` tag command shows changes based
1651on a tag. Here is an example that shows ``systemtap`` changes: $ git
1652show systemtap You can use the ``git branch --contains`` tag command to
1653show the branches that contain a particular feature. This command shows
1654the branches that contain the ``systemtap`` feature: $ git branch
1655--contains systemtap
1656
1657Adding Recipe-Space Kernel Features
1658===================================
1659
1660You can add kernel features in the
1661`recipe-space <#recipe-space-metadata>`__ by using the
1662```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
1663variable and by specifying the feature's ``.scc`` file path in the
1664```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement. When you
1665add features using this method, the OpenEmbedded build system checks to
1666be sure the features are present. If the features are not present, the
1667build stops. Kernel features are the last elements processed for
1668configuring and patching the kernel. Therefore, adding features in this
1669manner is a way to enforce specific features are present and enabled
1670without needing to do a full audit of any other layer's additions to the
1671``SRC_URI`` statement.
1672
1673You add a kernel feature by providing the feature as part of the
1674``KERNEL_FEATURES`` variable and by providing the path to the feature's
1675``.scc`` file, which is relative to the root of the kernel Metadata. The
1676OpenEmbedded build system searches all forms of kernel Metadata on the
1677``SRC_URI`` statement regardless of whether the Metadata is in the
1678"kernel-cache", system kernel Metadata, or a recipe-space Metadata (i.e.
1679part of the kernel recipe). See the "`Kernel Metadata
1680Location <#kernel-metadata-location>`__" section for additional
1681information.
1682
1683When you specify the feature's ``.scc`` file on the ``SRC_URI``
1684statement, the OpenEmbedded build system adds the directory of that
1685``.scc`` file along with all its subdirectories to the kernel feature
1686search path. Because subdirectories are searched, you can reference a
1687single ``.scc`` file in the ``SRC_URI`` statement to reference multiple
1688kernel features.
1689
1690Consider the following example that adds the "test.scc" feature to the
1691build.
1692
16931. *Create the Feature File:* Create a ``.scc`` file and locate it just
1694 as you would any other patch file, ``.cfg`` file, or fetcher item you
1695 specify in the ``SRC_URI`` statement.
1696
1697 .. note::
1698
1699 - You must add the directory of the ``.scc`` file to the
1700 fetcher's search path in the same manner as you would add a
1701 ``.patch`` file.
1702
1703 - You can create additional ``.scc`` files beneath the directory
1704 that contains the file you are adding. All subdirectories are
1705 searched during the build as potential feature directories.
1706
1707 Continuing with the example, suppose the "test.scc" feature you are
1708 adding has a ``test.scc`` file in the following directory: my_recipe
1709 \| +-linux-yocto \| +-test.cfg +-test.scc In this example, the
1710 ``linux-yocto`` directory has both the feature ``test.scc`` file and
1711 a similarly named configuration fragment file ``test.cfg``.
1712
17132. *Add the Feature File to ``SRC_URI``:* Add the ``.scc`` file to the
1714 recipe's ``SRC_URI`` statement: SRC_URI_append = " file://test.scc"
1715 The leading space before the path is important as the path is
1716 appended to the existing path.
1717
17183. *Specify the Feature as a Kernel Feature:* Use the
1719 ``KERNEL_FEATURES`` statement to specify the feature as a kernel
1720 feature: KERNEL_FEATURES_append = " test.scc" The OpenEmbedded build
1721 system processes the kernel feature when it builds the kernel.
1722
1723 .. note::
1724
1725 If other features are contained below "test.scc", then their
1726 directories are relative to the directory containing the
1727 test.scc
1728 file.
diff --git a/documentation/kernel-dev/kernel-dev-concepts-appx.rst b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
new file mode 100644
index 0000000000..e8addbd307
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
@@ -0,0 +1,409 @@
1************************
2Advanced Kernel Concepts
3************************
4
5.. _kernel-big-picture:
6
7Yocto Project Kernel Development and Maintenance
8================================================
9
10Kernels available through the Yocto Project (Yocto Linux kernels), like
11other kernels, are based off the Linux kernel releases from
12` <http://www.kernel.org>`__. At the beginning of a major Linux kernel
13development cycle, the Yocto Project team chooses a Linux kernel based
14on factors such as release timing, the anticipated release timing of
15final upstream ``kernel.org`` versions, and Yocto Project feature
16requirements. Typically, the Linux kernel chosen is in the final stages
17of development by the Linux community. In other words, the Linux kernel
18is in the release candidate or "rc" phase and has yet to reach final
19release. But, by being in the final stages of external development, the
20team knows that the ``kernel.org`` final release will clearly be within
21the early stages of the Yocto Project development window.
22
23This balance allows the Yocto Project team to deliver the most
24up-to-date Yocto Linux kernel possible, while still ensuring that the
25team has a stable official release for the baseline Linux kernel
26version.
27
28As implied earlier, the ultimate source for Yocto Linux kernels are
29released kernels from ``kernel.org``. In addition to a foundational
30kernel from ``kernel.org``, the available Yocto Linux kernels contain a
31mix of important new mainline developments, non-mainline developments
32(when no alternative exists), Board Support Package (BSP) developments,
33and custom features. These additions result in a commercially released
34Yocto Project Linux kernel that caters to specific embedded designer
35needs for targeted hardware.
36
37You can find a web interface to the Yocto Linux kernels in the `Source
38Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ at
39` <&YOCTO_GIT_URL;>`__. If you look at the interface, you will see to
40the left a grouping of Git repositories titled "Yocto Linux Kernel".
41Within this group, you will find several Linux Yocto kernels developed
42and included with Yocto Project releases:
43
44- *``linux-yocto-4.1``:* The stable Yocto Project kernel to use with
45 the Yocto Project Release 2.0. This kernel is based on the Linux 4.1
46 released kernel.
47
48- *``linux-yocto-4.4``:* The stable Yocto Project kernel to use with
49 the Yocto Project Release 2.1. This kernel is based on the Linux 4.4
50 released kernel.
51
52- *``linux-yocto-4.6``:* A temporary kernel that is not tied to any
53 Yocto Project release.
54
55- *``linux-yocto-4.8``:* The stable yocto Project kernel to use with
56 the Yocto Project Release 2.2.
57
58- *``linux-yocto-4.9``:* The stable Yocto Project kernel to use with
59 the Yocto Project Release 2.3. This kernel is based on the Linux 4.9
60 released kernel.
61
62- *``linux-yocto-4.10``:* The default stable Yocto Project kernel to
63 use with the Yocto Project Release 2.3. This kernel is based on the
64 Linux 4.10 released kernel.
65
66- *``linux-yocto-4.12``:* The default stable Yocto Project kernel to
67 use with the Yocto Project Release 2.4. This kernel is based on the
68 Linux 4.12 released kernel.
69
70- *``yocto-kernel-cache``:* The ``linux-yocto-cache`` contains patches
71 and configurations for the linux-yocto kernel tree. This repository
72 is useful when working on the linux-yocto kernel. For more
73 information on this "Advanced Kernel Metadata", see the "`Working
74 With Advanced Metadata
75 (``yocto-kernel-cache``) <#kernel-dev-advanced>`__" Chapter.
76
77- *``linux-yocto-dev``:* A development kernel based on the latest
78 upstream release candidate available.
79
80.. note::
81
82 Long Term Support Initiative (LTSI) for Yocto Linux kernels is as
83 follows:
84
85 - For Yocto Project releases 1.7, 1.8, and 2.0, the LTSI kernel is
86 ``linux-yocto-3.14``.
87
88 - For Yocto Project releases 2.1, 2.2, and 2.3, the LTSI kernel is
89 ``linux-yocto-4.1``.
90
91 - For Yocto Project release 2.4, the LTSI kernel is
92 ``linux-yocto-4.9``
93
94 - ``linux-yocto-4.4`` is an LTS kernel.
95
96Once a Yocto Linux kernel is officially released, the Yocto Project team
97goes into their next development cycle, or upward revision (uprev)
98cycle, while still continuing maintenance on the released kernel. It is
99important to note that the most sustainable and stable way to include
100feature development upstream is through a kernel uprev process.
101Back-porting hundreds of individual fixes and minor features from
102various kernel versions is not sustainable and can easily compromise
103quality.
104
105During the uprev cycle, the Yocto Project team uses an ongoing analysis
106of Linux kernel development, BSP support, and release timing to select
107the best possible ``kernel.org`` Linux kernel version on which to base
108subsequent Yocto Linux kernel development. The team continually monitors
109Linux community kernel development to look for significant features of
110interest. The team does consider back-porting large features if they
111have a significant advantage. User or community demand can also trigger
112a back-port or creation of new functionality in the Yocto Project
113baseline kernel during the uprev cycle.
114
115Generally speaking, every new Linux kernel both adds features and
116introduces new bugs. These consequences are the basic properties of
117upstream Linux kernel development and are managed by the Yocto Project
118team's Yocto Linux kernel development strategy. It is the Yocto Project
119team's policy to not back-port minor features to the released Yocto
120Linux kernel. They only consider back-porting significant technological
121jumps DASH and, that is done after a complete gap analysis. The reason
122for this policy is that back-porting any small to medium sized change
123from an evolving Linux kernel can easily create mismatches,
124incompatibilities and very subtle errors.
125
126The policies described in this section result in both a stable and a
127cutting edge Yocto Linux kernel that mixes forward ports of existing
128Linux kernel features and significant and critical new functionality.
129Forward porting Linux kernel functionality into the Yocto Linux kernels
130available through the Yocto Project can be thought of as a "micro
131uprev." The many “micro uprevs” produce a Yocto Linux kernel version
132with a mix of important new mainline, non-mainline, BSP developments and
133feature integrations. This Yocto Linux kernel gives insight into new
134features and allows focused amounts of testing to be done on the kernel,
135which prevents surprises when selecting the next major uprev. The
136quality of these cutting edge Yocto Linux kernels is evolving and the
137kernels are used in leading edge feature and BSP development.
138
139Yocto Linux Kernel Architecture and Branching Strategies
140========================================================
141
142As mentioned earlier, a key goal of the Yocto Project is to present the
143developer with a kernel that has a clear and continuous history that is
144visible to the user. The architecture and mechanisms, in particular the
145branching strategies, used achieve that goal in a manner similar to
146upstream Linux kernel development in ``kernel.org``.
147
148You can think of a Yocto Linux kernel as consisting of a baseline Linux
149kernel with added features logically structured on top of the baseline.
150The features are tagged and organized by way of a branching strategy
151implemented by the Yocto Project team using the Source Code Manager
152(SCM) Git.
153
154.. note::
155
156 - Git is the obvious SCM for meeting the Yocto Linux kernel
157 organizational and structural goals described in this section. Not
158 only is Git the SCM for Linux kernel development in ``kernel.org``
159 but, Git continues to grow in popularity and supports many
160 different work flows, front-ends and management techniques.
161
162 - You can find documentation on Git at
163 ` <http://git-scm.com/documentation>`__. You can also get an
164 introduction to Git as it applies to the Yocto Project in the
165 "`Git <&YOCTO_DOCS_OM_URL;#git>`__" section in the Yocto Project
166 Overview and Concepts Manual. The latter reference provides an
167 overview of Git and presents a minimal set of Git commands that
168 allows you to be functional using Git. You can use as much, or as
169 little, of what Git has to offer to accomplish what you need for
170 your project. You do not have to be a "Git Expert" in order to use
171 it with the Yocto Project.
172
173Using Git's tagging and branching features, the Yocto Project team
174creates kernel branches at points where functionality is no longer
175shared and thus, needs to be isolated. For example, board-specific
176incompatibilities would require different functionality and would
177require a branch to separate the features. Likewise, for specific kernel
178features, the same branching strategy is used.
179
180This "tree-like" architecture results in a structure that has features
181organized to be specific for particular functionality, single kernel
182types, or a subset of kernel types. Thus, the user has the ability to
183see the added features and the commits that make up those features. In
184addition to being able to see added features, the user can also view the
185history of what made up the baseline Linux kernel.
186
187Another consequence of this strategy results in not having to store the
188same feature twice internally in the tree. Rather, the kernel team
189stores the unique differences required to apply the feature onto the
190kernel type in question.
191
192.. note::
193
194 The Yocto Project team strives to place features in the tree such
195 that features can be shared by all boards and kernel types where
196 possible. However, during development cycles or when large features
197 are merged, the team cannot always follow this practice. In those
198 cases, the team uses isolated branches to merge features.
199
200BSP-specific code additions are handled in a similar manner to
201kernel-specific additions. Some BSPs only make sense given certain
202kernel types. So, for these types, the team creates branches off the end
203of that kernel type for all of the BSPs that are supported on that
204kernel type. From the perspective of the tools that create the BSP
205branch, the BSP is really no different than a feature. Consequently, the
206same branching strategy applies to BSPs as it does to kernel features.
207So again, rather than store the BSP twice, the team only stores the
208unique differences for the BSP across the supported multiple kernels.
209
210While this strategy can result in a tree with a significant number of
211branches, it is important to realize that from the developer's point of
212view, there is a linear path that travels from the baseline
213``kernel.org``, through a select group of features and ends with their
214BSP-specific commits. In other words, the divisions of the kernel are
215transparent and are not relevant to the developer on a day-to-day basis.
216From the developer's perspective, this path is the "master" branch in
217Git terms. The developer does not need to be aware of the existence of
218any other branches at all. Of course, value exists in the having these
219branches in the tree, should a person decide to explore them. For
220example, a comparison between two BSPs at either the commit level or at
221the line-by-line code ``diff`` level is now a trivial operation.
222
223The following illustration shows the conceptual Yocto Linux kernel.
224
225In the illustration, the "Kernel.org Branch Point" marks the specific
226spot (or Linux kernel release) from which the Yocto Linux kernel is
227created. From this point forward in the tree, features and differences
228are organized and tagged.
229
230The "Yocto Project Baseline Kernel" contains functionality that is
231common to every kernel type and BSP that is organized further along in
232the tree. Placing these common features in the tree this way means
233features do not have to be duplicated along individual branches of the
234tree structure.
235
236From the "Yocto Project Baseline Kernel", branch points represent
237specific functionality for individual Board Support Packages (BSPs) as
238well as real-time kernels. The illustration represents this through
239three BSP-specific branches and a real-time kernel branch. Each branch
240represents some unique functionality for the BSP or for a real-time
241Yocto Linux kernel.
242
243In this example structure, the "Real-time (rt) Kernel" branch has common
244features for all real-time Yocto Linux kernels and contains more
245branches for individual BSP-specific real-time kernels. The illustration
246shows three branches as an example. Each branch points the way to
247specific, unique features for a respective real-time kernel as they
248apply to a given BSP.
249
250The resulting tree structure presents a clear path of markers (or
251branches) to the developer that, for all practical purposes, is the
252Yocto Linux kernel needed for any given set of requirements.
253
254.. note::
255
256 Keep in mind the figure does not take into account all the supported
257 Yocto Linux kernels, but rather shows a single generic kernel just
258 for conceptual purposes. Also keep in mind that this structure
259 represents the Yocto Project
260 Source Repositories
261 that are either pulled from during the build or established on the
262 host development system prior to the build by either cloning a
263 particular kernel's Git repository or by downloading and unpacking a
264 tarball.
265
266Working with the kernel as a structured tree follows recognized
267community best practices. In particular, the kernel as shipped with the
268product, should be considered an "upstream source" and viewed as a
269series of historical and documented modifications (commits). These
270modifications represent the development and stabilization done by the
271Yocto Project kernel development team.
272
273Because commits only change at significant release points in the product
274life cycle, developers can work on a branch created from the last
275relevant commit in the shipped Yocto Project Linux kernel. As mentioned
276previously, the structure is transparent to the developer because the
277kernel tree is left in this state after cloning and building the kernel.
278
279Kernel Build File Hierarchy
280===========================
281
282Upstream storage of all the available kernel source code is one thing,
283while representing and using the code on your host development system is
284another. Conceptually, you can think of the kernel source repositories
285as all the source files necessary for all the supported Yocto Linux
286kernels. As a developer, you are just interested in the source files for
287the kernel on which you are working. And, furthermore, you need them
288available on your host system.
289
290Kernel source code is available on your host system several different
291ways:
292
293- *Files Accessed While using ``devtool``:* ``devtool``, which is
294 available with the Yocto Project, is the preferred method by which to
295 modify the kernel. See the "`Kernel Modification
296 Workflow <#kernel-modification-workflow>`__" section.
297
298- *Cloned Repository:* If you are working in the kernel all the time,
299 you probably would want to set up your own local Git repository of
300 the Yocto Linux kernel tree. For information on how to clone a Yocto
301 Linux kernel Git repository, see the "`Preparing the Build Host to
302 Work on the
303 Kernel <#preparing-the-build-host-to-work-on-the-kernel>`__" section.
304
305- *Temporary Source Files from a Build:* If you just need to make some
306 patches to the kernel using a traditional BitBake workflow (i.e. not
307 using the ``devtool``), you can access temporary kernel source files
308 that were extracted and used during a kernel build.
309
310The temporary kernel source files resulting from a build using BitBake
311have a particular hierarchy. When you build the kernel on your
312development system, all files needed for the build are taken from the
313source repositories pointed to by the
314```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable and gathered
315in a temporary work area where they are subsequently used to create the
316unique kernel. Thus, in a sense, the process constructs a local source
317tree specific to your kernel from which to generate the new kernel
318image.
319
320The following figure shows the temporary file structure created on your
321host system when you build the kernel using Bitbake. This `Build
322Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ contains all the
323source files used during the build.
324
325Again, for additional information on the Yocto Project kernel's
326architecture and its branching strategy, see the "`Yocto Linux Kernel
327Architecture and Branching
328Strategies <#yocto-linux-kernel-architecture-and-branching-strategies>`__"
329section. You can also reference the "`Using ``devtool`` to Patch the
330Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
331Kernel Development to Patch the
332Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
333sections for detailed example that modifies the kernel.
334
335Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase
336=======================================================================================
337
338This section describes part of the kernel configuration audit phase that
339most developers can ignore. For general information on kernel
340configuration including ``menuconfig``, ``defconfig`` files, and
341configuration fragments, see the "`Configuring the
342Kernel <#configuring-the-kernel>`__" section.
343
344During this part of the audit phase, the contents of the final
345``.config`` file are compared against the fragments specified by the
346system. These fragments can be system fragments, distro fragments, or
347user-specified configuration elements. Regardless of their origin, the
348OpenEmbedded build system warns the user if a specific option is not
349included in the final kernel configuration.
350
351By default, in order to not overwhelm the user with configuration
352warnings, the system only reports missing "hardware" options as they
353could result in a boot failure or indicate that important hardware is
354not available.
355
356To determine whether or not a given option is "hardware" or
357"non-hardware", the kernel Metadata in ``yocto-kernel-cache`` contains
358files that classify individual or groups of options as either hardware
359or non-hardware. To better show this, consider a situation where the
360``yocto-kernel-cache`` contains the following files:
361yocto-kernel-cache/features/drm-psb/hardware.cfg
362yocto-kernel-cache/features/kgdb/hardware.cfg
363yocto-kernel-cache/ktypes/base/hardware.cfg
364yocto-kernel-cache/bsp/mti-malta32/hardware.cfg
365yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg
366yocto-kernel-cache/bsp/qemuarma9/hardware.cfg
367yocto-kernel-cache/bsp/mti-malta64/hardware.cfg
368yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
369yocto-kernel-cache/bsp/common-pc/hardware.cfg
370yocto-kernel-cache/bsp/common-pc-64/hardware.cfg
371yocto-kernel-cache/features/rfkill/non-hardware.cfg
372yocto-kernel-cache/ktypes/base/non-hardware.cfg
373yocto-kernel-cache/features/aufs/non-hardware.kcf
374yocto-kernel-cache/features/ocf/non-hardware.kcf
375yocto-kernel-cache/ktypes/base/non-hardware.kcf
376yocto-kernel-cache/ktypes/base/hardware.kcf
377yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf The following list
378provides explanations for the various files:
379
380- ``hardware.kcf``: Specifies a list of kernel Kconfig files that
381 contain hardware options only.
382
383- ``non-hardware.kcf``: Specifies a list of kernel Kconfig files that
384 contain non-hardware options only.
385
386- ``hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options that
387 are hardware, regardless of whether or not they are within a Kconfig
388 file specified by a hardware or non-hardware Kconfig file (i.e.
389 ``hardware.kcf`` or ``non-hardware.kcf``).
390
391- ``non-hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options
392 that are not hardware, regardless of whether or not they are within a
393 Kconfig file specified by a hardware or non-hardware Kconfig file
394 (i.e. ``hardware.kcf`` or ``non-hardware.kcf``).
395
396Here is a specific example using the
397``kernel-cache/bsp/mti-malta32/hardware.cfg``: CONFIG_SERIAL_8250
398CONFIG_SERIAL_8250_CONSOLE CONFIG_SERIAL_8250_NR_UARTS
399CONFIG_SERIAL_8250_PCI CONFIG_SERIAL_CORE CONFIG_SERIAL_CORE_CONSOLE
400CONFIG_VGA_ARB The kernel configuration audit automatically detects
401these files (hence the names must be exactly the ones discussed here),
402and uses them as inputs when generating warnings about the final
403``.config`` file.
404
405A user-specified kernel Metadata repository, or recipe space feature,
406can use these same files to classify options that are found within its
407``.cfg`` files as hardware or non-hardware, to prevent the OpenEmbedded
408build system from producing an error or warning when an option is not in
409the final ``.config`` file.
diff --git a/documentation/kernel-dev/kernel-dev-faq.rst b/documentation/kernel-dev/kernel-dev-faq.rst
new file mode 100644
index 0000000000..b852769683
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-faq.rst
@@ -0,0 +1,43 @@
1**********************
2Kernel Development FAQ
3**********************
4
5.. _kernel-dev-faq-section:
6
7Common Questions and Solutions
8==============================
9
10The following lists some solutions for common questions. How do I use my
11own Linux kernel ``.config`` file? Refer to the "`Changing the
12Configuration <#changing-the-configuration>`__" section for information.
13How do I create configuration fragments? Refer to the "`Creating
14Configuration Fragments <#creating-config-fragments>`__" section for
15information. How do I use my own Linux kernel sources? Refer to the
16"`Working With Your Own Sources <#working-with-your-own-sources>`__"
17section for information. How do I install/not-install the kernel image
18on the rootfs? The kernel image (e.g. ``vmlinuz``) is provided by the
19``kernel-image`` package. Image recipes depend on ``kernel-base``. To
20specify whether or not the kernel image is installed in the generated
21root filesystem, override ``RDEPENDS_kernel-base`` to include or not
22include "kernel-image". See the "`Using .bbappend Files in Your
23Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
24Yocto Project Development Tasks Manual for information on how to use an
25append file to override metadata. How do I install a specific kernel
26module? Linux kernel modules are packaged individually. To ensure a
27specific kernel module is included in an image, include it in the
28appropriate machine
29```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ variable.
30These other variables are useful for installing specific modules:
31```MACHINE_ESSENTIAL_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RDEPENDS>`__
32```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
33```MACHINE_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RDEPENDS>`__
34```MACHINE_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RRECOMMENDS>`__
35For example, set the following in the ``qemux86.conf`` file to include
36the ``ab123`` kernel modules with images built for the ``qemux86``
37machine: MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123" For more
38information, see the "`Incorporating Out-of-Tree
39Modules <#incorporating-out-of-tree-modules>`__" section. How do I
40change the Linux kernel command line? The Linux kernel command line is
41typically specified in the machine config using the ``APPEND`` variable.
42For example, you can add some helpful debug information doing the
43following: APPEND += "printk.time=y initcall_debug debug"
diff --git a/documentation/kernel-dev/kernel-dev-intro.rst b/documentation/kernel-dev/kernel-dev-intro.rst
new file mode 100644
index 0000000000..0f87f8cf1d
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-intro.rst
@@ -0,0 +1,178 @@
1************
2Introduction
3************
4
5.. _kernel-dev-overview:
6
7Overview
8========
9
10Regardless of how you intend to make use of the Yocto Project, chances
11are you will work with the Linux kernel. This manual describes how to
12set up your build host to support kernel development, introduces the
13kernel development process, provides background information on the Yocto
14Linux kernel `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__, describes
15common tasks you can perform using the kernel tools, shows you how to
16use the kernel Metadata needed to work with the kernel inside the Yocto
17Project, and provides insight into how the Yocto Project team develops
18and maintains Yocto Linux kernel Git repositories and Metadata.
19
20Each Yocto Project release has a set of Yocto Linux kernel recipes,
21whose Git repositories you can view in the Yocto `Source
22Repositories <&YOCTO_GIT_URL;>`__ under the "Yocto Linux Kernel"
23heading. New recipes for the release track the latest Linux kernel
24upstream developments from ` <http://www.kernel.org>`__ and introduce
25newly-supported platforms. Previous recipes in the release are refreshed
26and supported for at least one additional Yocto Project release. As they
27align, these previous releases are updated to include the latest from
28the Long Term Support Initiative (LTSI) project. You can learn more
29about Yocto Linux kernels and LTSI in the "`Yocto Project Kernel
30Development and Maintenance <#kernel-big-picture>`__" section.
31
32Also included is a Yocto Linux kernel development recipe
33(``linux-yocto-dev.bb``) should you want to work with the very latest in
34upstream Yocto Linux kernel development and kernel Metadata development.
35
36.. note::
37
38 For more on Yocto Linux kernels, see the "
39 Yocto Project Kernel Development and Maintenance
40 section.
41
42The Yocto Project also provides a powerful set of kernel tools for
43managing Yocto Linux kernel sources and configuration data. You can use
44these tools to make a single configuration change, apply multiple
45patches, or work with your own kernel sources.
46
47In particular, the kernel tools allow you to generate configuration
48fragments that specify only what you must, and nothing more.
49Configuration fragments only need to contain the highest level visible
50``CONFIG`` options as presented by the Yocto Linux kernel ``menuconfig``
51system. Contrast this against a complete Yocto Linux kernel ``.config``
52file, which includes all the automatically selected ``CONFIG`` options.
53This efficiency reduces your maintenance effort and allows you to
54further separate your configuration in ways that make sense for your
55project. A common split separates policy and hardware. For example, all
56your kernels might support the ``proc`` and ``sys`` filesystems, but
57only specific boards require sound, USB, or specific drivers. Specifying
58these configurations individually allows you to aggregate them together
59as needed, but maintains them in only one place. Similar logic applies
60to separating source changes.
61
62If you do not maintain your own kernel sources and need to make only
63minimal changes to the sources, the released recipes provide a vetted
64base upon which to layer your changes. Doing so allows you to benefit
65from the continual kernel integration and testing performed during
66development of the Yocto Project.
67
68If, instead, you have a very specific Linux kernel source tree and are
69unable to align with one of the official Yocto Linux kernel recipes, an
70alternative exists by which you can use the Yocto Project Linux kernel
71tools with your own kernel sources.
72
73The remainder of this manual provides instructions for completing
74specific Linux kernel development tasks. These instructions assume you
75are comfortable working with
76`BitBake <http://openembedded.org/wiki/Bitbake>`__ recipes and basic
77open-source development tools. Understanding these concepts will
78facilitate the process of working with the kernel recipes. If you find
79you need some additional background, please be sure to review and
80understand the following documentation:
81
82- `Yocto Project Quick Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
83
84- `Yocto Project Overview and Concepts Manual <&YOCTO_DOCS_OM_URL;>`__.
85
86- ```devtool``
87 workflow <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__
88 as described in the Yocto Project Application Development and the
89 Extensible Software Development Kit (eSDK) manual.
90
91- The "`Understanding and Creating
92 Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
93 section in the Yocto Project Development Tasks Manual.
94
95- The "`Kernel Modification
96 Workflow <#kernel-modification-workflow>`__" section.
97
98Kernel Modification Workflow
99============================
100
101Kernel modification involves changing the Yocto Project kernel, which
102could involve changing configuration options as well as adding new
103kernel recipes. Configuration changes can be added in the form of
104configuration fragments, while recipe modification comes through the
105kernel's ``recipes-kernel`` area in a kernel layer you create.
106
107This section presents a high-level overview of the Yocto Project kernel
108modification workflow. The illustration and accompanying list provide
109general information and references for further information.
110
1111. *Set up Your Host Development System to Support Development Using the
112 Yocto Project*: See the "`Setting Up the Development Host to Use the
113 Yocto Project <&YOCTO_DOCS_DEV_URL;#dev-manual-start>`__" section in
114 the Yocto Project Development Tasks Manual for options on how to get
115 a build host ready to use the Yocto Project.
116
1172. *Set Up Your Host Development System for Kernel Development:* It is
118 recommended that you use ``devtool`` and an extensible SDK for kernel
119 development. Alternatively, you can use traditional kernel
120 development methods with the Yocto Project. Either way, there are
121 steps you need to take to get the development environment ready.
122
123 Using ``devtool`` and the eSDK requires that you have a clean build
124 of the image and that you are set up with the appropriate eSDK. For
125 more information, see the "`Getting Ready to Develop Using
126 ``devtool`` <#getting-ready-to-develop-using-devtool>`__" section.
127
128 Using traditional kernel development requires that you have the
129 kernel source available in an isolated local Git repository. For more
130 information, see the "`Getting Ready for Traditional Kernel
131 Development <#getting-ready-for-traditional-kernel-development>`__"
132 section.
133
1343. *Make Changes to the Kernel Source Code if applicable:* Modifying the
135 kernel does not always mean directly changing source files. However,
136 if you have to do this, you make the changes to the files in the
137 eSDK's Build Directory if you are using ``devtool``. For more
138 information, see the "`Using ``devtool`` to Patch the
139 Kernel <#using-devtool-to-patch-the-kernel>`__" section.
140
141 If you are using traditional kernel development, you edit the source
142 files in the kernel's local Git repository. For more information, see
143 the "`Using Traditional Kernel Development to Patch the
144 Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
145 section.
146
1474. *Make Kernel Configuration Changes if Applicable:* If your situation
148 calls for changing the kernel's configuration, you can use
149 ```menuconfig`` <#using-menuconfig>`__, which allows you to
150 interactively develop and test the configuration changes you are
151 making to the kernel. Saving changes you make with ``menuconfig``
152 updates the kernel's ``.config`` file.
153
154 .. note::
155
156 Try to resist the temptation to directly edit an existing
157 .config
158 file, which is found in the Build Directory among the source code
159 used for the build. Doing so, can produce unexpected results when
160 the OpenEmbedded build system regenerates the configuration file.
161
162 Once you are satisfied with the configuration changes made using
163 ``menuconfig`` and you have saved them, you can directly compare the
164 resulting ``.config`` file against an existing original and gather
165 those changes into a `configuration fragment
166 file <#creating-config-fragments>`__ to be referenced from within the
167 kernel's ``.bbappend`` file.
168
169 Additionally, if you are working in a BSP layer and need to modify
170 the BSP's kernel's configuration, you can use ``menuconfig``.
171
1725. *Rebuild the Kernel Image With Your Changes:* Rebuilding the kernel
173 image applies your changes. Depending on your target hardware, you
174 can verify your changes on actual hardware or perhaps QEMU.
175
176The remainder of this developer's guide covers common tasks typically
177used during kernel development, advanced Metadata usage, and Yocto Linux
178kernel maintenance concepts.
diff --git a/documentation/kernel-dev/kernel-dev-maint-appx.rst b/documentation/kernel-dev/kernel-dev-maint-appx.rst
new file mode 100644
index 0000000000..4653e51eba
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-maint-appx.rst
@@ -0,0 +1,226 @@
1******************
2Kernel Maintenance
3******************
4
5Tree Construction
6=================
7
8This section describes construction of the Yocto Project kernel source
9repositories as accomplished by the Yocto Project team to create Yocto
10Linux kernel repositories. These kernel repositories are found under the
11heading "Yocto Linux Kernel" at `YOCTO_GIT_URL <&YOCTO_GIT_URL;>`__ and
12are shipped as part of a Yocto Project release. The team creates these
13repositories by compiling and executing the set of feature descriptions
14for every BSP and feature in the product. Those feature descriptions
15list all necessary patches, configurations, branches, tags, and feature
16divisions found in a Yocto Linux kernel. Thus, the Yocto Project Linux
17kernel repository (or tree) and accompanying Metadata in the
18``yocto-kernel-cache`` are built.
19
20The existence of these repositories allow you to access and clone a
21particular Yocto Project Linux kernel repository and use it to build
22images based on their configurations and features.
23
24You can find the files used to describe all the valid features and BSPs
25in the Yocto Project Linux kernel in any clone of the Yocto Project
26Linux kernel source repository and ``yocto-kernel-cache`` Git trees. For
27example, the following commands clone the Yocto Project baseline Linux
28kernel that branches off ``linux.org`` version 4.12 and the
29``yocto-kernel-cache``, which contains stores of kernel Metadata: $ git
30clone git://git.yoctoproject.org/linux-yocto-4.12 $ git clone
31git://git.yoctoproject.org/linux-kernel-cache For more information on
32how to set up a local Git repository of the Yocto Project Linux kernel
33files, see the "`Preparing the Build Host to Work on the
34Kernel <#preparing-the-build-host-to-work-on-the-kernel>`__" section.
35
36Once you have cloned the kernel Git repository and the cache of Metadata
37on your local machine, you can discover the branches that are available
38in the repository using the following Git command: $ git branch -a
39Checking out a branch allows you to work with a particular Yocto Linux
40kernel. For example, the following commands check out the
41"standard/beagleboard" branch of the Yocto Linux kernel repository and
42the "yocto-4.12" branch of the ``yocto-kernel-cache`` repository: $ cd
43~/linux-yocto-4.12 $ git checkout -b my-kernel-4.12
44remotes/origin/standard/beagleboard $ cd ~/linux-kernel-cache $ git
45checkout -b my-4.12-metadata remotes/origin/yocto-4.12
46
47.. note::
48
49 Branches in the
50 yocto-kernel-cache
51 repository correspond to Yocto Linux kernel versions (e.g.
52 "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
53
54Once you have checked out and switched to appropriate branches, you can
55see a snapshot of all the kernel source files used to used to build that
56particular Yocto Linux kernel for a particular board.
57
58To see the features and configurations for a particular Yocto Linux
59kernel, you need to examine the ``yocto-kernel-cache`` Git repository.
60As mentioned, branches in the ``yocto-kernel-cache`` repository
61correspond to Yocto Linux kernel versions (e.g. ``yocto-4.12``).
62Branches contain descriptions in the form of ``.scc`` and ``.cfg``
63files.
64
65You should realize, however, that browsing your local
66``yocto-kernel-cache`` repository for feature descriptions and patches
67is not an effective way to determine what is in a particular kernel
68branch. Instead, you should use Git directly to discover the changes in
69a branch. Using Git is an efficient and flexible way to inspect changes
70to the kernel.
71
72.. note::
73
74 Ground up reconstruction of the complete kernel tree is an action
75 only taken by the Yocto Project team during an active development
76 cycle. When you create a clone of the kernel Git repository, you are
77 simply making it efficiently available for building and development.
78
79The following steps describe what happens when the Yocto Project Team
80constructs the Yocto Project kernel source Git repository (or tree)
81found at ` <&YOCTO_GIT_URL;>`__ given the introduction of a new
82top-level kernel feature or BSP. The following actions effectively
83provide the Metadata and create the tree that includes the new feature,
84patch, or BSP:
85
861. *Pass Feature to the OpenEmbedded Build System:* A top-level kernel
87 feature is passed to the kernel build subsystem. Normally, this
88 feature is a BSP for a particular kernel type.
89
902. *Locate Feature:* The file that describes the top-level feature is
91 located by searching these system directories:
92
93 - The in-tree kernel-cache directories, which are located in the
94 ```yocto-kernel-cache`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/bsp>`__
95 repository organized under the "Yocto Linux Kernel" heading in the
96 `Yocto Project Source
97 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
98
99 - Areas pointed to by ``SRC_URI`` statements found in kernel recipes
100
101 For a typical build, the target of the search is a feature
102 description in an ``.scc`` file whose name follows this format (e.g.
103 ``beaglebone-standard.scc`` and ``beaglebone-preempt-rt.scc``):
104 bsp_root_name-kernel_type.scc
105
1063. *Expand Feature:* Once located, the feature description is either
107 expanded into a simple script of actions, or into an existing
108 equivalent script that is already part of the shipped kernel.
109
1104. *Append Extra Features:* Extra features are appended to the top-level
111 feature description. These features can come from the
112 ```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
113 variable in recipes.
114
1155. *Locate, Expand, and Append Each Feature:* Each extra feature is
116 located, expanded and appended to the script as described in step
117 three.
118
1196. *Execute the Script:* The script is executed to produce files
120 ``.scc`` and ``.cfg`` files in appropriate directories of the
121 ``yocto-kernel-cache`` repository. These files are descriptions of
122 all the branches, tags, patches and configurations that need to be
123 applied to the base Git repository to completely create the source
124 (build) branch for the new BSP or feature.
125
1267. *Clone Base Repository:* The base repository is cloned, and the
127 actions listed in the ``yocto-kernel-cache`` directories are applied
128 to the tree.
129
1308. *Perform Cleanup:* The Git repositories are left with the desired
131 branches checked out and any required branching, patching and tagging
132 has been performed.
133
134The kernel tree and cache are ready for developer consumption to be
135locally cloned, configured, and built into a Yocto Project kernel
136specific to some target hardware.
137
138.. note::
139
140 - The generated ``yocto-kernel-cache`` repository adds to the kernel
141 as shipped with the Yocto Project release. Any add-ons and
142 configuration data are applied to the end of an existing branch.
143 The full repository generation that is found in the official Yocto
144 Project kernel repositories at
145 `http://git.yoctoproject.org <&YOCTO_GIT_URL;>`__ is the
146 combination of all supported boards and configurations.
147
148 - The technique the Yocto Project team uses is flexible and allows
149 for seamless blending of an immutable history with additional
150 patches specific to a deployment. Any additions to the kernel
151 become an integrated part of the branches.
152
153 - The full kernel tree that you see on ` <&YOCTO_GIT_URL;>`__ is
154 generated through repeating the above steps for all valid BSPs.
155 The end result is a branched, clean history tree that makes up the
156 kernel for a given release. You can see the script (``kgit-scc``)
157 responsible for this in the
158 ```yocto-kernel-tools`` <&YOCTO_GIT_URL;/cgit.cgi/yocto-kernel-tools/tree/tools>`__
159 repository.
160
161 - The steps used to construct the full kernel tree are the same
162 steps that BitBake uses when it builds a kernel image.
163
164Build Strategy
165==============
166
167Once you have cloned a Yocto Linux kernel repository and the cache
168repository (``yocto-kernel-cache``) onto your development system, you
169can consider the compilation phase of kernel development, which is
170building a kernel image. Some prerequisites exist that are validated by
171the build process before compilation starts:
172
173- The ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ points to the
174 kernel Git repository.
175
176- A BSP build branch with Metadata exists in the ``yocto-kernel-cache``
177 repository. The branch is based on the Yocto Linux kernel version and
178 has configurations and features grouped under the
179 ``yocto-kernel-cache/bsp`` directory. For example, features and
180 configurations for the BeagleBone Board assuming a
181 ``linux-yocto_4.12`` kernel reside in the following area of the
182 ``yocto-kernel-cache`` repository: yocto-kernel-cache/bsp/beaglebone
183
184 .. note::
185
186 In the previous example, the "yocto-4.12" branch is checked out in
187 the
188 yocto-kernel-cache
189 repository.
190
191The OpenEmbedded build system makes sure these conditions exist before
192attempting compilation. Other means, however, do exist, such as as
193bootstrapping a BSP.
194
195Before building a kernel, the build process verifies the tree and
196configures the kernel by processing all of the configuration "fragments"
197specified by feature descriptions in the ``.scc`` files. As the features
198are compiled, associated kernel configuration fragments are noted and
199recorded in the series of directories in their compilation order. The
200fragments are migrated, pre-processed and passed to the Linux Kernel
201Configuration subsystem (``lkc``) as raw input in the form of a
202``.config`` file. The ``lkc`` uses its own internal dependency
203constraints to do the final processing of that information and generates
204the final ``.config`` file that is used during compilation.
205
206Using the board's architecture and other relevant values from the
207board's template, kernel compilation is started and a kernel image is
208produced.
209
210The other thing that you notice once you configure a kernel is that the
211build process generates a build tree that is separate from your kernel's
212local Git source repository tree. This build tree has a name that uses
213the following form, where ``${MACHINE}`` is the metadata name of the
214machine (BSP) and "kernel_type" is one of the Yocto Project supported
215kernel types (e.g. "standard"): linux-${MACHINE}-kernel_type-build
216
217The existing support in the ``kernel.org`` tree achieves this default
218functionality.
219
220This behavior means that all the generated files for a particular
221machine or BSP are now in the build tree directory. The files include
222the final ``.config`` file, all the ``.o`` files, the ``.a`` files, and
223so forth. Since each machine or BSP has its own separate `Build
224Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in its own separate
225branch of the Git repository, you can easily switch between different
226builds.
diff --git a/documentation/kernel-dev/kernel-dev.rst b/documentation/kernel-dev/kernel-dev.rst
new file mode 100644
index 0000000000..734ed1881c
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev.rst
@@ -0,0 +1,14 @@
1=============================================
2Yocto Project Linux Kernel Development Manual
3=============================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 kernel-dev-intro
10 kernel-dev-common
11 kernel-dev-advanced
12 kernel-dev-concepts-appx
13 kernel-dev-maint-appx
14 kernel-dev-faq