path: root/documentation/kernel-dev/kernel-dev-common.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

<chapter id='kernel-dev-common'>

<title>Common Tasks</title>

<para>
    This chapter presents several common tasks that are performed when you
    work with the Yocto Project Linux kernel.
    These tasks include preparing a layer, modifying an existing recipe,
    iterative development, working with your own sources, and incorporating
    out-of-tree modules.
</para>

    <section id='preparing-a-layer'>
        <title>Preparing a Layer</title>

        <para>
            Customizing kernel recipes is best done in your own custom layer.
            Your layer will have its own BitBake append files
            (<filename>.bbappend</filename>) and provide a convenient
            mechanism to create your own recipe files (<filename>.bb</filename>).
        </para>

        <para>
            This guide is written with the assumption that you will be working
            from within your custom layer that is independent from layers
            released with the Yocto Project.
            For details on how to create and work with layers, see the following
            sections:
            <itemizedlist>
                <listitem><para>"<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>" for
                    general information on layers and how to create them.</para></listitem>
                <listitem><para>"<ulink url='&YOCTO_DOCS_DEV_URL;#get-your-layer-setup-for-the-build'>Get Your Layer Setup for the Build</ulink>" for
                    specific instructions on setting up a layer for kernel
                    development.</para></listitem>
            </itemizedlist>
        </para>
    </section>

    <section id='modifying-an-existing-recipe'>
        <title>Modifying an Existing Recipe</title>

        <para>
            In many cases, you can customize an existing linux-yocto recipe to
            meet the needs of your project.
            Each release of the Yocto Project provides a few Linux
            kernel recipes from which you can choose.
            These are located in the
            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>
            in <filename>meta/recipes-kernel/linux</filename>.
        </para>

        <para>
            To begin with, create a minimal, custom layer from which you
            can work.
            See the
            "<ulink url='&YOCTO_DOCS_DEV_URL;#get-your-layer-setup-for-the-build'>Get Your Layer Setup for the Build</ulink>" section
            of the Yocto Project Development Manual for information.
        </para>

        <para>
            Next comes the <filename>.bbappend</filename> file.
            You create this file in your custom layer.
            You also name it accordingly based on the linux-yocto recipe
            you are using.
            For example, if you are modifying the
            <filename>meta/recipes-kernel/linux/linux-yocto_3.4.bb</filename>
            recipe, the append file will typical be located as follows
            within your custom layer:
            <literallayout class='monospaced'>
     &lt;your-layer&gt;/recipes-kernel/linux/linux-yocto_3.4.bbappend
            </literallayout>
            The append file should initially contain the following text:
            <literallayout class='monospaced'>
     FILESEXTRAPATHS := "${THISDIR}/${PN}"
            </literallayout>
            The path <filename>${THISDIR}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename> expands
            to "linux-yocto" in the current directory for this example.
            If you add any new files that modify the kernel recipe, you need
            to place them in your layer in the following area:
            <literallayout class='monospaced'>
     &lt;your-layer&gt;/recipes-kernel/linux/linux-yocto/
            </literallayout>
            <note>If you are working on a new machine Board Support Package
                (BSP), be sure to refer to the
                <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.
            </note>
        </para>

        <para>
            Original text:
            <literallayout class='monospaced'>
In many cases, you can customize an existing linux-yocto recipe to meet the
needs of your project. Each release of the Yocto Project provides a few Linux
kernel recipes to choose from. To get started, prepare a layer and a bbappend
file corresponding to the recipe you wish to modify.

See [New example in Dev Manual] for instructions to create a minimal layer.

The bbappend will typical be located at the following location relative to the
layer (if modifying the linux-yocto_3.4.bb recipe):

	recipes-kernel/linux/linux-yocto_3.4.bbappend

And should contain the following text initially:

	FILESEXTRAPATHS := "${THISDIR}/${PN}"

The ${PN} will expand to "linux-yocto" in this example. Any new files added to
modify the recipe should then be added to the following path within the layer:

	recipes-kernel/linux/linux-yocto/

NOTE: If you are working on a new machine BSP, be sure to refer to the Yocto
Project Board Support Package Developer's Guide.
            </literallayout>
        </para>

        <section id='applying-patches'>
            <title>Applying Patches</title>

            <para>
                If you have a single patch or a small series of patches
                that you want to apply to the Linux kernel source, you
                can do so just as you would with any other recipe.
                You first copy the patches to the path added to
                <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
                in your <filename>.bbappend</filename> file as described in
                the previous section, and then reference them in
                <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
                statements.
            </para>

            <para>
                For example, you can apply a three-patch series by adding the
                following lines to your linux-yocto <filename>.bbappend</filename>
                file in your layer:
                <literallayout class='monospaced'>
     SRC_URI += "file://0001-first-change.patch"
     SRC_URI += "file://0002-first-change.patch"
     SRC_URI += "file://0003-first-change.patch"
                </literallayout>
                The next time you run BitBake to build the Linux kernel, BitBake
                detects the change in the recipe and fetches and applies the patches
                before building the kernel.
            </para>

            <para>
                Original Text:
                <literallayout class='monospaced'>
If you have a patch, or a small series of patches, to apply to the Linux kernel
source, you can do so just as you would with any other recipe. You first copy
the patches to the path added to FILESEXTRAPATHS in the bbappend file as
described in 2.2 and then reference them in the SRC_URI.

For example, you can apply a three patch series by adding the following lines to
your linux-yocto bbappend file in your layer:

	SRC_URI += "file://0001-first-change.patch"
	SRC_URI += "file://0002-first-change.patch"
	SRC_URI += "file://0003-first-change.patch"

At the next build, bitbake will detect the change in the recipe and fetch and
apply the patches before rebuilding the Linux kernel.
                </literallayout>
            </para>
        </section>

        <section id='changing-the-configuration'>
            <title>Changing the Configuration</title>

            <para>
                You can make wholesale or incremental changes to the Linux
                kernel <filename>.config</filename> file by including a
                <filename>defconfig</filename> or by specifying
                configuration fragments in the <filename>SRC_URI</filename>.
            </para>

            <para>
                If you have a complete Linux kernel <filename>.config</filename>
                file you want to use, copy it to the
                <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink><filename>}</filename>
                directory within your layer and name it "defconfig".
                Then, add the following line to your linux-yocto
                <filename>.bbappend</filename> file in your layer:
                <literallayout class='monospaced'>
     SRC_URI += "file://defconfig"
                </literallayout>
            </para>

            <para>
                Generally speaking, the preferred approach is to determine the
                incremental change you want to make and add that as a fragment.
                For example, if you wanted to add support for a basic serial
                console, create a file named <filename>8250.cfg</filename> in the
                <filename>${FILES}</filename> directory with the following
                content (without indentation):
                <literallayout class='monospaced'>
     CONFIG_SERIAL_8250=y
     CONFIG_SERIAL_8250_CONSOLE=y
     CONFIG_SERIAL_8250_PCI=y
     CONFIG_SERIAL_8250_NR_UARTS=4
     CONFIG_SERIAL_8250_RUNTIME_UARTS=4
     CONFIG_SERIAL_CORE=y
     CONFIG_SERIAL_CORE_CONSOLE=y
                </literallayout>
                Next, include this configuration fragment in a
                <filename>SRC_URI</filename> statement in your
                <filename>.bbappend</filename> file:
                <literallayout class='monospaced'>
     SRC_URI += "file://8250.cfg"
                </literallayout>
                The next time you run BitBake to build the Linux kernel, BitBake
                detects the change in the recipe and fetches and applies the
                new configuration before building the kernel.
            </para>

            <para>
                Original Text:
                <literallayout class='monospaced'>
Making wholesale or incremental changes to the Linux kernel config can be made
by including a defconfig or configuration fragments in the SRC_URI.

If you have a complete Linux kernel .config file you want to use, copy it as
"defconfig" to the ${FILES} directory and add the following line to your
linux-yocto bbappend file in your layer:

	SRC_URI += "file://defconfig"

Generally speaking, the preferred approach is to determine the incremental
change you want to make and add that as a fragment. For example, if you wanted
to add support for a basic serial console, create a file named "8250.cfg" in the
${FILES} directory with the following content (without indentation):

	CONFIG_SERIAL_8250=y
	CONFIG_SERIAL_8250_CONSOLE=y
	CONFIG_SERIAL_8250_PCI=y
	CONFIG_SERIAL_8250_NR_UARTS=4
	CONFIG_SERIAL_8250_RUNTIME_UARTS=4
	CONFIG_SERIAL_CORE=y
	CONFIG_SERIAL_CORE_CONSOLE=y

Then include this configuration fragment in the SRC_URI:

	SRC_URI += "file://8250.cfg"

At the next build, bitbake will detect the change in the recipe and fetch and
apply the new configuration before rebuilding the Linux kernel.
                 </literallayout>
            </para>
        </section>
    </section>

    <section id='iterative-development'>
        <title>Iterative Development</title>

        <para>
            If you do not have existing patches or configuration files,
            you can iteratively generate them from within the BitBake build
            environment as described within this section.
            During an iterative workflow, running a previously completed BitBake
            task causes BitBake to invalidate the tasks that follow that
            task in the build sequence.
            This invalidation causes the tasks to rebuild the next
            time you invoke the build.
        </para>

        <para>
            As you read this section, be sure to substitute "linux-yocto" with
            the name of the Linux kernel recipe with which you are working.
        </para>

        <para>
            Original Text:
            <literallayout class='monospaced'>
If you do not have existing patches or configuration files, you can easily
generate them from within the bitbake build environment, as will be described
below. As you do, running previously completed bitbake tasks will cause bitbake
to invalidate the tasks that follow them in the build sequence, causing them to
rebuild at the next invocation of the build. Throughout this section, be sure to
substitute "linux-yocto" with the name of the Linux kernel recipe you are
working with.
             </literallayout>
        </para>

        <section id='generating-configuration-files'>
            <title>Generating Configuration Files</title>

            <para>
                You can manipulate the <filename>.config</filename> file
                used to build a linux-yocto recipe with the
                <filename>menuconfig</filename> command as follows:
                <literallayout class='monospaced'>
     $ bitbake linux-yocto -c menuconfig
                </literallayout>
                This command starts the Linux kernel configuration tool,
                which allows you to prepare a new
                <filename>.config</filename> file for the build.
                When you exit the tool, be sure to save your changes
                at the prompt.
            </para>

            <para>
                The resulting <filename>.config</filename> file is
                located in
                <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename> under the
                <filename>linux-${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink><filename>}-${KTYPE}-build</filename> directory.
                You can use the entire <filename>.config</filename> file as the
                <filename>defconfig</filename> file as described in the
                "<link linkend='changing-the-configuration'>Changing the Configuration</link>" section.
            </para>

            <para>
                A better method is to create a configuration fragment using the
                differences between two configuration files: one previously
                created and saved, and one freshly created using the
                <filename>menuconfig</filename> tool.
            </para>

            <para>
                To create a configuration fragment using this method, follow
                these steps:
                <orderedlist>
                    <listitem><para>Complete a build at least through the kernel
                        configuration task as follows:
                        <literallayout class='monospaced'>
     $ bitbake linux-yocto -c kernel_configme -f
                        </literallayout></para></listitem>
                    <listitem><para>Create a renamed copy of the resulting
                        <filename>.config</filename> file (e.g.
                        <filename>config.orig</filename>).
                        </para></listitem>
                    <listitem><para>Run the <filename>menuconfig</filename>
                        command:
                        <literallayout class='monospaced'>
     $ bitbake linux-yocto -c menuconfig
                        </literallayout></para></listitem>
                    <listitem><para>Prepare a configuration fragment from
                        using the differences between the two files.
                        </para></listitem>
                </orderedlist>
            </para>

            <para>
                Ultimately, the configuration fragment file needs to be a
                list of Linux kernel <filename>CONFIG_</filename> assignments.
                It cannot be in <filename>diff</filename> format.
                Here is an example of a command that creates your
                configuration fragment file.
                Regardless of the exact command you use, plan on reviewing
                the output as you can usually remove some of the defaults:
                <literallayout class='monospaced'>
     $ diff -Nurp config.orig .config | sed -n "s/^\+//p" > frag.cfg
                </literallayout>
                See the "<link linkend='changing-the-configuration'>Changing the Configuration</link>"
                section for information on how to use the output as a
                configuration fragment.
                <note>
                    You can also use this method to create configuration
                    fragments for a BSP.
                    See the "<link linkend='bsp-descriptions'>BSP Descriptions</link>"
                    for more information.
                </note>
            </para>

            <para>
                The kernel tools also provide configuration validation.
                You can use these tools to produce warnings for when a
                requested configuration does not appear in the final
                <filename>.config</filename> file or when you override a
                policy configuration in a hardware configuration fragment.
                Following is the command that runs these tools and some
                sample output:
                <literallayout class='monospaced'>
     $ bitbake linux-yocto -c kernel_configcheck -f

     ...

     NOTE: validating kernel configuration
     This BSP sets 3 invalid/obsolete kernel options.
     These config options are not offered anywhere within this kernel.
     The full list can be found in your kernel src dir at:
     meta/cfg/standard/mybsp/invalid.cfg

     This BSP sets 21 kernel options that are possibly non-hardware related.
     The full list can be found in your kernel src dir at:
     meta/cfg/standard/mybsp/specified_non_hdw.cfg

     WARNING: There were 2 hardware options requested that do not
              have a corresponding value present in the final ".config" file.
              This probably means you aren't getting the config you wanted.
              The full list can be found in your kernel src dir at:
              meta/cfg/standard/mybsp/mismatch.cfg
                </literallayout>
            </para>

            <para>
                The output describes the various problems that you can
                encounter along with where to find the offending configuration
                items.
                You can use the information in the logs to adjust your
                configuration files and then repeat the
                <filename>kernel_configme</filename> and
                <filename>kernel_configcheck</filename> commands until
                they produce no warnings.
            </para>

            <para>
                Original Text:
                <literallayout class='monospaced'>
You can manipulate the config used to build a linux-yocto recipe with the
"menuconfig" command.

	$ bitbake linux-yocto -c menuconfig

This will start the Linux kernel configuration tool, allowing you to prepare a
new .config for the build. When you exit, be sure to save the changes when
prompted. The resulting .config file will be located in the ${WORKDIR} under
the linux-${MACHINE}-${KTYPE}-build directory. You can use this in its entirety
as the defconfig file described in 2.2.2.

Better still, would be to use the "menuconfig" command and take the difference
of the new .config file with the previous one to generate a configuration
fragment. To do this, be sure to complete a build at least through the kernel
configuration task:

	$ bitbake linux-yocto -c kernel_configme -f

Then make a copy of the .config file, calling it "config.orig", for example. Run
the "menuconfig" command as described above. Finally, prepare a configuration
fragment from the difference between the files. Ultimately you want a list of
Linux kernel CONFIG_ assignments, and not something in diff format. Something
like the following will do that automatically (but plan on reviewing the output
as you can usually remove some of the defaults):

	$ diff -Nurp config.orig .config | sed -n "s/^\+//p" > frag.cfg

You can use the output as a configuration fragment described in 2.2.2. This
method can also be used to define a BSP configuration fragment (See 3.3.5).

The Yocto Project kernel tools provide some configuration validation tools which
will warn when a configuration you requested does not appear in the final
config, or when you override a policy configuration in a hardware configuration
fragment. You can run these tools with the following command:

	$ bitbake linux-yocto -c kernel_configcheck -f

	...

	NOTE: validating kernel configuration
	This BSP sets 3 invalid/obsolete kernel options.
	These config options are not offered anywhere within this kernel.
	The full list can be found in your kernel src dir at:
	meta/cfg/standard/mybsp/invalid.cfg

	This BSP sets 21 kernel options that are possibly non-hardware related.
	The full list can be found in your kernel src dir at:
	meta/cfg/standard/mybsp/specified_non_hdw.cfg

	WARNING: There were 2 hardware options requested that do not
	         have a corresponding value present in the final ".config" file.
	         This probably means you aren't getting the config you wanted.
	The full list can be found in your kernel src dir at:
	meta/cfg/standard/mybsp/mismatch.cfg

The various problems that you can encounter are described in the output along
with where to find the offending configuration items. You can use these logs to
adjust your configuration files and repeat the "kernel_configme" and
"kernel_configcheck" commands until no warnings are produced.
                </literallayout>
            </para>
        </section>

        <section id='modifying-source-code'>
            <title>Modifying Source Code</title>

            <para>
            You can experiment with source code changes and create a simple patch without
leaving the bitbake environment. To get started, be sure to complete a build at
least through the kernel configuration task:

	$ bitbake linux-yocto -c kernel_configme -f

This step will ensure you have the sources prepared and the configuration
completed. You will find the sources in the ${WORKDIR}/linux directory.

You can edit the sources as you would any other Linux source tree, but keep in
mind that your changes will be lost if you trigger the fetch task for the
recipe. Avoid this by not issuing the "cleanall" or "cleansstate", or forcing
the "fetch" command. Also be sure not to modify the recipe itself while working
with temporary changes or bitbake may run the fetch command (depending on the
changes to the recipe).

To test your temporary changes, instruct bitbake to run the compile again. The
-f option forces the command to run again even though bitbake may think it has
already done so:

	$ bitbake linux-yocto -c compile -f

If the compile fails, you can update the sources and repeat the compile
command. Once it compiles successfully, you can inspect and test the resulting
build (kernel, modules, etc.) from the build directory at
${WORKDIR}/linux-${MACHINE}-${KTYPE}-build. Alternatively, you can run the
deploy command to place the kernel image in the tmp/deploy/images directory:

	$ bitbake linux-yocto -c deploy

And of course, you can run through all the remaining installation and packaging
steps by issuing:

	$ bitbake linux-yocto

For rapid iterative development, the edit-compile-repeat loop is preferable to
rebuilding the entire recipe as the installation and packaging tasks are very
time consuming.

Once you are happy with your modifications, you can make these permanent by
generating patches and applying them to the SRC_URI as described in section
2.2.1 Applying Patches. If you are not familiar with generating patches, refer
to the Yocto Project Development Manual, section 5.7.3 Creating the Patch.
            </para>
        </section>
    </section>

    <section id='working-with-your-own-sources'>
        <title>Working With Your Own Sources</title>

        <para>
        If you find yourself unable to work with one of the Linux kernel versions
supported by existing linux-yocto recipes, you can still make use of the Yocto
Project Linux kernel tooling while working with your own sources. You will not
be able to leverage the existing meta-data and stabilization work of the
linux-yocto sources, but you will be able to manage your own meta-data in the
same format as the linux-yocto sources which will facilitate converging with
linux-yocto on a future mutually-supported kernel version.

The linux-yocto-custom recipe, located in the poky repository at:

	meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb

is provided as an example recipe which uses kernel.org sources and the Yocto
Project Linux kernel tools for managing meta-data. To get started, copy this
recipe to your layer and provide it with a meaningful name, such as
linux-yocto-myproject_3.5.bb, where 3.5 is the base version of the Linux kernel
you will be working with. In the same directory, create a matching directory,
e.g. linux-yocto-myproject to store your patches and configuration files.

Edit the following variables in the recipe as appropriate for your project:

	o SRC_URI
	o LINUX_VERSION
	o LINUX_VERSION_EXTENSION
	o SRCREV
	o PR
	o PV
	o COMPATIBLE_MACHINE

The SRC_URI should be a git repository, using one of the supported git fetcher
protocols (file, git, http, etc.). The skeleton recipe provides an example
SRC_URI as a syntax reference.

Set LINUX_VERSION to the Linux kernel version you are using, such as "3.6.3".

LINUX_VERSION_EXTENSION is used to define the Linux kernel CONFIG_LOCALVERSION
which will be compiled in to the resulting kernel and visible via the uname
command.

Set SRCREV to the commit ID you wish to build from.

Treat the PR as you would the PR of any other recipe. Increment it to indicate
to the build system that the recipe has changed.

The default PV assignment is typically adequate. It combines the LINUX_VERSION
with the SCM revision (from the SRCPV variable) and results in a string
something like:

"3.4.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c60d32f0d770931e3cf343e2"

While lengthy, this extra verbosity helps ensure you are building from the exact
sources you intend.

Finally, the default COMPATIBLE_MACHINE assignment for linux-yocto-custom is set
to a regular expression matching only the empty string, "(^$)". This will
trigger an explicit build failure. You must change it to match a list of the
machines supported by your new recipe, such as "(qemux86|qemux86-64)"

With that in place, you can continue to customize this recipe as you would the
existing linux-yocto recipes. See Section 2.2 Modifying an Existing Recipe for
details.
        </para>
    </section>

    <section id='incorporating-out-of-tree-modules'>
        <title>Incorporating Out-of-Tree Modules</title>

        <para>
        While it is always preferable to work with sources integrated into the Linux
kernel sources, if you have need of an external kernel module, the hello-mod
recipe is available as a template to create your own out-of-tree Linux kernel
module recipe. It is available in the poky repository at:

	meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb

To get started, copy this recipe to your layer and provide it with a meaningful name, such as
mymodule_1.0.bb. In the same directory, create a directory named "files" where
you can store any source files, patches, or other files necessary for building
the module which do not come with the sources. Finally, update the recipe as
appropriate for the module. Typically you will need to set the following
variables:

	o DESCRIPTION
	o LICENSE*
	o SRC_URI
	o PV

Depending on the build system used by the module sources, you may need to make
adjustments. For example, a typical module Makefile will look much like that
provided with hello-mod:

	obj-m := hello.o

	SRC := $(shell pwd)

	all:
		$(MAKE) -C $(KERNEL_SRC) M=$(SRC)

	modules_install:
		$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install
	...

The important point to note here is the KERNEL_SRC variable. The module bbclass
class sets this, as well as KERNEL_PATH, to ${STAGING_KERNEL_DIR} with the
necessary Linux kernel build information to build modules. If your module
Makefile uses a different variable, you may want to override the do_compile()
step, or create a patch to the Makefile to work with the more typical KERNEL_SRC
or KERNEL_PATH variables.

After you have prepared the recipe, you will likely want to include the module
in your images. To do this, see the documentation for the following variables in
the Yocto Project Reference Manual and set one of them as appropriate in your
machine config file:

	MACHINE_ESSENTIAL_EXTRA_RDEPENDS
	MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
	MACHINE_EXTRA_RDEPENDS
	MACHINE_EXTRA_RRECOMMENDS

As modules are often not required for boot and may be excluded from certain
build configurations, the following allows for the most flexibility:

	MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule"

Where the value is derived by appending the module filename without the .ko
extension to the string "kernel-module-".

As it is an RRECOMMENDS (and not an RDEPENDS) variable, the build will not fail
if this module is not available to include in the image.
        </para>
    </section>
</chapter>
<!--
vim: expandtab tw=80 ts=4
-->