path: root/documentation/kernel-dev/kernel-dev-advanced.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

<chapter id='kernel-dev-advanced'>

<title>Working with Advanced Metadata</title>

    <para>
        In addition to configuration fragments and patches, the
        Yocto Project kernel tools support rich metadata that you can
        use to define complex policies and BSP support.
        The purpose of the metadata and the tools to manage it, known as
        the kern-tools (<filename>kern-tools-native_git.bb</filename>), is
        to assist in managing the complexity of the configuration and sources
        in support of multiple Board Support Packages (BSPs) and Linux kernel
        types.
    </para>

    <para>
        In particular, the kernel tools allow you to specify only what you
        must, and nothing more.
        Where a complete Linux kernel <filename>.config</filename> includes
        all the automatically selected <filename>CONFIG</filename> options,
        the configuration fragments only need to contain the highest level
        visible <filename>CONFIG</filename> options as presented by the Linux
        kernel <filename>menuconfig</filename> system.
        This reduces your maintenance effort and allows you
        to further separate your configuration in ways that make sense for
        your project.
        A common split is policy and hardware.
        For example, all your kernels might support
        the <filename>proc</filename> and <filename>sys</filename> filesystems,
        but only specific boards will require sound, USB, or specific drivers.
        Specifying these individually allows you to aggregate them
        together as needed, but maintain them in only one place.
        Similar logic applies to source changes.
    </para>

    <para>
        Original Text:
        <literallayout class='monospaced'>
In addition to configuration fragments and patches, the Yocto Project kernel
tools support rich metadata which you can use to define complex policies and
BSP support. The purpose of the metadata and the tools to manage it, known as
the kern-tools (kern-tools-native_git.bb), is to assist in managing the
complexity of the configuration and sources in support of multiple BSPs and
Linux kernel types.

In particular, the kernel tools allow you to specify only what you must, and
nothing more.  Where a complete Linux kernel .config includes all the
automatically selected CONFIG options, the configuration fragments only need to
contain the highest level visible CONFIG options as presented by the Linux
kernel menuconfig system. This reduces your maintenance effort and allows you
to further separate your configuration in ways that make sense for your project.
A common split is policy and hardware. For example, all your kernels may support
the proc and sys filesystems, but only specific boards will require sound, usb,
or specific drivers. Specifying these individually allows you to aggregate them
together as needed, but maintain them in only one place. Similar logic applies
to source changes.
        </literallayout>
    </para>

<section id='using-metadata-in-a-recipe'>
    <title>Using Metadata in a Recipe</title>

    <para>
        The metadata provided with any linux-yocto style Linux kernel sources
        must define a BSP that corresponds to the definition laid out in the
        recipe.
        A BSP consists of an aggregation of kernel policy and hardware specific
        feature enablement.
        This can be influenced from within the recipe.
    </para>

    <para>
        Every linux-yocto style recipe must define the following variable:
        <literallayout class='monospaced'>
     KMACHINE
        </literallayout>
        <filename>KMACHINE</filename> is typically set to the same value as
        used within the recipe-space BSP definition, such as "routerstationpro"
        or "fri2".
        However, multiple BSPs can reuse the same <filename>KMACHINE</filename>
        name if they are built using the same BSP description.
        See section 3.3.5 for more information.
        The <filename>meta-intel</filename> "fri2" and "fri2-noemgd" are good
        examples of such a situation where each specifies
        <filename>KMACHINE</filename> as "fri2".
    </para>

    <para>
        They may optionally define the following variables:
        <literallayout class='monospaced'>
     KBRANCH
     KERNEL_FEATURES
     KBRANCH_DEFAULT
     LINUX_KERNEL_TYPE
        </literallayout>
        <filename>KBRANCH_DEFAULT</filename> defines the default source branch
        within the Linux kernel source repository to be used to build the
        Linux kernel.
        It is used as the default value for <filename>KBRANCH</filename> which
        may define an alternate branch, typically with a machine override,
        such as:
        <literallayout class='monospaced'>
     KBRANCH_fri2 = "standard/fri2"
        </literallayout>
        Unless you specify otherwise, <filename>KBRANCH_DEFAULT</filename>
        is initialized to "master".
    </para>

    <para>
        <filename>LINUX_KERNEL_TYPE</filename> defines the kernel type to be
        used in assembling the configuration and defaults to "standard"
        if you do not specify otherwise.
        Together with <filename>KMACHINE</filename>, this defines the search
        arguments used by the Yocto Project Linux kernel tools to find the
        appropriate description within the metadata with which to build out
        the sources and configuration.
        The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
        kernel types.
        See section 3.3.4 for more inforation on kernel types.
    </para>

    <para>
        During the build, the kern-tools will search for the BSP description
        file that most closely matches the <filename>KMACHINE</filename>
        and <filename>LINUX_KERNEL_TYPE</filename> passed in from the
        recipe.
        It will use the first BSP description it finds matching both variables.
        Failing that it will issue a warning such as the following:
        <literallayout class='monospaced'>
     WARNING: Can't find any BSP hardware or required configuration fragments.
     WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
              meta/cfg/broken/fri2-broken/required_frags.txt in directory:
              meta/cfg/broken/fri2-broken
        </literallayout>
        In this example, <filename>KMACHINE</filename> was set to "fri2-broken"
        and <filename>LINUX_KERNEL_TYPE</filename> was set to "broken".
    </para>

    <para>
        It will then search first for the <filename>KMACHINE</filename> and
        then for the <filename>LINUX_KERNEL_TYPE</filename>.
        If it cannot find a partial match, it will use the
        sources from the <filename>KBRANCH</filename> and any configuration
        specified in the <filename>SRC_URI</filename>.
    </para>

    <para>
        <filename>KERNEL_FEATURES</filename> can be used to include features
        (configuration fragments, patches, or both) that are not already
        included by the <filename>KMACHINE</filename> and
        <filename>LINUX_KERNEL_TYPE</filename> combination.
        To include a feature specified as "features/netfilter.scc" for example,
        specify:
        <literallayout class='monospaced'>
     KERNEL_FEATURES += "features/netfilter.scc"
        </literallayout>
        To include a feature called "cfg/sound.scc" just for the
        <filename>qemux86</filename> machine, specify:
        <literallayout class='monospaced'>
     KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"
        </literallayout>
        The value of the entries in <filename>KERNEL_FEATURES</filename>
        are dependent on their location within the metadata itself.
        The examples here are taken from the
        <filename>linux-yocto-3.4</filename> repository where "features"
        and "cfg" are subdirectories of the <filename>metadata</filename>
        directory.
        For details, see section 3.3.
        <note>
            The processing of the these variables has evolved some between the
	        0.9 and 1.3 releases of the Yocto Project and associated
	        kern-tools sources.
            The descriptions in this section are accurate for 1.3 and later
	        releases of the Yocto Project.
        </note>
    </para>

    <para>
        Original Text.
        <literallayout class='monospaced'>
The metadata provided with any linux-yocto style Linux kernel sources must
define a BSP that corresponds to the definition laid out in the recipe. A BSP
consists of an aggregation of kernel policy and hardware specific feature
enablement. This can be influenced from within the recipe.

Every linux-yocto style recipe must define the following variables:

	KMACHINE

KMACHINE is typically set to the same value as used within the recipe-space BSP
definition, such as "routerstationpro" or "fri2". However, multiple BSPs can
reuse the same KMACHINE name if they are built using the same BSP description
(see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples of such
a situation where each specifies KMACHINE as "fri2".

They may optionally define the following variables:
	KBRANCH
	KERNEL_FEATURES
	KBRANCH_DEFAULT
	LINUX_KERNEL_TYPE

KBRANCH_DEFAULT defines the default source branch within the Linux kernel source
repository to be used to build the Linux kernel. It is used as the default value
for KBRANCH which may define an alternate branch, typically with a machine
override, such as:

KBRANCH_fri2 = "standard/fri2"

Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master".

LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the
configuration and defaults to "standard" if you do not specify otherwise.
Together with KMACHINE, this defines the search arguments used by the Yocto
Project Linux kernel tools to find the appropriate description within the
metadata with which to build out the sources and configuration. The linux-yocto
recipes define "standard", "tiny", and "preempt-rt" kernel types. See 3.3.4 for
more inforation on kernel types.

During the build, the kern-tools will search for the BSP description file that
most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the
recipe. It will use the first BSP description it finds matching both variables.
Failing that it will issue a warning such as the following:

	WARNING: Can't find any BSP hardware or required configuration fragments.
	WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
	         meta/cfg/broken/fri2-broken/required_frags.txt in directory:
	         meta/cfg/broken/fri2-broken

	In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE
	was set to "broken".

It will then search first for the KMACHINE and then
for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will use the
sources from the KBRANCH and any configuration specified in the SRC_URI.

KERNEL_FEATURES can be used to include features (configuration fragments,
patches, or both) that are not already included by the KMACHINE and
LINUX_KERNEL_TYPE combination. To include a feature specified as
"features/netfilter.scc" for example, specify:

KERNEL_FEATURES += "features/netfilter.scc"

To include a feature called "cfg/sound.scc" just for the qemux86 machine,
specify:

KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"

The value of the entries in KERNEL_FEATURES are dependent on their location
within the metadata itself. The examples here are taken from the
linux-yocto-3.4 repository where "features" and "cfg" are subdirectories of the
metadata directory. For details, see 3.3.

	Note: The processing of the these variables has evolved some between the
	      0.9 and 1.3 releases of the Yocto Project and associated
	      kern-tools sources. The above is accurate for 1.3 and later
	      releases of the Yocto Project.
        </literallayout>
    </para>
</section>

<section id='metadata-location'>
    <title>Metadata Location</title>

    <para>
        This metadata can be defined along with the Linux kernel
        recipe (recipe-space) as partially described in the
        "<link linkend='modifying-an-existing-recipe'>Modifying an Existing Recipe</link>"
        section as well as within the Linux kernel sources themselves
        (in-tree).
    </para>

    <para>
        Where you choose to store the metadata depends on what you want
        to do and how you intend to work.
        If you are unfamiliar with the Linux kernel and only wish
        to apply a config and possibly a couple of patches provided to
        you by others, you may find the recipe-space mechanism to be easier
        to work with.
        This is also a good approach if you are working with Linux kernel
        sources you do not control or if you just don't want to maintain a
        Linux kernel git repository on your own.
    </para>

    <para>
        If you are doing active kernel development and are already
        maintaining a Linux kernel git repository of your own, you may find
        it more convenient to work with the metadata in the same
        repository as the Linux kernel sources.
        This can make iterative development of the Linux kernel more efficient
        outside of the BitBake environment.

    </para>

    <para>
        Regardless of where the meta-data is stored, the syntax as
        described in the following sections applies equally.
    </para>

    <para>
        Original Text:
        <literallayout class='monospaced'>
This meta-data can be defined along with the Linux kernel recipe (recipe-space)
as partially described in section 2.2 as well as within the Linux kernel sources
themselves (in-tree).

Where you choose to store the meta-data depends on what you want to do and how
you intend to work. If you are unfamiliar with the Linux kernel and only wish
to apply a config and possibly a couple of patches provided to you by others,
you may find the recipe-space mechanism to be easier to work with. This is also
a good approach if you are working with Linux kernel sources you do not control
or if you just don't want to maintain a Linux kernel git repository on your own.

If you are doing active kernel development and are already maintaining a Linux
kernel git repository of your own, you may find it more convenient to work with
the meta-data in the same repository as the Linux kernel sources. This can make
iterative development of the Linux kernel more efficient outside of the bitbake
environment.

Regardless of where the meta-data is stored, the syntax as
described in the following sections applies equally.
        </literallayout>
    </para>

    <section id='recipe-space-metadata'>
        <title>Recipe-Space Metadata</title>

        <para>
            When stored in recipe-space, the metadata files reside in a
            directory hierarchy below
            <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>,
            which is typically set to
            <filename>${THISDIR}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
            for a linux-yocto or linux-yocto-custom derived Linux kernel
            recipe.
            See the "<link linkend='modifying-an-existing-recipe'>Modifying an Existing Recipe</link>"
            section for more information.
        </para>

        <para>
            By way of example, a trivial tree of metadata stored in
            recipe-space within a BSP layer might look like the following:
            <literallayout class='monospaced'>
     meta/
     `-- recipes-kernel
         `-- linux
             `-- linux-yocto
                 |-- bsp-standard.scc
                 |-- bsp.cfg
                 `-- standard.cfg
            </literallayout>
        </para>

        <para>
            When the metadata is stored in recipe-space, you must take
            steps to ensure BitBake has the necessary information to decide
            which files to fetch and when they need to be fetched again.
        </para>

        <para>
            It is only necessary to specify the <filename>.scc</filename>
            files on the
            <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
            BitBake parses them and fetches any files referenced in the
            <filename>.scc</filename> files by the <filename>include</filename>,
            <filename>patch</filename>, or <filename>kconf</filename> commands.
            Because of this, it is necessary to bump the recipe
            <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
            value when changing the content of files not explicitly listed
            in the SRC_URI.
        </para>

        <para>
            Original text:
            <literallayout class='monospaced'>
When stored in recipe-space, the meta-data files reside in a directory hierarchy
below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a
linux-yocto or linux-yocto-custom derived Linux kernel recipe. See 2.2.

By way of example, a trivial tree of meta-data stored in recipe-space within a
BSP layer might look like the following:

meta/
`-- recipes-kernel
    `-- linux
        `-- linux-yocto
            |-- bsp-standard.scc
            |-- bsp.cfg
            `-- standard.cfg

When the meta-data is stored in recipe-space, you must take steps to ensure
bitbake has the necessary information to decide which files to fetch and when
they need to be fetched again.

It is only necessary to specify the .scc files on the SRC_URI; bitbake will
parse them and fetch any files referenced in the .scc files by the include,
patch, or kconf commands. Because of this, it is necessary to bump the recipe PR
value when changing the content of files not explicitly listed in the SRC_URI.
            </literallayout>
        </para>
    </section>

    <section id='in-tree-metadata'>
        <title>In-Tree Metadata</title>

        <para>
            When stored in-tree, the metadata files reside in the
            "meta" directory of the Linux kernel sources.
            They may be present in the same branch as the sources,
            such as "master", or in their own orphan branch,
            typically named "meta".
            An orphan branch in Git is a branch with unique history and
            content to the other branches in the repository.
            This is useful to track metadata changes independently from the
            sources of the Linux kernel, while still keeping them
            together in the same repository.
            For the purposes of this document, we will discuss all
            in-tree metadata as residing below the
            <filename>meta/cfg/kernel-cache</filename> directory.
        </para>

        <para>
            By way of example, a trivial tree of metadata stored in a custom
            Linux kernel Git repository might look like the following:
            <literallayout class='monospaced'>
     meta/
     `-- cfg
         `-- kernel-cache
             |-- bsp-standard.scc
             |-- bsp.cfg
             `-- standard.cfg
            </literallayout>
        </para>

        <para>
            To use a specific branch for the metadata, specify the branch
            in the <filename>KMETA</filename> variable in your Linux kernel
            recipe, for example:
            <literallayout class='monospaced'>
     KMETA = "meta"
            </literallayout>
            To use the same branch as the sources, set
            <filename>KMETA</filename> to the empty string:
            <literallayout class='monospaced'>
     KMETA = ""
            </literallayout>
            If you are working with your own sources and want to create an
            orphan <filename>meta</filename> branch, you can do so using the
            following commands from within your Linux kernel Git repository:
            <literallayout class='monospaced'>
     $ git checkout --orphan meta
     $ git rm -rf .
     $ git commit --allow-empty -m "Create orphan meta branch"
            </literallayout>
        </para>

        <para>
            Original text:
            <literallayout class='monospaced'>
When stored in-tree, the meta-data files reside in the "meta" directory of the
Linux kernel sources. They may be present in the same branch as the sources,
such as "master", or in their own orphan branch, typically named "meta". An
orphan branch in git is a branch with unique history and content to the other
branches in the repository. This is useful to track meta-data changes
independently from the sources of the Linux kernel, while still keeping them
together in the same repository. For the purposes of this document, we will
discuss all in-tree meta-data as residing below the "meta/cfg/kernel-cache"
directory.

By way of example, a trivial tree of meta-data stored in a custom Linux kernel
git repository might look like the following:

meta/
`-- cfg
    `-- kernel-cache
        |-- bsp-standard.scc
        |-- bsp.cfg
        `-- standard.cfg

To use a specific branch for the meta-data, specify the branch in the KMETA
variable in your Linux kernel recipe, for example:

	KMETA = "meta"

To use the same branch as the sources, set KMETA to the empty string:

	KMETA = ""

If you are working with your own sources and want to create an orphan meta
branch, you can do so using the following commands from within your Linux kernel
git repository:

	$ git checkout --orphan meta
	$ git rm -rf .
	$ git commit --allow-empty -m "Create orphan meta branch"
            </literallayout>
        </para>
    </section>
</section>

<section id='metadata-syntax'>
    <title>Metadata Syntax</title>

    <para>
        The Yocto Project Linux kernel tools metadata consists of three
        primary types of files: <filename>scc</filename>
        <footnote>
            <para>
                <filename>scc</filename> stands for Series Configuration
                Control, but the naming has less significance in the
                current implementation of the tooling than it had in the
                past.
                Consider it to be a description file.
            </para>
        </footnote>
        description files, configuration fragments, and patches.
        The <filename>scc</filename> files define variables and include or
        otherwise reference any of the three file types.
        The description files are used to aggregate all types of metadata into
        what ultimately describes the sources and the configuration required
        to build a Linux kernel tailored to a specific machine.
    </para>

    <para>
        The <filename>scc</filename> description files are used to define two
        fundamental types of metadata:
        <itemizedlist>
            <listitem><para>Features</para></listitem>
            <listitem><para>Board Support Packages (BSPs)</para></listitem>
        </itemizedlist>
    </para>

    <para>
        Features aggregate sources in the form of patches and configuration
        in the form of configuration fragments into a modular reusable unit.
        Features are used to implement conceptually separate metadata
        descriptions like pure configuration fragments, simple patches,
        complex features, and kernel types (ktypes).
        Kernel types define general kernel features and policy to be reused
        in the BSPs.
    </para>

    <para>
        BSPs define hardware-specific features and aggregate them with kernel
        types to form the final description of what will be assembled and built.
    </para>

    <para>
        While the metadata syntax does not enforce any logical separation of
        configuration fragments, patches, features or kernel types, best
        practices dictate a logical separation of these types of meta-data.
        The following metadata file hierarchy is recommended:
        <literallayout class='monospaced'>
     &lt;base&gt;/
        bsp/
        cfg/
        features/
        ktypes/
        patches/
        </literallayout>
    </para>

    <para>
        The <filename>bsp</filename> directory should contain the
        BSP descriptions, described in detail in section 3.3.5.
        The remaining directories all contain "features"; the separation
        is meant to aid in conceptualizing their intended usage.
        A simple guide to determine where your <filename>scc</filename>
        description file should go is as follows.
        If it contains only configuration fragments, it belongs in
        <filename>cfg</filename>.
        If it contains only source-code fixes, it belongs in
        <filename>patches</filename>.
        If it encapsulates a major feature, often combining sources and
        configurations, it belongs in <filename>features</filename>.
        If it aggregates non-hardware configuration and patches
        in order to define a base kernel policy or major kernel type to
        be reused across multiple BSPs, it belongs in
        <filename>ktypes</filename>.
    </para>

    <para>
        The lines between these can easily become blurred, especially as
        out-of-tree features are slowly merged upstream over time.
        Also remember that this is purely logical organization and has
        no impact on the functionality of the metadata as
        all of <filename>cfg</filename>, <filename>features</filename>,
        <filename>patches</filename>, and <filename>ktypes</filename>,
        contain "features" as far as the Yocto Project Linux kernel
        tools are concerned.
    </para>

    <para>
        Paths used in metadata files are relative to
        <filename>&lt;base&gt;</filename>, which is either
        <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
        if you are creating metadata in recipe-space as described in
        section "<link linkend='recipe-space-metadata'>Recipe-Space Metadata</link>",
        or <filename>meta/cfg/kernel-cache/</filename> if you are creating
        metadata in-tree as described in
        the "<link linkend='in-tree-metadata'>In-Tree Metadata</link>" section.
    </para>            

    <para>
        Original text:
        <literallayout class='monospaced'>
The Yocto Project Linux kernel tools meta-data consists of three primary types
of files: scc* description files, configuration fragments, and patches. The scc
files define variables and include or otherwise reference any of the three file
types. The description files are used to aggregate all types of meta-data into
what ultimately describes the sources and the configuration required to build a
Linux kernel tailored to a specific machine.

The scc description files are used to define two fundamental types of meta-data:
	o Features
	o BSPs

Features aggregate sources in the form of patches and configuration in the form
of configuration fragments into a modular reusable unit. Features are used to
implement conceptually separate meta-data descriptions like pure configuration
fragments, simple patches, complex features, and kernel types (ktypes). Kernel
types define general kernel features and policy to be reused in the BSPs.

BSPs define hardware-specific features and aggregate them with kernel types to
form the final description of what will be assembled and built.

While the meta-data syntax does not enforce any logical separation of
configuration fragments, patches, features or kernel types, best practices
dictate a logical separation of these types of meta-data. The following
meta-data file hierarchy is recommended:

	&lt;base&gt;/
		bsp/
		cfg/
		features/
		ktypes/
		patches/

The bsp directory should contain the BSP descriptions, described in detail in
3.3.5. The remaining directories all contain "features"; the separation is meant
to aid in conceptualizing their intended usage.  A simple guide to determine
where your scc description file should go is as follows. If it contains only
configuration fragments, it belongs in cfg. If it contains only source-code
fixes, it belongs in patches. If it encapsulates a major feature, often
combining sources and configurations, it belongs in features. If it aggregates
non-hardware configuration and patches in order to define a base kernel policy
or major kernel type to be reused across multiple BSPs, it belongs in ktypes.
The line between these can easily become blurred, especially as out-of-tree
features are slowly merged upstream over time. Also remember that this is purely
logical organization and has no impact on the functionality of the meta-data as
all of cfg, features, patches, and ktypes, contain "features" as far as the
Yocto Project Linux kernel tools are concerned.

Paths used in meta-data files are relative to &lt;base&gt; which is either
FILESEXTRAPATHS if you are creating meta-data in recipe-space (see 3.2.1), or
meta/cfg/kernel-cache/ if you are creating meta-data in-tree (see 3.2.2).

* scc stands for Series Configuration Control, but the naming has less
  significance in the current implementation of the tooling than it had in the
  past. Consider it to be a description file.
        </literallayout>
    </para>

    <section id='configuration'>
        <title>Configuration</title>

        <para>
            The simplest unit of metadata is the configuration-only feature.
            It consists of one or more Linux kernel configuration parameters
            in a configuration fragment file (<filename>.cfg</filename>)
            and an <filename>scc</filename> file describing the fragment.
        </para>

        <para>
            The SMP fragment included in the linux-yocto-3.4 Git repository
            consists of the following two files:
            <literallayout class='monospaced'>
     cfg/smp.scc:
        define KFEATURE_DESCRIPTION "Enable SMP"
        kconf hardware smp.cfg

     cfg/smp.cfg:
        CONFIG_SMP=y
        CONFIG_SCHED_SMT=y
            </literallayout>
            You can find information on configuration fragment files in the
            "<ulink url='&YOCTO_DOCS_REF_URL;#creating-config-fragments'>Creating Configuration Fragments</ulink>"
            section of the Yocto Project Development Manual and in
            the "<link linkend='generating-configuration-files'>Generating Configuration Files</link>"
            section earlier in this manual.
        </para>

        <para>
            <filename>KFEATURE_DESCRIPTION</filename> provides a short
            description of the fragment, the primary use is for higher level
            tooling, such as the Yocto Project BSP Tools (TODO:Citation).
        </para>

        <para>
            The <filename>kconf</filename> command is used to include the
            actual configuration fragment in an <filename>scc</filename>
            file, and the "hardware" keyword identifies the fragment as
            being hardware enabling, as opposed to general policy,
            which would use the keyword "non-hardware".
            The distinction is made for the benefit of the configuration
            validation tools, which will warn you if a hardware fragment
            overrides a policy set by a non-hardware fragment.
        </para>

        <para>
            As described in the
            "<link linkend='generating-configuration-files'>Generating Configuration Files</link>"
            section, the following BitBake command can be used to audit your
            configuration:
            <literallayout class='monospaced'>
     $ bitbake linux-yocto -c kernel_configcheck -f
            </literallayout>
            The description file can include multiple <filename>kconf</filename>
            statements, one per fragment.
        </para>

        <para>
        Original text:
            <literallayout class='monospaced'>
The simplest unit of meta-data is the configuration-only feature. It consists of
one or more Linux kernel configuration parameters in a .cfg file (as described
in section XYZ) and an scc file describing the fragment. The SMP fragment
included in the linux-yocto-3.4 git repository consists of the following two
files:

cfg/smp.scc:
	define KFEATURE_DESCRIPTION "Enable SMP"
	kconf hardware smp.cfg

cfg/smp.cfg:
	CONFIG_SMP=y
	CONFIG_SCHED_SMT=y

See 2.3.1 for details on creating configuration fragments.

KFEATURE_DESCRIPTION provides a short description of the fragment, the
primary use is for higher level tooling, such as the Yocto Project BSP Tools
(TODO:Citation).

The "kconf" command is used to include the actual configuration fragment in an
scc file, and the "hardware" keyword identifies the fragment as being hardware
enabling, as opposed to general policy (which would use the keyword
"non-hardware"). The distinction is made for the benefit of the configuration
validation tools which will warn you if a hardware fragment overrides a policy
set by a non-hardware fragment.

As described in 2.3.1, the following bitbake command can be used to audit your
configuration:

	$ bitbake linux-yocto -c kernel_configcheck -f

The description file can include multiple kconf statements, one per fragment.
            </literallayout>
        </para>
    </section>

    <section id='patches'>
        <title>Patches</title>

        <para>
            Patches are described in a very similar way to configuration
            fragments, which are described in the previous section.
            Instead of a <filename>.cfg</filename> file, they work with
            source patches.
            A typical patch includes a description file and the patch itself:
            <literallayout class='monospaced'>
     patches/mypatch.scc:
        patch mypatch.patch

     patches/mypatch.patch:
        &lt;typical-patch&gt;
            </literallayout>
            For <filename>.patch</filename> files, the typical patch
            is created with <filename>diff -Nurp</filename> or
            <filename>git format-patch</filename>.
        </para>

        <para>
            The description file can include multiple patch statements,
            one per patch.
        </para>

        <para>
            Original text:
            <literallayout class='monospaced'>
Patches are described in a very similar way to configuration fragments (see
3.3.1). Instead of a .cfg file, they work with source patches. A typical patch
includes a description file and the patch itself:

patches/mypatch.scc:
	patch mypatch.patch

patches/mypatch.patch:
	&lt;typical patch created with 'diff -Nurp' or 'git format-patch'&gt;

The description file can include multiple patch statements, one per patch.
            </literallayout>
        </para>
    </section>

    <section id='features'>
        <title>Features</title>

        <para>
            Features are a combination of configuration fragments and patches.
            Or, more accurately, configuration fragments and patches are
            simple forms of a feature, which is a more complex metadata type.
            In addition to the <filename>kconf</filename> and
            <filename>patch</filename> commands, features often aggregate
            description files with the <filename>include</filename> command.
        </para>

        <para>
            A hypothetical example of a feature description file might look
            like the following:
            <literallayout class='monospaced'>
     features/myfeature.scc
        define KFEATURE_DESCRIPTION "Enable myfeature"

        patch 0001-myfeature-core.patch
        patch 0002-myfeature-interface.patch

        include cfg/myfeature_dependency.scc
        kconf non-hardware myfeature.cfg
            </literallayout>
        </para>

        <para>
            Features are typically less granular than configuration
            fragments and are more likely than configurations fragments
            and patches to be the types of things you will want to specify
            in the <filename>KERNEL_FEATURES</filename> variable of the
            Linux kernel recipe.
            See the "<link linkend='using-metadata-in-a-recipe'>Using Metadata in a Recipe</link>"
            section earlier in the manual.
        </para>

        <para>
        Original text:
            <literallayout class='monospaced'>
Features are a combination of configuration fragments and patches, or, more
accurately, configuration fragments and patches are simple forms of a feature, a
more complex meta-data type. In addition to the kconf and patch commands,
features often aggregate description files with the include command.

A hypothetical example of a feature description file might look like the
following:

features/myfeature.scc
	define KFEATURE_DESCRIPTION "Enable myfeature"

	patch 0001-myfeature-core.patch
	patch 0002-myfeature-interface.patch

	include cfg/myfeature_dependency.scc
	kconf non-hardware myfeature.cfg

Features are typically less granular than configuration fragments and are more
likely than configurations fragments and patches to be the types of things you
will want to specify in the KERNEL_FEATURES variable of the Linux kernel recipe
(see 3.1).
            </literallayout>
        </para>
    </section>

    <section id='kernel-types'>
        <title>Kernel Types</title>

        <para>
            Kernel types, or <filename>ktypes</filename>, are used to
            aggregate all non-hardware configuration fragments together
            with any patches you want to use for all Linux kernel builds
            of the specified <filename>ktype</filename>.
            In short, <filename>ktypes</filename> are where you define a
            high-level kernel policy.
            Syntactically, however, they are no different than features
            as described in the previous section.
            The <filename>ktype</filename> is selected by the
            <filename>LINUX_KERNEL_TYPE</filename> variable in the recipe.
            See the "<link linkend='using-metadata-in-a-recipe'>Using Metadata in a Recipe</link>"
            section for more information.
        </para>

        <para>
            By way of example, the linux-yocto-3.4 tree defines three
            <filename>ktypes</filename>: standard, tiny, and preempt-rt.
            <itemizedlist>
                <listitem><para>standard:
                    Includes the generic Linux kernel
                    policy of the Yocto Project linux-yocto kernel recipes.
                    This includes things like which file systems, which
                    networking options, which core kernel features, and which
                    debugging and tracing options are supported.
                    </para></listitem>
                <listitem><para>preempt-rt:
                    Applies the <filename>PREEMPT_RT</filename>
                    patches and the configuration options required to
                    build a real-time Linux kernel.
                    It inherits from standard.</para></listitem>
                <listitem><para>tiny:
                    Independent from the standard configuration
                    and defines a bare minimum configuration meant to serve as a
                    base for very small Linux kernels.
                    Tiny does not currently include any source changes, but it
                    might in the future.</para></listitem>
            </itemizedlist>
        </para>

        <para>
            The standard kernel type is defined by
            <filename>standard.scc</filename>:
            <literallayout class='monospaced'>
     # Include this kernel type fragment to get the standard features and
     # configuration values.
	
     # Include all standard features
     include standard-nocfg.scc
	
     kconf non-hardware standard.cfg
	
     # individual cfg block section
     include cfg/fs/devtmpfs.scc
     include cfg/fs/debugfs.scc
     include cfg/fs/btrfs.scc
     include cfg/fs/ext2.scc
     include cfg/fs/ext3.scc
     include cfg/fs/ext4.scc
	
     include cfg/net/ipv6.scc
     include cfg/net/ip_nf.scc
     include cfg/net/ip6_nf.scc
     include cfg/net/bridge.scc
            </literallayout>
        </para>

        <para>
            As with any <filename>scc</filename> file, a
            <filename>ktype</filename> definition can aggregate other
            <filename>scc</filename> files with the
            <filename>include</filename> command, or directly pull in
            configuration fragments and patches with the
            <filename>kconf</filename> and <filename>patch</filename>
            commands, respectively.
        </para>

        <note>
            It is not strictly necessary to create a
            <filename>ktype scc</filename> file.
            The BSP file can define the <filename>ktype</filename> implicitly
            with a <filename>define KTYPE myktype</filename> line. See the
            next section for more information.
        </note>

        <para>
            Original text:
            <literallayout class='monospaced'>
Kernel types, or ktypes, are used to aggregate all non-hardware configuration
fragments together with any patches you want to use for all Linux kernel builds
of the specified ktype. In short, ktypes are where you define a high-level
kernel policy. Syntactically, however, they are no different than features (see
3.3.3).  preempt-rt, and tiny. The ktype is selected by the LINUX_KERNEL_TYPE
variable in the recipe (see 3.1). 

By way of example, the linux-yocto-3.4 tree defines three ktypes: standard,
tiny, and preempt-rt. The standard kernel type includes the generic Linux kernel
policy of the Yocto Project linux-yocto kernel recipes. This includes things
like which filesystems, which networking options, which core kernel features,
and which debugging and tracing optoins are supported. The preempt-rt kernel
type applies the PREEMPT_RT patches and the configuration options required to
build a real-time Linux kernel. It inherits from standard. The tiny kernel type
is independent from the standard configuration and defines a bare minimum
configuration meant to serve as a base for very small Linux kernels. Tiny does
not currently include any source changes, but it may in the future.

The standard ktype is defined by standard.scc:
	# Include this kernel type fragment to get the standard features and
	# configuration values.
	
	# Include all standard features
	include standard-nocfg.scc
	
	kconf non-hardware standard.cfg
	
	# individual cfg block section
	include cfg/fs/devtmpfs.scc
	include cfg/fs/debugfs.scc
	include cfg/fs/btrfs.scc
	include cfg/fs/ext2.scc
	include cfg/fs/ext3.scc
	include cfg/fs/ext4.scc
	
	include cfg/net/ipv6.scc
	include cfg/net/ip_nf.scc
	include cfg/net/ip6_nf.scc
	include cfg/net/bridge.scc

As with any scc file, a ktype definition can aggregate other scc files with the
include command, or directly pull in configuration fragments and patches with
the kconf and patch commands, respectively.

Note: It is not strictly necessary to create a ktype scc file. The BSP file can
      define the ktype implicitly with a "define KTYPE myktype" line. See 3.3.5.
            </literallayout>
        </para>
    </section>

    <section id='bsp-descriptions'>
        <title>BSP Descriptions</title>

        <para>
3.3.5 BSP Descriptions
----------
BSP descriptions combine kernel types (see 3.3.4) with hardware-specific
features (see 3.3.3). The hardware specific portion is typically defined
independently, and then aggregated with each supported kernel type. Consider a
simple example:

mybsp.scc:
	define KMACHINE mybsp
	define KTYPE standard
	define KARCH i386

	kconf mybsp.cfg

Every BSP description should include the definition of the KMACHINE, KTYPE, and
KARCH variables. These variables allow the build-system to identify this
description as meeting the criteria set by the recipe being built. This
particular description can be said to support the "mybsp" machine for the
"standard" kernel type and the "i386" architecture. Note that there is no hard
link between the KTYPE and a ktype description file. If you do not have kernel
types defined in your meta-data, you only need to ensure that the recipe
LINUX_KERNEL_TYPE and the KTYPE here match.

NOTE: future versions of the tooling make the specification of KTYPE in the BSP
      optional.

If you did want to separate your kernel policy from your hardware configuration,
you could do so by specifying a kernel type, such as "standard" (see 3.3.4) and
including that description in the BSP description. You might also have multiple
hardware configurations that you aggregate into a single hardware description
file which you could include here, rather than referencing a single .cfg file.
Consider the following:

mybsp.scc:
	define KMACHINE mybsp
	define KTYPE standard
	define KARCH i386

	include standard.scc
	include mybsp.scc

In the above example standard.scc aggregates all the configuration fragments,
patches, and features that make up your standard kernel policy whereas mybsp.scc
aggregates all those necessary to support the hardware available on the mybsp
machine. For information on how to break a complete .config into the various
fragments, see 2.3.1.

Many real-world examples are more complex. Like any other scc file, BSP
descriptions can aggregate features. Consider the Fish River Island II (fri2)
BSP definitions from the linux-yocto-3.4 repository:

fri2.scc:
	kconf hardware fri2.cfg
	
	include cfg/x86.scc
	include features/eg20t/eg20t.scc
	include cfg/dmaengine.scc
	include features/ericsson-3g/f5521gw.scc
	include features/power/intel.scc
	include cfg/efi.scc
	include features/usb/ehci-hcd.scc
	include features/usb/ohci-hcd.scc
	include features/iwlwifi/iwlwifi.scc

The fri2.scc description file includes a hardware configuration fragment
(fri2.cfg) specific to the fri2 BSP as well as several more general
configuration fragments and features enabling hardware found on the fri2. This
description is then included in each of the three machine-ktype descriptions
(standard, preempt-rt, and tiny). Consider the fri2 standard description:

fri2-standard.scc:
	define KMACHINE fri2
	define KTYPE standard
	define KARCH i386
	
	include ktypes/standard/standard.scc
	branch fri2
	
	git merge emgd-1.14
	
	include fri2.scc
	
	# Extra fri2 configs above the minimal defined in fri2.scc
	include cfg/efi-ext.scc
	include features/drm-emgd/drm-emgd.scc
	include cfg/vesafb.scc
	
	# default policy for standard kernels
	include cfg/usb-mass-storage.scc

Note the "include fri2.scc" line about midway through the file. By defining all
hardware enablement common to the BSP for all kernel types, duplication is
significantly reduced.

This description introduces a few more variables and commands worthy of further
discussion.  Note the "branch" command which is used to create a
machine-specific branch into which source changes can be applied. With this
branch set up, the "git merge" command uses the git SCM to merge in a feature
branch "emgd-1.14".  This could also be handled with the patch command, but for
commonly used features such as this, feature branches can be a convenient
mechanism (see 3.5).

Next consider the fri2 tiny description:

fri2-tiny.scc:
	define KMACHINE fri2
	define KTYPE tiny
	define KARCH i386
	
	include ktypes/tiny/tiny.scc
	branch fri2
	
	include fri2.scc

As you might expect, the tiny description includes quite a bit less. In fact,
it includes only the minimal policy defined by the tiny ktype and the
hardware-specific configuration required for boot and the most basic
functionality of the system as defined in the base fri2 description file. Note
again the three critical variables: KMACHINE, KTYPE, and KARCH. Of these, only
the KTYPE has changed, now set to "tiny".
        </para>

        <para>
            Original text:
            <literallayout class='monospaced'>
3.3.5 BSP Descriptions
----------
BSP descriptions combine kernel types (see 3.3.4) with hardware-specific
features (see 3.3.3). The hardware specific portion is typically defined
independently, and then aggregated with each supported kernel type. Consider a
simple example:

mybsp.scc:
	define KMACHINE mybsp
	define KTYPE standard
	define KARCH i386

	kconf mybsp.cfg

Every BSP description should include the definition of the KMACHINE, KTYPE, and
KARCH variables. These variables allow the build-system to identify this
description as meeting the criteria set by the recipe being built. This
particular description can be said to support the "mybsp" machine for the
"standard" kernel type and the "i386" architecture. Note that there is no hard
link between the KTYPE and a ktype description file. If you do not have kernel
types defined in your meta-data, you only need to ensure that the recipe
LINUX_KERNEL_TYPE and the KTYPE here match.

NOTE: future versions of the tooling make the specification of KTYPE in the BSP
      optional.

If you did want to separate your kernel policy from your hardware configuration,
you could do so by specifying a kernel type, such as "standard" (see 3.3.4) and
including that description in the BSP description. You might also have multiple
hardware configurations that you aggregate into a single hardware description
file which you could include here, rather than referencing a single .cfg file.
Consider the following:

mybsp.scc:
	define KMACHINE mybsp
	define KTYPE standard
	define KARCH i386

	include standard.scc
	include mybsp.scc

In the above example standard.scc aggregates all the configuration fragments,
patches, and features that make up your standard kernel policy whereas mybsp.scc
aggregates all those necessary to support the hardware available on the mybsp
machine. For information on how to break a complete .config into the various
fragments, see 2.3.1.

Many real-world examples are more complex. Like any other scc file, BSP
descriptions can aggregate features. Consider the Fish River Island II (fri2)
BSP definitions from the linux-yocto-3.4 repository:

fri2.scc:
	kconf hardware fri2.cfg
	
	include cfg/x86.scc
	include features/eg20t/eg20t.scc
	include cfg/dmaengine.scc
	include features/ericsson-3g/f5521gw.scc
	include features/power/intel.scc
	include cfg/efi.scc
	include features/usb/ehci-hcd.scc
	include features/usb/ohci-hcd.scc
	include features/iwlwifi/iwlwifi.scc

The fri2.scc description file includes a hardware configuration fragment
(fri2.cfg) specific to the fri2 BSP as well as several more general
configuration fragments and features enabling hardware found on the fri2. This
description is then included in each of the three machine-ktype descriptions
(standard, preempt-rt, and tiny). Consider the fri2 standard description:

fri2-standard.scc:
	define KMACHINE fri2
	define KTYPE standard
	define KARCH i386
	
	include ktypes/standard/standard.scc
	branch fri2
	
	git merge emgd-1.14
	
	include fri2.scc
	
	# Extra fri2 configs above the minimal defined in fri2.scc
	include cfg/efi-ext.scc
	include features/drm-emgd/drm-emgd.scc
	include cfg/vesafb.scc
	
	# default policy for standard kernels
	include cfg/usb-mass-storage.scc

Note the "include fri2.scc" line about midway through the file. By defining all
hardware enablement common to the BSP for all kernel types, duplication is
significantly reduced.

This description introduces a few more variables and commands worthy of further
discussion.  Note the "branch" command which is used to create a
machine-specific branch into which source changes can be applied. With this
branch set up, the "git merge" command uses the git SCM to merge in a feature
branch "emgd-1.14".  This could also be handled with the patch command, but for
commonly used features such as this, feature branches can be a convenient
mechanism (see 3.5).

Next consider the fri2 tiny description:

fri2-tiny.scc:
	define KMACHINE fri2
	define KTYPE tiny
	define KARCH i386
	
	include ktypes/tiny/tiny.scc
	branch fri2
	
	include fri2.scc

As you might expect, the tiny description includes quite a bit less. In fact,
it includes only the minimal policy defined by the tiny ktype and the
hardware-specific configuration required for boot and the most basic
functionality of the system as defined in the base fri2 description file. Note
again the three critical variables: KMACHINE, KTYPE, and KARCH. Of these, only
the KTYPE has changed, now set to "tiny".
            </literallayout>
        </para>
    </section>
</section>

</chapter>
<!--
vim: expandtab tw=80 ts=4
-->