kernel-dev: Created file structure for new kernel-dev manual.

(From yocto-docs rev: 25be3ebb7713b875c4ec6e3723961b7dd860295d)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 392 insertions, 0 deletions

diff --git a/documentation/kernel-dev/kernel-dev-common.xml b/documentation/kernel-dev/kernel-dev-common.xml
new file mode 100644
index 0000000000..1290994257
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-common.xml
@@ -0,0 +1,392 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+
+<chapter id='kernel-concepts'>
+
+<title>Yocto Project Kernel Concepts</title>
+
+<section id='concepts-org'>
+    <title>Introduction</title>
+    <para>
+        This chapter provides conceptual information about the kernel:
+        <itemizedlist>
+            <listitem><para>Kernel Goals</para></listitem>
+            <listitem><para>Kernel Development and Maintenance Overview</para></listitem>
+            <listitem><para>Kernel Architecture</para></listitem>
+            <listitem><para>Kernel Tools</para></listitem>
+        </itemizedlist>
+    </para>
+</section>
+
+    <section id='kernel-goals'>
+        <title>Kernel Goals</title>
+        <para>
+            The complexity of embedded kernel design has increased dramatically.
+            Whether it is managing multiple implementations of a particular feature or tuning and
+            optimizing board specific features, both flexibility and maintainability are key concerns.
+            The Linux kernels available through the Yocto Project are presented with the embedded
+            developer's needs in mind and have evolved to assist in these key concerns.
+            For example, prior methods such as applying hundreds of patches to an extracted
+            tarball have been replaced with proven techniques that allow easy inspection,
+            bisection and analysis of changes.
+            Application of these techniques also creates a platform for performing integration and
+            collaboration with the thousands of upstream development projects.
+        </para>
+        <para>
+            With all these considerations in mind, the Yocto Project's kernel and development team
+            strives to attain these goals:
+        <itemizedlist>
+            <listitem><para>Allow the end user to leverage community best practices to seamlessly
+            manage the development, build and debug cycles.</para></listitem>
+            <listitem><para>Create a platform for performing integration and collaboration with the
+            thousands of upstream development projects that exist.</para></listitem>
+            <listitem><para>Provide mechanisms that support many different work flows, front-ends and
+            management techniques.</para></listitem>
+            <listitem><para>Deliver the most up-to-date kernel possible while still ensuring that
+            the baseline kernel is the most stable official release.</para></listitem>
+            <listitem><para>Include major technological features as part of the Yocto Project's
+            upward revision strategy.</para></listitem>
+            <listitem><para>Present a kernel Git repository that, similar to the upstream
+            <filename>kernel.org</filename> tree,
+            has a clear and continuous history.</para></listitem>
+            <listitem><para>Deliver a key set of supported kernel types, where each type is tailored
+            to meet a specific use (e.g. networking, consumer, devices, and so forth).</para></listitem>
+            <listitem><para>Employ a Git branching strategy that, from a developer's point of view,
+            results in a linear path from the baseline <filename>kernel.org</filename>,
+            through a select group of features and
+            ends with their BSP-specific commits.</para></listitem>
+        </itemizedlist>
+        </para>
+    </section>
+
+    <section id='kernel-big-picture'>
+        <title>Yocto Project Kernel Development and Maintenance Overview</title>
+        <para>
+            Kernels available through the Yocto Project, like other kernels, are based off the Linux
+            kernel releases from <ulink url='http://www.kernel.org'></ulink>.
+            At the beginning of a major development cycle, the Yocto Project team
+            chooses its kernel based on factors such as release timing, the anticipated release
+            timing of final upstream <filename>kernel.org</filename> versions, and Yocto Project
+            feature requirements.
+            Typically, the kernel chosen is in the
+            final stages of development by the community.
+            In other words, the kernel is in the release
+            candidate or "rc" phase and not yet a final release.
+            But, by being in the final stages of external development, the team knows that the
+            <filename>kernel.org</filename> final release will clearly be within the early stages of
+            the Yocto Project development window.
+        </para>
+        <para>
+            This balance allows the team to deliver the most up-to-date kernel
+            possible, while still ensuring that the team has a stable official release for
+            the baseline Linux kernel version.
+        </para>
+        <para>
+            The ultimate source for kernels available through the Yocto Project are released kernels
+            from <filename>kernel.org</filename>.
+            In addition to a foundational kernel from <filename>kernel.org</filename>, the
+            kernels available contain a mix of important new mainline
+            developments, non-mainline developments (when there is no alternative),
+            Board Support Package (BSP) developments,
+            and custom features.
+            These additions result in a commercially released Yocto Project Linux kernel that caters
+            to specific embedded designer needs for targeted hardware.
+        </para>
+        <para>
+            Once a kernel is officially released, the Yocto Project team goes into
+            their next development cycle, or upward revision (uprev) cycle, while still
+            continuing maintenance on the released kernel.
+            It is important to note that the most sustainable and stable way
+            to include feature development upstream is through a kernel uprev process.
+            Back-porting hundreds of individual fixes and minor features from various
+            kernel versions is not sustainable and can easily compromise quality.
+        </para>
+        <para>
+            During the uprev cycle, the Yocto Project team uses an ongoing analysis of
+            kernel development, BSP support, and release timing to select the best
+            possible <filename>kernel.org</filename> version.
+            The team continually monitors community kernel
+            development to look for significant features of interest.
+            The team does consider back-porting large features if they have a significant advantage.
+            User or community demand can also trigger a back-port or creation of new
+            functionality in the Yocto Project baseline kernel during the uprev cycle.
+        </para>
+        <para>
+            Generally speaking, every new kernel both adds features and introduces new bugs.
+            These consequences are the basic properties of upstream kernel development and are
+            managed by the Yocto Project team's kernel strategy.
+            It is the Yocto Project team's policy to not back-port minor features to the released kernel.
+            They only consider back-porting significant technological jumps - and, that is done
+            after a complete gap analysis.
+            The reason for this policy is that back-porting any small to medium sized change
+            from an evolving kernel can easily create mismatches, incompatibilities and very
+            subtle errors.
+        </para>
+        <para>
+            These policies result in both a stable and a cutting
+            edge kernel that mixes forward ports of existing features and significant and critical
+            new functionality.
+            Forward porting functionality in the kernels available through the Yocto Project kernel
+            can be thought of as a "micro uprev."
+            The many “micro uprevs” produce a kernel version with a mix of
+            important new mainline, non-mainline, BSP developments and feature integrations.
+            This kernel gives insight into new features and allows focused
+            amounts of testing to be done on the kernel, which prevents
+            surprises when selecting the next major uprev.
+            The quality of these cutting edge kernels is evolving and the kernels are used in leading edge
+            feature and BSP development.
+        </para>
+    </section>
+
+    <section id='kernel-architecture'>
+        <title>Kernel Architecture</title>
+        <para>
+            This section describes the architecture of the kernels available through the
+            Yocto Project and provides information
+            on the mechanisms used to achieve that architecture.
+        </para>
+
+        <section id='architecture-overview'>
+            <title>Overview</title>
+            <para>
+                As mentioned earlier, a key goal of the Yocto Project is to present the
+                developer with
+                a kernel that has a clear and continuous history that is visible to the user.
+                The architecture and mechanisms used achieve that goal in a manner similar to the
+                upstream <filename>kernel.org</filename>.
+            </para>
+            <para>
+                You can think of a Yocto Project kernel as consisting of a baseline Linux kernel with
+                added features logically structured on top of the baseline.
+                The features are tagged and organized by way of a branching strategy implemented by the
+                source code manager (SCM) Git.
+                For information on Git as applied to the Yocto Project, see the
+                "<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>" section in the
+                Yocto Project Development Manual.
+            </para>
+            <para>
+                The result is that the user has the ability to see the added features and
+                the commits that make up those features.
+                In addition to being able to see added features, the user can also view the history of what
+                made up the baseline kernel.
+            </para>
+            <para>
+                The following illustration shows the conceptual Yocto Project kernel.
+            </para>
+            <para>
+                <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" />
+            </para>
+            <para>
+                In the illustration, the "Kernel.org Branch Point"
+                marks the specific spot (or release) from
+                which the Yocto Project kernel is created.
+                From this point "up" in the tree, features and differences are organized and tagged.
+            </para>
+            <para>
+                The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
+                type and BSP that is organized further up the tree.
+                Placing these common features in the
+                tree this way means features don't have to be duplicated along individual branches of the
+                structure.
+            </para>
+            <para>
+                From the Yocto Project Baseline Kernel, branch points represent specific functionality
+                for individual BSPs as well as real-time kernels.
+                The illustration represents this through three BSP-specific branches and a real-time
+                kernel branch.
+                Each branch represents some unique functionality for the BSP or a real-time kernel.
+            </para>
+            <para>
+                In this example structure, the real-time kernel branch has common features for all
+                real-time kernels and contains
+                more branches for individual BSP-specific real-time kernels.
+                The illustration shows three branches as an example.
+                Each branch points the way to specific, unique features for a respective real-time
+                kernel as they apply to a given BSP.
+            </para>
+            <para>
+                The resulting tree structure presents a clear path of markers (or branches) to the
+                developer that, for all practical purposes, is the kernel needed for any given set
+                of requirements.
+            </para>
+        </section>
+
+        <section id='branching-and-workflow'>
+            <title>Branching Strategy and Workflow</title>
+            <para>
+                The Yocto Project team creates kernel branches at points where functionality is
+                no longer shared and thus, needs to be isolated.
+                For example, board-specific incompatibilities would require different functionality
+                and would require a branch to separate the features.
+                Likewise, for specific kernel features, the same branching strategy is used.
+            </para>
+            <para>
+                This branching strategy results in a tree that has features organized to be specific
+                for particular functionality, single kernel types, or a subset of kernel types.
+                This strategy also results in not having to store the same feature twice
+                internally in the tree.
+                Rather, the kernel team stores the unique differences required to apply the
+                feature onto the kernel type in question.
+                <note>
+                    The Yocto Project team strives to place features in the tree such that they can be
+                    shared by all boards and kernel types where possible.
+                    However, during development cycles or when large features are merged,
+                    the team cannot always follow this practice.
+                    In those cases, the team uses isolated branches to merge features.
+                </note>
+            </para>
+            <para>
+                BSP-specific code additions are handled in a similar manner to kernel-specific additions.
+                Some BSPs only make sense given certain kernel types.
+                So, for these types, the team creates branches off the end of that kernel type for all
+                of the BSPs that are supported on that kernel type.
+                From the perspective of the tools that create the BSP branch, the BSP is really no
+                different than a feature.
+                Consequently, the same branching strategy applies to BSPs as it does to features.
+                So again, rather than store the BSP twice, the team only stores the unique
+                differences for the BSP across the supported multiple kernels.
+            </para>
+            <para>
+                While this strategy can result in a tree with a significant number of branches, it is
+                important to realize that from the developer's point of view, there is a linear
+                path that travels from the baseline <filename>kernel.org</filename>, through a select
+                group of features and ends with their BSP-specific commits.
+                In other words, the divisions of the kernel are transparent and are not relevant
+                to the developer on a day-to-day basis.
+                From the developer's perspective, this path is the "master" branch.
+                The developer does not need to be aware of the existence of any other branches at all.
+                Of course, there is value in the existence of these branches
+                in the tree, should a person decide to explore them.
+                For example, a comparison between two BSPs at either the commit level or at the line-by-line
+                code <filename>diff</filename> level is now a trivial operation.
+            </para>
+            <para>
+                Working with the kernel as a structured tree follows recognized community best practices.
+                In particular, the kernel as shipped with the product, should be
+                considered an "upstream source" and viewed as a series of
+                historical and documented modifications (commits).
+                These modifications represent the development and stabilization done
+                by the Yocto Project kernel development team.
+            </para>
+            <para>
+                Because commits only change at significant release points in the product life cycle,
+                developers can work on a branch created
+                from the last relevant commit in the shipped Yocto Project kernel.
+                As mentioned previously, the structure is transparent to the developer
+                because the kernel tree is left in this state after cloning and building the kernel.
+            </para>
+        </section>
+
+        <section id='source-code-manager-git'>
+            <title>Source Code Manager - Git</title>
+            <para>
+                The Source Code Manager (SCM) is Git.
+                This SCM is the obvious mechanism for meeting the previously mentioned goals.
+                Not only is it the SCM for <filename>kernel.org</filename> but,
+                Git continues to grow in popularity and supports many different work flows,
+                front-ends and management techniques.
+            </para>
+            <para>
+                You can find documentation on Git at <ulink url='http://git-scm.com/documentation'></ulink>.
+                You can also get an introduction to Git as it applies to the Yocto Project in the
+                "<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>"
+                section in the Yocto Project Development Manual.
+                These referenced sections overview Git and describe a minimal set of
+                commands that allows you to be functional using Git.
+                <note>
+                    You can use as much, or as little, of what Git has to offer to accomplish what
+                    you need for your project.
+                    You do not have to be a "Git Master" in order to use it with the Yocto Project.
+                </note>
+            </para>
+        </section>
+    </section>
+
+    <section id='kernel-configuration'>
+        <title>Kernel Configuration</title>
+        <para>
+            Kernel configuration, along with kernel features, defines how a kernel
+            image is built for the Yocto Project.
+            Through configuration settings, you can customize a Yocto Project kernel to be
+            specific to particular hardware.
+            For example, you can specify sound support or networking support.
+            This section describes basic concepts behind Kernel configuration within the
+            Yocto Project and references you to other areas for specific configuration
+            applications.
+        </para>
+
+        <para>
+            Conceptually, configuration of a Yocto Project kernel occurs similarly to that needed for any
+            Linux kernel.
+            The build process for a Yocto Project kernel uses a <filename>.config</filename> file, which
+            is created through the Linux Kernel Configuration (LKC) tool.
+            You can directly set various configurations in the
+            <filename>.config</filename> file by using the <filename>menuconfig</filename>
+            tool as built by BitBake.
+            You can also define configurations in the file by using configuration fragments.
+            <note>
+                It is not recommended that you edit the <filename>.config</filename> file directly.
+            </note>
+            Here are some brief descriptions of the ways you can affect the
+            <filename>.config</filename> file:
+            <itemizedlist>
+                <listitem><para><emphasis>The <filename>menuconfig</filename> Tool:</emphasis>
+                    One of many front-ends that allows you to define kernel configurations.
+                    Some others are <filename>make config</filename>,
+                    <filename>make nconfig</filename>, and <filename>make gconfig</filename>.
+                    In the Yocto Project environment, you must use BitBake to build the
+                    <filename>menuconfig</filename> tool before you can use it to define
+                    configurations:
+                    <literallayout class='monospaced'>
+     $ bitbake linux-yocto -c menuconfig
+                    </literallayout>
+                    After the tool is built, you can interact with it normally.
+                    You can see how <filename>menuconfig</filename> is used to change a simple
+                    kernel configuration in the
+                    "<ulink url='&YOCTO_DOCS_DEV_URL;#configuring-the-kernel'>Configuring the Kernel</ulink>"
+                    section of the Yocto Project Development Manual.
+                    For general information on <filename>menuconfig</filename>, see
+                    <ulink url='http://en.wikipedia.org/wiki/Menuconfig'></ulink>.
+                    </para></listitem>
+                <listitem><para><emphasis>Configuration Fragments:</emphasis> A file with a
+                    list of kernel options just as they would appear syntactically in the
+                    <filename>.config</filename> file.
+                    Configuration fragments are typically logical groupings and are assembled
+                    by the OpenEmbedded build system to produce input used by the LKC
+                    that ultimately generates the <filename>.config</filename> file.</para>
+                    <para>The
+                    <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES'>KERNEL_FEATURES</ulink></filename>
+                    variable can be used to list configuration fragments.
+                    For further discussion on applying configuration fragments, see the
+                    "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout-kernel'>Linux Kernel Configuration</ulink>"
+                    section in the Yocto Project Board Support Package (BSP) Guide.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='kernel-tools'>
+        <title>Kernel Tools</title>
+        <para>
+            Since most standard workflows involve moving forward with an existing tree by
+            continuing to add and alter the underlying baseline, the tools that manage
+            the Yocto Project's kernel construction are largely hidden from the developer to
+            present a simplified view of the kernel for ease of use.
+        </para>
+        <para>
+            Fundamentally, the kernel tools that manage and construct the
+            Yocto Project kernel accomplish the following:
+            <itemizedlist>
+                <listitem><para>Group patches into named, reusable features.</para></listitem>
+                <listitem><para>Allow top-down control of included features.</para></listitem>
+                <listitem><para>Bind kernel configurations to kernel patches and features.</para></listitem>
+                <listitem><para>Present a seamless Git repository that blends Yocto Project value
+                    with the <filename>kernel.org</filename> history and development.</para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->