documentation: dev-manual, bsp-guide - Removing/Moving Appendix A

The kernel example appendix is being removed.  This broke a lot
of links.  For now I have moved the information into a new section
called "Patching the Kernel".  I have preserved the information
by adding the old appendix file as kerne-appendix-orig.xml.

(From yocto-docs rev: 994235a69362dfb0114ef9001ea7f2f2e2fdc5c3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 559 insertions, 0 deletions

diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index 918d884681..4ce0a94873 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -1590,6 +1590,565 @@
         </section>
     </section>
 
+
+
+
+
+
+
+
+
+
+
+
+    <section id="patching-the-kernel">
+        <title>Patching the Kernel</title>
+  
+        <para>
+            Kernel modification involves changing or adding configurations to an existing kernel,  
+            changing or adding recipes to the kernel that are needed to support specific hardware features, 
+            or even altering the source code itself.  
+            This appendix presents simple examples that modify the kernel source code, 
+            change the kernel configuration, and add a kernel source recipe.
+            <note>
+                You can use the <filename>yocto-kernel</filename> script
+                found in the <link linkend='source-directory'>Source Directory</link>
+                under <filename>scripts</filename> to manage kernel patches and configuration.
+                See the "<ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'>Managing kernel Patches and Config Items with yocto-kernel</ulink>" 
+                section in the Yocto Project Board Support Packages (BSP) Developer's Guide for
+                more information.</note>
+        </para>
+
+        <para>
+            This example adds some simple QEMU emulator console output at boot time by 
+            adding <filename>printk</filename> statements to the kernel's
+            <filename>calibrate.c</filename> source code file.
+            Booting the modified image causes the added messages to appear on the emulator's
+            console.
+        </para> 
+
+        <section id='understanding-the-files-you-need'>
+            <title>Understanding the Files You Need</title>
+          
+            <para>
+                Before you modify the kernel, you need to know what Git repositories and file 
+                structures you need.
+                Briefly, you need the following: 
+                <itemizedlist>
+                    <listitem><para>A local 
+                        <link linkend='source-directory'>Source Directory</link> for the 
+                        poky Git repository</para></listitem>
+                    <listitem><para>Local copies of the
+                        <link linkend='poky-extras-repo'><filename>poky-extras</filename></link>
+                        Git repository placed within the Source Directory.</para></listitem>  
+                    <listitem><para>A bare clone of the 
+                        <link linkend='local-kernel-files'>Yocto Project Kernel</link> upstream Git 
+                        repository to which you want to push your modifications.
+                        </para></listitem>
+                    <listitem><para>A copy of that bare clone in which you make your source 
+                        modifications</para></listitem>
+                </itemizedlist>
+            </para>
+
+            <para> 
+                The following figure summarizes these four areas.  
+                Within each rectangular that represents a data structure, a  
+                host development directory pathname appears at the 
+                lower left-hand corner of the box.  
+                These pathnames are the locations used in this example.
+                The figure also provides key statements and commands used during the kernel
+                modification process:
+            </para>
+
+            <para>
+                <imagedata fileref="figures/kernel-example-repos-generic.png" width="7in" depth="5in" 
+                    align="center" scale="100" />
+            </para>
+
+            <para>
+                Here is a brief description of the four areas:
+                <itemizedlist>
+                    <listitem><para><emphasis>Local Source Directory:</emphasis> 
+                        This area contains all the metadata that supports building images 
+                        using the OpenEmbedded build system.
+                        In this example, the 
+                        <link linkend='source-directory'>Source Directory</link> also 
+                        contains the 
+                        <link linkend='build-directory'>Build Directory</link>, 
+                        which contains the configuration directory 
+                        that lets you control the build.
+                        Also in this example, the Source Directory contains local copies of the 
+                        <filename>poky-extras</filename> Git repository.</para>
+                        <para>See the bulleted item
+                        "<link linkend='local-yp-release'>Yocto Project Release</link>"
+                        for information on how to get these files on your local system.</para></listitem>
+                    <listitem><para><emphasis>Local copies of the&nbsp;<filename>poky-extras</filename>&nbsp;Git Repository:</emphasis> 
+                        This area contains the <filename>meta-kernel-dev</filename> layer, 
+                        which is where you make changes that append the kernel build recipes.
+                        You edit <filename>.bbappend</filename> files to locate your 
+                        local kernel source files and to identify the kernel being built.
+                        This Git repository is a gathering place for extensions to the Yocto Project
+                        (or really any) kernel recipes that faciliate the creation and development
+                        of kernel features, BSPs or configurations.</para>
+                        <para>See the bulleted item
+                        "<link linkend='poky-extras-repo'>The 
+                        <filename>poky-extras</filename> Git Repository</link>" 
+                        for information on how to get these files.</para></listitem>                
+                    <listitem><para><emphasis>Bare Clone of the Yocto Project kernel:</emphasis> 
+                        This bare Git repository tracks the upstream Git repository of the Linux 
+                        Yocto kernel source code you are changing.
+                        When you modify the kernel you must work through a bare clone.
+                        All source code changes you make to the kernel must be committed and 
+                        pushed to the bare clone using Git commands.
+                        As mentioned, the <filename>.bbappend</filename> file in the 
+                        <filename>poky-extras</filename> repository points to the bare clone
+                        so that the build process can locate the locally changed source files.</para>
+                        <para>See the bulleted item
+                        "<link linkend='local-kernel-files'>Yocto Project Kernel</link>"
+                        for information on how to set up the bare clone.
+                        </para></listitem>
+                    <listitem><para><emphasis>Copy of the Yocto Project Kernel Bare Clone:</emphasis>
+                        This Git repository contains the actual source files that you modify.  
+                        Any changes you make to files in this location need to ultimately be pushed
+                        to the bare clone using the <filename>git push</filename> command.</para>
+                        <para>See the bulleted item
+                        "<link linkend='local-kernel-files'>Yocto Project Kernel</link>"
+                        for information on how to set up the bare clone.
+                        <note>Typically, Git workflows follow a scheme where changes made to a local area 
+                        are pulled into a Git repository.
+                        However, because the <filename>git pull</filename> command does not work 
+                        with bare clones, this workflow pushes changes to the 
+                        repository even though you could use other more complicated methods to
+                        get changes into the bare clone.</note>
+                        </para></listitem>
+                </itemizedlist>
+            </para>
+        </section>
+
+        <section id='setting-up-the-local-yocto-project-files-git-repository'>
+            <title>Setting Up the Local Source Directory</title>
+
+            <para>
+                You can set up the 
+                <link linkend='source-directory'>Source Directory</link>
+                through tarball extraction or by 
+                cloning the <filename>poky</filename> Git repository.  
+                This example uses <filename>poky</filename> as the root directory of the 
+                local Source Directory.
+                See the bulleted item
+                "<link linkend='local-yp-release'>Yocto Project Release</link>"
+                for information on how to get these files.
+            </para>
+
+            <para>
+                Once you have Source Directory set up, 
+                you have many development branches from which you can work. 
+                From inside the local repository you can see the branch names and the tag names used 
+                in the upstream Git repository by using either of the following commands:
+                <literallayout class='monospaced'>
+     $ cd poky
+     $ git branch -a
+     $ git tag -l
+                </literallayout> 
+                This example uses the Yocto Project &DISTRO; Release code named "&DISTRO_NAME;", 
+                which maps to the <filename>&DISTRO_NAME;</filename> branch in the repository. 
+                The following commands create and checkout the local <filename>&DISTRO_NAME;</filename>
+                branch:
+                <literallayout class='monospaced'>
+     $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
+     Branch &DISTRO_NAME; set up to track remote branch &DISTRO_NAME; from origin.
+     Switched to a new branch '&DISTRO_NAME;'
+                </literallayout>
+            </para>
+        </section>
+
+        <section id='setting-up-the-poky-extras-git-repository'>
+            <title>Setting Up the Local poky-extras Git Repository</title>
+
+            <para>
+                This example creates a local copy of the <filename>poky-extras</filename> Git 
+                repository inside the <filename>poky</filename> Source Directory.
+                See the bulleted item "<link linkend='poky-extras-repo'>The 
+                <filename>poky-extras</filename> Git Repository</link>" 
+                for information on how to set up a local copy of the
+                <filename>poky-extras</filename> repository.
+            </para>
+
+            <para>
+                Because this example uses the Yocto Project &DISTRO; Release code 
+                named "&DISTRO_NAME;", which maps to the <filename>&DISTRO_NAME;</filename> 
+                branch in the repository, you need to be sure you are using that 
+                branch for <filename>poky-extras</filename>.
+                The following commands create and checkout the local 
+                branch you are using for the <filename>&DISTRO_NAME;</filename>
+                branch:
+                <literallayout class='monospaced'>
+     $ cd ~/poky/poky-extras
+     $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
+     Branch &DISTRO_NAME; set up to track remote branch &DISTRO_NAME; from origin.
+     Switched to a new branch '&DISTRO_NAME;'
+                </literallayout>
+            </para>
+        </section>
+
+        <section id='setting-up-the-bare-clone-and-its-copy'>
+            <title>Setting Up the Bare Clone and its Copy</title>
+
+            <para>
+                This example modifies the <filename>linux-yocto-3.4</filename> kernel.
+                Thus, you need to create a bare clone of that kernel and then make a copy of the 
+                bare clone.
+                See the bulleted item
+                "<link linkend='local-kernel-files'>Yocto Project Kernel</link>" 
+                for information on how to do that.
+            </para>
+
+            <para>
+                The bare clone exists for the kernel build tools and simply as the receiving end 
+                of <filename>git push</filename>
+                commands after you make edits and commits inside the copy of the clone.
+                The copy (<filename>my-linux-yocto-3.4-work</filename> in this example) has to have 
+                a local branch created and checked out for your work.
+                This example uses <filename>common-pc-base</filename> as the local branch.
+                The following commands create and checkout the branch:
+                <literallayout class='monospaced'>
+     $ cd ~/my-linux-yocto-3.4-work
+     $ git checkout -b standard-common-pc-base origin/standard/common-pc/base
+     Branch standard-common-pc-base set up to track remote branch 
+        standard/common-pc/base from origin.
+     Switched to a new branch 'standard-common-pc-base'
+                </literallayout>
+            </para>
+        </section>
+
+        <section id='building-and-booting-the-default-qemu-kernel-image'>
+            <title>Building and Booting the Default QEMU Kernel Image</title>
+
+            <para>
+                Before we make changes to the kernel source files, this example first builds the 
+                default image and then boots it inside the QEMU emulator.
+                <note>
+                    Because a full build can take hours, you should check two variables in the 
+                    <filename>build</filename> directory that is created after you source the 
+                    <filename>&OE_INIT_FILE;</filename> script.
+                    You can find these variables
+                    <filename>BB_NUMBER_THREADS</filename> and <filename>PARALLEL_MAKE</filename>
+                    in the <filename>build/conf</filename> directory in the 
+                    <filename>local.conf</filename> configuration file.
+                    By default, these variables are commented out.
+                    If your host development system supports multi-core and multi-thread capabilities,
+                    you can uncomment these statements and set the variables to significantly shorten
+                    the full build time.
+                    As a guideline, set both <filename>BB_NUMBER_THREADS</filename> and 
+                    <filename>PARALLEL_MAKE</filename> to twice the number 
+                    of cores your machine supports.
+                </note>
+                The following two commands <filename>source</filename> the build environment setup script
+                and build the default <filename>qemux86</filename> image.
+                If necessary, the script creates the build directory:
+                <literallayout class='monospaced'>
+     $ cd ~/poky
+     $ source &OE_INIT_FILE;
+     You had no conf/local.conf file. This configuration file has therefore been
+     created for you with some default values. You may wish to edit it to use a 
+     different MACHINE (target hardware) or enable parallel build options to take 
+     advantage of multiple cores for example. See the file for more information as 
+     common configuration options are commented.
+
+     The Yocto Project has extensive documentation about OE including a reference manual
+     which can be found at:
+         http://yoctoproject.org/documentation
+
+     For more information about OpenEmbedded see their website:
+         http://www.openembedded.org/
+
+     You had no conf/bblayers.conf file. The configuration file has been created for
+     you with some default values. To add additional metadata layers into your
+     configuration please add entries to this file.
+
+     The Yocto Project has extensive documentation about OE including a reference manual
+     which can be found at:
+         http://yoctoproject.org/documentation
+
+     For more information about OpenEmbedded see their website:
+         http://www.openembedded.org/
+
+
+
+     ### Shell environment set up for builds. ###
+
+     You can now run 'bitbake &lt;target&gt;>'
+
+     Common targets are:
+         core-image-minimal
+         core-image-sato
+         meta-toolchain
+         meta-toolchain-sdk
+         adt-installer
+         meta-ide-support
+
+     You can also run generated qemu images with a command like 'runqemu qemux86'
+                </literallayout>
+            </para>
+
+            <para>
+                The following <filename>bitbake</filename> command starts the build:
+                <literallayout class='monospaced'>
+     $ bitbake -k core-image-minimal
+                </literallayout>
+                <note>Be sure to check the settings in the <filename>local.conf</filename>
+                before starting the build.</note>
+            </para>
+
+            <para>  
+                After the build completes, you can start the QEMU emulator using the resulting image
+                <filename>qemux86</filename> as follows:
+                <literallayout class='monospaced'>
+     $ runqemu qemux86
+                </literallayout>
+            </para>
+
+            <para>
+                As the image boots in the emulator, console message and status output appears
+                across the terminal window.
+                Because the output scrolls by quickly, it is difficult to read.
+                To examine the output, you log into the system using the 
+                login <filename>root</filename> with no password.
+                Once you are logged in, issue the following command to scroll through the 
+                console output:
+                <literallayout class='monospaced'>
+     # dmesg | less
+                </literallayout>
+            </para>
+
+            <para>
+                Take note of the output as you will want to look for your inserted print command output
+                later in the example.
+            </para>
+        </section>
+
+        <section id='changing-the-source-code-and-pushing-it-to-the-bare-clone'>
+            <title>Changing the Source Code and Pushing it to the Bare Clone</title>
+
+            <para>
+                The file you change in this example is named <filename>calibrate.c</filename>
+                and is located in the <filename>my-linux-yocto-3.4-work</filename> Git repository
+                (the copy of the bare clone) in <filename>init</filename>.
+                This example simply inserts several <filename>printk</filename> statements
+                at the beginning of the <filename>calibrate_delay</filename> function.
+            </para>
+
+            <para>
+                Here is the unaltered code at the start of this function:
+                <literallayout class='monospaced'>
+     void __cpuinit calibrate_delay(void)
+     {
+        unsigned long lpj;
+        static bool printed;
+        int this_cpu = smp_processor_id();
+
+        if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
+               .
+               .
+               .
+               </literallayout>
+           </para>
+
+           <para>
+               Here is the altered code showing five new <filename>printk</filename> statements 
+               near the top of the function:
+                <literallayout class='monospaced'>
+     void __cpuinit calibrate_delay(void)
+     {
+         unsigned long lpj;
+         static bool printed;
+         int this_cpu = smp_processor_id();
+
+         printk("*************************************\n");
+         printk("*                                   *\n");
+         printk("*        HELLO YOCTO KERNEL         *\n");
+         printk("*                                   *\n");
+         printk("*************************************\n");
+
+        if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
+               .
+               .
+               .
+               </literallayout>
+           </para>
+
+           <para>
+               After making and saving your changes, you need to stage them for the push.
+               The following Git commands are one method of staging and committing your changes:
+               <literallayout class='monospaced'>
+     $ git add calibrate.c
+     $ git commit --signoff
+               </literallayout>
+           </para>
+
+           <para>
+               Once the source code has been modified, you need to use Git to push the changes to 
+               the bare clone.  
+               If you do not push the changes, then the OpenEmbedded build system will not pick 
+               up the changed source files.
+           </para>
+
+           <para>
+               The following command pushes the changes to the bare clone:
+               <literallayout class='monospaced'>
+     $ git push origin standard-common-pc-base:standard/default/common-pc/base
+               </literallayout>
+           </para>
+       </section>
+
+       <section id='changing-build-parameters-for-your-build'>
+           <title>Changing Build Parameters for Your Build</title>
+
+           <para>
+               At this point, the source has been changed and pushed.
+               The example now defines some variables used by the OpenEmbedded build system 
+               to locate your kernel source.
+               You essentially need to identify where to find the kernel recipe and the changed source code.
+               You also need to be sure some basic configurations are in place that identify the 
+               type of machine you are building and to help speed up the build should your host support
+               multiple-core and thread capabilities.
+           </para>
+
+           <para>
+               Do the following to make sure the build parameters are set up for the example.
+               Once you set up these build parameters, they do not have to change unless you 
+               change the target architecture of the machine you are building or you move
+               the bare clone, copy of the clone, or the <filename>poky-extras</filename> repository:
+               <itemizedlist>
+                   <listitem><para><emphasis>Build for the Correct Target Architecture:</emphasis> The 
+                       <filename>local.conf</filename> file in the build directory defines the build's 
+                       target architecture.
+                       By default, <filename>MACHINE</filename> is set to 
+                       <filename>qemux86</filename>, which specifies a 32-bit 
+                       <trademark class='registered'>Intel</trademark> Architecture 
+                       target machine suitable for the QEMU emulator.  
+                       In this example, <filename>MACHINE</filename> is correctly configured.
+                       </para></listitem>
+                   <listitem><para><emphasis>Optimize Build Time:</emphasis> Also in the 
+                       <filename>local.conf</filename> file are two variables that can speed your 
+                       build time if your host supports multi-core and multi-thread capabilities:
+                       <filename>BB_NUMBER_THREADS</filename> and <filename>PARALLEL_MAKE</filename>.
+                       If the host system has multiple cores then you can optimize build time 
+                       by setting both these variables to twice the number of 
+                       cores.</para></listitem>
+                   <listitem><para><emphasis>Identify Your <filename>meta-kernel-dev</filename>
+                       Layer:</emphasis> The <filename>BBLAYERS</filename> variable in the 
+                       <filename>bblayers.conf</filename> file found in the 
+                       <filename>poky/build/conf</filename> directory needs to have the path to your local
+                       <filename>meta-kernel-dev</filename> layer.  
+                       By default, the <filename>BBLAYERS</filename> variable contains paths to  
+                       <filename>meta</filename> and <filename>meta-yocto</filename> in the 
+                       <filename>poky</filename> Git repository.
+                       Add the path to your <filename>meta-kernel-dev</filename> location.
+                       Be sure to substitute your user information in the statement.
+                       Here is an example:
+                       <literallayout class='monospaced'>
+     BBLAYERS = " \
+       /home/scottrif/poky/meta \
+       /home/scottrif/poky/meta-yocto \
+       /home/scottrif/poky/meta-yocto-bsp \
+       /home/scottrif/poky/poky-extras/meta-kernel-dev \
+       "
+                       </literallayout></para></listitem>
+                   <listitem><para><emphasis>Identify Your Source Files:</emphasis> In the 
+                       <filename>linux-yocto_3.4.bbappend</filename> file located in the 
+                       <filename>poky-extras/meta-kernel-dev/recipes-kernel/linux</filename>
+                       directory, you need to identify the location of the 
+                       local source code, which in this example is the bare clone named
+                       <filename>linux-yocto-3.4.git</filename>.
+                       To do this, set the <filename>KSRC_linux_yocto</filename> variable to point to your
+                       local <filename>linux-yocto-3.4.git</filename> Git repository by adding the 
+                       following statement.
+                       Also, be sure the <filename>SRC_URI</filename> variable is pointing to
+                       your kernel source files by removing the comment.
+                       Finally, be sure to substitute your user information in the statement:
+                       <literallayout class='monospaced'>
+     KSRC_linux_yocto_3_4 ?= "/home/scottrif/linux-yocto-3.4.git"
+     SRC_URI = "git://${KSRC_linux_yocto_3_4};protocol=file;nocheckout=1;branch=${KBRANCH},meta;name=machine,meta"
+                       </literallayout></para></listitem>
+               </itemizedlist>
+           </para>
+
+           <note>
+               <para>Before attempting to build the modified kernel, there is one more set of changes you 
+               need to make in the <filename>meta-kernel-dev</filename> layer.
+               Because all the kernel <filename>.bbappend</filename> files are parsed during the 
+               build process regardless of whether you are using them or not, you should either 
+               comment out the <filename>COMPATIBLE_MACHINE</filename> statements in all 
+               unused <filename>.bbappend</filename> files, or simply remove (or rename) all the files 
+               except the one your are using for the build 
+               (i.e. <filename>linux-yocto_3.4.bbappend</filename> in this example).</para>
+               <para>If you do not make one of these two adjustments, your machine will be compatible
+               with all the kernel recipes in the <filename>meta-kernel-dev</filename> layer. 
+               When your machine is comapatible with all the kernel recipes, the build attempts
+               to build all kernels in the layer.
+               You could end up with build errors blocking your work.</para>
+           </note>            
+       </section>
+
+        <section id='building-and-booting-the-modified-qemu-kernel-image'>
+            <title>Building and Booting the Modified QEMU Kernel Image</title>
+
+            <para>
+                Next, you need to build the modified image.
+                Do the following:
+                <orderedlist>
+                    <listitem><para>Your environment should be set up since you previously sourced
+                        the <filename>&OE_INIT_FILE;</filename> script.
+                        If it isn't, source the script again from <filename>poky</filename>.
+                        <literallayout class='monospaced'>
+     $ cd ~/poky
+     $ source &OE_INIT_FILE;
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>Be sure old images are cleaned out by running the 
+                        <filename>cleanall</filename> BitBake task as follows from your build directory:
+                        <literallayout class='monospaced'>
+     $ bitbake -c cleanall linux-yocto
+                        </literallayout></para>
+                        <para><note>Never remove any files by hand from the <filename>tmp/deploy</filename>
+                        directory insided the build directory.
+                        Always use the BitBake <filename>cleanall</filename> task to clear
+                        out previous builds.</note></para></listitem>
+                    <listitem><para>Next, build the kernel image using this command:
+                        <literallayout class='monospaced'>
+     $ bitbake -k core-image-minimal
+                        </literallayout></para></listitem>
+                    <listitem><para>Finally, boot the modified image in the QEMU emulator 
+                        using this command:
+                        <literallayout class='monospaced'>
+     $ runqemu qemux86
+                        </literallayout></para></listitem>
+                </orderedlist>
+            </para>
+
+            <para>
+                Log into the machine using <filename>root</filename> with no password and then 
+                use the following shell command to scroll through the console's boot output.
+                <literallayout class='monospaced'>
+     # dmesg | less
+                </literallayout>
+            </para>
+
+            <para>
+                You should see the results of your <filename>printk</filename> statements 
+                as part of the output.
+            </para>
+        </section>
+    </section>
+
+
+
+
+
+
+
+
     <section id="usingpoky-changes-updatingimages">
         <title>Updating Existing Images</title>