documentation/poky-ref-manual: New chapter introduced

Long-term strategy for the YP Reference Manual is that it contains
reference material and not "how-to-information".  A step in this
direction is to isolate any discussions on components and other
areas of YP that need talked about.  So to start with, I have created
a new chapter for now named "Technical Details" that so far has
a discussion of some components and shared state.  This is a
step in the direction of making this manual a reference manual and
not a "how to" manual.

Changes included removing redundant material from the 'usingpoky.xml'
chapter and also adding the new chapter 'technical-details' into the
'poky-ref-manual.xml' file used for the make.

(From yocto-docs rev: 567823392d903016d314ae9fcc1b4d7abf59d9ae)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

3 files changed, 232 insertions, 219 deletions

diff --git a/documentation/poky-ref-manual/poky-ref-manual.xml b/documentation/poky-ref-manual/poky-ref-manual.xml
index 8c81d2bee0..01907134c7 100644
--- a/documentation/poky-ref-manual/poky-ref-manual.xml
+++ b/documentation/poky-ref-manual/poky-ref-manual.xml
@@ -92,6 +92,8 @@
 
     <xi:include href="extendpoky.xml"/>
 
+    <xi:include href="technical-details.xml"/>
+
     <xi:include href="../bsp-guide/bsp.xml"/>
 
     <xi:include href="development.xml"/>
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml
new file mode 100644
index 0000000000..8f83ecb3a1
--- /dev/null
+++ b/documentation/poky-ref-manual/technical-details.xml
@@ -0,0 +1,229 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+<chapter id='technical-details'>
+<title>Technical Details</title>
+
+    <para>
+        This chapter provides technical details for various parts of the Yocto Project. 
+        Currently, topics include Yocto Project components and shared state (sstate) cache.
+    </para>
+
+<section id='usingpoky-components'>
+    <title>Yocto Project Components</title>
+
+    <para>
+        The BitBake task executor together with various types of configuration files form the 
+        Yocto Project core.
+        This section overviews the BitBake task executor and the
+        configuration files by describing what they are used for and how they interact.
+    </para>
+    
+    <para>  
+        BitBake handles the parsing and execution of the data files. 
+        The data itself is of various types:
+    <itemizedlist>
+        <listitem><para><emphasis>Recipes:</emphasis>  Provides details about particular 
+            pieces of software</para></listitem>
+        <listitem><para><emphasis>Class Data:</emphasis>  An abstraction of common build 
+            information (e.g. how to build a Linux kernel).</para></listitem>
+        <listitem><para><emphasis>Configuration Data:</emphasis>  Defines machine-specific settings, 
+            policy decisions, etc.
+            Configuration data acts as the glue to bind everything together.</para></listitem>
+    </itemizedlist>
+        For more information on data, see the
+        <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#yocto-project-terms'>
+        Yocto Project Terms</ulink> section in 
+        <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html'>
+        The Yocto Project Development Manual</ulink>.
+    </para>
+
+    <para> 
+        BitBake knows how to combine multiple data sources together and refers to each data source
+        as a <link linkend='usingpoky-changes-layers'>'layer'</link>.
+    </para>
+
+    <para>
+        Following are some brief details on these core components.
+        For more detailed information on these components see the 
+        <link linkend='ref-structure'>'Reference: Directory Structure'</link>
+        appendix.
+    </para>
+
+    <section id='usingpoky-components-bitbake'>
+        <title>BitBake</title>
+
+        <para>
+            BitBake is the tool at the heart of the Yocto Project and is responsible
+            for parsing the metadata, generating a list of tasks from it,
+            and then executing those tasks. 
+            To see a list of the options BitBake supports, use the following help command:
+            <literallayout class='monospaced'>
+     $ bitbake --help
+            </literallayout>
+        </para>
+
+        <para>
+            The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
+            <filename>packagename</filename> is the name of the package you want to build 
+            (referred to as the "target" in this manual). 
+            The target often equates to the first part of a <filename>.bb</filename> filename.
+            So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you
+            might type the following:
+            <literallayout class='monospaced'>
+     $ bitbake matchbox-desktop
+            </literallayout>
+            Several different versions of <filename>matchbox-desktop</filename> might exist.
+            BitBake chooses the one selected by the distribution configuration.
+            You can get more details about how BitBake chooses between different 
+            target versions and providers in the 
+            <link linkend='ref-bitbake-providers'>Preferences and Providers</link> section.
+        </para>
+
+        <para>
+            BitBake also tries to execute any dependent tasks first.
+            So for example, before building <filename>matchbox-desktop</filename>, BitBake
+            would build a cross compiler and <filename>eglibc</filename> if they had not already 
+            been built.
+            <note>This release of the Yocto Project does not support the <filename>glibc</filename>
+                GNU version of the Unix standard C library.  By default, the Yocto Project builds with
+                <filename>eglibc</filename>.</note>
+        </para>
+
+        <para>
+            A useful BitBake option to consider is the <filename>-k</filename> or 
+            <filename>--continue</filename> option.  
+            This option instructs BitBake to try and continue processing the job as much 
+            as possible even after encountering an error.  
+            When an error occurs, the target that
+            failed and those that depend on it cannot be remade.  
+            However, when you use this option other dependencies can still be processed.
+        </para>
+    </section>
+
+    <section id='usingpoky-components-metadata'>
+        <title>Metadata (Recipes)</title>
+
+        <para>
+            The <filename>.bb</filename> files are usually referred to as "recipes." 
+            In general, a recipe contains information about a single piece of software.
+            The information includes the location from which to download the source patches 
+            (if any are needed), which special configuration options to apply, 
+            how to compile the source files, and how to package the compiled output. 
+        </para>
+
+        <para>
+            The term "package" can also be used to describe recipes.
+            However, since the same word is used for the packaged output from the Yocto 
+            Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), 
+            this document avoids using the term "package" to refer to recipes.
+        </para>
+    </section>
+
+    <section id='usingpoky-components-classes'>
+        <title>Classes</title>
+
+        <para>
+            Class files (<filename>.bbclass</filename>) contain information that is useful to share
+            between metadata files. 
+            An example is the Autotools class, which contains
+            common settings for any application that Autotools uses.
+            The <link linkend='ref-classes'>Reference: Classes</link> appendix provides details
+            about common classes and how to use them.
+        </para>
+    </section>
+
+    <section id='usingpoky-components-configuration'>
+        <title>Configuration</title>
+
+        <para>
+            The configuration files (<filename>.conf</filename>) define various configuration variables
+            that govern the Yocto Project build process. 
+            These files fall into several areas that define machine configuration options, 
+            distribution configuration options, compiler tuning options, general common configuration
+            options and user configuration options (<filename>local.conf</filename>, which is found
+            in the Yocto Project files build directory).
+        </para>
+    </section>
+</section>
+
+<section id="considering-shared-state-cache">
+    <title>Considering Shared State Cache</title>
+
+    <para>
+        By design, the Yocto Project builds everything from scratch unless it can determine that
+        a given task's inputs have not changed.  
+        While building from scratch ensures that everything is current, it does also
+        mean that a lot of time could be spent rebuiding things that don't necessarily need built.
+    </para>
+
+    <para>
+        The Yocto Project build process uses a shared state caching scheme to avoid having to 
+        rebuild software when it is not necessary.  
+        Because the build time for a Yocto image can be significant, it is helpful to try and 
+        determine what really needs built and what can be skipped given a particular project's
+        build process.
+    </para>
+
+    <para>
+        The scheme that the Yocto Project uses involves checksum generation and comparison for
+        a task's inputs.
+        The scheme also employs an area of memory called the shared state cache that is 
+        pointed to by the <filename>SSTATE_DIR</filename> variable.
+        This area contains task output generated from a previous build.  
+        If a given task's checksum matches the checksum of a previous build for the same  
+        task, the build process uses the state of the cache rather than rerunning that 
+        task.
+    </para>
+
+    <para>
+        The previous paragraph is a simplistic explanation of how the build process 
+        uses checksums and shared state memory cache to avoide building tasks that 
+        don't need built.
+        If you want a bit more explanation on the topic, 
+        see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared
+        State - What does it mean and why should I care?</ulink>" from the Yocto 
+        Project discussion archives.
+    </para>
+
+    <para>
+        As with all schemes, this one has some drawbacks.
+        It is possible that you could make implicit changes that are not factored into the checksum 
+        calculation, but do affect a task's output. 
+        A good example is perhaps when a tool changes its output.
+        Let's say that the output of <filename>rpmdeps</filename> needed to change.
+        The result of the change should be that all the "package", "package_write_rpm",
+        and "package_deploy-rpm" sstate-cache items would become invalid.
+        But, because this is a change that is external to the code and therefore implicit,
+        the associated sstate-cache items do not become invalidated.
+        In this case, the build process would use the cache items rather than running the 
+        task again. 
+        Obviously, these types of implicit changes can cause problems.
+    </para>
+
+    <para>
+        To avoid these problems during the build, you need to understand the effects of any
+        change you make.
+        Note that any changes you make directly to a function automatically are factored into 
+        the checksum calculation and thus, will invalidate the associated area of sstate cache.
+        You need to be aware of any implicit changes that are not obvious changes to the 
+        code and could affect the output of a given task. 
+        Once you are aware of such a change, you can take steps to invalidate the cache 
+        and force the task to run. 
+        The step to take is as simple as changing a function's comments in the source code. 
+        For example, to invalidate package sstate files, change the comment statments
+        of <filename>do_package</filename> or one of the functions it calls.
+        The change is purely cosmetic, but it causes the checksum to be recalculated and  
+        forces the task to be run again.
+    </para>
+
+    <note>
+        For an example of a commit that makes a cosmetic change to invalidate an sstate, 
+        see this
+        <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
+    </note>
+</section>
+
+</chapter>
+<!-- 
+vim: expandtab tw=80 ts=4 
+-->
diff --git a/documentation/poky-ref-manual/usingpoky.xml b/documentation/poky-ref-manual/usingpoky.xml
index 9a338b3c89..ea316595b1 100644
--- a/documentation/poky-ref-manual/usingpoky.xml
+++ b/documentation/poky-ref-manual/usingpoky.xml
@@ -4,150 +4,9 @@
 <title>Using the Yocto Project</title>
 
     <para>
-        This section gives an overview of the components that make up the Yocto Project 
-        followed by information about Yocto Project builds and dealing with any
-        problems that might arise.
+        This section describes the common usage for the Yocto Project.
     </para>
 
-<section id='usingpoky-components'>
-    <title>Yocto Project Components</title>
-
-    <para>
-        The BitBake task executor together with various types of configuration files form the 
-        Yocto Project core.
-        This section overviews the BitBake task executor and the
-        configuration files by describing what they are used for and how they interact.
-    </para>
-    
-    <para>  
-        BitBake handles the parsing and execution of the data files. 
-        The data itself is of various types:
-    <itemizedlist>
-        <listitem><para><emphasis>Recipes:</emphasis>  Provides details about particular 
-            pieces of software</para></listitem>
-        <listitem><para><emphasis>Class Data:</emphasis>  An abstraction of common build 
-            information (e.g. how to build a Linux kernel).</para></listitem>
-        <listitem><para><emphasis>Configuration Data:</emphasis>  Defines machine-specific settings, 
-            policy decisions, etc.
-            Configuration data acts as the glue to bind everything together.</para></listitem>
-    </itemizedlist>
-        For more information on data, see the
-        <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#yocto-project-terms'>
-        Yocto Project Terms</ulink> section in 
-        <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html'>
-        The Yocto Project Development Manual</ulink>.
-    </para>
-
-    <para> 
-        BitBake knows how to combine multiple data sources together and refers to each data source
-        as a <link linkend='usingpoky-changes-layers'>'layer'</link>.
-    </para>
-
-    <para>
-        Following are some brief details on these core components.
-        For more detailed information on these components see the 
-        <link linkend='ref-structure'>'Reference: Directory Structure'</link>
-        appendix.
-    </para>
-
-    <section id='usingpoky-components-bitbake'>
-        <title>BitBake</title>
-
-        <para>
-            BitBake is the tool at the heart of the Yocto Project and is responsible
-            for parsing the metadata, generating a list of tasks from it,
-            and then executing those tasks. 
-            To see a list of the options BitBake supports, use the following help command:
-            <literallayout class='monospaced'>
-     $ bitbake --help
-            </literallayout>
-        </para>
-
-        <para>
-            The most common usage for BitBake is <filename>bitbake &lt;packagename&gt;</filename>, where
-            <filename>packagename</filename> is the name of the package you want to build 
-            (referred to as the "target" in this manual). 
-            The target often equates to the first part of a <filename>.bb</filename> filename.
-            So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you
-            might type the following:
-            <literallayout class='monospaced'>
-     $ bitbake matchbox-desktop
-            </literallayout>
-            Several different versions of <filename>matchbox-desktop</filename> might exist.
-            BitBake chooses the one selected by the distribution configuration.
-            You can get more details about how BitBake chooses between different 
-            target versions and providers in the 
-            <link linkend='ref-bitbake-providers'>Preferences and Providers</link> section.
-        </para>
-
-        <para>
-            BitBake also tries to execute any dependent tasks first.
-            So for example, before building <filename>matchbox-desktop</filename>, BitBake
-            would build a cross compiler and <filename>eglibc</filename> if they had not already 
-            been built.
-            <note>This release of the Yocto Project does not support the <filename>glibc</filename>
-                GNU version of the Unix standard C library.  By default, the Yocto Project builds with
-                <filename>eglibc</filename>.</note>
-        </para>
-
-        <para>
-            A useful BitBake option to consider is the <filename>-k</filename> or 
-            <filename>--continue</filename> option.  
-            This option instructs BitBake to try and continue processing the job as much 
-            as possible even after encountering an error.  
-            When an error occurs, the target that
-            failed and those that depend on it cannot be remade.  
-            However, when you use this option other dependencies can still be processed.
-        </para>
-    </section>
-
-    <section id='usingpoky-components-metadata'>
-        <title>Metadata (Recipes)</title>
-
-        <para>
-            The <filename>.bb</filename> files are usually referred to as "recipes." 
-            In general, a recipe contains information about a single piece of software.
-            The information includes the location from which to download the source patches 
-            (if any are needed), which special configuration options to apply, 
-            how to compile the source files, and how to package the compiled output. 
-        </para>
-
-        <para>
-            The term "package" can also be used to describe recipes.
-            However, since the same word is used for the packaged output from the Yocto 
-            Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), 
-            this document avoids using the term "package" to refer to recipes.
-        </para>
-    </section>
-
-    <section id='usingpoky-components-classes'>
-        <title>Classes</title>
-
-        <para>
-            Class files (<filename>.bbclass</filename>) contain information that is useful to share
-            between metadata files. 
-            An example is the Autotools class, which contains
-            common settings for any application that Autotools uses.
-            The <link linkend='ref-classes'>Reference: Classes</link> appendix provides details
-            about common classes and how to use them.
-        </para>
-    </section>
-
-    <section id='usingpoky-components-configuration'>
-        <title>Configuration</title>
-
-        <para>
-            The configuration files (<filename>.conf</filename>) define various configuration variables
-            that govern the Yocto Project build process. 
-            These files fall into several areas that define machine configuration options, 
-            distribution configuration options, compiler tuning options, general common configuration
-            options and user configuration options (<filename>local.conf</filename>, which is found
-            in the Yocto Project files build directory).
-        </para>
-    </section>
-</section>
-
-
 <section id='usingpoky-build'>
     <title>Running a Build</title>
 
@@ -221,83 +80,6 @@
             due to some components not being rebuilt.
         </para>
     </section>
-
-    <section id="considering-shared-state-cache">
-        <title>Considering Shared State Cache</title>
-
-        <para>
-            By design, the Yocto Project builds everything from scratch unless it can determine that
-            a given task's inputs have not changed.  
-            While building from scratch ensures that everything is current, it does also
-            mean that a lot of time could be spent rebuiding things that don't necessarily need built.
-        </para>
-
-        <para>
-            The Yocto Project build process uses a shared state caching scheme to avoid having to 
-            rebuild software when it is not necessary.  
-            Because the build time for a Yocto image can be significant, it is helpful to try and 
-            determine what really needs built and what can be skipped given a particular project's
-            build process.
-        </para>
-
-        <para>
-            The scheme that the Yocto Project uses involves checksum generation and comparison for
-            a task's inputs.
-            The scheme also employs an area of memory called the shared state cache that is 
-            pointed to by the <filename>SSTATE_DIR</filename> variable.
-            This area contains task output generated from a previous build.  
-            If a given task's checksum matches the checksum of a previous build for the same  
-            task, the build process uses the state of the cache rather than rerunning that 
-            task.
-        </para>
-
-        <para>
-            The previous paragraph is a simplistic explanation of how the build process 
-            uses checksums and shared state memory cache to avoide building tasks that 
-            don't need built.
-            If you want a bit more explanation on the topic, 
-            see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared
-            State - What does it mean and why should I care?</ulink>" from the Yocto 
-            Project discussion archives.
-        </para>
-
-        <para>
-            As with all schemes, this one has some drawbacks.
-            It is possible that you could make implicit changes that are not factored into the checksum 
-            calculation, but do affect a task's output. 
-            A good example is perhaps when a tool changes its output.
-            Let's say that the output of <filename>rpmdeps</filename> needed to change.
-            The result of the change should be that all the "package", "package_write_rpm",
-            and "package_deploy-rpm" sstate-cache items would become invalid.
-            But, because this is a change that is external to the code and therefore implicit,
-            the associated sstate-cache items do not become invalidated.
-            In this case, the build process would use the cache items rather than running the 
-            task again. 
-            Obviously, these types of implicit changes can cause problems.
-        </para>
-
-        <para>
-            To avoid these problems during the build, you need to understand the effects of any
-            change you make.
-            Note that any changes you make directly to a function automatically are factored into 
-            the checksum calculation and thus, will invalidate the associated area of sstate cache.
-            You need to be aware of any implicit changes that are not obvious changes to the 
-            code and could affect the output of a given task. 
-            Once you are aware of such a change, you can take steps to invalidate the cache 
-            and force the task to run. 
-            The step to take is as simple as changing a function's comments in the source code. 
-            For example, to invalidate package sstate files, change the comment statments
-            of <filename>do_package</filename> or one of the functions it calls.
-            The change is purely cosmetic, but it causes the checksum to be recalculated and  
-            forces the task to be run again.
-        </para>
-
-        <note>
-            For an example of a commit that makes a cosmetic change to invalidate an sstate, 
-            see this
-            <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
-        </note>
-    </section>
 </section>
 
 <section id='usingpoky-install'>