dev-manual: First draft of "multi-builds" section.

Fixes [YOCTO #5400]

This is the first draft of a new section on multi-machine builds.

(From yocto-docs rev: a47fd17295d943bff482a0747f3b2e69831aa985)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 174 insertions, 0 deletions

diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index 87f14a046e..a710efe6d5 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -5973,6 +5973,180 @@ Gateways via their Web Interfaces</ulink>"</emphasis>
         </section>
     </section>
 
+    <section id='building-images-for-more-than-one-machine'>
+        <title>Building Images for More than One Machine</title>
+
+        <para>
+            A common scenario developers face is creating images for several
+            different machines that use the same software environment.
+            In this situation, it is tempting to set the
+            tunings and optimization flags for each build specifically for
+            the targeted hardware (i.e. "maxing out" the tunings).
+            Doing so can considerably add to build times and package feed
+            maintenance collectively for the machines.
+            For example, selecting tunes that are extremely specific to a
+            CPU core used in a system might enable some micro optimizations
+            in GCC for that particular system but would otherwise not gain
+            you much of a performance difference across the other systems
+            as compared to using a more general tuning across all the builds.
+            Another example is setting
+            <ulink url='var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink>
+            specifically for each machine's build.
+            Rather than "max out" each build's tunings, you can take steps that
+            cause the OpenEmbedded build system to reuse software across the
+            various machines where it makes sense.
+        </para>
+        <para>
+            If build speed and package feed maintenance are considerations,
+            you should consider the points in this section that can help you
+            optimize your tunings to best consider build times and package
+            feed maintenance.
+            <itemizedlist>
+                <listitem><para><emphasis>Share the Build Directory:</emphasis>
+                    If at all possible, share the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
+                    across builds.
+                    The Yocto Project supports switching between different
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
+                    values in the same <filename>TMPDIR</filename>.
+                    This practice is well supported and regularly used by
+                    developers when building for multiple machines.
+                    When you use the same <filename>TMPDIR</filename> for
+                    multiple machine builds, the OpenEmbedded build system can
+                    reuse the existing native and often cross-recipes for
+                    multiple machines.
+                    Thus, build time decreases.
+                    <note>
+                        If
+                        <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
+                        settings change or fundamental configuration settings
+                        such as the filesystem layout, you need to work with
+                        a clean <filename>TMPDIR</filename>.
+                        Sharing <filename>TMPDIR</filename> under these
+                        circumstances might work but since it is not
+                        guaranteed, you should use a clean
+                        <filename>TMPDIR</filename>.
+                    </note>
+                    </para></listitem>
+                <listitem><para><emphasis>Enable the Appropriate Package Architecture:</emphasis>
+                    By default, the OpenEmbedded build system enables three
+                    levels of package architectures: "all", "tune" or "package",
+                    and "machine".
+                    Any given recipe usually selects one of these package
+                    architectures (types) for its output.
+                    Depending for what a given recipe creates packages, making
+                    sure you enable the appropriate package architecture can
+                    directly impact the build time.</para>
+                    <para>A recipe that just generates scripts can enable
+                    "all" architecture because there are no binaries to build.
+                    To specifically enable "all" architecture, be sure your
+                    recipe inherits the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
+                    class.
+                    This class is useful for "all" architectures because it
+                    contains many other variables that can be used across
+                    multiple architectures.</para>
+                    <para>If your recipe needs to generate packages that are
+                    machine-specific or when one of the build or runtime
+                    dependencies is already machine-architecture dependent,
+                    which makes your recipe also machine-architecture dependent,
+                    make sure your recipe enables the "machine" package
+                    architecture through the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>
+                    variable:
+                    <literallayout class='monospaced'>
+     PACKAGE_ARCH = "${MACHINE_ARCH}"
+                    </literallayout>
+                    If you select a specific tuning for the recipe,
+                    the OpenEmbedded build system enables the package
+                    architecture defined by that tuning file.</para>
+                    <para>When you do not specifically enable a package
+                    architecture through the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>,
+                    The OpenEmbedded build system defaults to the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_ARCH'><filename>TUNE_ARCH</filename></ulink>
+                    setting:
+                    <literallayout class='monospaced'>
+     PACKAGE_ARCH == TUNE_ARCH
+                    </literallayout>
+                    </para></listitem>
+                <listitem><para><emphasis>Choose a Generic Tuning File if Possible:</emphasis>
+                    Some tunes are more generic and can run on multiple targets
+                    (e.g. an <filename>armv5</filename> set of packages could
+                    run on <filename>armv6</filename> and
+                    <filename>armv7</filename> processors in most cases).
+                    Similarly, <filename>i486</filename> binaries could work
+                    on <filename>i586</filename> and higher processors.</para>
+                    <para>If you select the same tune for several different
+                    machines, the OpenEmbedded build system reuses software
+                    previously built, thus speeding up the overall build time.
+                    Realize that even though a new sysroot for each machine is
+                    generated, the software is not recompiled and only one
+                    package feed exists.
+                    </para></listitem>
+                <listitem><para><emphasis>Manage Granular Level Packaging:</emphasis>
+                    Sometimes cases exist where where injecting another level
+                    of package architecture beyond the three higher levels
+                    noted earlier can be useful.
+                    For example, consider the <filename>emgd</filename>
+                    graphics stack in the
+                    <filename>meta-intel</filename> layer.
+                    In this layer, a subset of software exists that is
+                    compiled against something different from the rest of the
+                    generic packages.
+                    You can examine the key code in the
+                    <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories</ulink>
+                    in <filename>classes/emgd-gl.bbclass</filename>.
+                    For a specific set of packages, the code redefines
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>.
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXTRA_ARCHS'><filename>PACKAGE_EXTRA_ARCHS</filename></ulink>
+                    is then appended with this extra tune name in
+                    <filename>meta-intel-emgd.inc</filename>.
+                    The result is that when searching for packages, the
+                    build system uses a four-level search and the packages
+                    in this new level are preferred as compared to the standard
+                    tune.
+                    The overall result is that the build system reuses most
+                    software from the common tune except for specific cases
+                    as needed.
+                    </para></listitem>
+                <listitem><para><emphasis>Use Tools to Debug Issues:</emphasis>
+                    Sometimes you can run into situations where software is
+                    being rebuilt when you think it should not be.
+                    For example, the OpenEmbedded build system might not be
+                    using shared state between machines when you think it
+                    should be.
+                    These types of situations are usually due to references
+                    to variables such as <filename>MACHINE</filename>
+                    in code that is supposed to only be "tune" specific.</para>
+                    <note>
+                        Patches to fix any issues identified are most welcome
+                        as these issues occasionally do occur.
+                    </note>
+                    <para>For such cases, you can use some tools to help you
+                    sort out the situation:
+                    <itemizedlist>
+                        <listitem><para><emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis>
+                            You can find this tool in the
+                            <filename>scripts</filename> directory of the
+                            Source Repositories.
+                            See the comments in the script for information on
+                            how to use the tool.
+                            </para></listitem>
+                        <listitem><para><emphasis>BitBake's "-S printdiff" Option:</emphasis>
+                            Using this option causes BitBake to try to
+                            establish the closest signature match it can
+                            (e.g. in the shared state cache) and then run
+                            <filename>bitbake-diffsigs</filename> over the
+                            matches to determine the stamps and delta where
+                            these two stamp trees diverge.
+                            </para></listitem>
+                    </itemizedlist>
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
     <section id='working-with-packages'>
         <title>Working with Packages</title>