dev-manual: Added section on upgrading recipes.

Section covers AUH, devtool and manual.  Still need to add
manual.

(From yocto-docs rev: 56f04b1fcc8673e20df6d8f5c65120b03cad31e7)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 456 insertions, 0 deletions

diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index 745e1c0ab6..8264331dd1 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -4068,6 +4068,462 @@
         </section>
     </section>
 
+    <section id='gs-upgrading-recipes'>
+        <title>Upgrading Recipes</title>
+
+        <para>
+            Over time, upstream developers publish new versions for software
+            built by layer recipes.
+            It is recommended to keep recipes up-to-date with upstream
+            version releases.
+            You can use the Automated Upgrade Helper (AUH) to set up
+            automatic version upgrades.
+            Alternatively, you can use <filename>devtool upgrade</filename>
+            to set up semi-automatic version upgrades.
+            Finally, you can even manually upgrade a recipe by editing the
+            recipe itself.
+        </para>
+
+        <section id='gs-using-the-auto-upgrade-helper'>
+            <title>Using the Auto Upgrade Helper (AUH)</title>
+
+            <para>
+                The AUH utility works in conjunction with the
+                OpenEmbedded build system in order to automatically generate
+                upgrades for recipes based on new versions being
+                published upstream.
+                Use AUH when you want to create a service that performs the
+                upgrades automatically and optionally sends you an email with
+                the results.
+            </para>
+
+            <para>
+                AUH allows you to update several recipes with a single use.
+                You can also optionally perform build and integration tests
+                using images with the results saved to your hard drive and
+                emails of results optionally sent to recipe maintainers.
+                Finally, AUH creates Git commits with appropriate commit
+                messages in the layer's tree for the changes made to recipes.
+                <note>
+                    Conditions do exist when you should not use AUH to upgrade
+                    recipes and you should instead use either
+                    <filename>devtool upgrade</filename> or upgrade your
+                    recipes manually:
+                    <itemizedlist>
+                        <listitem><para>
+                            When AUH cannot complete the upgrade sequence.
+                            This situation usually results because custom
+                            patches carried by the recipe cannot be
+                            automatically rebased to the new version.
+                            In this case, <filename>devtool upgrade</filename>
+                            allows you to manually resolve conflicts.
+                            </para></listitem>
+                        <listitem><para>
+                            When for any reason you want fuller control over
+                            the upgrade process.
+                            For example, when you want special arrangements
+                            for testing.
+                            </para></listitem>
+                    </itemizedlist>
+                </note>
+            </para>
+
+            <para>
+                The following steps describe how to set up the AUH utility:
+                <orderedlist>
+                    <listitem><para>
+                        <emphasis>Be Sure the Development Host is Set Up:</emphasis>
+                        You need to be sure that your development host is
+                        set up to use the Yocto Project.
+                        For information on how to set up your host, see the
+                        "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-the-development-host-to-use-the-yocto-project'>Setting Up the Development Host to Use the Yocto Project</ulink>"
+                        section in the Yocto Project Development Tasks Manual.
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Make Sure Git is Configured:</emphasis>
+                        The AUH utility requires Git to be configured because
+                        AUH uses Git to save upgrades.
+                        Thus, you must have Git user and email configured.
+                        The following command shows your configurations:
+                        <literallayout class='monospaced'>
+     $ git config --list
+                        </literallayout>
+                        If you do not have the user and email configured, you
+                        can use the following commands to do so:
+                        <literallayout class='monospaced'>
+     $ git config --global user.name <replaceable>some_name</replaceable>
+     $ git config --global user.email <replaceable>username</replaceable>@<replaceable>domain</replaceable>.com
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Clone the AUH Respository:</emphasis>
+                        To use AUH, you must clone the repository onto your
+                        development host.
+                        The following command uses Git to create a local
+                        copy of the repository on your system:
+                        <literallayout class='monospaced'>
+     $ git clone git://git.yoctoproject.org/auto-upgrade-helper
+     Cloning into 'auto-upgrade-helper'...
+     remote: Counting objects: 768, done.
+     remote: Compressing objects: 100% (300/300), done.
+     remote: Total 768 (delta 499), reused 703 (delta 434)
+     Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
+     Resolving deltas: 100% (499/499), done.
+     Checking connectivity... done.
+                        </literallayout>
+                        AUH is not part of the
+                        <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core (OE-Core)</ulink>
+                        or
+                        <ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink>
+                        repositories.
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Create a Dedicated Build Directory:</emphasis>
+                        Run the
+                        <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink>
+                        script to create a fresh build directory that you
+                        use exclusively for running the AUH utility:
+                        <literallayout class='monospaced'>
+     $ cd ~/poky
+     $ source oe-init-build-env <replaceable>your_AUH_build_directory</replaceable>
+                        </literallayout>
+                        Re-using an existing build directory and its
+                        configurations is not recommended as existing settings
+                        could cause AUH to fail or behave undesirably.
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Make Configurations in Your Local Configuration File:</emphasis>
+                        Several settings need to exist in the
+                        <filename>local.conf</filename> file in the build
+                        directory you just created for AUH.
+                        Make these following configurations:
+                        <itemizedlist>
+                            <listitem><para>
+                                Enable "distrodata" as follows:
+                                <literallayout class='monospaced'>
+     INHERIT =+ "distrodata"
+                                </literallayout>
+                                </para></listitem>
+                            <listitem><para>
+                                If you want to enable
+                                <ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality'>Build History</ulink>,
+                                which is optional, you need the following
+                                lines in the
+                                <filename>conf/local.conf</filename> file:
+                                <literallayout class='monospaced'>
+     INHERIT =+ "buildhistory"
+     BUILDHISTORY_COMMIT = "1"
+                                </literallayout>
+                                With this configuration and a successful
+                                upgrade, a build history "diff" file appears in
+                                the
+                                <filename>upgrade-helper/work/recipe/buildhistory-diff.txt</filename>
+                                file found in your build directory.
+                                </para></listitem>
+                            <listitem><para>
+                                If you want to enable testing through the
+                                <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink>
+                                class, which is optional, you need to have the
+                                following set in your
+                                <filename>conf/local.conf</filename> file:
+                                <literallayout class='monospaced'>
+     INHERIT += "testimage"
+                                </literallayout>
+                                <note>
+                                    If your distro does not enable by default
+                                    ptest, which Poky does, you need the
+                                    following in your
+                                    <filename>local.conf</filename> file:
+                                    <literallayout class='monospaced'>
+     DISTRO_FEATURES_append = " ptest"
+                                    </literallayout>
+                                </note>
+                                </para></listitem>
+                        </itemizedlist>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Optionally Start a vncserver:</emphasis>
+                        If you are running in a server without an X11 session,
+                        you need to start a vncserver:
+                        <literallayout class='monospaced'>
+     $ vncserver :1
+     $ export DISPLAY=:1
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Create and Edit an AUH Configuration File:</emphasis>
+                        You need to have the
+                        <filename>upgrade-helper/upgrade-helper.conf</filename>
+                        configuration file in your build directory.
+                        You can find a sample configuration file in the
+                        <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/'>AUH source repository</ulink>.
+                        </para>
+
+                        <para>Read through the sample file and make
+                        configurations as needed.
+                        For example, if you enabled build history in your
+                        <filename>local.conf</filename> as described earlier,
+                        you must enable it in
+                        <filename>upgrade-helper.conf</filename>.</para>
+
+                        <para>Also, if you are using the default
+                        <filename>maintainers.inc</filename> file supplied
+                        with Poky and located in
+                        <filename>meta-yocto</filename> and you do not set a
+                        "maintainers_whitelist" or "global_maintainer_override"
+                        in the <filename>upgrade-helper.conf</filename>
+                        configuration, and you specify "-e all" on the
+                        AUH command-line, the utility automatically sends out
+                        emails to all the default maintainers.
+                        Please avoid this.
+                        </para></listitem>
+                </orderedlist>
+            </para>
+
+            <para>
+                This next set of examples describes how to use the AUH:
+                <itemizedlist>
+                    <listitem><para>
+                        <emphasis>Upgrading a Specific Recipe:</emphasis>
+                        To upgrade a specific recipe, use the following
+                        form:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py <replaceable>recipe_name</replaceable>
+                        </literallayout>
+                        For example, this command upgrades the
+                        <filename>xmodmap</filename> recipe:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py xmodmap
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Upgrading a Specific Recipe to a Particular Version:</emphasis>
+                        To upgrade a specific recipe to a particular version,
+                        use the following form:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py <replaceable>recipe_name</replaceable> -t <replaceable>version</replaceable>
+                        </literallayout>
+                        For example, this command upgrades the
+                        <filename>xmodmap</filename> recipe to version
+                        1.2.3:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py xmodmap -t 1.2.3
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Upgrading all Recipes to the Latest Versions and Suppressing Email Notifications:</emphasis>
+                        To upgrade all recipes to their most recent versions
+                        and suppress the email notifications, use the following
+                        command:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py all
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        <emphasis>Upgrading all Recipes to the Latest Versions and Send Email Notifications:</emphasis>
+                        To upgrade all recipes to their most recent versions
+                        and send email messages to maintainers for each
+                        attempted recipe as well as a status email, use the
+                        following command:
+                        <literallayout class='monospaced'>
+     $ upgrade-helper.py -e all
+                        </literallayout>
+                        </para></listitem>
+                </itemizedlist>
+            </para>
+
+            <para>
+                Once you have run the AUH utility, you can find the results
+                in the AUH build directory:
+                <literallayout class='monospaced'>
+     ${BUILDDIR}/upgrade-helper/<replaceable>timestamp</replaceable>
+                </literallayout>
+                The AUH utility also creates recipe update commits from
+                successful upgrade attempts in the layer tree.
+            </para>
+
+            <para>
+                You can easily set up to run the AUH utility on a regular
+                basis by using a cron job.
+                See the
+                <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh'><filename>weeklyjob.sh</filename></ulink>
+                file distributed with the utility for an example.
+            </para>
+        </section>
+
+        <section id='gs-using-devtool-upgrade'>
+            <title>Using <filename>devtool upgrade</filename></title>
+
+            <para>
+                As mentioned earlier, an alternative method for upgrading
+                recipes to newer versions is to use
+                <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool upgrade</filename></ulink>.
+                You can read about <filename>devtool upgrade</filename> in
+                general in the
+                "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software'>Use <filename>devtool upgrade</filename> to Create a Version of the Recipe that Supports a Newer Version of the Software</ulink>"
+                section in the Yocto Project Application Development and the
+                Extensible Software Development Kit (eSDK) Manual.
+            </para>
+
+            <para>
+                To see all the command-line options available with
+                <filename>devtool upgrade</filename>, use the following help
+                command:
+                <literallayout class='monospaced'>
+     $ devtool upgrade -h
+                </literallayout>
+            </para>
+
+            <para>
+                If you want to find out what version a recipe is currently at
+                upstream without any attempt to upgrade your local version of
+                the recipe, you can use the following command:
+                <literallayout class='monospaced'>
+     $ devtool latest-version <replaceable>recipe_name</replaceable>
+                </literallayout>
+            </para>
+
+            <para>
+                As mentioned in the previous section describing AUH,
+                <filename>devtool upgrade</filename> works in a
+                less-automated manner than AUH.
+                Specifically, <filename>devtool upgrade</filename> only
+                works on a single recipe that you name on the command line,
+                cannot perform build and integration testing using images,
+                and does not automatically generate commits for changes in
+                the source tree.
+                Despite all these "limitations",
+                <filename>devtool upgrade</filename> updates the recipe file
+                to the new upstream version and attempts to rebase custom
+                patches contained by the recipe as needed.
+                <note>
+                    AUH uses much of <filename>devtool upgrade</filename>
+                    behind the scenes making AUH somewhat of a "wrapper"
+                    application for <filename>devtool upgrade</filename>.
+                </note>
+            </para>
+
+            <para>
+                A typical scenario involves having used Git to clone an
+                upstream repository that you use during build operations.
+                Because you are (or have) built the recipe in the past, the
+                layer is likely added to your configuration already.
+                If for some reason, the layer is not added, you could add
+                it easily using the
+                <ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'><filename>bitbake-layers</filename></ulink>
+                script.
+                For example, suppose you use the <filename>nano.bb</filename>
+                recipe from the <filename>meta-oe</filename> layer in the
+                <filename>meta-openembedded</filename> repository.
+                For this example, assume that the layer has been cloned into
+                following area:
+                <literallayout class='monospaced'>
+     /home/scottrif/meta-openembedded
+                </literallayout>
+                The following command from your
+                <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
+                adds the layer to your build configuration (i.e.
+                <filename>${BUILDDIR}/conf/bblayers.conf</filename>):
+                <literallayout class='monospaced'>
+     $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
+     NOTE: Starting bitbake server...
+     Parsing recipes: 100% |##########################################| Time: 0:00:55
+     Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
+     Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
+     Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
+     Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
+     Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
+                </literallayout>
+                For this example, assume that the <filename>nano.bb</filename>
+                recipe that is upstream has a 2.9.3 version number.
+                However, the version in the local repository is 2.7.4.
+                The following command from your build directory automatically
+                upgrades the recipe for you:
+                <note>
+                    Using the <filename>-V</filename> option is not necessary.
+                    Omitting the version number causes
+                    <filename>devtool upgrade</filename> to upgrade the recipe
+                    to the most recent version.
+                </note>
+                <literallayout class='monospaced'>
+     $ devtool upgrade nano -V 2.9.3
+     NOTE: Starting bitbake server...
+     NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
+     Parsing recipes: 100% |##########################################| Time: 0:00:46
+     Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
+     NOTE: Extracting current version source...
+     NOTE: Resolving any missing task queue dependencies
+            .
+            .
+            .
+     NOTE: Executing SetScene Tasks
+     NOTE: Executing RunQueue Tasks
+     NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
+     Adding changed files: 100% |#####################################| Time: 0:00:00
+     NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
+     NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
+                </literallayout>
+                Continuing with this example, you can use
+                <filename>devtool build</filename> to build the newly upgraded
+                recipe:
+                <literallayout class='monospaced'>
+     $ devtool build nano
+     NOTE: Starting bitbake server...
+     Loading cache: 100% |################################################################################################| Time: 0:00:01
+     Loaded 2040 entries from dependency cache.
+     Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
+     Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
+     NOTE: Resolving any missing task queue dependencies
+            .
+            .
+            .
+     NOTE: Executing SetScene Tasks
+     NOTE: Executing RunQueue Tasks
+     NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
+     NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
+                </literallayout>
+                Within the <filename>devtool upgrade</filename> workflow,
+                opportunity exists to deploy and test your rebuilt software.
+                For this example, however, running
+                <filename>devtool finish</filename> cleans up the workspace
+                once the source in your workspace is clean.
+                This usually means using Git to stage and submit commits
+                for the changes generated by the upgrade process.
+            </para>
+
+            <para>
+                Once the tree is clean, you can clean things up in this
+                example with the following command from the
+                <filename>${BUILDDIR}/workspace/sources/nano</filename>
+                directory:
+                <literallayout class='monospaced'>
+     $ devtool finish nano meta-oe
+     NOTE: Starting bitbake server...
+     Loading cache: 100% |################################################################################################| Time: 0:00:00
+     Loaded 2040 entries from dependency cache.
+     Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
+     Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
+     NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
+     NOTE: Updating recipe nano_2.9.3.bb
+     NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
+     NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
+     NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
+                </literallayout>
+                Using the <filename>devtool finish</filename> command cleans
+                up the workspace and creates a patch file based on your
+                commits.
+                The tool puts all patch files back into the source directory
+                in a sub-directory named <filename>nano</filename> in this
+                case.
+            </para>
+        </section>
+
+        <section id='dev-manually-upgrading-a-recipe'>
+            <title>Manually Upgrading a Recipe</title>
+
+        </section>
+    </section>
+
     <section id='finding-the-temporary-source-code'>
         <title>Finding Temporary Source Code</title>