ref-manual: Added the migration section for 2.1

(From yocto-docs rev: ed1eec9139d8a9caf32f0cc896f651cafecece4f)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 540 insertions, 12 deletions

diff --git a/documentation/ref-manual/migration.xml b/documentation/ref-manual/migration.xml
index 670e1c1f7d..e6c0aa36b8 100644
--- a/documentation/ref-manual/migration.xml
+++ b/documentation/ref-manual/migration.xml
@@ -2447,9 +2447,10 @@
                     </literallayout>
                     </para></listitem>
                 <listitem><para>
-                    <filename>d.delVar('VARNAME')</filename> and
-                    <filename>d.setVar('VARNAME', None)</filename> result
-                    in the variable and all of its overrides being cleared out.
+                    <filename>d.delVar('</filename><replaceable>varname</replaceable><filename>')</filename> and
+                    <filename>d.setVar('</filename><replaceable>varname</replaceable><filename>', None)</filename>
+                    result in the variable and all of its overrides being
+                    cleared out.
                     Before the change, only the non-overridden values
                     were cleared.
                     </para></listitem>
@@ -2734,23 +2735,550 @@
         Yocto Project 2.1 Release from the prior release.
     </para>
 
-    <section id='migration-2.1-placeholder'>
-        <title>Stuff to Remember When Creating this Section</title>
+    <section id='migration-2.1-variable-expansion-in-python-functions'>
+        <title>Variable Expansion in Python Functions</title>
 
         <para>
-            This is a note from Alexander Kanavin regarding qt4 stuff.
+            Variable expressions, such as
+            <filename>${</filename><replaceable>varname</replaceable><filename>}</filename>
+            no longer expand automatically within Python functions.
+            Suppressing expansion was done to allow Python functions to
+            construct shell scripts or other code for situations in which you
+            do not want such expressions expanded.
+            For any existing code that relies on these expansions, you need to
+            change the expansions to either expand the value of individual
+            variables through <filename>d.getVar()</filename>.
+            To alternatively expand more complex expressions,
+            use <filename>d.expand()</filename>.
+        </para>
+    </section>
+
+    <section id='migration-2.1-overrides-must-now-be-lower-case'>
+        <title>Overrides Must Now be Lower-Case</title>
+
+        <para>
+            The convention for overrides has always been for them to be
+            lower-case characters.
+            This practice is now a requirement as BitBake's datastore now
+            assumes lower-case characters in order to give a slight performance
+            boost during parsing.
+            In practical terms, this requirement means that anything that ends
+            up in
+            <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
+            must now appear in lower-case characters (e.g. values for
+            <filename>MACHINE</filename>, <filename>TARGET_ARCH</filename>,
+            <filename>DISTRO</filename>, and also recipe names if
+            <filename>_pn-</filename><replaceable>recipename</replaceable>
+            overrides are to be effective).
+        </para>
+    </section>
+
+    <section id='migration-2.1-expand-parameter-to-getvar-and-getvarflag-now-mandatory'>
+        <title>Expand Parameter to <filename>getVar()</filename> and
+            <filename>getVarFlag()</filename> is Now Mandatory</title>
+
+        <para>
+            The expand parameter to <filename>getVar()</filename> and
+            <filename>getVarFlag()</filename> previously defaulted to
+            False if not specified.
+            Now, however, no default exists so one must be specified.
+            You must change any <filename>getVar()</filename> calls that
+            do not specify the final expand parameter to calls that do specify
+            the parameter.
+            You can run the following <filename>sed</filename> command at the
+            base of a layer to make this change:
+            <literallayout class='monospaced'>
+     sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
+     sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVarFlag *`
+            </literallayout>
             <note>
-                Qt4 recipes and classes have been removed in YP 2.1 because
-                Qt4.x has reached end of life upstream. For legacy applications
-                and LSB compliancy purposes Qt4 can be found in meta-qt4
-                repository, and for any new development Qt5 is available at
-                meta-qt5 repository(provide links to both).
+                The reason for this change is that it prepares the way for
+                changing the default to True in a future Yocto Project release.
+                This future change is a much more sensible default than False.
+                However, the change needs to be made gradually as a sudden
+                change of the default would potentially cause side-effects
+                that would be difficult to detect.
             </note>
         </para>
     </section>
-</section>
 
+    <section id='migration-2.1-makefile-environment-changes'>
+        <title>Makefile Environment Changes</title>
+
+        <para>
+            <link linkend='var-EXTRA_OEMAKE'><filename>EXTRA_OEMAKE</filename></link>
+            now defaults to "" instead of "-e MAKEFLAGS=".
+            Setting <filename>EXTRA_OEMAKE</filename> to "-e MAKEFLAGS=" by
+            default was a historical accident that has required many classes
+            (e.g. <filename>autotools</filename>, <filename>module</filename>)
+            and recipes to override this default in order to work with
+            sensible build systems.
+            When upgrading to the release, you must edit any recipe that
+            relies upon this old default by either setting
+            <filename>EXTRA_OEMAKE</filename> back to "-e MAKEFLAGS=" or by
+            explicitly setting any required variable value overrides using
+            <filename>EXTRA_OEMAKE</filename>, which is typically only needed
+            when a Makefile sets a default value for a variable that is
+            inappropriate for cross-compilation using the "=" operator rather
+            than the "?=" operator.
+        </para>
+    </section>
+
+    <section id='migration-2.1-libexecdir-reverted-to-prefix-libexec'>
+        <title><filename>libexecdir</filename> Reverted to <filename>${prefix}/libexec</filename></title>
+
+        <para>
+            The use of <filename>${libdir}/${BPN}</filename> as
+            <filename>libexecdir</filename> is different as compared to all
+            other mainstream distributions, which either uses
+            <filename>${prefix}/libexec</filename> or
+            <filename>${libdir}</filename>.
+            The use is also contrary to the GNU Coding Standards
+            (i.e. <ulink url='https://www.gnu.org/prep/standards/html_node/Directory-Variables.html'></ulink>)
+            that suggest <filename>${prefix}/libexec</filename> and also
+            notes that any package-specific nesting should be done by the
+            package itself.
+            Finally, having <filename>libexecdir</filename> change between
+            recipes makes it very difficult for different recipes to invoke
+            binaries that have been installed into
+            <filename>libexecdir</filename>.
+            The Filesystem Hierarchy Standard
+            (i.e. <ulink url='http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html'></ulink>)
+            now recognizes the use of <filename>${prefix}/libexec/</filename>,
+            giving distributions the choice between
+            <filename>${prefix}/lib</filename> or
+            <filename>${prefix}/libexec</filename> without breaking FHS.
+        </para>
+    </section>
+
+    <section id='migration-2.1-ac-cv-sizeof-off-t-no-longer-cached-in-site-files'>
+        <title><filename>ac_cv_sizeof_off_t</filename> is No Longer Cached in Site Files</title>
+
+        <para>
+            For recipes inheriting the
+            <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
+            class, <filename>ac_cv_sizeof_off_t</filename> is no longer cached
+            in the site files for <filename>autoconf</filename>.
+            The reason for this change is because the
+            <filename>ac_cv_sizeof_off_t</filename> value is not necessarily
+            static per architecture as was previously assumed.
+            Rather, the value changes based on whether large file support is
+            enabled.
+            For most software that uses <filename>autoconf</filename>, this
+            change should not be a problem.
+            However, if you have a recipe that bypasses the standard
+            <link linkend='ref-tasks-configure'><filename>do_configure</filename></link>
+            task from the <filename>autotools</filename> class and the software
+            the recipe is building uses a very old version of
+            <filename>autoconf</filename>, the recipe might be incapable of
+            determining the correct size of <filename>off_t</filename> during
+            <filename>do_configure</filename>.
+        </para>
+
+        <para>
+            The best course of action is to patch the software as necessary
+            to allow the default implementation from the
+            <filename>autotools</filename> class to work such that
+            <filename>autoreconf</filename> succeeds and produces a working
+            configure script), and to remove the
+            overridden <filename>do_configure</filename> task such that the
+            default implementation does get used.
+        </para>
+    </section>
+
+    <section id='migration-2.1-image-generation-split-out-from-filesystem-generation'>
+        <title>Image Generation is Now Split Out from Filesystem Generation</title>
+
+        <para>
+            Previously, for image recipes the
+            <link linkend='ref-tasks-rootfs'><filename>do_rootfs</filename></link>
+            task assembled the filesystem and then from that filesystem
+            generated images.
+            With this Yocto Project release, image generation is split into
+            separate
+            <link linkend='ref-tasks-image'><filename>do_image_*</filename></link>
+            tasks for clarity both in operation and in the code.
+        </para>
+
+        <para>
+            For most cases, this change does not present any problems.
+            However, if you have made customizations that directly modify the
+            <filename>do_rootfs</filename> task or that mention
+            <filename>do_rootfs</filename>, you might need to update those
+            changes.
+            In particular, if you had added any tasks after
+            <filename>do_rootfs</filename>, you should make edits so that
+            those tasks are after the
+            <link linkend='ref-tasks-image-complete'><filename>do_image_complete</filename></link>
+            task rather than before the task so that the your added tasks
+            run at the correct time.
+        </para>
+
+        <para>
+            A minor part of this restructuring is that the post-processing
+            definitions and functions have been moved from the
+            <link linkend='ref-classes-image'><filename>image</filename></link>
+            class to the
+            <link linkend='ref-classes-rootfs*'><filename>rootfs-postcommands</filename></link>
+            class.
+            Functionally, however, they remain unchanged.
+        </para>
+    </section>
+
+    <section id='migration-2.1-removed-recipes'>
+        <title>Removed Recipes</title>
+
+        <para>
+            The following recipes have been removed in the 2.1 release:
+            <itemizedlist>
+                <listitem><para><filename>gcc</filename> version 4.8:
+                    Versions 4.9 and 5.3 remain.
+                    </para></listitem>
+                <listitem><para><filename>qt4</filename>:
+                    All support for Qt 4.x has been moved out to a separate
+                    <filename>meta-qt4</filename> layer because Qt 4 is no
+                    longer supported upstream.
+                    </para></listitem>
+                <listitem><para><filename>x11vnc</filename>:
+                    Moved to the <filename>meta-oe</filename> layer.
+                    </para></listitem>
+                <listitem><para><filename>linux-yocto-3.14</filename>:
+                    No longer supported.
+                    </para></listitem>
+                <listitem><para><filename>linux-yocto-3.19</filename>:
+                    No longer supported.
+                    </para></listitem>
+                <listitem><para><filename>libjpeg</filename>:
+                    Replaced by the <filename>libjpeg-turbo</filename> recipe.
+                    </para></listitem>
+                <listitem><para><filename>pth</filename>:
+                    Became obsolete.
+                    </para></listitem>
+                <listitem><para><filename>liboil</filename>:
+                    Recipe is no longer needed and has been moved to the
+                    <filename>meta-multimedia</filename> layer.
+                    </para></listitem>
+                <listitem><para><filename>gtk-theme-torturer</filename>:
+                    Recipe is no longer needed and has been moved to the
+                    <filename>meta-gnome</filename> layer.
+                    </para></listitem>
+                <listitem><para><filename>gnome-mime-data</filename>:
+                    Recipe is no longer needed and has been moved to the
+                    <filename>meta-gnome</filename> layer.
+                    </para></listitem>
+                <listitem><para><filename>udev</filename>:
+                    Replaced by the <filename>eudev</filename> recipe for
+                    compatibility when using <filename>sysvinit</filename>
+                    with newer kernels.
+                    </para></listitem>
+                <listitem><para><filename>python-pygtk</filename>:
+                    Recipe became obsolete.
+                    </para></listitem>
+                <listitem><para><filename>adt-installer</filename>:
+                    Recipe became obsolete.
+                    See the
+                    "<link linkend='migration-2.1-adt-removed'>ADT Removed</link>"
+                    section for more information.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-class-changes'>
+        <title>Class Changes</title>
 
+        <para>
+            The following classes have changed:
+            <itemizedlist>
+                <listitem><para><filename>autotools_stage</filename>:
+                    Removed because the
+                    <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
+                    class now provides its functionality.
+                    Recipes that inherited from
+                    <filename>autotools_stage</filename> should now inherit
+                    from <filename>autotools</filename> instead.
+                    </para></listitem>
+                <listitem><para><filename>boot-directdisk</filename>:
+                    Merged into the
+                    <link linkend='ref-classes-image-vm'><filename>image-vm</filename></link>
+                    class.
+                    The <filename>boot-directdisk</filename> class was rarely
+                    directly used.
+                    Consequently, this change should not cause any issues.
+                    </para></listitem>
+                <listitem><para><filename>bootimg</filename>:
+                    Merged into the
+                    <link linkend='ref-classes-image-live'><filename>image-live</filename></link>
+                    class.
+                    The <filename>bootimg</filename> class was rarely
+                    directly used.
+                    Consequently, this change should not cause any issues.
+                    </para></listitem>
+                <listitem><para><filename>packageinfo</filename>:
+                    Removed due to its limited use by the Hob UI, which has
+                    itself been removed.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-build-system-ui-changes'>
+        <title>Build System User Interface Changes</title>
+
+        <para>
+            The following changes have been made to the build system user
+            interface:
+            <itemizedlist>
+                <listitem><para><emphasis>Hob GTK+-based UI</emphasis>:
+                    Removed because it is unmaintained and based on the
+                    outdated GTK+ 2 library.
+                    The Toaster web-based UI is much more capable and is
+                    actively maintained.
+                    See the
+                    "<ulink url='&YOCTO_DOCS_TOAST_URL;#using-the-toaster-web-interface'>Using the Toaster Web Interface</ulink>"
+                    section in the Yocto Project Toaster User Manual for more
+                    information on this interface.
+                    </para></listitem>
+                <listitem><para><emphasis>"puccho" BitBake UI</emphasis>:
+                    Removed because is unmaintained and no longer useful.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-adt-removed'>
+        <title>ADT Removed</title>
+
+        <para>
+            The Application Development Toolkit (ADT) has been removed
+            because its functionality almost completely overlapped with the
+            <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-using-the-standard-sdk'>standard SDK</ulink>
+            and the
+            <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-extensible'>extensible SDK</ulink>.
+            For information on these SDKs and how to build and use them, see the
+            <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>.
+        </para>
+    </section>
+
+    <section id='migration-2.1-poky-reference-distribution-changes'>
+        <title>Poky Reference Distribution Changes</title>
+
+        <para>
+            The following changes have been made for the Poky distribution:
+            <itemizedlist>
+                <listitem><para>
+                    The <filename>meta-yocto</filename> layer has been renamed
+                    to <filename>meta-poky</filename> to better match its
+                    purpose, which is to provide the Poky reference
+                    distribution.
+                    The <filename>meta-yocto-bsp</filename> layer retains its
+                    original name since it provides reference machines for
+                    the Yocto Project and it is otherwise unrelated to Poky.
+                    References to <filename>meta-yocto</filename> in your
+                    <filename>conf/bblayers.conf</filename> should
+                    automatically be updated, so you should not need to change
+                    anything unless you are relying on this naming elsewhere.
+                    </para></listitem>
+                <listitem><para>
+                    The
+                    <link linkend='ref-classes-uninative'><filename>uninative</filename></link>
+                    class is now enabled by default in Poky.
+                    This class attempts to isolate the build system from the
+                    host distribution's C library and makes re-use of native
+                    shared state artifacts across different host distributions
+                    practical.
+                    With this class enabled, a tarball containing a pre-built
+                    C library is downloaded at the start of the build.</para>
+
+                    <para>The <filename>uninative</filename> class is enabled
+                    through the
+                    <filename>meta/conf/distro/include/yocto-uninative.inc</filename>
+                    file, which for those not using the Poky distribution, can
+                    include to easily enable the same functionality.</para>
+
+                    <para>Alternatively, if you wish to build your own
+                    <filename>uninative</filename> tarball, you can do so by
+                    building the <filename>uninative-tarball</filename> recipe,
+                    making it available to your build machines
+                    (e.g. over HTTP/HTTPS) and setting a similar configuration
+                    as the one set by <filename>yocto-uninative.inc</filename>.
+                    </para></listitem>
+                <listitem><para>
+                    Static library generation, for most cases, is now disabled
+                    by default in the Poky distribution.
+                    Disabling this generation saves some build time as well
+                    as the size used for build output artifacts.</para>
+
+                    <para>Disabling this library generation is accomplished
+                    through a
+                    <filename>meta/conf/distro/include/no-static-libs.inc</filename>,
+                    which for those not using the Poky distribution can
+                    easily include to enable the same functionality.</para>
+
+                    <para>Any recipe that needs to opt-out of having the
+                    "--disable-static" option specified on the configure
+                    command line either because it is not a supported option
+                    for the configure script or because static libraries are
+                    needed should set the following variable:
+                    <literallayout class='monospaced'>
+     DISABLE_STATIC = ""
+                    </literallayout>
+                    </para></listitem>
+                <listitem><para>
+                    The separate <filename>poky-tiny</filename> distribution
+                    now uses the musl C library instead of a heavily pared
+                    down <filename>glibc</filename>.
+                    Using <filename>glibc</filename> results in a smaller
+                    distribution and facilitates much greater maintainability
+                    because musl is designed to have a small footprint.</para>
+
+                    <para>If you have used <filename>poky-tiny</filename> and
+                    have customized the <filename>glibc</filename>
+                    configuration you will need to redo those customizations
+                    with musl when upgrading to the new release.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-packaging-changes'>
+        <title>Packaging Changes</title>
+
+        <para>
+            The following changes have been made to packaging:
+            <itemizedlist>
+                <listitem><para>
+                    The <filename>runuser</filename> and
+                    <filename>mountpoint</filename> binaries, which were
+                    previously in the main <filename>util-linux</filename>
+                    package, have been split out into the
+                    <filename>util-linux-runuser</filename> and
+                    <filename>util-linux-mountpoint</filename> packages,
+                    respectively.
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>python-elementtree</filename> package has
+                    been merged into the <filename>python-xml</filename>
+                    package.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-tuning-file-changes'>
+        <title>Tuning File Changes</title>
+
+        <para>
+            The following changes have been made to the tuning files:
+            <itemizedlist>
+                <listitem><para>
+                    The "no-thumb-interwork" tuning feature has been dropped
+                    from the ARM tune include files.
+                    Because interworking is required for ARM EABI, attempting
+                    to disable it through a tuning feature no longer makes
+                    sense.
+                    <note>
+                        Support for ARM OABI was deprecated in gcc 4.7.
+                    </note>
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>tune-cortexm*.inc</filename> and
+                    <filename>tune-cortexr4.inc</filename> files have been
+                    removed because they are poorly tested.
+                    Until the OpenEmbedded build system officially gains
+                    support for CPUs without an MMU, these tuning files would
+                    probably be better maintained in a separate layer
+                    if needed.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-2.1-miscellaneous-changes'>
+        <title>Miscellaneous Changes</title>
+
+        <para>
+            These additional changes exist:
+            <itemizedlist>
+                <listitem><para>
+                    The minimum Git version has been increased to 1.8.3.1.
+                    If your host distribution does not provide a sufficiently
+                    recent version, you can install the buildtools, which
+                    will provide it.
+                    </para></listitem>
+                <listitem><para>
+                    The buggy and incomplete support for the RPM version 4
+                    package manager has been removed.
+                    The well-tested and maintained support for RPM version 5
+                    remains.
+                    </para></listitem>
+                <listitem><para>
+                    The
+                    <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component'><filename>devtool modify</filename></ulink>
+                    command now defaults to extracting the source since that
+                    is most commonly expected.
+                    The "-x" or "--extract" options are now no-ops.
+                    If you wish to provide your own existing source tree, you
+                    will now need to specify either the "-n" or
+                    "--no-extract" option when running
+                    <filename>devtool modify</filename>.
+                    </para></listitem>
+                <listitem><para>
+                    If the formfactor for a machine is either not supplied
+                    or does not specify whether a keyboard is attached, then
+                    the default is to assume a keyboard is attached rather
+                    than assume no keyboard.
+                    <note>
+                        This change primarily affects the Sato UI.
+                    </note>
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>.debug</filename> directory packaging is
+                    now automatic.
+                    If your recipe builds software that installs binaries into
+                    directories other than the standard ones, you no longer
+                    need to take care of setting
+                    <filename>FILES_${PN}-dbg</filename> to pick up the
+                    resulting <filename>.debug</filename> directories as these
+                    directories are automatically found and added.
+                    </para></listitem>
+                <listitem><para>
+                    Inaccurate disk and CPU percentage data has been dropped
+                    from <filename>buildstats</filename> output.
+                    This data has been replaced with
+                    <filename>getrusage()</filename> data and corrected IO
+                    statistics.
+                    You will probably need to update code that reads the
+                    <filename>buildstats</filename> data.
+                    </para></listitem>
+                <listitem><para>
+                    The
+                    <filename>meta/conf/distro/include/package_regex.inc</filename>
+                    is now deprecated.
+                    The contents of this file have been moved to individual
+                    recipes.
+                    <note><title>Tip</title>
+                        Because this file will likely be removed in a future
+                        Yocto Project release, it is suggested that you remove
+                        any references to the file that might be in your
+                        configuration.
+                    </note>
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>v86d/uvesafb</filename> has been removed from
+                    the <filename>genericx86</filename> and
+                    <filename>genericx86-64</filename> reference machines,
+                    which are provided by the
+                    <filename>meta-yocto-bsp</filename> layer.
+                    Most modern x86 boards do not rely on this file and it only
+                    adds kernel error messages during startup.
+                    If you do still need the file, you can simply add
+                    <filename>v86d</filename> to your image.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+</section>
 
 </chapter>
 <!--