overview-manual, ref-manual: Moved "Licenses" to overview-manual

Fixes [YOCTO #12370]

The "Licenses" section in the technical details chapter of the
ref-manual was concepts and needed moved to the new overview-manual.
Some links were broke during the move and they were fixed in the
BSP and dev-manual.

(From yocto-docs rev: 34c013f055736dcde2fe12daea1aaf2beaee97c5)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 390 insertions, 0 deletions

diff --git a/documentation/overview-manual/overview-concepts.xml b/documentation/overview-manual/overview-concepts.xml
index a7d6d2f6fb..2f5c14eef6 100644
--- a/documentation/overview-manual/overview-concepts.xml
+++ b/documentation/overview-manual/overview-concepts.xml
@@ -1460,6 +1460,396 @@
         </section>
     </section>
 
+    <section id="overview-licenses">
+        <title>Licenses</title>
+
+        <para>
+            This section describes the mechanism by which the OpenEmbedded
+            build system tracks changes to licensing text.
+            The section also describes how to enable commercially licensed
+            recipes, which by default are disabled.
+        </para>
+
+        <para>
+            For information that can help you maintain compliance with
+            various open source licensing during the lifecycle of the product,
+            see the
+            "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>"
+            section in the Yocto Project Development Tasks Manual.
+        </para>
+
+        <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
+            <title>Tracking License Changes</title>
+
+            <para>
+                The license of an upstream project might change in the future.
+                In order to prevent these changes going unnoticed, the
+                <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
+                variable tracks changes to the license text. The checksums are
+                validated at the end of the configure step, and if the
+                checksums do not match, the build will fail.
+            </para>
+
+            <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
+                <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
+
+                <para>
+                    The <filename>LIC_FILES_CHKSUM</filename>
+                    variable contains checksums of the license text in the
+                    source code for the recipe.
+                    Following is an example of how to specify
+                    <filename>LIC_FILES_CHKSUM</filename>:
+                    <literallayout class='monospaced'>
+     LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
+                         file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
+                         file://licfile2.txt;endline=50;md5=zzzz \
+                         ..."
+                    </literallayout>
+                    <note><title>Notes</title>
+                        <itemizedlist>
+                            <listitem><para>
+                                When using "beginline" and "endline", realize
+                                that line numbering begins with one and not
+                                zero.
+                                Also, the included lines are inclusive (i.e.
+                                lines five through and including 29 in the
+                                previous example for
+                                <filename>licfile1.txt</filename>).
+                                </para></listitem>
+                            <listitem><para>
+                                When a license check fails, the selected license
+                                text is included as part of the QA message.
+                                Using this output, you can determine the exact
+                                start and finish for the needed license text.
+                                </para></listitem>
+                        </itemizedlist>
+                    </note>
+                </para>
+
+                <para>
+                    The build system uses the
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
+                    variable as the default directory when searching files
+                    listed in <filename>LIC_FILES_CHKSUM</filename>.
+                    The previous example employs the default directory.
+                </para>
+
+                <para>
+                    Consider this next example:
+                    <literallayout class='monospaced'>
+     LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
+                                         md5=bb14ed3c4cda583abc85401304b5cd4e"
+     LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
+                    </literallayout>
+                </para>
+
+                <para>
+                    The first line locates a file in
+                    <filename>${S}/src/ls.c</filename> and isolates lines five
+                    through 16 as license text.
+                    The second line refers to a file in
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.
+                </para>
+
+                <para>
+                    Note that <filename>LIC_FILES_CHKSUM</filename> variable is
+                    mandatory for all recipes, unless the
+                    <filename>LICENSE</filename> variable is set to "CLOSED".
+                </para>
+            </section>
+
+            <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
+                <title>Explanation of Syntax</title>
+
+                <para>
+                    As mentioned in the previous section, the
+                    <filename>LIC_FILES_CHKSUM</filename> variable lists all
+                    the important files that contain the license text for the
+                    source code.
+                    It is possible to specify a checksum for an entire file,
+                    or a specific section of a file (specified by beginning and
+                    ending line numbers with the "beginline" and "endline"
+                    parameters, respectively).
+                    The latter is useful for source files with a license
+                    notice header, README documents, and so forth.
+                    If you do not use the "beginline" parameter, then it is
+                    assumed that the text begins on the first line of the file.
+                    Similarly, if you do not use the "endline" parameter,
+                    it is assumed that the license text ends with the last
+                    line of the file.
+                </para>
+
+                <para>
+                    The "md5" parameter stores the md5 checksum of the license
+                    text.
+                    If the license text changes in any way as compared to
+                    this parameter then a mismatch occurs.
+                    This mismatch triggers a build failure and notifies
+                    the developer.
+                    Notification allows the developer to review and address
+                    the license text changes.
+                    Also note that if a mismatch occurs during the build,
+                    the correct md5 checksum is placed in the build log and
+                    can be easily copied to the recipe.
+                </para>
+
+                <para>
+                    There is no limit to how many files you can specify using
+                    the <filename>LIC_FILES_CHKSUM</filename> variable.
+                    Generally, however, every project requires a few
+                    specifications for license tracking.
+                    Many projects have a "COPYING" file that stores the
+                    license information for all the source code files.
+                    This practice allows you to just track the "COPYING"
+                    file as long as it is kept up to date.
+                    <note><title>Tips</title>
+                        <itemizedlist>
+                            <listitem><para>
+                                If you specify an empty or invalid "md5"
+                                parameter, BitBake returns an md5 mis-match
+                                error and displays the correct "md5" parameter
+                                value during the build.
+                                The correct parameter is also captured in
+                                the build log.
+                                </para></listitem>
+                            <listitem><para>
+                                If the whole file contains only license text,
+                                you do not need to use the "beginline" and
+                               "endline" parameters.
+                               </para></listitem>
+                        </itemizedlist>
+                    </note>
+                </para>
+            </section>
+        </section>
+
+        <section id="enabling-commercially-licensed-recipes">
+            <title>Enabling Commercially Licensed Recipes</title>
+
+            <para>
+                By default, the OpenEmbedded build system disables
+                components that have commercial or other special licensing
+                requirements.
+                Such requirements are defined on a
+                recipe-by-recipe basis through the
+                <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></ulink>
+                variable definition in the affected recipe.
+                For instance, the
+                <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
+                recipe contains the following statement:
+                <literallayout class='monospaced'>
+     LICENSE_FLAGS = "commercial"
+                </literallayout>
+                Here is a slightly more complicated example that contains both
+                an explicit recipe name and version (after variable expansion):
+                <literallayout class='monospaced'>
+     LICENSE_FLAGS = "license_${PN}_${PV}"
+                </literallayout>
+                    In order for a component restricted by a
+                <filename>LICENSE_FLAGS</filename> definition to be enabled and
+                included in an image, it needs to have a matching entry in the
+                global
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></ulink>
+                variable, which is a variable typically defined in your
+                <filename>local.conf</filename> file.
+                For example, to enable the
+                <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
+                    package, you could add either the string
+                    "commercial_gst-plugins-ugly" or the more general string
+                    "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
+                See the
+                "<link linkend='license-flag-matching'>License Flag Matching</link>"
+                section for a full
+                explanation of how <filename>LICENSE_FLAGS</filename> matching
+                works.
+                Here is the example:
+                <literallayout class='monospaced'>
+     LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
+                </literallayout>
+                    Likewise, to additionally enable the package built from the
+                recipe containing
+                    <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>,
+                and assuming that the actual recipe name was
+                <filename>emgd_1.10.bb</filename>, the following string would
+                enable that package as well as the original
+                <filename>gst-plugins-ugly</filename> package:
+                <literallayout class='monospaced'>
+     LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
+                </literallayout>
+                    As a convenience, you do not need to specify the complete
+                license string in the whitelist for every package.
+                You can use an abbreviated form, which consists
+                of just the first portion or portions of the license
+                string before the initial underscore character or characters.
+                A partial string will match any license that contains the
+                given string as the first portion of its license.
+                For example, the following whitelist string will also match
+                both of the packages previously mentioned as well as any other
+                packages that have licenses starting with "commercial" or
+                "license".
+                <literallayout class='monospaced'>
+     LICENSE_FLAGS_WHITELIST = "commercial license"
+                </literallayout>
+            </para>
+
+            <section id="license-flag-matching">
+                <title>License Flag Matching</title>
+
+                <para>
+                            License flag matching allows you to control what recipes
+                    the OpenEmbedded build system includes in the build.
+                    Fundamentally, the build system attempts to match
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></ulink>
+                    strings found in recipes against
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></ulink>
+                    strings found in the whitelist.
+                    A match causes the build system to include a recipe in the
+                    build, while failure to find a match causes the build
+                    system to exclude a recipe.
+                </para>
+
+                <para>
+                    In general, license flag matching is simple.
+                    However, understanding some concepts will help you
+                    correctly and effectively use matching.
+                </para>
+
+                <para>
+                    Before a flag
+                    defined by a particular recipe is tested against the
+                    contents of the whitelist, the expanded string
+                    <filename>_${PN}</filename> is appended to the flag.
+                    This expansion makes each
+                    <filename>LICENSE_FLAGS</filename> value recipe-specific.
+                    After expansion, the string is then matched against the
+                    whitelist.
+                    Thus, specifying
+                    <filename>LICENSE_FLAGS = "commercial"</filename>
+                    in recipe "foo", for example, results in the string
+                    <filename>"commercial_foo"</filename>.
+                    And, to create a match, that string must appear in the
+                    whitelist.
+                </para>
+
+                <para>
+                    Judicious use of the <filename>LICENSE_FLAGS</filename>
+                    strings and the contents of the
+                    <filename>LICENSE_FLAGS_WHITELIST</filename> variable
+                    allows you a lot of flexibility for including or excluding
+                    recipes based on licensing.
+                    For example, you can broaden the matching capabilities by
+                    using license flags string subsets in the whitelist.
+                    <note>
+                        When using a string subset, be sure to use the part of
+                        the expanded string that precedes the appended
+                        underscore character (e.g.
+                        <filename>usethispart_1.3</filename>,
+                        <filename>usethispart_1.4</filename>, and so forth).
+                    </note>
+                    For example, simply specifying the string "commercial" in
+                    the whitelist matches any expanded
+                    <filename>LICENSE_FLAGS</filename> definition that starts
+                    with the string "commercial" such as "commercial_foo" and
+                    "commercial_bar", which are the strings the build system
+                    automatically generates for hypothetical recipes named
+                    "foo" and "bar" assuming those recipes simply specify the
+                    following:
+                    <literallayout class='monospaced'>
+     LICENSE_FLAGS = "commercial"
+                    </literallayout>
+                    Thus, you can choose to exhaustively
+                    enumerate each license flag in the whitelist and
+                    allow only specific recipes into the image, or
+                    you can use a string subset that causes a broader range of
+                    matches to allow a range of recipes into the image.
+                </para>
+
+                <para>
+                    This scheme works even if the
+                    <filename>LICENSE_FLAGS</filename> string already
+                    has <filename>_${PN}</filename> appended.
+                    For example, the build system turns the license flag
+                    "commercial_1.2_foo" into "commercial_1.2_foo_foo" and
+                    would match both the general "commercial" and the specific
+                    "commercial_1.2_foo" strings found in the whitelist, as
+                    expected.
+                </para>
+
+                <para>
+                    Here are some other scenarios:
+                    <itemizedlist>
+                        <listitem><para>
+                            You can specify a versioned string in the recipe
+                            such as "commercial_foo_1.2" in a "foo" recipe.
+                            The build system expands this string to
+                            "commercial_foo_1.2_foo".
+                            Combine this license flag with a whitelist that has
+                            the string "commercial" and you match the flag
+                            along with any other flag that starts with the
+                            string "commercial".
+                            </para></listitem>
+                        <listitem><para>
+                            Under the same circumstances, you can use
+                            "commercial_foo" in the whitelist and the build
+                            system not only matches "commercial_foo_1.2" but
+                            also matches any license flag with the string
+                            "commercial_foo", regardless of the version.
+                            </para></listitem>
+                        <listitem><para>
+                            You can be very specific and use both the
+                            package and version parts in the whitelist (e.g.
+                            "commercial_foo_1.2") to specifically match a
+                            versioned recipe.
+                            </para></listitem>
+                    </itemizedlist>
+                </para>
+            </section>
+
+            <section id="other-variables-related-to-commercial-licenses">
+                <title>Other Variables Related to Commercial Licenses</title>
+
+                <para>
+                    Other helpful variables related to commercial
+                    license handling exist and are defined in the
+                    <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
+                    <literallayout class='monospaced'>
+     COMMERCIAL_AUDIO_PLUGINS ?= ""
+     COMMERCIAL_VIDEO_PLUGINS ?= ""
+                    </literallayout>
+                    If you want to enable these components, you can do so by
+                    making sure you have statements similar to the following
+                    in your <filename>local.conf</filename> configuration file:
+                    <literallayout class='monospaced'>
+     COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
+        gst-plugins-ugly-mpegaudioparse"
+     COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
+        gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
+     LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
+                    </literallayout>
+                    Of course, you could also create a matching whitelist
+                    for those components using the more general "commercial"
+                    in the whitelist, but that would also enable all the
+                    other packages with
+                    <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></ulink>
+                    containing "commercial", which you may or may not want:
+                    <literallayout class='monospaced'>
+     LICENSE_FLAGS_WHITELIST = "commercial"
+                    </literallayout>
+                </para>
+
+                <para>
+                    Specifying audio and video plug-ins as part of the
+                    <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
+                    <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
+                    (along with the enabling
+                    <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
+                    plug-ins or components into built images, thus adding
+                    support for media formats or components.
+                </para>
+            </section>
+        </section>
+    </section>
+
     <section id='x32'>
         <title>x32 psABI</title>