From 15fa336a1fc55511b347f390c38fedaa8f8ab7e1 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Tue, 15 May 2018 11:14:50 -0700 Subject: overview-manual, dev-manual: Moved licensing how-to stuff to dev-manual The section on licensing in the overview-manual was really "how-to" information. I moved this to a new section in the dev-manual for "working with licenses". I fixed some references in the ref-manual and in the bsp-guide as well. (From yocto-docs rev: f150a1ea2da900aae88fc5fa60f4115cc213ba2d) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../dev-manual/dev-manual-common-tasks.xml | 793 +++++++++++++++------ 1 file changed, 591 insertions(+), 202 deletions(-) (limited to 'documentation/dev-manual') diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index 61ed8f1cee..14cf1a1685 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -2187,9 +2187,8 @@ containing the current checksum. For more explanation and examples of how to set the LIC_FILES_CHKSUM variable, see the - "Tracking License Changes" - section in the Yocto Project Overview and Concepts - Manual. + "Tracking License Changes" + section. To determine the correct checksum string, you can list the appropriate files in the @@ -3400,7 +3399,7 @@ The variable LIC_FILES_CHKSUM is used to track source license changes as described in the - "Tracking License Changes" + "Tracking License Changes" section in the Yocto Project Overview and Concepts Manual. You can quickly create Autotool-based recipes in a manner similar to the previous example. @@ -13501,127 +13500,515 @@ Some notes from Cal: -
- Maintaining Open Source License Compliance During Your Product's Lifecycle +
+ Working With Licenses - One of the concerns for a development organization using open source - software is how to maintain compliance with various open source - licensing during the lifecycle of the product. - While this section does not provide legal advice or - comprehensively cover all scenarios, it does - present methods that you can use to - assist you in meeting the compliance requirements during a software - release. + As mentioned in the + "Licensing" + section in the Yocto Project Overview and Concepts Manual, + open source projects are open to the public and they + consequently have different licensing structures in place. + This section describes the mechanism by which the + OpenEmbedded build system + tracks changes to licensing text and covers how to maintain open + source license compliance during your project's lifecycle. + The section also describes how to enable commercially licensed + recipes, which by default are disabled. - - With hundreds of different open source licenses that the Yocto - Project tracks, it is difficult to know the requirements of each - and every license. - However, the requirements of the major FLOSS licenses can begin - to be covered by - assuming that three main areas of concern exist: - - Source code must be provided. - License text for the software must be - provided. - Compilation scripts and modifications to the - source code must be provided. - - - There are other requirements beyond the scope of these - three and the methods described in this section - (e.g. the mechanism through which source code is distributed). - +
+ Tracking License Changes - - As different organizations have different methods of complying with - open source licensing, this section is not meant to imply that - there is only one single way to meet your compliance obligations, - but rather to describe one method of achieving compliance. - The remainder of this section describes methods supported to meet the - previously mentioned three requirements. - Once you take steps to meet these requirements, - and prior to releasing images, sources, and the build system, - you should audit all artifacts to ensure completeness. - - The Yocto Project generates a license manifest during - image creation that is located - in ${DEPLOY_DIR}/licenses/image_name-datestamp - to assist with any audits. - - + + The license of an upstream project might change in the future. + In order to prevent these changes going unnoticed, the + LIC_FILES_CHKSUM + variable tracks changes to the license text. The checksums are + validated at the end of the configure step, and if the + checksums do not match, the build will fail. + + +
+ Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable + + + The LIC_FILES_CHKSUM + variable contains checksums of the license text in the + source code for the recipe. + Following is an example of how to specify + LIC_FILES_CHKSUM: + + LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ + file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ + file://licfile2.txt;endline=50;md5=zzzz \ + ..." + + Notes + + + When using "beginline" and "endline", realize + that line numbering begins with one and not + zero. + Also, the included lines are inclusive (i.e. + lines five through and including 29 in the + previous example for + licfile1.txt). + + + When a license check fails, the selected license + text is included as part of the QA message. + Using this output, you can determine the exact + start and finish for the needed license text. + + + + + + + The build system uses the + S + variable as the default directory when searching files + listed in LIC_FILES_CHKSUM. + The previous example employs the default directory. + + + + Consider this next example: + + LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ + md5=bb14ed3c4cda583abc85401304b5cd4e" + LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" + + + + + The first line locates a file in + ${S}/src/ls.c and isolates lines five + through 16 as license text. + The second line refers to a file in + WORKDIR. + + + + Note that LIC_FILES_CHKSUM variable is + mandatory for all recipes, unless the + LICENSE variable is set to "CLOSED". + +
+ +
+ Explanation of Syntax + + + As mentioned in the previous section, the + LIC_FILES_CHKSUM variable lists all + the important files that contain the license text for the + source code. + It is possible to specify a checksum for an entire file, + or a specific section of a file (specified by beginning and + ending line numbers with the "beginline" and "endline" + parameters, respectively). + The latter is useful for source files with a license + notice header, README documents, and so forth. + If you do not use the "beginline" parameter, then it is + assumed that the text begins on the first line of the file. + Similarly, if you do not use the "endline" parameter, + it is assumed that the license text ends with the last + line of the file. + + + + The "md5" parameter stores the md5 checksum of the license + text. + If the license text changes in any way as compared to + this parameter then a mismatch occurs. + This mismatch triggers a build failure and notifies + the developer. + Notification allows the developer to review and address + the license text changes. + Also note that if a mismatch occurs during the build, + the correct md5 checksum is placed in the build log and + can be easily copied to the recipe. + + + + There is no limit to how many files you can specify using + the LIC_FILES_CHKSUM variable. + Generally, however, every project requires a few + specifications for license tracking. + Many projects have a "COPYING" file that stores the + license information for all the source code files. + This practice allows you to just track the "COPYING" + file as long as it is kept up to date. + Tips + + + If you specify an empty or invalid "md5" + parameter, + BitBake + returns an md5 mis-match + error and displays the correct "md5" parameter + value during the build. + The correct parameter is also captured in + the build log. + + + If the whole file contains only license text, + you do not need to use the "beginline" and + "endline" parameters. + + + + +
+
-
- Providing the Source Code +
+ Enabling Commercially Licensed Recipes - Compliance activities should begin before you generate the - final image. - The first thing you should look at is the requirement that - tops the list for most compliance groups - providing - the source. - The Yocto Project has a few ways of meeting this - requirement. + By default, the OpenEmbedded build system disables + components that have commercial or other special licensing + requirements. + Such requirements are defined on a + recipe-by-recipe basis through the + LICENSE_FLAGS + variable definition in the affected recipe. + For instance, the + poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly + recipe contains the following statement: + + LICENSE_FLAGS = "commercial" + + Here is a slightly more complicated example that contains both + an explicit recipe name and version (after variable expansion): + + LICENSE_FLAGS = "license_${PN}_${PV}" + + In order for a component restricted by a + LICENSE_FLAGS definition to be enabled and + included in an image, it needs to have a matching entry in the + global + LICENSE_FLAGS_WHITELIST + variable, which is a variable typically defined in your + local.conf file. + For example, to enable the + poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly + package, you could add either the string + "commercial_gst-plugins-ugly" or the more general string + "commercial" to LICENSE_FLAGS_WHITELIST. + See the + "License Flag Matching" + section for a full + explanation of how LICENSE_FLAGS matching + works. + Here is the example: + + LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly" + + Likewise, to additionally enable the package built from the + recipe containing + LICENSE_FLAGS = "license_${PN}_${PV}", + and assuming that the actual recipe name was + emgd_1.10.bb, the following string would + enable that package as well as the original + gst-plugins-ugly package: + + LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10" + + As a convenience, you do not need to specify the complete + license string in the whitelist for every package. + You can use an abbreviated form, which consists + of just the first portion or portions of the license + string before the initial underscore character or characters. + A partial string will match any license that contains the + given string as the first portion of its license. + For example, the following whitelist string will also match + both of the packages previously mentioned as well as any other + packages that have licenses starting with "commercial" or + "license". + + LICENSE_FLAGS_WHITELIST = "commercial license" + +
+ License Flag Matching + + + License flag matching allows you to control what recipes + the OpenEmbedded build system includes in the build. + Fundamentally, the build system attempts to match + LICENSE_FLAGS strings found in recipes + against LICENSE_FLAGS_WHITELIST + strings found in the whitelist. + A match causes the build system to include a recipe in the + build, while failure to find a match causes the build + system to exclude a recipe. + + + + In general, license flag matching is simple. + However, understanding some concepts will help you + correctly and effectively use matching. + + + + Before a flag + defined by a particular recipe is tested against the + contents of the whitelist, the expanded string + _${PN} is appended to the flag. + This expansion makes each + LICENSE_FLAGS value recipe-specific. + After expansion, the string is then matched against the + whitelist. + Thus, specifying + LICENSE_FLAGS = "commercial" + in recipe "foo", for example, results in the string + "commercial_foo". + And, to create a match, that string must appear in the + whitelist. + + + + Judicious use of the LICENSE_FLAGS + strings and the contents of the + LICENSE_FLAGS_WHITELIST variable + allows you a lot of flexibility for including or excluding + recipes based on licensing. + For example, you can broaden the matching capabilities by + using license flags string subsets in the whitelist. + + When using a string subset, be sure to use the part of + the expanded string that precedes the appended + underscore character (e.g. + usethispart_1.3, + usethispart_1.4, and so forth). + + For example, simply specifying the string "commercial" in + the whitelist matches any expanded + LICENSE_FLAGS definition that starts + with the string "commercial" such as "commercial_foo" and + "commercial_bar", which are the strings the build system + automatically generates for hypothetical recipes named + "foo" and "bar" assuming those recipes simply specify the + following: + + LICENSE_FLAGS = "commercial" + + Thus, you can choose to exhaustively + enumerate each license flag in the whitelist and + allow only specific recipes into the image, or + you can use a string subset that causes a broader range of + matches to allow a range of recipes into the image. + + + + This scheme works even if the + LICENSE_FLAGS string already + has _${PN} appended. + For example, the build system turns the license flag + "commercial_1.2_foo" into "commercial_1.2_foo_foo" and + would match both the general "commercial" and the specific + "commercial_1.2_foo" strings found in the whitelist, as + expected. + + + + Here are some other scenarios: + + + You can specify a versioned string in the recipe + such as "commercial_foo_1.2" in a "foo" recipe. + The build system expands this string to + "commercial_foo_1.2_foo". + Combine this license flag with a whitelist that has + the string "commercial" and you match the flag + along with any other flag that starts with the + string "commercial". + + + Under the same circumstances, you can use + "commercial_foo" in the whitelist and the build + system not only matches "commercial_foo_1.2" but + also matches any license flag with the string + "commercial_foo", regardless of the version. + + + You can be very specific and use both the + package and version parts in the whitelist (e.g. + "commercial_foo_1.2") to specifically match a + versioned recipe. + + + +
+ + +
+ +
+ Maintaining Open Source License Compliance During Your Product's Lifecycle + - One of the easiest ways to meet this requirement is - to provide the entire - DL_DIR - used by the build. - This method, however, has a few issues. - The most obvious is the size of the directory since it includes - all sources used in the build and not just the source used in - the released image. - It will include toolchain source, and other artifacts, which - you would not generally release. - However, the more serious issue for most companies is accidental - release of proprietary software. - The Yocto Project provides an - archiver - class to help avoid some of these concerns. + One of the concerns for a development organization using open source + software is how to maintain compliance with various open source + licensing during the lifecycle of the product. + While this section does not provide legal advice or + comprehensively cover all scenarios, it does + present methods that you can use to + assist you in meeting the compliance requirements during a software + release. - Before you employ DL_DIR or the - archiver class, you need to decide how - you choose to provide source. - The source archiver class can generate - tarballs and SRPMs and can create them with various levels of - compliance in mind. + With hundreds of different open source licenses that the Yocto + Project tracks, it is difficult to know the requirements of each + and every license. + However, the requirements of the major FLOSS licenses can begin + to be covered by + assuming that three main areas of concern exist: + + Source code must be provided. + License text for the software must be + provided. + Compilation scripts and modifications to the + source code must be provided. + + + There are other requirements beyond the scope of these + three and the methods described in this section + (e.g. the mechanism through which source code is distributed). - One way of doing this (but certainly not the only way) is to - release just the source as a tarball. - You can do this by adding the following to the - local.conf file found in the - Build Directory: - + As different organizations have different methods of complying with + open source licensing, this section is not meant to imply that + there is only one single way to meet your compliance obligations, + but rather to describe one method of achieving compliance. + The remainder of this section describes methods supported to meet the + previously mentioned three requirements. + Once you take steps to meet these requirements, + and prior to releasing images, sources, and the build system, + you should audit all artifacts to ensure completeness. + + The Yocto Project generates a license manifest during + image creation that is located + in ${DEPLOY_DIR}/licenses/image_name-datestamp + to assist with any audits. + + + +
+ Providing the Source Code + + + Compliance activities should begin before you generate the + final image. + The first thing you should look at is the requirement that + tops the list for most compliance groups - providing + the source. + The Yocto Project has a few ways of meeting this + requirement. + + + + One of the easiest ways to meet this requirement is + to provide the entire + DL_DIR + used by the build. + This method, however, has a few issues. + The most obvious is the size of the directory since it includes + all sources used in the build and not just the source used in + the released image. + It will include toolchain source, and other artifacts, which + you would not generally release. + However, the more serious issue for most companies is accidental + release of proprietary software. + The Yocto Project provides an + archiver + class to help avoid some of these concerns. + + + + Before you employ DL_DIR or the + archiver class, you need to decide how + you choose to provide source. + The source archiver class can generate + tarballs and SRPMs and can create them with various levels of + compliance in mind. + + + + One way of doing this (but certainly not the only way) is to + release just the source as a tarball. + You can do this by adding the following to the + local.conf file found in the + Build Directory: + INHERIT += "archiver" ARCHIVER_MODE[src] = "original" - - During the creation of your image, the source from all - recipes that deploy packages to the image is placed within - subdirectories of - DEPLOY_DIR/sources based on the - LICENSE - for each recipe. - Releasing the entire directory enables you to comply with - requirements concerning providing the unmodified source. - It is important to note that the size of the directory can - get large. - + + During the creation of your image, the source from all + recipes that deploy packages to the image is placed within + subdirectories of + DEPLOY_DIR/sources based on the + LICENSE + for each recipe. + Releasing the entire directory enables you to comply with + requirements concerning providing the unmodified source. + It is important to note that the size of the directory can + get large. + - - A way to help mitigate the size issue is to only release - tarballs for licenses that require the release of - source. - Let us assume you are only concerned with GPL code as - identified by running the following script: - + + A way to help mitigate the size issue is to only release + tarballs for licenses that require the release of + source. + Let us assume you are only concerned with GPL code as + identified by running the following script: + # Script to archive a subset of packages matching specific license(s) # Source and license files are copied into sub folders of package folder # Must be run from build folder @@ -13644,96 +14031,97 @@ Some notes from Cal: cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null fi done - done - At this point, you could create a tarball from the - gpl_source_release directory and - provide that to the end user. - This method would be a step toward achieving compliance - with section 3a of GPLv2 and with section 6 of GPLv3. - -
+ done + + At this point, you could create a tarball from the + gpl_source_release directory and + provide that to the end user. + This method would be a step toward achieving compliance + with section 3a of GPLv2 and with section 6 of GPLv3. + +
-
- Providing License Text +
+ Providing License Text - - One requirement that is often overlooked is inclusion - of license text. - This requirement also needs to be dealt with prior to - generating the final image. - Some licenses require the license text to accompany - the binary. - You can achieve this by adding the following to your - local.conf file: - + + One requirement that is often overlooked is inclusion + of license text. + This requirement also needs to be dealt with prior to + generating the final image. + Some licenses require the license text to accompany + the binary. + You can achieve this by adding the following to your + local.conf file: + COPY_LIC_MANIFEST = "1" COPY_LIC_DIRS = "1" LICENSE_CREATE_PACKAGE = "1" - - Adding these statements to the configuration file ensures - that the licenses collected during package generation - are included on your image. - - Setting all three variables to "1" results in the - image having two copies of the same license file. - One copy resides in - /usr/share/common-licenses and - the other resides in - /usr/share/license. - - The reason for this behavior is because - COPY_LIC_DIRS - and - COPY_LIC_MANIFEST - add a copy of the license when the image is built but do - not offer a path for adding licenses for newly installed - packages to an image. - LICENSE_CREATE_PACKAGE - adds a separate package and an upgrade path for adding - licenses to an image. - - + + Adding these statements to the configuration file ensures + that the licenses collected during package generation + are included on your image. + + Setting all three variables to "1" results in the + image having two copies of the same license file. + One copy resides in + /usr/share/common-licenses and + the other resides in + /usr/share/license. + + The reason for this behavior is because + COPY_LIC_DIRS + and + COPY_LIC_MANIFEST + add a copy of the license when the image is built but do + not offer a path for adding licenses for newly installed + packages to an image. + LICENSE_CREATE_PACKAGE + adds a separate package and an upgrade path for adding + licenses to an image. + + - - As the source archiver class has already - archived the original - unmodified source that contains the license files, - you would have already met the requirements for inclusion - of the license information with source as defined by the GPL - and other open source licenses. - -
+ + As the source archiver class has already + archived the original + unmodified source that contains the license files, + you would have already met the requirements for inclusion + of the license information with source as defined by the GPL + and other open source licenses. + +
-
- Providing Compilation Scripts and Source Code Modifications +
+ Providing Compilation Scripts and Source Code Modifications - - At this point, we have addressed all we need to - prior to generating the image. - The next two requirements are addressed during the final - packaging of the release. - + + At this point, we have addressed all we need to + prior to generating the image. + The next two requirements are addressed during the final + packaging of the release. + - - By releasing the version of the OpenEmbedded build system - and the layers used during the build, you will be providing both - compilation scripts and the source code modifications in one - step. - + + By releasing the version of the OpenEmbedded build system + and the layers used during the build, you will be providing both + compilation scripts and the source code modifications in one + step. + - - If the deployment team has a - BSP layer - and a distro layer, and those those layers are used to patch, - compile, package, or modify (in any way) any open source - software included in your released images, you - might be required to release those layers under section 3 of - GPLv2 or section 1 of GPLv3. - One way of doing that is with a clean - checkout of the version of the Yocto Project and layers used - during your build. - Here is an example: - + + If the deployment team has a + BSP layer + and a distro layer, and those those layers are used to patch, + compile, package, or modify (in any way) any open source + software included in your released images, you + might be required to release those layers under section 3 of + GPLv2 or section 1 of GPLv3. + One way of doing that is with a clean + checkout of the version of the Yocto Project and layers used + during your build. + Here is an example: + # We built using the &DISTRO_NAME_NO_CAP; branch of the poky repo $ git clone -b &DISTRO_NAME_NO_CAP; git://git.yoctoproject.org/poky $ cd poky @@ -13742,15 +14130,15 @@ Some notes from Cal: $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer # clean up the .git repos $ find . -name ".git" -type d -exec rm -rf {} \; - - One thing a development organization might want to consider - for end-user convenience is to modify - meta-poky/conf/bblayers.conf.sample to - ensure that when the end user utilizes the released build - system to build an image, the development organization's - layers are included in the bblayers.conf - file automatically: - + + One thing a development organization might want to consider + for end-user convenience is to modify + meta-poky/conf/bblayers.conf.sample to + ensure that when the end user utilizes the released build + system to build an image, the development organization's + layers are included in the bblayers.conf + file automatically: + # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf # changes incompatibly POKY_BBLAYERS_CONF_VERSION = "2" @@ -13764,14 +14152,15 @@ Some notes from Cal: ##OEROOT##/meta-yocto-bsp \ ##OEROOT##/meta-mylayer \ " - - Creating and providing an archive of the - Metadata - layers (recipes, configuration files, and so forth) - enables you to meet your - requirements to include the scripts to control compilation - as well as any modifications to the original source. - + + Creating and providing an archive of the + Metadata + layers (recipes, configuration files, and so forth) + enables you to meet your + requirements to include the scripts to control compilation + as well as any modifications to the original source. + +
-- cgit v1.2.3-54-g00ecf