From 5e1255f80910f87bd59c37736b52fdcb303246f2 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Thu, 8 Dec 2011 13:22:23 -0800 Subject: documentation/poky-ref-manual/extendpoky.xml: intro changed and order changed Beefed up the introductory paragraph and I re-ordered the "Making and Maintaining Changes" section towards the end of the chapter. (From yocto-docs rev: b166fbfdccc971f077f9d0e598604f761a820b4f) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/poky-ref-manual/extendpoky.xml | 1170 +++++++++++++------------- 1 file changed, 586 insertions(+), 584 deletions(-) (limited to 'documentation') diff --git a/documentation/poky-ref-manual/extendpoky.xml b/documentation/poky-ref-manual/extendpoky.xml index 6f24ff14b1..4d3eba4c6b 100644 --- a/documentation/poky-ref-manual/extendpoky.xml +++ b/documentation/poky-ref-manual/extendpoky.xml @@ -3,13 +3,14 @@ -Extending the Yocto Project +Common Tasks - This chapter provides information about how to extend the functionality - already present in the Yocto Project. - The chapter also documents standard tasks such as adding new + This chapter describes standard tasks such as adding new software packages, extending or customizing images or porting the Yocto Project to new hardware (adding a new machine). + The chapter also describes ways to modify package source code, combine multiple + versions of library files into a single image, track license changes, and handle + a package name alias. Finally, the chapter contains advice about how to make changes to the Yocto Project to achieve the best results. @@ -658,729 +659,730 @@ -
- Making and Maintaining Changes +
+ Modifying Package Source Code - Because the Yocto Project is extremely configurable and flexible, - we recognize that developers will want - to extend, configure or optimize it for their specific uses. - To best keep pace with future Yocto Project changes, - we recommend you make controlled changes to the Yocto Project. + Although the Yocto Project is usually used to build software, you can use it to modify software. - The Yocto Project supports a "layers" concept. - If you use layers properly, you can ease future upgrades and allow segregation - between the Yocto Project core and a given developer's changes. - The following section provides more advice on managing changes to the Yocto Project. + During a build, source is available in the + WORKDIR directory. + The actual location depends on the type of package and the architecture of the target device. + For a standard recipe not related to + MACHINE, the location is + tmp/work/PACKAGE_ARCH-poky-TARGET_OS/PN-PV-PR/. + For target device-dependent packages, you should use the MACHINE + variable instead of + PACKAGE_ARCH + in the directory name. -
- BitBake Layers - - Often, developers want to extend the Yocto Project either by adding packages - or by overriding files contained within the Yocto Project to add their own - functionality. - BitBake has a powerful mechanism called - "layers", which provides a way to handle this extension in a fully - supported and non-invasive fashion. - + + Be sure the package recipe sets the + S variable to something + other than the standard WORKDIR/PN-PV/ value. + - - The Yocto Project files include several additional layers such as - meta-rt and meta-yocto - that demonstrate this functionality. - The meta-rt layer is not enabled by default. - However, the meta-yocto layer is. - + + After building a package, you can modify the package source code without problems. + The easiest way to test your changes is by calling the + compile task as shown in the following example: + + $ bitbake -c compile -f NAME_OF_PACKAGE + + - - To enable a layer, you simply add the layer's path to the - BBLAYERS variable in your - bblayers.conf file, which is found in the Yocto Project file's - build directory. - The following example shows how to enable the meta-rt: - - LCONF_VERSION = "1" + + The -f or --force + option forces re-execution of the specified task. + You can call other tasks this way as well. + But note that all the modifications in + WORKDIR + are gone once you execute -c clean for a package. + +
- BBFILES ?= "" - BBLAYERS = " \ - /path/to/poky/meta \ - /path/to/poky/meta-yocto \ - /path/to/poky/meta-rt \ - " - - +
+ Modifying Package Source Code with Quilt + + + By default Poky uses Quilt + to manage patches in the do_patch task. + This is a powerful tool that you can use to track all modifications to package sources. + - - BitBake parses each conf/layer.conf file for each layer in - BBLAYERS - and adds the recipes, classes and configurations contained within the layer to - the Yocto Project. - To create your own layer, independent of the Yocto Project files, - simply create a directory with a conf/layer.conf file and - add the directory to your bblayers.conf file. - + + Before modifying source code, it is important to notify Quilt so it can track the changes + into the new patch file: + + $ quilt new NAME-OF-PATCH.patch + + - - The meta-yocto/conf/layer.conf file demonstrates the - required syntax: - - # We have a conf and classes directory, add to BBPATH - BBPATH := "${BBPATH}:${LAYERDIR}" + + After notifying Quilt, add all modified files into that patch: + + $ quilt add file1 file2 file3 + + - # We have a packages directory, add to BBFILES - BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \ - ${LAYERDIR}/recipes-*/*/*.bbappend" + + You can now start editing. + Once you are done editing, you need to use Quilt to generate the final patch that + will contain all your modifications. + + $ quilt refresh + + - BBFILE_COLLECTIONS += "yocto" - BBFILE_PATTERN_yocto := "^${LAYERDIR}/" - BBFILE_PRIORITY_yocto = "5" - - + + You can find the resulting patch file in the + patches/ subdirectory of the source + (S) directory. + For future builds, you should copy the patch into the Yocto Project metadata and add it into the + SRC_URI of a recipe. + Here is an example: + + SRC_URI += "file://NAME-OF-PATCH.patch" + + - - In the previous example, the recipes for the layers are added to - BBFILES. - The BBFILE_COLLECTIONS - variable is then appended with the layer name. - The BBFILE_PATTERN variable - immediately expands with a regular expression used to match files from - BBFILES into - a particular layer, in this case by using the base pathname. - The BBFILE_PRIORITY variable - then assigns different priorities to the files in different layers. - Applying priorities is useful in situations where the same package might appear in multiple - layers and allows you to choose what layer should take precedence. - + + Finally, don't forget to 'bump' the + PR value in the same recipe since + the resulting packages have changed. + +
+ +
+ Combining Multiple versions of Library Files into One Image + + + The build system offers the ability to build libraries with different + target optimizations or architecture formats and combine these together + into one system image. + You can link different binaries in the image + against the different libraries as needed for specific use cases. + This feature is called "Multilib." + + + + An example would be where you have most of a system compiled in 32-bit + mode using 32-bit libraries, but you have something large, like a database + engine, that needs to be a 64-bit application and use 64-bit libraries. + Multilib allows you to get the best of both 32-bit and 64-bit libraries. + + + + While the Multilib feature is most commonly used for 32 and 64-bit differences, + the approach the build system uses facilitates different target optimizations. + You could compile some binaries to use one set of libraries and other binaries + to use other different sets of libraries. + The libraries could differ in architecture, compiler options, or other + optimizations. + + + + This section overviews the Multilib process only. + For more details on how to implement Multilib, see the + Multilib wiki + page. + + +
+ Preparing to use Multilib - Note the use of the LAYERDIR - variable with the immediate expansion operator. - The LAYERDIR variable expands to the directory of the current layer and - requires the immediate expansion operator so that BitBake does not wait to expand the variable - when it's parsing a different directory. + User-specific requirements drive the Multilib feature, + Consequently, there is no one "out-of-the-box" configuration that likely + exists to meet your needs. - BitBake can locate where other .bbclass and configuration files - are applied through the BBPATH environment variable. - For these cases, BitBake uses the first file with the matching name found in - BBPATH. - This is similar to the way the PATH variable is used for binaries. - We recommend, therefore, that you use unique .bbclass - and configuration file names in your custom layer. + In order to enable Multilib, you first need to ensure your recipe is + extended to support multiple libraries. + Many standard recipes are already extended and support multiple libraries. + You can check in the meta/conf/multilib.conf + configuration file in the Yocto Project files directory to see how this is + done using the BBCLASSEXTEND variable. + Eventually, all recipes will be covered and this list will be unneeded. - + - We also recommend the following: - - Store custom layers in a Git repository that uses the - meta-prvt-XXXX format. - Clone the repository alongside other meta - directories in the Yocto Project source files area. - - Following these recommendations keeps your Yocto Project files area and - its configuration entirely inside the Yocto Project's core base. + For the most part, the Multilib class extension works automatically to + extend the package name from ${PN} to + ${MLPREFIX}${PN}, where MLPREFIX + is the particular multilib (e.g. "lib32-" or "lib64-"). + Standard variables such as DEPENDS, + RDEPENDS, RPROVIDES, + RRECOMMENDS, PACKAGES, and + PACKAGES_DYNAMIC are automatically extended by the system. + If you are extending any manual code in the recipe, you can use the + ${MLPREFIX} variable to ensure those names are extended + correctly. + This automatic extension code resides in multilib.bbclass.
-
- Committing Changes - - - Modifications to the Yocto Project are often managed under some kind of source - revision control system. - Because some simple practices can significantly improve usability, policy for committing changes - is important. - It helps to use a consistent documentation style when committing changes. - The Yocto Project development team has found the following practices work well: - - The first line of the commit summarizes the change and begins with the - name of the affected package or packages. - However, not all changes apply to specific packages. - Consequently, the prefix could also be a machine name or class name. - The second part of the commit (if needed) is a longer more detailed - description of the changes. - Placing a blank line between the first and second parts helps with - readability. - - +
+ Using Multilib - Following is an example commit: + After you have set up the recipes, you need to define the actual + combination of multiple libraries you want to build. + You accomplish this through your local.conf + configuration file in the Yocto Project build directory. + An example configuration would be as follows: - bitbake/data.py: Add emit_func() and generate_dependencies() functions - - These functions allow generation of dependency data between functions and - variables allowing moves to be made towards generating checksums and allowing - use of the dependency information in other parts of BitBake. - - Signed-off-by: Richard Purdie richard.purdie@linuxfoundation.org + MACHINE = "qemux86-64" + require conf/multilib.conf + MULTILIBS = "multilib:lib32" + DEFAULTTUNE_virtclass-multilib-lib32 = "x86" + MULTILIB_IMAGE_INSTALL = "lib32-connman" + This example enables an + additional library named lib32 alongside the + normal target packages. + When combining these "lib32" alternatives, the example uses "x86" for tuning. + For information on this particular tuning, see + meta/conf/machine/include/ia32/arch-ia32.inc. - All commits should be self-contained such that they leave the - metadata in a consistent state that builds both before and after the - commit is made. - Besides being a good practice to follow, it helps ensure autobuilder test results - are valid. + The example then includes lib32-connman + in all the images, which illustrates one method of including a + multiple library dependency. + You can use a normal image build to include this dependency, + for example: + + $ bitbake core-image-sato + + You can also build Multilib packages specifically with a command like this: + + $ bitbake lib32-connman +
-
- Package Revision Incrementing +
+ Additional Implementation Details - If a committed change results in changing the package output, - then the value of the - PR variable needs to be increased - (or "bumped") as part of that commit. - This means that for new recipes you must be sure to add the PR - variable and set its initial value equal to "r0". - Failing to define PR makes it easy to miss when you bump a package. - Note that you can only use integer values following the "r" in the - PR variable. + Different packaging systems have different levels of native Multilib + support. + For the RPM Package Management System, the following implementation details + exist: + + A unique architecture is defined for the Multilib packages, + along with creating a unique deploy folder under + tmp/deploy/rpm in the Yocto + Project build directory. + For example, consider lib32 in a + qemux86-64 image. + The possible architectures in the system are "all", "qemux86_64", + "lib32_qemux86_64", and "lib32_x86". + The ${MLPREFIX} variable is stripped from + ${PN} during RPM packaging. + The naming for a normal RPM package and a Multilib RPM package in a + qemux86-64 system resolves to something similar to + bash-4.1-r2.x86_64.rpm and + bash-4.1.r2.lib32_x86.rpm, respectively. + + When installing a Multilib image, the RPM backend first + installs the base image and then installs the Multilib libraries. + + The build system relies on RPM to resolve the identical files in the + two (or more) Multilib packages. + - If you are sharing a common .inc file with multiple recipes, - you can also use the - INC_PR variable to ensure that - the recipes sharing the .inc file are rebuilt when the - .inc file itself is changed. - The .inc file must set INC_PR - (initially to "r0"), and all recipes referring to it should set PR - to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. - If the .inc file is changed then its - INC_PR should be incremented. + For the IPK Package Management System, the following implementation details exist: + + The ${MLPREFIX} is not stripped from + ${PN} during IPK packaging. + The naming for a normal RPM package and a Multilib IPK package in a + qemux86-64 system resolves to something like + bash_4.1-r2.x86_64.ipk and + lib32-bash_4.1-rw_x86.ipk, respectively. + + The IPK deploy folder is not modified with + ${MLPREFIX} because packages with and without + the Multilib feature can exist in the same folder due to the + ${PN} differences. + IPK defines a sanity check for Multilib installation + using certain rules for file comparison, overridden, etc. + + +
+
- - When upgrading the version of a package, assuming the - PV changes, - the PR variable should be reset to "r0" - (or "$(INC_PR).0" if you are using INC_PR). - +
+ Tracking License Changes - - Usually, version increases occur only to packages. - However, if for some reason PV changes but does not - increase, you can increase the - PE variable (Package Epoch). - The PE variable defaults to "0". - + + The license of an upstream project might change in the future. In order to prevent these changes + going unnoticed, the Yocto Project provides a + LIC_FILES_CHKSUM + variable to track changes to the license text. The checksums are validated at the end of the + configure step, and if the checksums do not match, the build will fail. + - - Version numbering strives to follow the - - Debian Version Field Policy Guidelines. - These guidelines define how versions are compared and what "increasing" a version means. - +
+ Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable - There are two reasons for following the previously mentioned guidelines. - First, to ensure that when a developer updates and rebuilds, they get all the changes to - the repository and do not have to remember to rebuild any sections. - Second, to ensure that target users are able to upgrade their - devices using package manager commands such as opkg upgrade - (or similar commands for dpkg/apt or rpm-based systems). + The LIC_FILES_CHKSUM + variable contains checksums of the license text in the source code for the recipe. + Following is an example of how to specify LIC_FILES_CHKSUM: + + LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ + file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ + file://licfile2.txt;endline=50;md5=zzzz \ + ..." + - The goal is to ensure the Yocto Project has packages that can be upgraded in all cases. + The Yocto Project uses the + S variable as the + default directory used when searching files listed in + LIC_FILES_CHKSUM. + The previous example employs the default directory. -
- -
- Using The Yocto Project in a Team Environment - It might not be immediately clear how you can use the Yocto Project in a team environment, - or scale it for a large team of developers. - The specifics of any situation determine the best solution. - Granted that the Yocto Project offers immense flexibility regarding this, practices do exist - that experience has shown work well. + You can also use relative paths as shown in the following example: + + LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\ + md5=bb14ed3c4cda583abc85401304b5cd4e" + LIC_FILES_CHKSUM = "file://../license.html;md5=5c94767cedb5d6987c902ac850ded2c6" + - The core component of any development effort with the Yocto Project is often an - automated build and testing framework along with an image generation process. - You can use these core components to check that the metadata can be built, - highlight when commits break the build, and provide up-to-date images that - allow developers to test the end result and use it as a base platform for further - development. - Experience shows that buildbot is a good fit for this role. - What works well is to configure buildbot to make two types of builds: - incremental and full (from scratch). - See the buildbot for the - Yocto Project for an example implementation that uses buildbot. + In this example, the first line locates a file in + S/src/ls.c. + The second line refers to a file in + WORKDIR, which is the parent + of S. - - You can tie incremental builds to a commit hook that triggers the build - each time a commit is made to the metadata. - This practice results in useful acid tests that determine whether a given commit - breaks the build in some serious way. - Associating a build to a commit can catch a lot of simple errors. - Furthermore, the tests are fast so developers can get quick feedback on changes. + Note that this variable is mandatory for all recipes, unless the + LICENSE variable is set to "CLOSED". +
+
+ Explanation of Syntax - Full builds build and test everything from the ground up. - These types of builds usually happen at predetermined times like during the - night when the machine load is low. + As mentioned in the previous section, the + LIC_FILES_CHKSUM variable lists all the + important files that contain the license text for the source code. + It is possible to specify a checksum for an entire file, or a specific section of a + file (specified by beginning and ending line numbers with the "beginline" and "endline" + parameters, respectively). + The latter is useful for source files with a license notice header, + README documents, and so forth. + If you do not use the "beginline" parameter, then it is assumed that the text begins on the + first line of the file. + Similarly, if you do not use the "endline" parameter, it is assumed that the license text + ends with the last line of the file. - Most teams have many pieces of software undergoing active development at any given time. - You can derive large benefits by putting these pieces under the control of a source - control system that is compatible with the Yocto Project (i.e. Git or Subversion (SVN). - You can then set the autobuilder to pull the latest revisions of the packages - and test the latest commits by the builds. - This practice quickly highlights issues. - The Yocto Project easily supports testing configurations that use both a - stable known good revision and a floating revision. - The Yocto Project can also take just the changes from specific source control branches. - This capability allows you to track and test specific changes. + The "md5" parameter stores the md5 checksum of the license text. + If the license text changes in any way as compared to this parameter + then a mismatch occurs. + This mismatch triggers a build failure and notifies the developer. + Notification allows the developer to review and address the license text changes. + Also note that if a mismatch occurs during the build, the correct md5 + checksum is placed in the build log and can be easily copied to the recipe. - Perhaps the hardest part of setting this up is defining the software project or - the Yocto Project metadata policies that surround the different source control systems. - Of course circumstances will be different in each case. - However, this situation reveals one of the Yocto Project's advantages - - the system itself does not - force any particular policy on users, unlike a lot of build systems. - The system allows the best policies to be chosen for the given circumstances. + There is no limit to how many files you can specify using the + LIC_FILES_CHKSUM variable. + Generally, however, every project requires a few specifications for license tracking. + Many projects have a "COPYING" file that stores the license information for all the source + code files. + This practice allows you to just track the "COPYING" file as long as it is kept up to date. -
-
- Updating Existing Images + + If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match + error and displays the correct "md5" parameter value during the build. + The correct parameter is also captured in the build log. + - - Often, rather than re-flashing a new image, you might wish to install updated - packages into an existing running system. - You can do this by first sharing the tmp/deploy/ipk/ directory - through a web server and then by changing /etc/opkg/base-feeds.conf - to point at the shared server. - Following is an example: - - $ src/gz all http://www.mysite.com/somedir/deploy/ipk/all - $ src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a - $ src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard - - + + If the whole file contains only license text, you do not need to use the "beginline" and + "endline" parameters. +
-
- Modifying Package Source Code - - Although the Yocto Project is usually used to build software, you can use it to modify software. - - - - During a build, source is available in the - WORKDIR directory. - The actual location depends on the type of package and the architecture of the target device. - For a standard recipe not related to - MACHINE, the location is - tmp/work/PACKAGE_ARCH-poky-TARGET_OS/PN-PV-PR/. - For target device-dependent packages, you should use the MACHINE - variable instead of - PACKAGE_ARCH - in the directory name. - - - - Be sure the package recipe sets the - S variable to something - other than the standard WORKDIR/PN-PV/ value. - - - - After building a package, you can modify the package source code without problems. - The easiest way to test your changes is by calling the - compile task as shown in the following example: - - $ bitbake -c compile -f NAME_OF_PACKAGE - - - - - The -f or --force - option forces re-execution of the specified task. - You can call other tasks this way as well. - But note that all the modifications in - WORKDIR - are gone once you execute -c clean for a package. - -
- -
- Modifying Package Source Code with Quilt - - - By default Poky uses Quilt - to manage patches in the do_patch task. - This is a powerful tool that you can use to track all modifications to package sources. - - - - Before modifying source code, it is important to notify Quilt so it can track the changes - into the new patch file: - - $ quilt new NAME-OF-PATCH.patch - - - - - After notifying Quilt, add all modified files into that patch: - - $ quilt add file1 file2 file3 - - - +
+ Handling a Package Name Alias - You can now start editing. - Once you are done editing, you need to use Quilt to generate the final patch that - will contain all your modifications. - - $ quilt refresh - + Sometimes a package name you are using might exist under an alias or as a similarly named + package in a different distribution. + The Yocto Project implements a distro_check + task that automatically connects to major distributions + and checks for these situations. + If the package exists under a different name in a different distribution, you get a + distro_check mismatch. + You can resolve this problem by defining a per-distro recipe name alias using the + DISTRO_PN_ALIAS variable. - You can find the resulting patch file in the - patches/ subdirectory of the source - (S) directory. - For future builds, you should copy the patch into the Yocto Project metadata and add it into the - SRC_URI of a recipe. - Here is an example: + Following is an example that shows how you specify the DISTRO_PN_ALIAS + variable: - SRC_URI += "file://NAME-OF-PATCH.patch" + DISTRO_PN_ALIAS_pn-PACKAGENAME = "distro1=package_name_alias1 \ + distro2=package_name_alias2 \ + distro3=package_name_alias3 \ + ..." - + - Finally, don't forget to 'bump' the - PR value in the same recipe since - the resulting packages have changed. + If you have more than one distribution alias, separate them with a space. + Note that the Yocto Project currently automatically checks the + Fedora, OpenSuSE, Debian, Ubuntu, + and Mandriva distributions for source package recipes without having to specify them + using the DISTRO_PN_ALIAS variable. + For example, the following command generates a report that lists the Linux distributions + that include the sources for each of the Yocto Project recipes. + + $ bitbake world -f -c distro_check + + The results are stored in the build/tmp/log/distro_check-${DATETIME}.results + file found in the Yocto Project files area.
-
- Combining Multiple versions of Library Files into One Image - +
+ Making and Maintaining Changes - The build system offers the ability to build libraries with different - target optimizations or architecture formats and combine these together - into one system image. - You can link different binaries in the image - against the different libraries as needed for specific use cases. - This feature is called "Multilib." + Because the Yocto Project is extremely configurable and flexible, + we recognize that developers will want + to extend, configure or optimize it for their specific uses. + To best keep pace with future Yocto Project changes, + we recommend you make controlled changes to the Yocto Project. - An example would be where you have most of a system compiled in 32-bit - mode using 32-bit libraries, but you have something large, like a database - engine, that needs to be a 64-bit application and use 64-bit libraries. - Multilib allows you to get the best of both 32-bit and 64-bit libraries. + The Yocto Project supports a "layers" concept. + If you use layers properly, you can ease future upgrades and allow segregation + between the Yocto Project core and a given developer's changes. + The following section provides more advice on managing changes to the Yocto Project. - - While the Multilib feature is most commonly used for 32 and 64-bit differences, - the approach the build system uses facilitates different target optimizations. - You could compile some binaries to use one set of libraries and other binaries - to use other different sets of libraries. - The libraries could differ in architecture, compiler options, or other - optimizations. - +
+ BitBake Layers + + Often, developers want to extend the Yocto Project either by adding packages + or by overriding files contained within the Yocto Project to add their own + functionality. + BitBake has a powerful mechanism called + "layers", which provides a way to handle this extension in a fully + supported and non-invasive fashion. + - - This section overviews the Multilib process only. - For more details on how to implement Multilib, see the - Multilib wiki - page. - + + The Yocto Project files include several additional layers such as + meta-rt and meta-yocto + that demonstrate this functionality. + The meta-rt layer is not enabled by default. + However, the meta-yocto layer is. + -
- Preparing to use Multilib + + To enable a layer, you simply add the layer's path to the + BBLAYERS variable in your + bblayers.conf file, which is found in the Yocto Project file's + build directory. + The following example shows how to enable the meta-rt: + + LCONF_VERSION = "1" - - User-specific requirements drive the Multilib feature, - Consequently, there is no one "out-of-the-box" configuration that likely - exists to meet your needs. - + BBFILES ?= "" + BBLAYERS = " \ + /path/to/poky/meta \ + /path/to/poky/meta-yocto \ + /path/to/poky/meta-rt \ + " + + - - In order to enable Multilib, you first need to ensure your recipe is - extended to support multiple libraries. - Many standard recipes are already extended and support multiple libraries. - You can check in the meta/conf/multilib.conf - configuration file in the Yocto Project files directory to see how this is - done using the BBCLASSEXTEND variable. - Eventually, all recipes will be covered and this list will be unneeded. + + BitBake parses each conf/layer.conf file for each layer in + BBLAYERS + and adds the recipes, classes and configurations contained within the layer to + the Yocto Project. + To create your own layer, independent of the Yocto Project files, + simply create a directory with a conf/layer.conf file and + add the directory to your bblayers.conf file. + + + + The meta-yocto/conf/layer.conf file demonstrates the + required syntax: + + # We have a conf and classes directory, add to BBPATH + BBPATH := "${BBPATH}:${LAYERDIR}" + + # We have a packages directory, add to BBFILES + BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + + BBFILE_COLLECTIONS += "yocto" + BBFILE_PATTERN_yocto := "^${LAYERDIR}/" + BBFILE_PRIORITY_yocto = "5" + + + + + In the previous example, the recipes for the layers are added to + BBFILES. + The BBFILE_COLLECTIONS + variable is then appended with the layer name. + The BBFILE_PATTERN variable + immediately expands with a regular expression used to match files from + BBFILES into + a particular layer, in this case by using the base pathname. + The BBFILE_PRIORITY variable + then assigns different priorities to the files in different layers. + Applying priorities is useful in situations where the same package might appear in multiple + layers and allows you to choose what layer should take precedence. - + - For the most part, the Multilib class extension works automatically to - extend the package name from ${PN} to - ${MLPREFIX}${PN}, where MLPREFIX - is the particular multilib (e.g. "lib32-" or "lib64-"). - Standard variables such as DEPENDS, - RDEPENDS, RPROVIDES, - RRECOMMENDS, PACKAGES, and - PACKAGES_DYNAMIC are automatically extended by the system. - If you are extending any manual code in the recipe, you can use the - ${MLPREFIX} variable to ensure those names are extended - correctly. - This automatic extension code resides in multilib.bbclass. + Note the use of the LAYERDIR + variable with the immediate expansion operator. + The LAYERDIR variable expands to the directory of the current layer and + requires the immediate expansion operator so that BitBake does not wait to expand the variable + when it's parsing a different directory. -
- -
- Using Multilib - After you have set up the recipes, you need to define the actual - combination of multiple libraries you want to build. - You accomplish this through your local.conf - configuration file in the Yocto Project build directory. - An example configuration would be as follows: - - MACHINE = "qemux86-64" - require conf/multilib.conf - MULTILIBS = "multilib:lib32" - DEFAULTTUNE_virtclass-multilib-lib32 = "x86" - MULTILIB_IMAGE_INSTALL = "lib32-connman" - - This example enables an - additional library named lib32 alongside the - normal target packages. - When combining these "lib32" alternatives, the example uses "x86" for tuning. - For information on this particular tuning, see - meta/conf/machine/include/ia32/arch-ia32.inc. + BitBake can locate where other .bbclass and configuration files + are applied through the BBPATH environment variable. + For these cases, BitBake uses the first file with the matching name found in + BBPATH. + This is similar to the way the PATH variable is used for binaries. + We recommend, therefore, that you use unique .bbclass + and configuration file names in your custom layer. - The example then includes lib32-connman - in all the images, which illustrates one method of including a - multiple library dependency. - You can use a normal image build to include this dependency, - for example: - - $ bitbake core-image-sato - - You can also build Multilib packages specifically with a command like this: - - $ bitbake lib32-connman - + We also recommend the following: + + Store custom layers in a Git repository that uses the + meta-prvt-XXXX format. + Clone the repository alongside other meta + directories in the Yocto Project source files area. + + Following these recommendations keeps your Yocto Project files area and + its configuration entirely inside the Yocto Project's core base.
-
- Additional Implementation Details +
+ Committing Changes - Different packaging systems have different levels of native Multilib - support. - For the RPM Package Management System, the following implementation details - exist: + Modifications to the Yocto Project are often managed under some kind of source + revision control system. + Because some simple practices can significantly improve usability, policy for committing changes + is important. + It helps to use a consistent documentation style when committing changes. + The Yocto Project development team has found the following practices work well: - A unique architecture is defined for the Multilib packages, - along with creating a unique deploy folder under - tmp/deploy/rpm in the Yocto - Project build directory. - For example, consider lib32 in a - qemux86-64 image. - The possible architectures in the system are "all", "qemux86_64", - "lib32_qemux86_64", and "lib32_x86". - The ${MLPREFIX} variable is stripped from - ${PN} during RPM packaging. - The naming for a normal RPM package and a Multilib RPM package in a - qemux86-64 system resolves to something similar to - bash-4.1-r2.x86_64.rpm and - bash-4.1.r2.lib32_x86.rpm, respectively. - - When installing a Multilib image, the RPM backend first - installs the base image and then installs the Multilib libraries. - - The build system relies on RPM to resolve the identical files in the - two (or more) Multilib packages. + The first line of the commit summarizes the change and begins with the + name of the affected package or packages. + However, not all changes apply to specific packages. + Consequently, the prefix could also be a machine name or class name. + The second part of the commit (if needed) is a longer more detailed + description of the changes. + Placing a blank line between the first and second parts helps with + readability. - For the IPK Package Management System, the following implementation details exist: - - The ${MLPREFIX} is not stripped from - ${PN} during IPK packaging. - The naming for a normal RPM package and a Multilib IPK package in a - qemux86-64 system resolves to something like - bash_4.1-r2.x86_64.ipk and - lib32-bash_4.1-rw_x86.ipk, respectively. - - The IPK deploy folder is not modified with - ${MLPREFIX} because packages with and without - the Multilib feature can exist in the same folder due to the - ${PN} differences. - IPK defines a sanity check for Multilib installation - using certain rules for file comparison, overridden, etc. - - + Following is an example commit: + + bitbake/data.py: Add emit_func() and generate_dependencies() functions + + These functions allow generation of dependency data between functions and + variables allowing moves to be made towards generating checksums and allowing + use of the dependency information in other parts of BitBake. + + Signed-off-by: Richard Purdie richard.purdie@linuxfoundation.org + + + + + All commits should be self-contained such that they leave the + metadata in a consistent state that builds both before and after the + commit is made. + Besides being a good practice to follow, it helps ensure autobuilder test results + are valid. -
-
- -
- Tracking License Changes +
- - The license of an upstream project might change in the future. In order to prevent these changes - going unnoticed, the Yocto Project provides a - LIC_FILES_CHKSUM - variable to track changes to the license text. The checksums are validated at the end of the - configure step, and if the checksums do not match, the build will fail. - +
+ Package Revision Incrementing -
- Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable + + If a committed change results in changing the package output, + then the value of the + PR variable needs to be increased + (or "bumped") as part of that commit. + This means that for new recipes you must be sure to add the PR + variable and set its initial value equal to "r0". + Failing to define PR makes it easy to miss when you bump a package. + Note that you can only use integer values following the "r" in the + PR variable. + - The LIC_FILES_CHKSUM - variable contains checksums of the license text in the source code for the recipe. - Following is an example of how to specify LIC_FILES_CHKSUM: - - LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ - file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ - file://licfile2.txt;endline=50;md5=zzzz \ - ..." - + If you are sharing a common .inc file with multiple recipes, + you can also use the + INC_PR variable to ensure that + the recipes sharing the .inc file are rebuilt when the + .inc file itself is changed. + The .inc file must set INC_PR + (initially to "r0"), and all recipes referring to it should set PR + to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. + If the .inc file is changed then its + INC_PR should be incremented. + + + + When upgrading the version of a package, assuming the + PV changes, + the PR variable should be reset to "r0" + (or "$(INC_PR).0" if you are using INC_PR). - The Yocto Project uses the - S variable as the - default directory used when searching files listed in - LIC_FILES_CHKSUM. - The previous example employs the default directory. + Usually, version increases occur only to packages. + However, if for some reason PV changes but does not + increase, you can increase the + PE variable (Package Epoch). + The PE variable defaults to "0". - You can also use relative paths as shown in the following example: - - LIC_FILES_CHKSUM = "file://src/ls.c;startline=5;endline=16;\ - md5=bb14ed3c4cda583abc85401304b5cd4e" - LIC_FILES_CHKSUM = "file://../license.html;md5=5c94767cedb5d6987c902ac850ded2c6" - + Version numbering strives to follow the + + Debian Version Field Policy Guidelines. + These guidelines define how versions are compared and what "increasing" a version means. - In this example, the first line locates a file in - S/src/ls.c. - The second line refers to a file in - WORKDIR, which is the parent - of S. + There are two reasons for following the previously mentioned guidelines. + First, to ensure that when a developer updates and rebuilds, they get all the changes to + the repository and do not have to remember to rebuild any sections. + Second, to ensure that target users are able to upgrade their + devices using package manager commands such as opkg upgrade + (or similar commands for dpkg/apt or rpm-based systems). + - Note that this variable is mandatory for all recipes, unless the - LICENSE variable is set to "CLOSED". + The goal is to ensure the Yocto Project has packages that can be upgraded in all cases.
-
- Explanation of Syntax +
+ Using The Yocto Project in a Team Environment + - As mentioned in the previous section, the - LIC_FILES_CHKSUM variable lists all the - important files that contain the license text for the source code. - It is possible to specify a checksum for an entire file, or a specific section of a - file (specified by beginning and ending line numbers with the "beginline" and "endline" - parameters, respectively). - The latter is useful for source files with a license notice header, - README documents, and so forth. - If you do not use the "beginline" parameter, then it is assumed that the text begins on the - first line of the file. - Similarly, if you do not use the "endline" parameter, it is assumed that the license text - ends with the last line of the file. + It might not be immediately clear how you can use the Yocto Project in a team environment, + or scale it for a large team of developers. + The specifics of any situation determine the best solution. + Granted that the Yocto Project offers immense flexibility regarding this, practices do exist + that experience has shown work well. - The "md5" parameter stores the md5 checksum of the license text. - If the license text changes in any way as compared to this parameter - then a mismatch occurs. - This mismatch triggers a build failure and notifies the developer. - Notification allows the developer to review and address the license text changes. - Also note that if a mismatch occurs during the build, the correct md5 - checksum is placed in the build log and can be easily copied to the recipe. + The core component of any development effort with the Yocto Project is often an + automated build and testing framework along with an image generation process. + You can use these core components to check that the metadata can be built, + highlight when commits break the build, and provide up-to-date images that + allow developers to test the end result and use it as a base platform for further + development. + Experience shows that buildbot is a good fit for this role. + What works well is to configure buildbot to make two types of builds: + incremental and full (from scratch). + See the buildbot for the + Yocto Project for an example implementation that uses buildbot. - There is no limit to how many files you can specify using the - LIC_FILES_CHKSUM variable. - Generally, however, every project requires a few specifications for license tracking. - Many projects have a "COPYING" file that stores the license information for all the source - code files. - This practice allows you to just track the "COPYING" file as long as it is kept up to date. + You can tie incremental builds to a commit hook that triggers the build + each time a commit is made to the metadata. + This practice results in useful acid tests that determine whether a given commit + breaks the build in some serious way. + Associating a build to a commit can catch a lot of simple errors. + Furthermore, the tests are fast so developers can get quick feedback on changes. - - If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match - error and displays the correct "md5" parameter value during the build. - The correct parameter is also captured in the build log. - + + Full builds build and test everything from the ground up. + These types of builds usually happen at predetermined times like during the + night when the machine load is low. + - - If the whole file contains only license text, you do not need to use the "beginline" and - "endline" parameters. - + + Most teams have many pieces of software undergoing active development at any given time. + You can derive large benefits by putting these pieces under the control of a source + control system that is compatible with the Yocto Project (i.e. Git or Subversion (SVN). + You can then set the autobuilder to pull the latest revisions of the packages + and test the latest commits by the builds. + This practice quickly highlights issues. + The Yocto Project easily supports testing configurations that use both a + stable known good revision and a floating revision. + The Yocto Project can also take just the changes from specific source control branches. + This capability allows you to track and test specific changes. + + + + Perhaps the hardest part of setting this up is defining the software project or + the Yocto Project metadata policies that surround the different source control systems. + Of course circumstances will be different in each case. + However, this situation reveals one of the Yocto Project's advantages - + the system itself does not + force any particular policy on users, unlike a lot of build systems. + The system allows the best policies to be chosen for the given circumstances. +
-
-
- Handling a Package Name Alias - - Sometimes a package name you are using might exist under an alias or as a similarly named - package in a different distribution. - The Yocto Project implements a distro_check - task that automatically connects to major distributions - and checks for these situations. - If the package exists under a different name in a different distribution, you get a - distro_check mismatch. - You can resolve this problem by defining a per-distro recipe name alias using the - DISTRO_PN_ALIAS variable. - +
+ Updating Existing Images - - Following is an example that shows how you specify the DISTRO_PN_ALIAS - variable: - - DISTRO_PN_ALIAS_pn-PACKAGENAME = "distro1=package_name_alias1 \ - distro2=package_name_alias2 \ - distro3=package_name_alias3 \ - ..." - - - - - If you have more than one distribution alias, separate them with a space. - Note that the Yocto Project currently automatically checks the - Fedora, OpenSuSE, Debian, Ubuntu, - and Mandriva distributions for source package recipes without having to specify them - using the DISTRO_PN_ALIAS variable. - For example, the following command generates a report that lists the Linux distributions - that include the sources for each of the Yocto Project recipes. - - $ bitbake world -f -c distro_check - - The results are stored in the build/tmp/log/distro_check-${DATETIME}.results - file found in the Yocto Project files area. - + + Often, rather than re-flashing a new image, you might wish to install updated + packages into an existing running system. + You can do this by first sharing the tmp/deploy/ipk/ directory + through a web server and then by changing /etc/opkg/base-feeds.conf + to point at the shared server. + Following is an example: + + $ src/gz all http://www.mysite.com/somedir/deploy/ipk/all + $ src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a + $ src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard + + +
+