From e5702bce4a741d47c169509f7ee01cab89da372e Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Mon, 19 Mar 2018 12:37:09 -0700 Subject: dev-manual: Completed rewrite of runtime package management Fixes [YOCTO #12419] This section was unclear and needed some work. I added text to help clear things up. (From yocto-docs rev: 54358a39a5710dbd0cbcfe8dafd2f641ec3863ef) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../dev-manual/dev-manual-common-tasks.xml | 285 ++++++++++++++------- 1 file changed, 197 insertions(+), 88 deletions(-) (limited to 'documentation') diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index 670b01a4e8..3b8e091963 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -34,11 +34,11 @@ It is very easy to create your own layers to use with the OpenEmbedded build system. - The Yocto Project ships with scripts that speed up creating - general layers and BSP layers. + The Yocto Project ships with tools that speed up creating + layers. This section describes the steps you perform by hand to create a layer so that you can better understand them. - For information about the layer-creation scripts, see the + For information about the layer-creation tools, see the "Creating a New BSP Layer Using the bitbake-layers Script" section in the Yocto Project Board Support Package (BSP) Developer's Guide and the @@ -47,9 +47,11 @@ - Follow these general steps to create your layer without the aid of a script: + Follow these general steps to create your layer without using + tools: - Check Existing Layers: + + Check Existing Layers: Before creating a new layer, you should be sure someone has not already created a layer containing the Metadata you need. @@ -57,19 +59,31 @@ OpenEmbedded Metadata Index for a list of layers from the OpenEmbedded community that can be used in the Yocto Project. + You could find a layer that is identical or close to + what you need. - Create a Directory: + + Create a Directory: Create the directory for your layer. While not strictly required, prepend the name of the - folder with the string meta-. + directory with the string "meta-". For example: meta-mylayer meta-GUI_xyz meta-mymachine + Realize that the name of your layer does not include + the "meta-" string. + For example, the names of the previous three layers + are "mylayer", "GUI_xyz", and "mymachine". + Following this layer directory naming convention can + save you trouble later when tools or components + "assume" your layer resides in a directory whose name + starts with "meta-". - Create a Layer Configuration + + Create a Layer Configuration File: Inside your new layer folder, you need to create a conf/layer.conf file. @@ -77,9 +91,14 @@ file and copy that to your layer's conf directory and then modify the file as needed. + The meta-yocto-bsp/conf/layer.conf file - demonstrates the required syntax: + in the Yocto Project + Source Repositories + demonstrates the required syntax. + For your layer, you need to replace "yoctobsp" with the + root name of your layer: # We have a conf and classes directory, add to BBPATH BBPATH .= ":${LAYERDIR}" @@ -91,35 +110,41 @@ BBFILE_COLLECTIONS += "yoctobsp" BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" BBFILE_PRIORITY_yoctobsp = "5" - LAYERVERSION_yoctobsp = "3" + LAYERVERSION_yoctobsp = "4" + LAYERSERIES_COMPAT_yoctobsp = "&DISTRO_NAME_NO_CAP;" + Here is an explanation of the example: - The configuration and - classes directory is appended to + + The configuration and classes directory is + appended to BBPATH. All non-distro layers, which include all BSP layers, are expected to append the layer - directory to the - BBPATH. + directory to BBPATH. On the other hand, distro layers, such as meta-poky, can choose to enforce their own precedence over BBPATH. For an example of that syntax, see the - layer.conf file for - the meta-poky layer. - - The recipes for the layers are - appended to + meta-poky/conf/layer.conf + file for in the Source Repositories. + + + + The recipes for the layers are appended to BBFILES. - The + + The BBFILE_COLLECTIONS - variable is then appended with the layer name. + variable is then appended with the layer name, + which is "yoctobsp" in this example. - The + + The BBFILE_PATTERN variable is set to a regular expression and is used to match files from @@ -127,24 +152,34 @@ layer. In this case, LAYERDIR - is used to make BBFILE_PATTERN match within the - layer's path. - The + is used to make + BBFILE_PATTERN match + within the layer's path. + + + The BBFILE_PRIORITY variable then assigns a priority to the layer. Applying priorities is useful in situations where the same recipe might appear in multiple - layers and allows you to choose the layer - that takes precedence. - The + layers. + A priority allows you to choose the layer + that takes precedence. + + + The LAYERVERSION variable optionally specifies the version of a - layer as a single number. + layer as a single number. + + + The + LAYERDIR + variable expands to the directory of the + current layer + - Note the use of the - LAYERDIR - variable, which expands to the directory of the current - layer. + Through the use of the BBPATH variable, BitBake locates class files (.bbclass), @@ -156,10 +191,11 @@ This is similar to the way the PATH variable is used for binaries. It is recommended, therefore, that you use unique - class and configuration - filenames in your custom layer. - Add Content: Depending - on the type of layer, add the content. + class and configuration filenames in your custom layer. + + + Add Content: + Depending on the type of layer, add the content. If the layer adds support for a machine, add the machine configuration in a conf/machine/ file within the layer. @@ -169,7 +205,8 @@ If the layer introduces new recipes, put the recipes you need in recipes-* subdirectories within the layer. - In order to be compliant with the Yocto Project, + + In order to be compliant with the Yocto Project, a layer must contain a README file. @@ -8835,11 +8872,13 @@ Some notes from Cal: During a build, BitBake always transforms a recipe into one or more packages. For example, BitBake takes the bash recipe - and currently produces the bash-dbg, - bash-staticdev, - bash-dev, bash-doc, - bash-locale, and - bash packages. + and produces a number of packages (e.g. + bash, bash-bashbug, + bash-completion, + bash-completion-dbg, + bash-completion-dev, + bash-completion-extra, + bash-dbg, and so forth). Not all generated packages are included in an image. @@ -8883,7 +8922,7 @@ Some notes from Cal: In order to use runtime package management, you - need a host/server machine that serves up the pre-compiled + need a host or server machine that serves up the pre-compiled packages plus the required metadata. You also need package manipulation tools on the target. The build machine is a likely candidate to act as the server. @@ -8891,6 +8930,10 @@ Some notes from Cal: package server. The build machine could push its artifacts to another machine that acts as the server (e.g. Internet-facing). + In fact, doing so is advantageous for a production + environment as getting the packages away from the + development system's build directory prevents accidental + overwrites. @@ -8900,11 +8943,11 @@ Some notes from Cal: out into a couple of different package groupings based on criteria such as the target's CPU architecture, the target board, or the C library used on the target. - For example, a build targeting the qemuarm + For example, a build targeting the qemux86 device produces the following three package databases: - all, armv5te, and - qemuarm. - If you wanted your qemuarm device to be + noarch, i586, and + qemux86. + If you wanted your qemux86 device to be aware of all the packages that were available to it, you would need to point it to each of these databases individually. @@ -8950,10 +8993,10 @@ Some notes from Cal: PACKAGE_CLASSES ?= “package_packageformat” where packageformat - can be "ipk", "rpm", and "deb", which are the + can be "ipk", "rpm", "deb", or "tar" which are the supported package formats. - Because the Yocto Project supports three + Because the Yocto Project supports four different package formats, you can set the variable with more than one argument. However, the OpenEmbedded build system only @@ -8972,12 +9015,12 @@ Some notes from Cal: "package-management" in the IMAGE_FEATURES variable. - Including "package-management" in this - configuration variable ensures that when the image - is assembled for your target, the image includes - the currently-known package databases as well as - the target-specific tools required for runtime - package management to be performed on the target. + Including "package-management" in this configuration + variable ensures that when the image is assembled for your + target, the image includes the currently-known package + databases as well as the target-specific tools required + for runtime package management to be performed on the + target. However, this is not strictly necessary. You could start your image off without any databases but only include the required on-target package @@ -8991,21 +9034,26 @@ Some notes from Cal: Whenever you perform any sort of build step that can - potentially generate a package or modify an existing + potentially generate a package or modify existing package, it is always a good idea to re-generate the - package index with: + package index after the build by using the following + command: $ bitbake package-index - Realize that it is not sufficient to simply do the - following: + It might be tempting to build the package and the + package index at the same time with a command such as + the following: $ bitbake some-package package-index - The reason for this restriction is because BitBake does not - properly schedule the package-index - target fully after any other target has completed. - Thus, be sure to run the package update step separately. + Do not do this as BitBake does not schedule the package + index for after the completion of the package you are + building. + Consequently, you cannot be sure of the package index + including information for the package you just built. + Thus, be sure to run the package update step separately + after building any packages. @@ -9030,8 +9078,8 @@ Some notes from Cal: For example, if ${TMPDIR} is tmp and your selected package type - is IPK, then your IPK packages are available in - tmp/deploy/ipk. + is RPM, then your RPM packages are available in + tmp/deploy/rpm. @@ -9086,17 +9134,39 @@ Some notes from Cal: Using RPM - The dnf application performs - runtime package management of RPM packages. - You must perform an initial setup for - dnf on the target machine - if the - PACKAGE_FEED_ARCHS, - PACKAGE_FEED_BASE_PATHS, - and - PACKAGE_FEED_URIS - variables have not been set or the target image was - built before the variables were set. + The + Dandified Packaging Tool + (DNF) performs runtime package management of RPM + packages. + In order to use DNF for runtime package management, + you must perform an initial setup on the target + machine for cases where the + PACKAGE_FEED_* variables were not + set as part of the image that is running on the + target. + This means if you built your image and did not not use + these variables as part of the build and your image is + now running on the target, you need to perform the + steps in this section if you want to use runtime + package management. + + For information on the + PACKAGE_FEED_* variables, see + PACKAGE_FEED_ARCHS, + PACKAGE_FEED_BASE_PATHS, + and + PACKAGE_FEED_URIS + in the Yocto Project Reference Manual variables + glossary. + + + + + On the target, you must inform DNF that package + databases are available. + You do this by creating a file named + /etc/yum.repos.d/oe-packages.repo + and defining the oe-packages. @@ -9105,21 +9175,60 @@ Some notes from Cal: all, i586, and qemux86 from a server named my.server. - You must inform dnf of the - availability of these databases by creating a - /etc/yum.repos.d/oe-packages.repo - file with the following content: - + The specifics for setting up the web server are up to + you. + The critical requirement is that the URIs in the + target repository configuration point to the + correct remote location for the feeds. + This example uses a location outside of the build + system's deploy directory. + Realize that it is possible to run a web server + that gets the databases from + deploy directory where the image + was built. + + + + When telling DNF where to look for the package + databases, you must declare individual locations + per architecture or a single location used for all + architectures. + You cannot do both: + + + Create an Explicit List of Architectures: + Define individual base URLs to identify where + each package database is located: + [oe-packages] - baseurl=http://my.server http://my.server/rpm/qemux86 http://my.server/rpm/all - - From the target machine, fetch the repository: + baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all + + This example informs DNF about individual + package databases for all three architectures. + + + Create a Single (Full) Package Index: + Define a single base URL that identifies where + a full package database is located: + + [oe-packages] + baseurl=http://my.server/rpm + + This example informs DNF about a single package + database that contains all the package index + information for all supported architectures. + + + + + + Once you have informed DNF where to find the package + databases, you need to fetch them: # dnf makecache - After everything is set up, dnf - is able to find, install, and upgrade packages from - the specified repository. + DNF is now able to find, install, and upgrade packages + from the specified repository or repositories. See the DNF documentation -- cgit v1.2.3-54-g00ecf