From 3db33448592f737515289edef6ed25d407fd8039 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Thu, 13 Apr 2017 10:18:33 -0700 Subject: ref-manual, dev-manual: Updates to support recipe-specific sysroots Changes covered several areas. Version 2.3 of the YP now supports recipe-specific sysroots and is not limited to machine-specific as was prior releases. Manual changes were as follows: dev-manual: "Sharing Files Between Recipes" - Wordings were changed to support the new functionality. ref-manual: do_prepare_recipe_sysroot task added to the list of tasks described in "Configuration and Compilation". ref-manual: Extensive re-write of the "staging.bbclass" section. ref-manual: Added a section to the build structure for build/tmp/work/tunearch/recipename/version/. This section breaks down the recipe-specific subdirectories used to create (stage) the sysroot. ref-manual: New section (entry) for the do_prepare_recipe_sysroot task in the task chapter. ref-manual: Added a variable glossary description for the SSTATE_SCAN_FILES variable. In addition to these changes, I sprinkled in a liberal amount of cross-referencing for the readers pleasure. (From yocto-docs rev: 3a8ca96861f4c5d3badb91d0cdc5a3df513d4e59) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/ref-manual/closer-look.xml | 13 +++ documentation/ref-manual/ref-classes.xml | 145 +++++++++++++++++++++++++++-- documentation/ref-manual/ref-structure.xml | 118 ++++++++++++++++++++++- documentation/ref-manual/ref-tasks.xml | 12 +++ documentation/ref-manual/ref-variables.xml | 29 ++++++ 5 files changed, 305 insertions(+), 12 deletions(-) (limited to 'documentation/ref-manual') diff --git a/documentation/ref-manual/closer-look.xml b/documentation/ref-manual/closer-look.xml index d7b47d83da..a6f1b23257 100644 --- a/documentation/ref-manual/closer-look.xml +++ b/documentation/ref-manual/closer-look.xml @@ -867,6 +867,19 @@ This step in the build process consists of three tasks: + + do_prepare_recipe_sysroot: + This task sets up the two sysroots in + ${WORKDIR} + (i.e. recipe-sysroot and + recipe-sysroot-native) so that + the sysroots contain the contents of the + do_populate_sysroot + tasks of the recipes on which the recipe + containing the tasks depends. + A sysroot exists for both the target and for the native + binaries, which run on the host system. + do_configure: This task configures the source by enabling and disabling any build-time and configuration options for diff --git a/documentation/ref-manual/ref-classes.xml b/documentation/ref-manual/ref-classes.xml index 3d9fbf695d..8f25c521c6 100644 --- a/documentation/ref-manual/ref-classes.xml +++ b/documentation/ref-manual/ref-classes.xml @@ -3190,13 +3190,144 @@ <filename>staging.bbclass</filename> - The staging class provides the - do_populate_sysroot - task, which stages files into the sysroot to make them available to - other recipes at build time. - The class is enabled by default because it is inherited by the - base - class. + The staging class installs files into individual + recipe work directories for sysroots. + The class contains the following key tasks: + + + The + do_populate_sysroot + task, which is responsible for handing the files that end up + in the recipe sysroots. + + + The + do_prepare_recipe_sysroot + task (a "partner" task to the + populate_sysroot task), which installs + the files into the individual recipe work directories (i.e. + WORKDIR). + + + + + + The code in the staging class is complex and + basically works in two stages: + + + Stage One: + The first stage addresses recipes that have files they want + to share with other recipes that have dependencies on the + originating recipe. + Normally these dependencies are installed through the + do_install + task into + ${D}. + The do_populate_sysroot task copies + a subset of these files into + ${SYSROOT_DESTDIR}. + This subset of files is controlled by the + SYSROOT_DIRS, + SYSROOT_DIRS_NATIVE, + and + SYSROOT_DIRS_BLACKLIST + variables. + + Additionally, a recipe can customize the files further by + declaring a processing function in the + SYSROOT_PREPROCESS_FUNCS + variable. + + + + + A shared state (sstate) object is built from these files + and the files are placed into a subdirectory of + tmp/sysroot-components/. + The files are scanned for hardcoded paths to the original + installation location. + If the location is found in text files, the hardcoded + locations are replaced by tokens and a list of the files + needing such replacements is created. + These adjustments are referred to as "FIXMEs". + The list of files that are scanned for paths is controlled by + the + SSTATE_SCAN_FILES + variable. + + + Stage Two: + The second stage addresses recipes that want to use something + from another recipe and declare a dependency on that recipe + through the + DEPENDS + variable. + The recipe will have a + do_prepare_recipe_sysroot + task and when + this task executes, it creates the + recipe-sysroot and + recipe-sysroot-native in the recipe + work directory (i.e. + WORKDIR. + The OpenEmbedded build system creates hard links to copies of the + relevant files from sysroot-components + into the recipe work directory. + + If hard links are not possible, the build system uses + actual copies. + + The build system then addresses any "FIXMEs" to paths as + defined from the list created in the first stage. + + + + Finally, any files in ${bindir} + within the sysroot that have the prefix + "postinst-" are executed. + + Although these files are not recommended for general use, + the files do allow some issues such as user creation + and module indexes to be addressed. + + + + + Because recipes can have other dependencies outside of + DEPENDS (e.g. + do_unpack[depends] += "tar-native:do_populate_sysroot"), + the sysroot creation function + extend_recipe_sysroot is also added as + a pre-function for those tasks whose dependencies are not + through DEPENDS but operate similarly. + + + + When installing dependencies into the sysroot, the code + traverses the dependency graph and processes dependencies + in exactly the same way as the dependencies would or would not + be when installed from sstate. + This processing means, for example, a native tool would have + its native dependencies added but a target library would not + have its dependencies traversed or installed. + The same sstate dependency code is used so that + builds should be identical regardless of whether sstate + was used or not. + For a closer look, see the + setscene_depvalid() function in the + sstate + class. + + + + The build system is careful to maintain manifests of the files + it installs so that any given dependency can be installed as + needed. + The sstate hash of the installed item is also stored so that + if it changes, the build system can reinstall it. + + diff --git a/documentation/ref-manual/ref-structure.xml b/documentation/ref-manual/ref-structure.xml index 541a47e556..b0c4bfeaa4 100644 --- a/documentation/ref-manual/ref-structure.xml +++ b/documentation/ref-manual/ref-structure.xml @@ -801,11 +801,24 @@ <filename>build/tmp/sysroots/</filename> - This directory contains shared header files and libraries as well as other shared - data. - Packages that need to share output with other packages do so within this directory. - The directory is subdivided by architecture so multiple builds can run within - the one Build Directory. + Previous versions of the OpenEmbedded build system used to + create a global shared sysroot per machine along with a native + sysroot. + Beginning with the &DISTRO; version of the Yocto Project, + sysroots exist in recipe-specific + WORKDIR + directories. + Thus, the build/tmp/sysroots/ directory + is unused. + + The build/tmp/sysroots/ directory + can still be populated using the + bitbake build-sysroots command and can + be used for compatibility in some cases. + However, in general it is not recommended to populate + this directory. + Individual recipe-specific sysroots should be used. + @@ -894,6 +907,101 @@ +
+ <filename>build/tmp/work/<replaceable>tunearch</replaceable>/<replaceable>recipename</replaceable>/<replaceable>version</replaceable>/</filename> + + + The recipe work directory (recipe_work_directory). + + + + As described earlier in the + "build/tmp/sysroots/" + section, beginning with the &DISTRO; release of the Yocto + Project, the OpenEmbedded build system builds each recipe in its + own work directory (i.e. + WORKDIR). + The path to the work directory is constructed using the + architecture of the given build (e.g. + TUNE_PKGARCH, + MACHINE_ARCH, + or "allarch"), the recipe name, and the version of the recipe (i.e. + PE:PV-PR). + + + + A number of key subdirectories exist within each recipe + work directory: + + + recipe_work_directory/temp: + Contains the log files of each task executed for this + recipe, the "run" files for each executed task, which + contain the code run, and a + log.task_order file, which lists the + order in which tasks were executed. + + + recipe_work_directory/image: + Contains the output of the + do_install + task, which corresponds to the + ${D} + variable in that task. + + + recipe_work_directory/pseudo: + Contains the pseudo database and log for any tasks executed + under pseudo for the recipe. + + + recipe_work_directory/sysroot-destdir: + Contains the output of the + do_populate_sysroot + task. + + + recipe_work_directory/package: + Contains the output of the + do_package + task before the output is split into individual packages. + + + recipe_work_directory/packages-split: + Contains the output of the do_package + task after the output has been split into individual + packages. + Subdirectories exist for each individual package created + by the recipe. + + + recipe_work_directory/recipe-sysroot: + A directory populated with the target dependencies of the + recipe. + This directory looks like the target filesystem and + contains libraries that the recipe might need to link + against (e.g. the C library). + + + recipe_work_directory/recipe-sysroot-native: + A directory populated with the native dependencies of the + recipe. + This directory contains the tools the recipe needs to build + (e.g. the compiler, Autoconf, libtool, and so forth). + + + recipe_work_directory/build: + This subdirectory applies only to recipes that support + builds where the source is separate from the + build artifacts. + The OpenEmbedded build system uses this directory as a + separate build directory (i.e. + ${B}). + + + +
+
<filename>build/tmp/work-shared/</filename> diff --git a/documentation/ref-manual/ref-tasks.xml b/documentation/ref-manual/ref-tasks.xml index a3a41ebcc1..99e9f52b86 100644 --- a/documentation/ref-manual/ref-tasks.xml +++ b/documentation/ref-manual/ref-tasks.xml @@ -490,6 +490,18 @@
+
+ <filename>do_prepare_recipe_sysroot</filename> + + + Installs the files into the individual recipe work directories + (i.e. WORKDIR). + See the + "staging" + class for more information. + +
+
<filename>do_rm_work</filename> diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml index 55f7a28c21..3699ae75bc 100644 --- a/documentation/ref-manual/ref-variables.xml +++ b/documentation/ref-manual/ref-variables.xml @@ -12776,6 +12776,35 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3" + SSTATE_SCAN_FILES + + SSTATE_SCAN_FILES[doc] = "Controls the list of files the OpenEmbedded build system scans for hardcoded installation paths." + + + + + Controls the list of files the OpenEmbedded build system + scans for hardcoded installation paths. + + + + During a build, the OpenEmbedded build system creates a + shared state (sstate) object during the first stage of + preparing the sysroots. + During the build, the object is scanned for hardcoded paths + for original installation locations. + The list of files that are scanned for paths is controlled + by the SSTATE_SCAN_FILES variable. + + + + For details on the process, see the + staging + class. + + + + STAGING_BASE_LIBDIR_NATIVE STAGING_BASE_LIBDIR_NATIVE[doc] = "Specifies the path to the /lib subdirectory of the sysroot directory for the build host." -- cgit v1.2.3-54-g00ecf