summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorScott Rifenbark <srifenbark@gmail.com>2017-04-13 10:18:33 -0700
committerRichard Purdie <richard.purdie@linuxfoundation.org>2017-04-18 15:51:44 +0100
commit3db33448592f737515289edef6ed25d407fd8039 (patch)
treeb1d5e4cc2ca19d4827210f304c244506bfc74853
parent1fa1a7f174593e41b8bcf6c2f19565d6da44e991 (diff)
downloadpoky-3db33448592f737515289edef6ed25d407fd8039.tar.gz
ref-manual, dev-manual: Updates to support recipe-specific sysroots
Changes covered several areas. Version 2.3 of the YP now supports recipe-specific sysroots and is not limited to machine-specific as was prior releases. Manual changes were as follows: dev-manual: "Sharing Files Between Recipes" - Wordings were changed to support the new functionality. ref-manual: do_prepare_recipe_sysroot task added to the list of tasks described in "Configuration and Compilation". ref-manual: Extensive re-write of the "staging.bbclass" section. ref-manual: Added a section to the build structure for build/tmp/work/tunearch/recipename/version/. This section breaks down the recipe-specific subdirectories used to create (stage) the sysroot. ref-manual: New section (entry) for the do_prepare_recipe_sysroot task in the task chapter. ref-manual: Added a variable glossary description for the SSTATE_SCAN_FILES variable. In addition to these changes, I sprinkled in a liberal amount of cross-referencing for the readers pleasure. (From yocto-docs rev: 3a8ca96861f4c5d3badb91d0cdc5a3df513d4e59) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml34
-rw-r--r--documentation/ref-manual/closer-look.xml13
-rw-r--r--documentation/ref-manual/ref-classes.xml145
-rw-r--r--documentation/ref-manual/ref-structure.xml118
-rw-r--r--documentation/ref-manual/ref-tasks.xml12
-rw-r--r--documentation/ref-manual/ref-variables.xml29
6 files changed, 322 insertions, 29 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index d2bd56dc13..e12fa32516 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -3193,10 +3193,11 @@
3193 access to the library itself and its associated headers. 3193 access to the library itself and its associated headers.
3194 The way this access is accomplished is by populating sysroot 3194 The way this access is accomplished is by populating sysroot
3195 with files. 3195 with files.
3196 One sysroot exists per "machine" for which the image is 3196 Each recipe has two sysroots in its work directory, one for
3197 being built. 3197 target files
3198 In practical terms, this means a sysroot exists for the target 3198 (<filename>recipe-sysroot</filename>) and one for files that
3199 machine, and a sysroot exists for the build host. 3199 are native to the build host
3200 (<filename>recipe-sysroot-native</filename>).
3200 <note> 3201 <note>
3201 You could find the term "staging" used within the Yocto 3202 You could find the term "staging" used within the Yocto
3202 project regarding files populating sysroot (e.g. the 3203 project regarding files populating sysroot (e.g. the
@@ -3214,12 +3215,6 @@
3214 task within the 3215 task within the
3215 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename> 3216 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
3216 directory. 3217 directory.
3217 </para>
3218
3219 <para>
3220 A subset of these files, as defined by the
3221 the <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink>
3222 variable, automatically populates the sysroot.
3223 The reason for this limitation is that almost all files that 3218 The reason for this limitation is that almost all files that
3224 populate the sysroot are cataloged in manifests in order to 3219 populate the sysroot are cataloged in manifests in order to
3225 ensure the files can be removed later when a recipe is either 3220 ensure the files can be removed later when a recipe is either
@@ -3228,6 +3223,13 @@
3228 </para> 3223 </para>
3229 3224
3230 <para> 3225 <para>
3226 A subset of the files installed by the
3227 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3228 task are used by the
3229 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3230 task as defined by the the
3231 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink>
3232 variable to automatically populate the sysroot.
3231 It is possible to modify the list of directories that populate 3233 It is possible to modify the list of directories that populate
3232 the sysroot. 3234 the sysroot.
3233 The following example shows how you could add the 3235 The following example shows how you could add the
@@ -3239,13 +3241,11 @@
3239 </para> 3241 </para>
3240 3242
3241 <para> 3243 <para>
3242 For information on variables you can use to help control how 3244 for a more complete description of the
3243 files sysroot is populated, see the 3245 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3244 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink>, 3246 task and its associated functions, see the
3245 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS_NATIVE'><filename>SYSROOT_DIRS_NATIVE</filename></ulink>, 3247 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-staging'><filename>staging</filename></ulink>
3246 and 3248 class.
3247 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS_BLACKLIST'><filename>SYSROOT_DIRS_BLACKLIST</filename></ulink>
3248 variables.
3249 </para> 3249 </para>
3250 </section> 3250 </section>
3251 3251
diff --git a/documentation/ref-manual/closer-look.xml b/documentation/ref-manual/closer-look.xml
index d7b47d83da..a6f1b23257 100644
--- a/documentation/ref-manual/closer-look.xml
+++ b/documentation/ref-manual/closer-look.xml
@@ -867,6 +867,19 @@
867 <para> 867 <para>
868 This step in the build process consists of three tasks: 868 This step in the build process consists of three tasks:
869 <itemizedlist> 869 <itemizedlist>
870 <listitem><para>
871 <emphasis><link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>:</emphasis>
872 This task sets up the two sysroots in
873 <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}</filename>
874 (i.e. <filename>recipe-sysroot</filename> and
875 <filename>recipe-sysroot-native</filename>) so that
876 the sysroots contain the contents of the
877 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>
878 tasks of the recipes on which the recipe
879 containing the tasks depends.
880 A sysroot exists for both the target and for the native
881 binaries, which run on the host system.
882 </para></listitem>
870 <listitem><para><emphasis><filename>do_configure</filename>:</emphasis> 883 <listitem><para><emphasis><filename>do_configure</filename>:</emphasis>
871 This task configures the source by enabling and 884 This task configures the source by enabling and
872 disabling any build-time and configuration options for 885 disabling any build-time and configuration options for
diff --git a/documentation/ref-manual/ref-classes.xml b/documentation/ref-manual/ref-classes.xml
index 3d9fbf695d..8f25c521c6 100644
--- a/documentation/ref-manual/ref-classes.xml
+++ b/documentation/ref-manual/ref-classes.xml
@@ -3190,13 +3190,144 @@
3190 <title><filename>staging.bbclass</filename></title> 3190 <title><filename>staging.bbclass</filename></title>
3191 3191
3192 <para> 3192 <para>
3193 The <filename>staging</filename> class provides the 3193 The <filename>staging</filename> class installs files into individual
3194 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link> 3194 recipe work directories for sysroots.
3195 task, which stages files into the sysroot to make them available to 3195 The class contains the following key tasks:
3196 other recipes at build time. 3196 <itemizedlist>
3197 The class is enabled by default because it is inherited by the 3197 <listitem><para>
3198 <link linkend='ref-classes-base'><filename>base</filename></link> 3198 The
3199 class. 3199 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>
3200 task, which is responsible for handing the files that end up
3201 in the recipe sysroots.
3202 </para></listitem>
3203 <listitem><para>
3204 The
3205 <link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>
3206 task (a "partner" task to the
3207 <filename>populate_sysroot</filename> task), which installs
3208 the files into the individual recipe work directories (i.e.
3209 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>).
3210 </para></listitem>
3211 </itemizedlist>
3212 </para>
3213
3214 <para>
3215 The code in the <filename>staging</filename> class is complex and
3216 basically works in two stages:
3217 <itemizedlist>
3218 <listitem><para>
3219 <emphasis>Stage One:</emphasis>
3220 The first stage addresses recipes that have files they want
3221 to share with other recipes that have dependencies on the
3222 originating recipe.
3223 Normally these dependencies are installed through the
3224 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
3225 task into
3226 <filename>${</filename><link linkend='var-D'><filename>D</filename></link><filename>}</filename>.
3227 The <filename>do_populate_sysroot</filename> task copies
3228 a subset of these files into
3229 <filename>${SYSROOT_DESTDIR}</filename>.
3230 This subset of files is controlled by the
3231 <link linkend='var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></link>,
3232 <link linkend='var-SYSROOT_DIRS_NATIVE'><filename>SYSROOT_DIRS_NATIVE</filename></link>,
3233 and
3234 <link linkend='var-SYSROOT_DIRS_BLACKLIST'><filename>SYSROOT_DIRS_BLACKLIST</filename></link>
3235 variables.
3236 <note>
3237 Additionally, a recipe can customize the files further by
3238 declaring a processing function in the
3239 <link linkend='var-SYSROOT_PREPROCESS_FUNCS'><filename>SYSROOT_PREPROCESS_FUNCS</filename></link>
3240 variable.
3241 </note>
3242 </para>
3243
3244 <para>
3245 A shared state (sstate) object is built from these files
3246 and the files are placed into a subdirectory of
3247 <filename>tmp/sysroot-components/</filename>.
3248 The files are scanned for hardcoded paths to the original
3249 installation location.
3250 If the location is found in text files, the hardcoded
3251 locations are replaced by tokens and a list of the files
3252 needing such replacements is created.
3253 These adjustments are referred to as "FIXMEs".
3254 The list of files that are scanned for paths is controlled by
3255 the
3256 <link linkend='var-SSTATE_SCAN_FILES'><filename>SSTATE_SCAN_FILES</filename></link>
3257 variable.
3258 </para></listitem>
3259 <listitem><para>
3260 <emphasis>Stage Two:</emphasis>
3261 The second stage addresses recipes that want to use something
3262 from another recipe and declare a dependency on that recipe
3263 through the
3264 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
3265 variable.
3266 The recipe will have a
3267 <link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>
3268 task and when
3269 this task executes, it creates the
3270 <filename>recipe-sysroot</filename> and
3271 <filename>recipe-sysroot-native</filename> in the recipe
3272 work directory (i.e.
3273 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
3274 The OpenEmbedded build system creates hard links to copies of the
3275 relevant files from <filename>sysroot-components</filename>
3276 into the recipe work directory.
3277 <note>
3278 If hard links are not possible, the build system uses
3279 actual copies.
3280 </note>
3281 The build system then addresses any "FIXMEs" to paths as
3282 defined from the list created in the first stage.
3283 </para>
3284
3285 <para>
3286 Finally, any files in <filename>${bindir}</filename>
3287 within the sysroot that have the prefix
3288 "<filename>postinst-</filename>" are executed.
3289 <note>
3290 Although these files are not recommended for general use,
3291 the files do allow some issues such as user creation
3292 and module indexes to be addressed.
3293 </note>
3294 </para>
3295
3296 <para>
3297 Because recipes can have other dependencies outside of
3298 <filename>DEPENDS</filename> (e.g.
3299 <filename>do_unpack[depends] += "tar-native:do_populate_sysroot"</filename>),
3300 the sysroot creation function
3301 <filename>extend_recipe_sysroot</filename> is also added as
3302 a pre-function for those tasks whose dependencies are not
3303 through <filename>DEPENDS</filename> but operate similarly.
3304 </para>
3305
3306 <para>
3307 When installing dependencies into the sysroot, the code
3308 traverses the dependency graph and processes dependencies
3309 in exactly the same way as the dependencies would or would not
3310 be when installed from sstate.
3311 This processing means, for example, a native tool would have
3312 its native dependencies added but a target library would not
3313 have its dependencies traversed or installed.
3314 The same sstate dependency code is used so that
3315 builds should be identical regardless of whether sstate
3316 was used or not.
3317 For a closer look, see the
3318 <filename>setscene_depvalid()</filename> function in the
3319 <link linkend='ref-classes-sstate'><filename>sstate</filename></link>
3320 class.
3321 </para>
3322
3323 <para>
3324 The build system is careful to maintain manifests of the files
3325 it installs so that any given dependency can be installed as
3326 needed.
3327 The sstate hash of the installed item is also stored so that
3328 if it changes, the build system can reinstall it.
3329 </para></listitem>
3330 </itemizedlist>
3200 </para> 3331 </para>
3201</section> 3332</section>
3202 3333
diff --git a/documentation/ref-manual/ref-structure.xml b/documentation/ref-manual/ref-structure.xml
index 541a47e556..b0c4bfeaa4 100644
--- a/documentation/ref-manual/ref-structure.xml
+++ b/documentation/ref-manual/ref-structure.xml
@@ -801,11 +801,24 @@
801 <title><filename>build/tmp/sysroots/</filename></title> 801 <title><filename>build/tmp/sysroots/</filename></title>
802 802
803 <para> 803 <para>
804 This directory contains shared header files and libraries as well as other shared 804 Previous versions of the OpenEmbedded build system used to
805 data. 805 create a global shared sysroot per machine along with a native
806 Packages that need to share output with other packages do so within this directory. 806 sysroot.
807 The directory is subdivided by architecture so multiple builds can run within 807 Beginning with the &DISTRO; version of the Yocto Project,
808 the one Build Directory. 808 sysroots exist in recipe-specific
809 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
810 directories.
811 Thus, the <filename>build/tmp/sysroots/</filename> directory
812 is unused.
813 <note>
814 The <filename>build/tmp/sysroots/</filename> directory
815 can still be populated using the
816 <filename>bitbake build-sysroots</filename> command and can
817 be used for compatibility in some cases.
818 However, in general it is not recommended to populate
819 this directory.
820 Individual recipe-specific sysroots should be used.
821 </note>
809 </para> 822 </para>
810 </section> 823 </section>
811 824
@@ -894,6 +907,101 @@
894 </para> 907 </para>
895 </section> 908 </section>
896 909
910 <section id='structure-build-tmp-work-tunearch-recipename-version'>
911 <title><filename>build/tmp/work/<replaceable>tunearch</replaceable>/<replaceable>recipename</replaceable>/<replaceable>version</replaceable>/</filename></title>
912
913 <para>
914 The recipe work directory (<replaceable>recipe_work_directory</replaceable>).
915 </para>
916
917 <para>
918 As described earlier in the
919 "<link linkend='structure-build-tmp-sysroots'><filename>build/tmp/sysroots/</filename></link>"
920 section, beginning with the &DISTRO; release of the Yocto
921 Project, the OpenEmbedded build system builds each recipe in its
922 own work directory (i.e.
923 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>).
924 The path to the work directory is constructed using the
925 architecture of the given build (e.g.
926 <link linkend='var-TUNE_PKGARCH'><filename>TUNE_PKGARCH</filename></link>,
927 <link linkend='var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></link>,
928 or "allarch"), the recipe name, and the version of the recipe (i.e.
929 <link linkend='var-PE'><filename>PE</filename></link><filename>:</filename><link linkend='var-PV'><filename>PV</filename></link><filename>-</filename><link linkend='var-PR'><filename>PR</filename></link>).
930 </para>
931
932 <para>
933 A number of key subdirectories exist within each recipe
934 work directory:
935 <itemizedlist>
936 <listitem><para>
937 <filename><replaceable>recipe_work_directory</replaceable>/temp</filename>:
938 Contains the log files of each task executed for this
939 recipe, the "run" files for each executed task, which
940 contain the code run, and a
941 <filename>log.task_order</filename> file, which lists the
942 order in which tasks were executed.
943 </para></listitem>
944 <listitem><para>
945 <filename><replaceable>recipe_work_directory</replaceable>/image</filename>:
946 Contains the output of the
947 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
948 task, which corresponds to the
949 <filename>${</filename><link linkend='var-D'><filename>D</filename></link><filename>}</filename>
950 variable in that task.
951 </para></listitem>
952 <listitem><para>
953 <filename><replaceable>recipe_work_directory</replaceable>/pseudo</filename>:
954 Contains the pseudo database and log for any tasks executed
955 under pseudo for the recipe.
956 </para></listitem>
957 <listitem><para>
958 <filename><replaceable>recipe_work_directory</replaceable>/sysroot-destdir</filename>:
959 Contains the output of the
960 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>
961 task.
962 </para></listitem>
963 <listitem><para>
964 <filename><replaceable>recipe_work_directory</replaceable>/package</filename>:
965 Contains the output of the
966 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
967 task before the output is split into individual packages.
968 </para></listitem>
969 <listitem><para>
970 <filename><replaceable>recipe_work_directory</replaceable>/packages-split</filename>:
971 Contains the output of the <filename>do_package</filename>
972 task after the output has been split into individual
973 packages.
974 Subdirectories exist for each individual package created
975 by the recipe.
976 </para></listitem>
977 <listitem><para>
978 <filename><replaceable>recipe_work_directory</replaceable>/recipe-sysroot</filename>:
979 A directory populated with the target dependencies of the
980 recipe.
981 This directory looks like the target filesystem and
982 contains libraries that the recipe might need to link
983 against (e.g. the C library).
984 </para></listitem>
985 <listitem><para>
986 <filename><replaceable>recipe_work_directory</replaceable>/recipe-sysroot-native</filename>:
987 A directory populated with the native dependencies of the
988 recipe.
989 This directory contains the tools the recipe needs to build
990 (e.g. the compiler, Autoconf, libtool, and so forth).
991 </para></listitem>
992 <listitem><para>
993 <filename><replaceable>recipe_work_directory</replaceable>/build</filename>:
994 This subdirectory applies only to recipes that support
995 builds where the source is separate from the
996 build artifacts.
997 The OpenEmbedded build system uses this directory as a
998 separate build directory (i.e.
999 <filename>${</filename><link linkend='var-B'><filename>B</filename></link><filename>}</filename>).
1000 </para></listitem>
1001 </itemizedlist>
1002 </para>
1003 </section>
1004
897 <section id='structure-build-work-shared'> 1005 <section id='structure-build-work-shared'>
898 <title><filename>build/tmp/work-shared/</filename></title> 1006 <title><filename>build/tmp/work-shared/</filename></title>
899 1007
diff --git a/documentation/ref-manual/ref-tasks.xml b/documentation/ref-manual/ref-tasks.xml
index a3a41ebcc1..99e9f52b86 100644
--- a/documentation/ref-manual/ref-tasks.xml
+++ b/documentation/ref-manual/ref-tasks.xml
@@ -490,6 +490,18 @@
490 </para> 490 </para>
491 </section> 491 </section>
492 492
493 <section id='ref-tasks-prepare_recipe_sysroot'>
494 <title><filename>do_prepare_recipe_sysroot</filename></title>
495
496 <para>
497 Installs the files into the individual recipe work directories
498 (i.e. WORKDIR).
499 See the
500 "<link linkend='ref-classes-staging'><filename>staging</filename></link>"
501 class for more information.
502 </para>
503 </section>
504
493 <section id='ref-tasks-rm_work'> 505 <section id='ref-tasks-rm_work'>
494 <title><filename>do_rm_work</filename></title> 506 <title><filename>do_rm_work</filename></title>
495 507
diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 55f7a28c21..3699ae75bc 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -12776,6 +12776,35 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
12776 </glossdef> 12776 </glossdef>
12777 </glossentry> 12777 </glossentry>
12778 12778
12779 <glossentry id='var-SSTATE_SCAN_FILES'><glossterm>SSTATE_SCAN_FILES</glossterm>
12780 <info>
12781 SSTATE_SCAN_FILES[doc] = "Controls the list of files the OpenEmbedded build system scans for hardcoded installation paths."
12782 </info>
12783 <glossdef>
12784 <para role="glossdeffirst">
12785<!-- <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
12786 Controls the list of files the OpenEmbedded build system
12787 scans for hardcoded installation paths.
12788 </para>
12789
12790 <para>
12791 During a build, the OpenEmbedded build system creates a
12792 shared state (sstate) object during the first stage of
12793 preparing the sysroots.
12794 During the build, the object is scanned for hardcoded paths
12795 for original installation locations.
12796 The list of files that are scanned for paths is controlled
12797 by the <filename>SSTATE_SCAN_FILES</filename> variable.
12798 </para>
12799
12800 <para>
12801 For details on the process, see the
12802 <link linkend='ref-classes-staging'><filename>staging</filename></link>
12803 class.
12804 </para>
12805 </glossdef>
12806 </glossentry>
12807
12779 <glossentry id='var-STAGING_BASE_LIBDIR_NATIVE'><glossterm>STAGING_BASE_LIBDIR_NATIVE</glossterm> 12808 <glossentry id='var-STAGING_BASE_LIBDIR_NATIVE'><glossterm>STAGING_BASE_LIBDIR_NATIVE</glossterm>
12780 <info> 12809 <info>
12781 STAGING_BASE_LIBDIR_NATIVE[doc] = "Specifies the path to the /lib subdirectory of the sysroot directory for the build host." 12810 STAGING_BASE_LIBDIR_NATIVE[doc] = "Specifies the path to the /lib subdirectory of the sysroot directory for the build host."